Justyna Toton Oct 12, 2023 - Robotics

Intersections of Potential: Game Engines for Robotic Applications

What do gaming and robotics have in common? 3D engines, which we constantly use in our work, are typically associated with gaming, but their applications go beyond the gaming industry. While many of the games we love are built in Open 3D Engine (O3DE), Unity, and Unreal, these three 3D engines also make weaves in the robotic industry, as they offer plenty of value for simulation users. With a robust ecosystem, modularity, and ease of use, they are a compelling solution for those developing robotic simulations.

The use of gaming engines in robotics

Gamers and robotics simulation developers desire the same thing: a highly photorealistic world where multiple agents can interact without any hindrances to graphics quality and real-time performance.

Many robotic use cases require high performance, and game engines offer assets for fast, photorealistic robotic world-building that meets the criteria. With rendering capabilities such as dynamic lighting, global illumination, shadow mapping, post-processing effects, and advanced shader systems, they can significantly enhance the visual fidelity of robotic simulations. As a result, engineers can develop complex simulations that mimic real-world physics, lighting conditions, and object interactions with great accuracy and at a relatively low cost.

The list does not end here: Game engines are designed to simulate and control multiple agents within a virtual environment, which is helpful for scenarios that involve large numbers of robots, such as fleet automation. It is crucial for the logistics and mining industry, where optimizing entire robotic systems often determines productivity.

Fruit-picking robots (Apple Krakens) in a scenic orchard, integrated with ROS 2 navigation stack and custom automation.

Among all game engines, O3DE holds a special place in the hearts of our Robotec team. Back in March 2023, Robotec joined The Open 3D Foundation (O3DF) to help build the open-source future for robotic applications. At Robotec, we use O3DE in many of our simulation projects because it perfectly addresses the challenges of modern robotics.

The O3DE Engine had a certain appeal to the robotic community from the start because, like ROS, it is an open-source project. For the robotic community that largely operates on Linux, the community-oriented nature of the engine is a huge advantage. Additionally, as it is C++-based and modular, ROS developers face little problems with adaptation to the O3DE environment.

Why do many simulation developers favor O3DE? It significantly reduces development time in many cases, as it offers a vast array of pre-built assets and tools.

On top of that, the physics engine of O3DE can be great for testing the dynamics of robot behavior, especially during movement and collisions of all sorts. While there are many clear advantages of using game engines for robotic simulation purposes, we must remember that many simulation challenges are irrelevant in the gaming world. The timing of operations and determinism of simulation that plays a significant role in robotics is not necessarily important in gaming. For a long time, things such as a robot description format were not very compatible with game engines.

Luckily, things have changed, and now O3DE has its own ROS 2 GEM, a direct ROS 2 integration in O3DE. To see ROS 2 GEM in action, head to our blog post.

Robotic simulations with the O3DE Engine

ROS 2 Gem is a set of components that enable easier engine use in robotic simulations. Our Robotec team developed this Gem in cooperation with AWS Game Tech to simplify the workflow of simulation developers and build something for the benefit of the entire robotic community. We took advantage of the fact that O3DE is C++-based, just like ROS, and bridged the gap between them.

The Gem enables robotic simulation based on modern ROS, eliminating the need for bridges or wrappers. Developers can write ROS 2 code directly in their simulation projects, benefit from workspace overlays, handle simulation time, and access useful components for tasks like publishing transform frames and applying namespaces. Multi-robot simulation is supported by default.

The O3DE engine is relatively new and under constant development, which comes with its own challenges when teams encounter complex problems. On the other hand, we are granted unprecedented control over our projects and unique chances for problem-solving.

Necessity has always been a mother of invention, so working with O3DE pushed our teams to think outside the box and effectively tackle non-standard problems.

To contribute to ROS 2 Gem, visit our GitHub repository.

Interested in our latest projects? Follow us on LinkedIn to always stay updated!