Open-source simulation for robotics with O3DE

Our focus

Robotec.ai has been working on simulations for robotics since its inception. We worked with several engines and built simulation platforms on top of them.

From the beginning, we also decided to work with modern ROS (ROS 2), the most important and quickly growing standard in robotics. We benefit from using ROS in our projects and have contributed to its development. We care about our simulation platforms and modules being well integrated with ROS to provide robotics and software engineers the best development experience.

Goals of simulation

Simulation for robotics is a complex subject. There are many solutions on the market and in the open source. Picking the optimal solution for your use case might require quite some research and deliberation. Typically, simulation serves one or more of the following goals:

Development of robotics stack, including perception, planning, and varying degrees of autonomy. Thanks to simulation, teams can prototype faster, which is further enhanced using ROS.
Validation and testing. Simulation can be used to validate behavior and changes in the software stack. This is often done through validation scenarios, integrating with CI/CD for regression testing, providing dashboards with test results, etc.
Synthetic data generation. Simulation data is easier, cheaper, safer to obtain, and unconstrained by various limitations to the acquisition of real data. Machine learning models for tasks such as detection can be trained on a mix of synthetic and real data.

Applying simulation to each of these goals comes with its specific challenges, and there is plenty to learn about how to overcome them best.

Simulation platform is not a Highlander

We should not think that “there can be only one”. For example, a high-fidelity simulation of robots’ tracks on rough, deformable terrain will have a conflicting set of requirements with a simulation of an operating fleet of agricultural robots. One needs to focus on a dedicated model and plenty of precision, while the other aims to combine multiple elements crucial to the operation, demanding in terms of performance, such as sensor simulation, and vehicle dynamics.

As with everything in engineering, it is important to pick the right tools for the job.

Game Engines for simulation

Despite limitations such as lower accuracy of physical simulation and limited control loop frequency, game engines are widely used for robotic simulation. Performance, rendering, a library of assets, and ease of development are among the reasons why we see them applied to robotics.

The most dominant ones are Unity3D and Unreal Engine, and they are used for diverse types of simulations such as automotive, robotic arms, warehouse logistics, and other types of robotic applications. They are typically used with robotic stacks based on ROS since it is best to replace the robot’s body and its environment with a simulation seamlessly.

Unfortunately, from the perspective of a ROS developer, these game engines did not seem to get it quite right. Mapping out concepts between the ROS ecosystem and such simulation can be challenging. Game engines by default use bridges or wrappers for ROS communication, which introduces an additional layer to maintain and affects performance. They are, unlike ROS, closed source and have associated licensing costs. With Unity3D, there is an added issue due to C#, introducing a new competence requirement for a robotics team.

O3DE (Open 3D Engine): a rising star

O3DE is an open-source game and simulation engine governed by the Linux Foundation. It is available under Apache 2.0 license and free to use for any purpose. The engine is written in C++. You can read more about O3DE features in this guide and check its GitHub repository here.

O3DE embraces the modular approach through Gems, optional extensions to the engine. At Robotec.ai, we are developing the ROS 2 Gem in cooperation with AWS Game Tech.

We have decided that robotic simulation enabled by this Gem will be based on modern ROS, and thus, no wrappers or bridges are needed. Yes, you can write ROS 2 code in your simulation project! O3DE with this Gem will respect your workspace overlays, handle simulation time, and supply useful components for common tasks such as publishing transform frames and applying namespaces. It supports multi-robot simulation by default.

We strive to incorporate our experience with simulation and ROS 2 into this Gem so that it gets the best of two worlds: Gazebo and game engines.

Interested in trying it out? The Gem works with Galactic and Humble. It is under active development and already quite useful, as demonstrated by a couple of projects which you can check out:

Fruit-picking robots (Apple Krakens) in a scenic orchard, integrated with ROS 2 navigation stack and custom automation. This demo was showcased during ROSCon 2022 in Kyoto and prepared in collaboration with AWS Game Tech and Open Robotics.
A robot vacuum in a beautiful apartment, integrated with ROS 2 navigation stack. The demo was prepared in collaboration with AWS Game Tech and showcased during the re:MARS event.