We developed a ROS2 module that allows to integrate the Robot Operating System 2 with Unity3D for performance-oriented AV/ADAS simulation.
Enter the search phrase
The importance of simulation for the development of autonomous vehicles and other robots is well understood. As we were creating a simulation based on the Unity3D engine for one of our customers, our goal was to interface with ROS, a middleware and package library which offers a lot of added value for robotic projects. The vehicle automation that we were developing together was to run on ROS2 and it was our goal for the simulation to provide data at the same standard and to work with the same perception, location and path planning algorithms.
ROS and Unity3D don’t easily mix because the programming languages, key concepts and coordinate frames are all different. The problem was temporarily addressed by a third-party bridge, which connected both worlds through WebSockets. Apart from only working with the old version of ROS, the bridge had significant reliability and communication issues. Since our simulation involves multiple vehicles and high-resolution sensors, performance of communication is essential. The bridge was failing even with 20% of our minimal requirements and the issue was at the core of its design. We clearly needed something much better.
We have developed a high-performance communication module for Unity3D, based on our advanced fork of ROS2 C# client. The results not only met our expectations but exceeded them:
Prepared and applied full laboratory setup for generating highly accurate ground truth datasets for vehicle interior sensing including driver monitoring in reference to distraction detection. We used a high-end motion capture system in the manufactured vehicle mock-up.Read more
Dedicated driving simulator dataset collection campaign for generating ground truth datasets drowsiness detection. Our team applied a combination of eye closure metrics, driving performance and physiological data analysis (ECG and EEG).Read more