F-18 cockpit mixed reality pilot trainer
F-18 cockpit mixed reality pilot trainer

Mixed Reality Tracking System for Future Pilot Training by Vrgineers and ART

Vrgineers and Advanced Realtime Tracking demonstrate the combination of XTAL 3 headset and SMARTTRACK3/M in a mixed reality pilot trainer. The partnership between these two technological companies started in 2018. At IT2EC 2023 in Rotterdam, the integrated SMARTTRACK3/M into an F-35-like Classroom Trainer manufactured and delivered to USAF and RAF will be for display. This unique combination of the latest ART infrared all-in-one hardware and Vrgineers algorithms for cockpit motion compensation creates an unseen immersion for every mixed reality training. One of the challenges in next-generation pilot training using virtual technology and motion platforms is the alignment of the pilot’s position in the cockpit. By overcoming this issue, the simulator industry is moving forward to eliminate the disadvantages of simulated training.

“We are continuously working on removing the technological challenges of modern simulators, one of which is caused by front-facing camera position distance from users’ eyes. We are developing advanced algorithms for motion compensation to minimize the shift between virtual and physical scene, making experience realistic. The durability and compact size of SMARTTRACK3/M, which was optimized for using in cockpits, allows us as training device integrator to make it a comprehensive part of a simulation,” says Marek Polcak, CEO of Vrgineers.

“This is the application SMARTTRACK3/M was designed for. ,We have taken the proven hardware from the SMARTTRACK3 and adapted it to the limited space available. As a result, we have the precision and the reliability of a seasoned system in a form factor fitting to simulator cockpits” says Andreas Werner, business development manager for simulations at ART.

F-18 cockpit mixed reality pilot trainer. (Photo by Vrgineers)

SMARTTRACK3/M embedded two infrared cameras with LEDs is a pre-calibrated system, ready to go with durability to sustain continuous usage for 5+ years without needing service. It creates the perfect solution to be embedded in professional training platforms. It is a fully capable 6DOF system that provides real-time transitional and rotational data up to 240Hz. All the data motion and rotation calculations are performed by the SMARTTRACK3/M device itself, easing the computer performance, which can be utilized to run more graphically demanding applications. Thanks to its ethernet connection, it offloads USB traffic, often overloaded when using high-resolution mixed reality cameras.

Advanced Realtime Tracking GmbH & Co. KG (ART) offers extensive options for object tracking and localization for a wide variety of applications, including customer-specific requirements. Since 1999, Advanced Realtime Tracking GmbH & Co. KG is one of the leading manufacturers of premium optical motion tracking systems for Virtual and Augmented Reality (VR/AR).

Vrgineers is a leader in developing immersive technologies focused on training professional fighter jets and helicopter pilots. The company develops and manufactures its XTAL headset for virtual and mixed reality. The XTAL headset is developed based on the priorities of professional pilots and instructors from the international aviation community to meet the most demanding requirements for realistic simulation. The company achieved this through the unique combination of a wide field of view and high image quality that XTAL brings.