A Raytheon BBN-led team will develop an artificial intelligence tool, Medical Assistance, Guidance, Instruction and Correction, or MAGIC, to assist combat field medics in 50 medical skills applied during common procedures. The medical assistant is being developed under a DARPA award. The built-in cameras of the AR goggles observe the user’s actions, and AI verifies that steps are followed correctly, providing assistance—on request or when intervention is needed—in the form of auditory suggestions or visual overlays in the headset.
The tool will use DARPA Perceptually-enabled Task Guidance program’s consumer off-the-shelf augmented reality goggles, which harness the AI to use visual and audible sensors that track tasks and guide field medics in performing procedures correctly and efficiently. To train the AI, the team will collect and label combat medical data sets comprising:
more than 2,500 videos recorded in stereo to provide 3D data.
nearly 50 million images to identify the correct sequence of procedures.
“The combat medical environment is challenging and chaotic. Our goal for the Raytheon BBN MAGIC AI tool is to help support personnel to provide guidance as needed without disrupting their concentration,” explained Brian VanVoorst, Raytheon BBN scientist.
Those prompts can include next steps, follow-on actions, or critical measurements such as elapsed time or dosing. Raytheon-BBN is teamed with Valkyries Austere Medical Solutions for this work. The first demonstration of the new tool is planned in 18 months. Training data collected for the program will be shared with the research community. Raytheon Intelligence & Space, a Raytheon Technologies business, delivers the disruptive technologies our customers need to succeed in any domain, against any challenge. A developer of advanced sensors, cyber services and software solutions, Raytheon Intelligence & Space provides a decisive advantage to civil, military and commercial customers around the world.
