How augmented reality could guide facial reconstruction surgeries

A Japanese research team has created an augmented reality (AR) system that allows them to create 3D simulations of facial reconstructive procedure results, projecting the images over a patient’s face during surgery.

Researchers at Osaka Medical College in Japan reported that they have developed and tested the AR system for evaluating improvements of the body surface, according to a news release from the American Society of Plastic Surgeons and Wolters Kluwer Health. They used it in eight different cases and found that AR could be a helpful guide to planning, performing and evaluating the results of facial reconstruction surgeries.

“Augmented reality is a technology that combines computer-generated images on a screen with a real object or scene,” Drs. Koichi Ueda and Daisuke Mitsuno, researchers involved in the study, said in a press release. “We sought to develop a sophisticated yet simple and modifiable AR technique for use during plastic and reconstructive surgery.”

Using a high-definition digital camera, the researchers got a 3D image of the facial surface and used computed tomography scans to get digital information about the underlying bones for each patient. The digital data was used to create 3D simulations of the desired facial reconstruction results.

The surgeon used commercially-available smart glasses to overlay the 3D digital simulation image over the patient’s face during the procedures. The research group used free and open source software to solve technical problems like manipulating and displaying the simulations and lining them up properly with the projected surgical location.

The researchers were able to change the technical settings for each surgical procedure that was performed to address the limitations of the AR system. While the system was not used as a surgical guide, it was useful for seeing the planned correction to confirm the final outcome of the procedure.

“In all cases in this study, the body surface contour after the procedure and the ideal postoperative image almost coincided,” Dr. Ueda said.

Ueda, Mitsuno and colleagues hope to perform more studies of the system during plastic and reconstructive surgeries to make the display method better. They hope that future developments will allow them to evaluate improvements on the body surface to enable simple navigation of internal organs.

“Our findings are not only useful for body surface evaluation but also for effective evaluation of AR technology in the field of plastic surgery,” Ueda and Mitsuno said.

The research was published in the American Society of Plastic Surgeons journal.

(Learn from some of the medical device industry’s top executives and experts at DeviceTalks Boston on Oct. 2.)