Use of AR, VR and Navigation Linked Heads-Up Display throughout the Neurosurgical Workflow
Anthony Costa, Leslie Schlachter, Dominic Nistal, Joshua Bederson, Raj Shrivastava, Holly Oemke Madarash, Margaret Pain
Introduction: Virtual reality (VR), augmented reality (AR), and heads up display (HUD) can improve situational awareness of patient-specific anatomy. How best to integrate AR/VR in the neurosurgical workflow is still unclear.
Objective: Develop image acquisition and review strategies for optimal use of segmentation software to create 3D representations of patient-specific anatomy for intraoperative use. Define the features that optimize each phase of the AR/VR workflow: preoperative preparation, surgical planning, intraoperative practice and navigation update.
Methods: Pre-operative imaging is based on a standardized rubric (figure 1). VR reconstructions are used to define critical structures and surgical approach. This guides an AR segmentation strategy to narrow the scope of the segmented model. The AR model is applied to the operative field via image injection and HUD projection through the microscope, navigation accuracy is adjusted intraoperatively using intracranial landmarks when required. (Figure 2)
Results: VR reconstructions and pre-operative imaging were used to guide AR model creation in a series of 200 patients. The most common diagnoses were meningioma (n=73), aneurysm (n=21), and AVM (n=15). Critical structures identified using the AR system were lesion (n=204), blood vessels (n=117) and cranial nerves (n=136). AR image injection via HUD was useful in distinct phases of operation: skin incision, craniotomy design, approach, and resection. (Figure 3) Navigation update allowed refinement of the registration in 20 cases (10%).
Conclusions: VR can improve understanding of critical anatomical relationships. Successful AR models tend to have 2-4 different structures painted and are useful in multiple phases of each operation. Navigation update appears to be a significant advance in utility of HUD.