Virtual Reality for Synergistic Surgical Training and Data Generation
Authors:
Adnan Munawar,
Zhaoshuo Li,
Punit Kunjam,
Nimesh Nagururu,
Andy S. Ding,
Peter Kazanzides,
Thomas Looi,
Francis X. Creighton,
Russell H. Taylor,
Mathias Unberath
Abstract:
Surgical simulators not only allow planning and training of complex procedures, but also offer the ability to generate structured data for algorithm development, which may be applied in image-guided computer assisted interventions. While there have been efforts on either develo** training platforms for surgeons or data generation engines, these two features, to our knowledge, have not been offer…
▽ More
Surgical simulators not only allow planning and training of complex procedures, but also offer the ability to generate structured data for algorithm development, which may be applied in image-guided computer assisted interventions. While there have been efforts on either develo** training platforms for surgeons or data generation engines, these two features, to our knowledge, have not been offered together. We present our developments of a cost-effective and synergistic framework, named Asynchronous Multibody Framework Plus (AMBF+), which generates data for downstream algorithm development simultaneously with users practicing their surgical skills. AMBF+ offers stereoscopic display on a virtual reality (VR) device and haptic feedback for immersive surgical simulation. It can also generate diverse data such as object poses and segmentation maps. AMBF+ is designed with a flexible plugin setup which allows for unobtrusive extension for simulation of different surgical procedures. We show one use case of AMBF+ as a virtual drilling simulator for lateral skull-base surgery, where users can actively modify the patient anatomy using a virtual surgical drill. We further demonstrate how the data generated can be used for validating and training downstream computer vision algorithms
△ Less
Submitted 15 November, 2021;
originally announced November 2021.
