Pedicle Screw Placement Using an Augmented Reality Head-Mounted Display in a Porcine Model
Summary
The augmented reality head-mounted display, Magic Leap, was used in combination with a conventional navigation system to place pedicle screws in a porcine model by adhering to a novel workflow. With a median insertion time of <2.5 min, submillimeter technical accuracy and 100% clinical accuracy were achieved according to Gertzbein.
Abstract
This protocol helps assess the accuracy and workflow of an augmented reality (AR) hybrid navigation system using the Magic Leap head-mounted display (HMD) for minimally invasive pedicle screw placement. The cadaveric porcine specimens were placed on a surgical table and draped with sterile covers. The levels of interest were identified using fluoroscopy, and a dynamic reference frame was attached to the spinous process of a vertebra in the region of interest. Cone beam computerized tomography (CBCT) was performed, and a 3D rendering was automatically generated, which was used for the subsequent planning of the pedicle screw placements. Each surgeon was fitted with an HMD that was individually eye-calibrated and connected to the spinal navigation system.
Navigated instruments, tracked by the navigation system and displayed in 2D and 3D in the HMD, were used for 33 pedicle cannulations, each with a diameter of 4.5 mm. Postprocedural CBCT scans were assessed by an independent reviewer to measure the technical (deviation from the planned path) and clinical (Gertzbein grade) accuracy of each cannulation. The navigation time for each cannulation was measured. The technical accuracy was 1.0 mm ± 0.5 mm at the entry point and 0.8 mm ± 0.1 mm at the target. The angular deviation was 1.5° ± 0.6°, and the mean insertion time per cannulation was 141 s ± 71 s. The clinical accuracy was 100% according to the Gertzbein grading scale (32 grade 0; 1 grade 1). When used for minimally invasive pedicle cannulations in a porcine model, submillimeter technical accuracy and 100% clinical accuracy could be achieved with this protocol.
Introduction
The correct placement of pedicle screws is important to avoid damage to neurovascular structures in and around the spine. The placement accuracy using the free-hand technique is highly variable1. By using 3D navigation, the accuracy is improved compared to traditional image-guided methods based on intraoperative fluoroscopy. Higher accuracy reduces the risk of revision surgery2,3.
With the estimation that mean life expectancy will continue to increase, an increasing number of elderly patients will need surgical spine procedures for various pathologies4. Minimally invasive approaches are gaining ground due to their lower morbidity, especially in the elderly5,6. However, these approaches are dependent on accurate navigational solutions. As navigation is image-based, efforts are being made to reduce the intraoperative radiation exposure of patients and staff7,8,9,10.
Augmented reality (AR) is an emerging technology in surgical navigation that aims to improve the accuracy and efficacy in the operating room (OR)11. AR superimposes computer-generated information onto a real-world view. This works especially well when the superimposed information is seen through an HMD. For that purpose, HMDs using head-up display technology have gained attention due to their small size, portability, and the possibility to maintain a direct line of sight. Several HMDs are available on the market today for AR navigation12,13,14,15,16.
The Magic Leap headset is an optical see-through HMD, including several cameras, a depth sensor, and inertial measurement units, which are used to determine the position and orientation of the headset in the environment. The purpose of this study was to evaluate the workflow of the Magic Leap HMD, combined with a conventional navigation system and a state-of-the-art mobile CBCT device, for intraoperative imaging in a realistic surgical environment.
Protocol
The procedure was performed in a conventional OR, equipped with a radiolucent OR table, a navigation platform, and a mobile CBCT device providing both 2D fluoroscopy and 3D CBCT images of high quality for AR navigation. Two porcine cadavers, which were approximately 80 cm in length and 45 kg, were used for the purpose of this study. The specimens were purchased commercially, and their use for this experiment did not require an ethical permit. All the devices, instruments, and software used within the described workflow are listed in the Table of Materials. The following step-by-step procedure was performed and repeated for each specimen.
1. Porcine cadaver specimen
- Place the porcine cadaver specimen on the operating table in the operating room.
- Drape the porcine cadaver specimen in sterile covers. Use incision film to cover the skin in the surgical field.
2. Identification of the vertebral levels of interest
- Using the CBCT scanner, identify the vertebral levels of interest by fluoroscopy. Use the wireless control tablet of the CBCT scanner to move the scanner to the desired position, align the x-ray beam, and to perform the fluoroscopy scan (Figure 1).
NOTE: The 2D scans can be immediately reviewed on the tablet. The vertebral levels are identified by looking for ribs on the fluoroscopy scan and counting upward or downward. - Attach the radiolucent navigation dynamic reference clamp to a spinous process in the area of interest by exposing the spinous process and fastening the clamp using the dedicated screwdriver. Then, attach the reflective spheres of the reference frame to the clamp (Figure 2).
- Perform a CBCT scan, and transfer the scan to the navigation platform (via LAN) (Figure 3). The navigation system camera tracks the CBCT scanner and the dynamic reference frame, enabling automatic patient registration using the Brainlab Loop-X Automatic Registration software on the navigation platform.
- Start the Spine & Trauma Navigation software on the navigation platform. Use the spinal pointer and the 2D navigation views to verify the accuracy of the patient registration on anatomical landmarks.
Figure 1: The wireless control tablet of the CBCT scanner. The tablet showing the fluoroscopy images from the CBCT. Please click here to view a larger version of this figure.
Figure 2: A schematic picture of the clamp attached to the spinous process. Please click here to view a larger version of this figure.
Figure 3: The Loop-X CBCT. The CBCT performing a scan on the draped pig cadaver with the reference attached. Please click here to view a larger version of this figure.
3. Instrument calibration
- Calibrate a navigated drill guide and a screw driver to the navigation system. For this purpose, select the instrument in the Brainlab Spine & Trauma Instrument Setup software, and then present the real instrument to the camera of the navigation system along with a calibration device. Move the instrument in a rotating motion while in contact with the calibration device until the navigation system recognizes the instrument. Once calibrated, track and visualize the instrument on both the 2D images and the 3D model in the HMD.
4. Head-mounted device fitting
- Ensure that each surgeon is fitted with a Magic Leap headset (the HMD). Ensure the HMD and the navigation platform are connected to the same network (WLAN connection for the HMD and LAN connection for the navigation platform).
- To establish the communication between the HMD and the Spine & Trauma Navigation software, look at the QR code displayed on the screen of the navigation platform. This starts the corresponding Mixed Reality application running on the HMD and transfer of data to the HMD.
- Perform the mixed reality alignment by looking at the spine reference array through the HMD for a few seconds. Wait for a 3D model of the spine, rendered based on the CBCT scan, to be accurately augmented onto the specimen in the HMD. In addition to the 3D overlay, look at the 2D navigation views and a second 3D model above the 2D navigation views (hover view) that are displayed in the HMD.
Figure 4: The view through the HMD. The surgeon's view through the HMD presenting both 2D and 3D information. The 3D overlay shows the planned 3D screws with protruding trajectory lines assisting instrument alignment. The lower 3D model is augmented onto the spine of the pig; additional information is provided in the 2D and 3D representations floating above, which can be positioned freely in the virtual space and switched on and off. Please click here to view a larger version of this figure.
5. Planning the pedicle screw placement
- Plan the pedicle screw paths based on the 3D registered augmented model, aligning them with the anatomy of the spine, and visualize in the HMD (Figure 5). Perform fine-tuning of the screw paths on the touchscreen of the navigation platform.
Figure 5: Pedicle screw path planning. The paths for the pedicle screws being planned using the HMD and the navigation pointer. Please click here to view a larger version of this figure.
6. Beginning the pedicle screw placement
- Make small skin incisions approximately 2 cm long with a scalpel for minimally invasive access to the pedicles based on the superimposed 3D model visible through the HMD (Figure 6).
- Using a minimally invasive technique, dissect the soft tissue, and dilate the canal with dilators until the pedicle entry point on the vertebral surface is reached.
- Adjust the depth of the drill guide to match the length of the screw planned for the pedicle. The planned screw length is displayed on the screen of the navigation system. Position and align the navigated drill guide to the planned path.
- Drill the pedicle using a power drill with a 4.5 mm drill bit (Figure 7). Drill according to the planned path; the drill guide stops the drill from going deeper than the planned depth.
- Estimate the time from the skin incision until the canal is drilled for each pedicle.
Figure 6: Minimally invasive incisions. The pig cadaver from above showing the minimally invasive incisions along the spine. To the right is the reference with the reflective spheres clamped to the spinous process. Please click here to view a larger version of this figure.
Figure 7: Drilling the pedicles. The pedicle is drilled with a power drill using the navigation visible through the HMD to align the drill guide to the pre-planned path. Please click here to view a larger version of this figure.
7. Visualization of the screw placement
NOTE: No screws were placed to avoid metal artifacts during evaluation.
- Perform a second CBCT to acquire X-ray images of the drilled vertebrae for the accuracy analysis. Ensure the drilled canal in the vertebra is clearly visible before using it for subsequent accuracy analyses.
8. Cannulating the spine
- Repeat the above procedure described in section 2, section 4, section 6, and section 7 to cover the next region of interest until the whole spine is cannulated.
- Repeat the same procedure (sections 1-8) using the second specimen.
9. Image analysis
- Match the obtained CBCT images to the navigation plan and make corrections according to the laboratory notes taken during the procedure.
- Have an independent reviewer evaluate all the images and grade the cannulations according to the Gertzbein grading scale, from 0 to 3. Grades 0 or 1 are considered accurate. Grades 2 or 3 are considered inaccurate.
- Fuse the trajectories of the planned paths and the cannulations, and define the technical accuracy as the deviation from the path at the entry and target. Measure the angular deviation.
Representative Results
In total, 33 navigated cannulations were performed. The time per cannulation and the clinical and technical accuracy were assessed on the postoperative CBCT scans (Figure 8).
Figure 8: The postoperative scan of a Gertzbein grade 0 cannulation. The scan includes the surgical plan for the pedicle cannulation, presented in the coronal, axial, and sagittal views. Note the close alignment of the virtual screw and the cannulated canal. Please click here to view a larger version of this figure.
The mean insertion time per cannulation was 141 s ± 71 s (median [range]: 151 [43-471]; Figure 9).
Figure 9: Histogram and box of the distribution of the pedicle cannulation times. Top, histogram of the distribution of the pedicle cannulation times (n = 33); bottom, the corresponding box plot showing the median, interquartile range, and an outlier. Please click here to view a larger version of this figure.
All 33 cannulations were considered clinically accurate according to the Gertzbein grading scale (32 grade 0; 1 grade 1; Table 1).
| Gertzbein Grade 0 | Gertzbein Grade 1 | Gertzbein Grade 2 | Gertzbein Grade 3 | Clinically Accurate | Clinically Inaccurate | Accuracy | |
| Number of screws | 32 | 1 | 0 | 0 | 33 | 0 | 100% |
Table 1: Clinical accuracy of implanted screws according to the Gertzbein grading scale. Grades 0 or 1 were considered accurate. Grades 2 or 3 were considered inaccurate.
To assess the technical accuracy, the deviation of each cannulation from its planned path was measured at the bone entry and at the bottom of the drill canal (Figure 10). The 3D measurements were performed by fusing the intraoperative scan, including the planned cannulation paths, with the postoperative scan of the cannulations. The angular deviation was calculated based on these data.
Figure 10: Overview of the measurement model for technical accuracy. Please click here to view a larger version of this figure.
This method was previously described by Frisk et al.12. For the 33 pedicle cannulations performed, the technical accuracy was 1.0 mm ± 0.5 mm (median [range]: 1.0 [0.4-3.3]) at the entry point (Figure 11) and 0.8 mm ± 0.1 mm (median [range]: 0.8 [0.6-4.6]) at the bottom of the drill canal (Figure 12). The angular deviation was 1.5° ± 0.6° (median [range]: 1.5 [0.3-5.0]; Figure 13).
Figure 11: Technical accuracy at the bone entry point. Top, the technical accuracy at the entry; bottom, the corresponding box plot showing the median, interquartile range, and an outlier. Please click here to view a larger version of this figure.
Figure 12: Technical accuracy at the target (tip of drill canal). Top, technical accuracy at the target (tip of the drill canal); bottom, the corresponding box plot showing the median, interquartile range, and outliers. Please click here to view a larger version of this figure.
Figure 13: Angular deviation compared to the planned path. Top, angle deviation from the planned path; bottom, the corresponding box plot showing the median and interquartile range. Please click here to view a larger version of this figure.
Discussion
In this study, a novel workflow for minimally invasive pedicle screw placement using an HMD in sterile conditions is described and its accuracy evaluated. There are several scientific reports on HMD systems for cranial and spinal navigation, two of which have gained FDA approval for clinical use17,18. Other studies have shown promising results in the usability of HMDs in sterile environments19,20, as well as good accuracy in phantom and cadaver studies12,13,21. The results of the current study support the usefulness and feasibility of the workflow in a sterile environment and can serve as an important basis for the clinical introduction of the current device.
This study is distinguished by the step-by-step description of the procedure in the OR. Using an integrated navigation concept, including intraoperative CBCT and HMD, the patient registration and image overlay can be automatized to save time and effort in the OR. Once the setup is completed and the surgeons are equipped with the eye-calibrated HMD, all the other steps can be performed seamlessly. A great advantage of the pre-planning of the screw trajectories is that any deviation from the correct path can immediately be visualized and corrected.
When the planning is complete, the trajectories can be seen through the pedicles and will match the anatomical angulations of the pedicles. Any trajectories not matching the angulation of the others will become evident, and the surgeon can then correct them to facilitate the subsequent rod placement. The planned trajectories are saved, and they can then be used to assess the technical accuracy after fusion to the postoperative scans. In this context, technical accuracy is a combination of the inbound error of the navigation system and the surgeon's ability to adhere to the planned path. Importantly, the possibility to perform a confirmation CBCT allows for the intraoperative revision of any screw that, despite navigation, may be incorrectly placed.
CBCT is a well-known and widely used imaging device for intraoperative navigation and postoperative verification. CBCT provides 3D images of superior quality compared to the 2D images from a C-arm, a device commonly used in spinal surgery. The image quality and diagnostic accuracy of CBCT are comparable to conventional CT. The time requirement for the setup and sterile draping is similar to that of a standard C-arm but with much better diagnostic quality imaging22,23,24,25.
The difference in technical accuracy between the entry point and the target point is a result of the fact that the accuracy at the entry point is highly dependent on the anatomy at the chosen entry point. If the entry point is placed on a slope on the bone surface, there always is a risk of skiving26,27. When the pedicle is entered, the rigid cortical walls will guide the device, and, hence, the deviation at the target will be smaller due to there being no room for wiggling.
The HMD provides a 3D model that is rendered from the intraoperative CBCT or preoperative imaging and augmented onto the actual spine. In addition, it displays 2D images in the axial, sagittal, and coronal planes, as well as a second 3D model that the surgeon can rotate and position anywhere in the virtual space, based on personal preference. Interaction with the display software is currently performed using a remote control. To use this remote control in a sterile environment, it would have to be placed in a sterile plastic bag. This is standard practice with several non-sterile handheld devices that have to be used in sterile environments. However, in a clinical environment, hand gestures or voice commands would be preferred. During navigation, virtual representations of the tracked instruments in the 2D and 3D views provide visual feedback to aid the surgeon.
The HMD itself has evolved, and the second generation of Magic Leap is lighter and has a larger field of view. The field of view is an important factor in the use of HMDs and represents one of the features that is constantly being developed further. The field of view of the Magic Leap was fully efficient for conducting this experiment and did not pose any limitations to the workflow. Each HMD has its own small computer that the surgeon needs to wear underneath their sterile gowns. The communication between the HMD and the navigation system is via Wi-Fi, and network limitations may result in latency. Despite this product being the first prototype, the current results indicate excellent clinical accuracy and submillimeter technical accuracy.
The limitations of this study are the small sample size and the porcine, cadaveric model. The possible effects of breathing and bleeding on the accuracy could not be evaluated. Although a minimally invasive technique was used, no screws were inserted. However, the screw canals were readily visible and allowed for an accurate assessment of accuracy without interference from metal artifacts.In conclusion, this paper provides a detailed description of a novel workflow for HMD AR navigation. When used for minimally invasive pedicle cannulations in a porcine model, submillimeter technical accuracy and 100% clinical accuracy could be achieved.
Disclosures
The authors have nothing to disclose.
Acknowledgements
None.
Materials
| Instrument tracking array spine & trauma 4-marker | Brainlab | ||
| Curve Navigation System | Brainlab | Navigation System |
|
| Disposable clip-on remote control | Brainlab | SmartClip | |
| Drill guide tube, handle with marker spheres, drill guide depth control insertable, drill bits | Brainlab | Drill guide and accessories | |
| Expedium | DePuy Synthes | Screwdriver |
|
| Instrument calibration matrix | Brainlab | Instrument Calibration Matrix |
|
| Loop-X | Brainlab | CBCT scanner |
|
| Magic Leap 2 | Magic leap Inc. | Mixed Reality headset |
|
| Navigation pointer spine | Brainlab | Navigation Pointer |
|
| Spine reference array for reference clamp carbon (4-Sphere Geometry) | Brainlab | Spine Reference Array |
|
| Spine reference clamp carbon with slider | Brainlab | Spine Reference Clamp  |
|
| TruSystem 7500 | Trumpf | Operating table | |
| Software | |||
| Mixed Reality Spine Navigation App for Magic Leap | Brainlab | Run on Curve Navigation System Version: 2.0 |
|
| PDM | Brainlab | Run on Curve Navigation System Version: 4.2 |
|
| Spine & Trauma Instrument Setup | Brainlab | Run on Curve Navigation System Version: 6.2 |
|
| Spine & Trauma Navigation 2.0 | Brainlab | Run on Curve Navigation System Version: 1.6 |
References
- Kosmopoulos, V., Schizas, C. Pedicle screw placement accuracy: A meta-analysis. Spine. 32 (3), 111-1120 (2007).
- Perdomo-Pantoja, A., et al. Accuracy of current techniques for placement of pedicle screws in the spine: A comprehensive systematic review and meta-analysis of 51,161 screws. World Neurosurgery. 126, 664-678 (2019).
- Elmi-Terander, A., et al. Augmented reality navigation with intraoperative 3D imaging vs fluoroscopy-assisted free-hand surgery for spine fixation surgery: A matched-control study comparing accuracy. Scientific Reports. 10, 707 (2020).
- Deyo, R. A., et al. Trends, major medical complications, and charges associated with surgery for lumbar spinal stenosis in older adults. JAMA. 303 (13), 1259-1265 (2010).
- Lee, D. G., Park, C. K., Lee, D. C. Clinical and radiological comparison of 2 level anterior lumbar interbody fusion with posterolateral fusion and percutaneous pedicle screw in elderly patients with osteoporosis. Medicine. 99 (10), 19205 (2020).
- Shamji, M. F., Goldstein, C. L., Wang, M., Uribe, J. S., Fehlings, M. G. Minimally invasive spinal surgery in the elderly: Does it make sense. Neurosurgery. 77, 108-115 (2015).
- Bourgeois, A. C., et al. Improved accuracy of minimally invasive transpedicular screw placement in the lumbar spine with 3-dimensional stereotactic image guidance: A comparative meta-analysis. Journal of Spinal Disorders and Techniques. 28 (9), 324-329 (2015).
- Innocenzi, G., Bistazzoni, S., D’Ercole, M., Cardarelli, G., Ricciardi, F. Does navigation improve pedicle screw placement accuracy? Comparison between navigated and non-navigated percutaneous and open fixations. Acta Neurochirurgica. Supplement. 124, 289-295 (2017).
- Fomekong, E., Safi, S. E., Raftopoulos, C. Spine navigation based on 3-dimensional robotic fluoroscopy for accurate percutaneous pedicle screw placement: A prospective study of 66 consecutive cases. World Neurosurgery. 108, 76-83 (2017).
- Yu, E., Khan, S. N. Does less invasive spine surgery result in increased radiation exposure? A systematic review. Clinical Orthopaedics and Related Research. 472 (6), 1738-1748 (2014).
- Burström, G., Persson, O., Edström, E., Elmi-Terander, A. J. A. N. Augmented reality navigation in spine surgery: A systematic review. Acta Neurochirurgica. 163 (3), 843-852 (2021).
- Frisk, H., et al. Feasibility and accuracy of thoracolumbar pedicle screw placement using an augmented reality head mounted device. Sensors. 22 (2), 522 (2022).
- Liebmann, F., et al. Pedicle screw navigation using surface digitization on the Microsoft HoloLens. International Journal of Computer Assisted Radiology and Surgery. 14 (7), 1157-1165 (2019).
- Molina, C. A., et al. Augmented reality-assisted pedicle screw insertion: A cadaveric proof-of-concept study. Journal of Neurosurgery. Spine. 31 (1), 139-146 (2019).
- Yanni, D. S., et al. Real-time navigation guidance with intraoperative CT imaging for pedicle screw placement using an augmented reality head-mounted display: A proof-of-concept study. Neurosurgical Focus. 51 (2), 11 (2021).
- Liu, H., et al. Percutaneous placement of lumbar pedicle screws via intraoperative CT image-based augmented reality-guided technology. Journal of Neurosurgery. Spine. 32 (4), 542-547 (2019).
- Bhatt, F. R., et al. Augmented reality-assisted spine surgery: An early experience demonstrating safety and accuracy with 218 screws. Global Spine Journal. , (2022).
- Felix, B., et al. Augmented reality spine surgery navigation: Increasing pedicle screw insertion accuracy for both open and minimally invasive spine surgeries. Spine. 47 (12), 865-872 (2022).
- Dennler, C., et al. Augmented reality in the operating room: A clinical feasibility study. BMC Musculoskeletal Disorders. 22 (1), 451 (2021).
- Cofano, F., et al. Augmented reality in medical practice: From spine surgery to remote assistance. Frontiers in Surgery. 8, 657901 (2021).
- Muller, F., et al. Augmented reality navigation for spinal pedicle screw instrumentation using intraoperative 3D imaging. Spine Journal. 20 (4), 621-628 (2020).
- Burström, G., et al. Intraoperative cone beam computed tomography is as reliable as conventional computed tomography for identification of pedicle screw breach in thoracolumbar spine surgery. European Radiology. 31 (4), 2349-2356 (2020).
- Edström, E., Burström, G., Nachabe, R., Gerdhem, P., Elmi Terander, A. A novel augmented-reality-based surgical navigation system for spine surgery in a hybrid operating room: Design, workflow, and clinical applications. Operative Neurosurgery. 18 (5), 496-502 (2020).
- Feng, W., Wang, W., Chen, S., Wu, K., Wang, H. O-arm navigation versus C-arm guidance for pedicle screw placement in spine surgery: A systematic review and meta-analysis. International Orthopaedics. 44 (5), 919-926 (2020).
- Cewe, P., Vorbau, R., Omar, A., Elmi-Terander, A., Edstrom, E. Radiation distribution in a hybrid operating room, utilizing different X-ray imaging systems: Investigations to minimize occupational exposure. Journal of Neurointerventional Surgery. 14 (11), 1139-1144 (2021).
- Su, B. W., et al. An anatomical study of the mid-lateral pars relative to the pedicle footprint in the lower lumbar spine. Spine. 34 (13), 1355-1362 (2009).
- Burström, G., Nachabe, R., Persson, O., Edstrom, E., Elmi Terander, A. Augmented and virtual reality instrument tracking for minimally invasive spine surgery: A feasibility and accuracy study. Spine. 44 (15), 1097-1104 (2019).
