Multimodal Cross-Device and Marker-Free Co-Registration of Preclinical Imaging Modalities

A method for multimodal image co-registration that does not require fiducial markers for production scans is presented. The phantom-based approach generates a differential transformation between the coordinate systems of two imaging modalities.

Residual of fusion and validating the differential transformation
Upon calculating the differential transformation, the software displays a residual of fusion in millimeters, representing the root mean square error¹⁹ of the transformation. If this residual exceeds the order of magnitude of the voxel size, it is advisable to inspect the datasets for general issues. However, as all images have slight distortions, the residual cannot become arbitrarily small; it only reflects the fit of the markers used. For example, a co-registration with three markers may result in a smaller residual on the same datasets than a transformation with four well-distributed markers. This occurs because the markers themselves may be overfitted when fewer fiducials are employed. The accuracy across the entire dataset improves with a greater number of markers.

The quantitative accuracy of the method depends on the specific pair of devices used. The calculated differential transformation between the coordinate systems of two devices can be validated following these steps: Adhering to step 4 of the protocol, but using the phantom with fiducial markers as the "sample" again. Placing the phantom in any position, ensuring it's different from the one used for estimating the differential transformation. It's also possible to use a different phantom suitable for the respective modalities if one is available. Next, applying the differential transformation determined earlier (step 4.2.5) to align the two modalities. Then, placing markers on the images from both modalities as per step 3.2 of the protocol. To calculate the fusion residual for these markers, click on Menu Fusion > Register Overlay to Underlay > Showing Residual Score.

The residual error describes the average misplacement of the signal and should be in the order of the voxel size. Concrete acceptance thresholds are application-dependent and may depend on several factors, such as the stiffness and accuracy of the imaging systems but can also be affected by image reconstruction artifacts.

Troubleshooting self-consistency
Often, difficulties with self-consistency arise from unreliable placement. A common error is placing the carrier in a laterally reversed position. Ideally, it should be mechanically inserted into the imaging device in only one direction. If this is not feasible, comprehensible markings should be added for the user. Another frequent issue is the possibility of movement in the longitudinal axis, making axial positioning unreliable. Using a spacer that can be attached at one end to secure the mouse bed in place is recommended. Custom spacers can, for example, be quickly and easily created by 3D printing them. However, some devices cannot provide self-consistency with varying fields of view. In such cases, contacting the vendor is advised, who should confirm the incompatibility and potentially address it in a future update. Otherwise, the method remains reliable if an identical field of view is maintained for all scans, including calibration and production imaging.

For some production scans with deviating placement, transformation to the calibrated position is possible, if sufficient carrier structure is discernible. For in vivo imaging, the sedated animal must remain in one carrier, and constructing a single carrier that fits securely in both devices is not always achievable. Often, a mouse bed for a tracer-based modality is used, and then placement is improvised into a CT device. For instance, in Figure 5A, an MPI mouse bed was placed atop a CT mouse bed due to mechanical constraints. Axial leeway and the possibility of rolling make this positioning unreliable. In such cases, it is recommended to design an adapter that replaces the lower mouse bed and allows for an interlocking fit. It may, for example, use trunnions attached to the lower part and additional holes in the bottom of the upper mouse bed.

However, retrospective correction for existing images is possible, as the mouse bed is detectable in the CT image. The protocol necessitates calibration scans, followed by calculating a differential transformation of the overlay to the underlay. The procedure is similar but must also map each individual production CT scan to the calibration scan, using the mouse bed structures as fiducials.

Figure 5: Troubleshooting placement. (A) An MPI mouse bed is placed on top of a CT mouse bed. Hence, the position in the CT cannot be reliably reproduced. Self-consistency can be achieved by fusing each CT image to the reference CT image used for estimating the differential transformation. (B–D) Simplified to 2D. (B) Each production CT image is loaded as an overlay and registered to the reference CT image (underlay) using structures of the mouse bed visible in the CT. The corrected production CT image is now consistent with the reference CT and can be used with the differential transformation T. (C) An MPI overlay is registered to the reference CT image using the fiducial markers of a phantom. (D) The multimodal images are assembled. For this purpose, each CT image is mapped to the reference position with its individual differential transformation. Subsequently, the MPI overlay is also registered to the reference position using the differential transformation, which is valid for all images of the device. Please click here to view a larger version of this figure.

To map the production CT scans to the calibration scan, refer to section 3 of the protocol, incorporating the following modifications. For clarity, the description continues using the example of a CT underlay and MPI overlay: In step 3.1, load the CT calibration scan (image A) as the underlay and the CT scan to be corrected as the overlay. Utilize structures of the MPI mouse bed either as markers for step 3.2 or as visual references for step 3.3. Bypass step 3.4, but save the overlay represents the corrected CT volume (Menu File > Overlay > Saving overlay as). In the subsequent dialog box, input a new name and click on save. Close the overlay by navigating to Menu File > Overlay > Closing overlay. Load the next CT scan that requires correction as the overlay and resume the procedure from step 3.2 of the protocol. The concept underlying this step is illustrated in Figure 5B.

The mouse bed is now virtually aligned identically to the calibration scan in all the recently saved CT volumes. As part of the standard procedure, the calibration scan is registered to the MPI images using the differential transformation T (Figure 5C). To subsequently merge the CT image with MPI, always use the corrected CT volume (Figure 5D).

Troubleshooting flipped images and scaling
The registration method introduced here assumes reasonably accurate image quality and only adjusts rotation and translation. It does not correct for flipped images or incorrect scaling. However, these two issues can be addressed manually prior to calculating the differential transformation.

Inconsistencies between data formats from different manufacturers may cause some datasets, particularly those in DICOM format, to be displayed as mirror-inverted in the software. As phantoms and mouse beds are often symmetrical, this issue may not be immediately apparent. Detecting flipped images is easier when the scan contains recognizable lettering in the respective modality, such as the raised lettering in the correct orientation seen in the phantom in Figure 3H. In the example illustrated in Figure 6, CT data is loaded as the underlay, and MPI data is loaded as the overlay. It is an in vivo scan of a mouse placed in an MPI mouse bed with attached fiducial markers. The MPI mouse bed is situated on top of a µCT mouse bed (Figure 6A). By adhering to the protocol and marking the fiducials in both the underlay and overlay in a consistent direction of rotation, a visibly incongruous result is produced (Figure 6B). Upon closer inspection, however, the problem can be identified. The fiducials form an asymmetrical triangle. Observing the sides of the triangle in the axial view (Figure 6C, D) from the shortest to the middle to the longest, a clockwise rotation is evident in the CT data, while a counterclockwise rotation is apparent in the MPI data. This demonstrates that one of the images is laterally inverted. In this instance, we assume the CT data to be accurate. To rectify the MPI overlay, the image is flipped: to do that, switch the selected layer to overlay and click on Menu Edit > Flip > Flip X. The differential transformation calculated by the software encompasses all necessary rotations, so "Flip X" is sufficient even if the image appears flipped in another direction.

Figure 6: Troubleshooting transformation. CT data are loaded as underlay with a voxel size of 0.240 mm, and MPI data as overlay with a voxel size of 0.249 mm. The mouse bed contains fiducial markers. (A) 3D view of the uncorrected overlay image. The fiducials in the CT underlay are indicated by arrows. The fiducials in the MPI overlay are visible as spheres in the NIH color table. (B) Mismatched result of a transformation performed without appropriate corrections. Residual of fusion = 6.94 mm. (C) Measurement of the distances between the fiducials in CT. Clockwise rotation from the shortest to the longest distance. (D) Measurement of the distances between the fiducials in MPI. Counter-clockwise rotation from the shortest to the longest distance. Comparison with the CT measurements results in a scaling factor of 0.928774. (E) Corrected overlay after flipping and scaling. (F) Transformation with matching results in 3D view. (G) Transformation with matching results in axial view. Residual of fusion = 0.528 mm. Please click here to view a larger version of this figure.

Datasets with incorrect voxel sizes can also be corrected manually. Since the dimensions of the phantom should be known, this can be verified in the image. The simplest method is using an edge of known length. Press [Ctrl + right mouse button] at one end of an edge, and while holding the button down, move the mouse pointer to the other end of the edge and release the button. In the subsequent dialog, the software displays the length of the measured distance in the image. In the example illustrated in Figure 6, it is apparent that the sizes are not congruent when comparing the distances between the fiducials in both modalities (Figure 6C,D). Again, the CT data is assumed to be accurate. To modify the scaling, a scaling factor (SF) is computed. As the ratio of the lengths (CT/MPI) is not precisely identical for each side of the triangle, the mean quotient is calculated: SF = ((l_1CT/l_1MPI) + (l_2CT/l_2MPI) + (l_2CT/l_2MPI)) / 3.

Subsequently, adjust the voxel size of the overlay by multiplying each dimension by SF. To achieve this, switch the selected layer to overlay and open Menu Edit > Change Voxel Sizes. Compute each dimension, enter the value, and then click on OK. The result of both corrections is shown in Figure 6E. Following this, the overlay is registered to the underlay according to the protocol. The resulting alignment is displayed in Figure 6F,G. While this provides a quick solution for correcting an existing scan, we recommend calibrating the imaging device for production use.

Limitations
This method is limited to spatial co-registration of existing volumetric data composed of cube-shaped voxels. It does not include a reconstruction process that computes the volume from raw data generated by the imaging device (e.g., projections in CT). Various image enhancement techniques are associated with this step, such as iterative methods^20,21 and the application of artificial intelligence²¹. Although the described method is, in principle, applicable to all modalities that produce 3D images with cube-shaped voxels, it cannot be employed for fusing 3D data with 2D data, like an MRI volume combined with 2D infrared thermography²² or fluorescence imaging, which may be relevant in image-guided surgery applications. The registration of 3D data does not correct for distortions, such as those that occur in MRI images at the coil edge. While not mandatory, optimal results are achieved when distortions are corrected during the reconstruction process. The automated transformation also does not address flipped images or incorrect scaling. However, these two issues can be manually resolved as outlined in the troubleshooting section.

Significance of the method
The proposed method eliminates the need for fiducial markers in production scans, offering several advantages. It benefits modalities for which marker maintenance or frequent replacement is required. For example, most MRI markers are based on moisture but tend to dry out over time, and radioactive PET markers decay. By removing the necessity for fiducials in production scans, the field of view can be reduced, leading to shorter acquisition times. This is helpful in high-throughput settings to reduce costs and to minimize x-ray dose in CT scanning. A decreased dose is desirable because radiation can impact the biological pathways of test animals in longitudinal imaging studies²³.

Moreover, the method is not limited to specific modalities. The trade-off for this versatility is that fewer steps are automated. A previously published method for fusing µCT and FMT data employs built-in markers in a mouse bed for every scan and can perform automated marker detection and distortion correction during reconstruction²⁴. Other methods eliminate the need for markers by utilizing image similarity. While this approach yields good results and can also correct distortions²⁵, it is only applicable if the two modalities provide similar enough images. This is usually not the case in the combination of an anatomically detailed modality and a tracer-based modality. However, these combinations are necessary for assessing the pharmacokinetics of targeted agents²⁶, which have applications in areas such as anticancer nanotherapy^{27, 28}.

Because quality control is less rigorous in preclinical compared to clinical applications, misalignment of combined imaging devices is a recognized issue²⁹. Data affected by this misalignment could be improved retrospectively by scanning a phantom and determining the differential transformation, potentially reducing costs and minimizing animal harm. In addition to the demonstrated method that employs fiducial markers to calculate a differential transformation, which is then applied to production scans, further possibilities for image fusion are described and used. An overview, which includes references to various available software, can be found in Birkfellner et al.³⁰.

In conclusion, the presented method offers an effective solution for multimodal image co-registration. The protocol is readily adaptable for various imaging modalities, and the provided troubleshooting techniques enhance the method's robustness against typical issues.