Bone erosions are an important pathological feature of rheumatoid arthritis. The purpose of this work is to introduce a training tool to provide users with guidance on identifying pathological cortical breaks on high resolution peripheral quantitative computed tomography images for erosion analysis.
Bone erosions are a pathological feature of several forms of inflammatory arthritis including rheumatoid arthritis (RA). The increased presence and size of erosions are associated with poor outcomes, joint function, and disease progression. High-resolution peripheral quantitative computed tomography (HR-pQCT) provides unparalleled in vivo visualization of bone erosions. However, at this resolution, discontinuities in the cortical shell (cortical breaks) that are associated with normal physiological processes and pathology are also visible. The Study grouP for xtrEme Computed Tomography in Rheumatoid Arthritis previously used a consensus process to develop a definition of pathological erosion in HR-pQCT: a cortical break detected in at least two consecutive slices, in at least two perpendicular planes, non-linear in shape, with underlying trabecular bone loss. However, despite the availability of a consensus definition, erosion identification is a demanding task with challenges in inter-rater variability. The purpose of this work is to introduce a training tool to provide users with guidance on identifying pathological cortical breaks on HR-pQCT images for erosion analysis. The protocol presented here uses a custom-built module (Bone Analysis Module (BAM) – Training), implemented as an extension to an open-source image processing software (3D Slicer). Using this module, users can practice identifying erosions and compare their results to erosions annotated by expert rheumatologists.
Bone erosions occur when inflammation causes localized bone loss at the cortical bone surface. These erosions extend into the underlying trabecular bone region. They are a pathological feature of several forms of inflammatory arthritis, including rheumatoid arthritis (RA)1. Erosion presence and size are associated with poor outcomes, patient function, and disease progression2,3,4,5. While plain radiography remains the clinical standard for erosion assessment, high-resolution peripheral quantitative computed tomography (HR-pQCT) provides 3D images and superior sensitivity and specificity for erosion detection6,7. For inflammatory arthritis, such as RA, HR-pQCT is commonly performed on the 2nd and 3rd metacarpophalangeal joints – the most affected joints of the hand8. Because HR-pQCT images have high spatial resolution, physiological interruptions in the cortical surface are also observed in healthy individuals without RA9. These cortical interruptions are often associated with vascular channels or nutrient foramen passing through the bone10. Thus, the challenge is to distinguish cortical interruptions associated with a disease process (i.e., pathological erosions) from non-pathological features.
The consensus definition of a pathological bone erosion was published by the Study grouP for xtrEme Computed Tomography in Rheumatoid Arthritis (SPECTRA) as the presence of a definite interruption in the cortical layer of the bone that extends over at least two consecutive slices and is detectable in two or more perpendicular planes11. Further, the interruption must be non-linear in shape and accompanied by a loss in the trabecular region. Visual examples of cortical interruptions that do and do not meet the criteria of erosions are shown in Klose-Jensen et al.12.
However, not all cortical interruptions that meet the above criteria are classified as erosions. Interruptions are sometimes caused by physiological processes such as vascular channels (Figure 1). These can be identified and differentiated from erosions due to their predictable anatomical locations, parallel and straight margins and sub-millimetric size13. Cysts are another form of cortical interruption that is not considered to be an erosion. They often have a rounded trabecular structure with a clear cystic wall 13. In contrast to the sharp edges and open trabecular structure displayed by erosions. However, it is possible for erosions to form within cystic sites, making it ambiguous to delineate the volume of bone loss caused by the erosions and not the cysts. While resolving this ambiguity with further criteria is not the purpose of this study, there is a need to provide comprehensive examples of pathological erosion and physiological cortical interruptions.
Figure 1: Example of cortical interruptions that were not caused solely by erosions. (A) A drawing illustrating the common location of vascular channels at the base of the metacarpal head. Examples of vascular channels in (B) coronal, (C) sagittal, (D) and (E) axial planes. (F) Example of a cortical interruption caused by a cyst. (G) Example of a void volume within trabecular region of the bone involving both cysts and erosions. Please click here to view a larger version of this figure.
Despite the challenges in erosion identification, there are currently no training tools in place to provide less experienced users with guidance on interpreting HR-pQCT images for erosion analysis. Recently, an open-source module for erosion analysis called bone analysis module (BAM) – Erosion Volume was developed, implemented as an extension to an open-source image processing software to enable erosion visualization and volumetric analyses14. The protocol presented here describes the use of a training module added to BAM (BAM – Training), which compares a users' erosion identification attempts by comparing the erosion identification with erosions annotated by expert rheumatologists. This training tool provides users with feedback on erosion identification in order to guide improvements in erosion analysis. Software installation instructions are provided in step 1. For new data acquisition, see steps 3 – 5.3. For training module use only, see step 2.
All methods in this protocol follow the guidelines set by the Conjoint Health Research Ethics Board at the University of Calgary (REB19-0387).
1. Install 3D Slicer 15 and bone analysis modules
Figure 2: Example of settings window after adding bone analysis modules to an installation of 3D Slicer. The image shows a screenshot of the settings window with the modules highlighted in the red box. Please click here to view a larger version of this figure.
2. Training module
Figure 3: 3D Slicer's dropdown menu. The dropdown menu to find the bone analysis modules and select the training module. Please click here to view a larger version of this figure.
Figure 4: Identification of the volume inside the bone's periosteal surface. (A) Example of a mask. The mask is visualized as a binary image. (B) Example of a segmentation. The segmentation refers to the visualization of the binary image overlaid with the grey scale image. These distinctions are made by 3D Slicer. Please click here to view a larger version of this figure.
Figure 5: Example screen shot of training module within 3D Slicer. (A) Click to add new seed points. (B) Click to compute erosion volumes. (C) Click to import images. (D) Click to reveal seed points placed by experts. Please click here to view a larger version of this figure.
3. Image acquisition and export to use in erosion analysis tool
4. File conversion and bone mask generation
NOTE: Depending on the image format, follow step 4.1 for AIM (HR-pQCT proprietary image format), MHA (ITK MetaImage format), nii (NIfTI – Neuroimaging Informatics Technology Initiative), NRRD (Nearly Raw Raster Data) images or step 4.2 for DICOM images.
5. Identification of erosions
6. Erosion Statistics
Using the training tool, users can practice identifying erosion sites while receiving feedback on their results. This feedback loop can improve the user's ability to identify erosions and potentially use the BAM modules to identify erosions on their own images. Feedback after seed point placement is based on the following criteria. 1) If the number of seed points placed does not match the number of reference erosions, the user is then prompted to delete or add the appropriate number of seed points. 2) If the seed point location cannot be matched with a reference erosion, feedback stating that no erosion exists at that seed point's location is displayed. 3) If a seed point is matched with a reference pathological/physiological cortical interruption such as a cyst or a vascular channel, the user is informed about the type of cortical interruption they attempted to identify as an erosion and is asked to remove the seed point. 4) If the location of the seed point overlaps a reference erosion, the algorithm may still not detect the erosion. This may occur when the seed point has not been centered in the erosion. In these cases, the user is prompted to adjust the position of the seed point. 5) If a seed point is placed too far from any erosion, the user is informed of their incorrect placement and encouraged to try again. 6) When a seed point location matches the reference erosion, a prompt is displayed informing the user of their successful attempt to identify the erosion at that specific seed point.
The following section illustrates examples of how the module works based on different inputs. Correct and incorrect inputs will be demonstrated in the following examples. Figure 6A shows the seed point location which is located within the erosion. Only one erosion exists within this image, therefore computing the erosions with the seed point will yield the expected results. Figure 6B shows the prompt displayed to users when their attempt to identify the erosions matches the expertly annotated image. The module also displays the results as segmentations on the grey-scale image (Figure 6C). If the user placed a seed point at a location without an erosion, such as Figure 7A, the module would display an error prompt (Figure 7B) stating that no erosion exists at this location and suggests that the user relocate/remove the seed points.
Figure 6: Example of correct erosion identification. (A) Example of a user correctly placing a seed point within the erosion site. (B) Example of feedback prompt when all erosions were identified correctly. (C) Example of displayed erosion segmentation when an erosion is computed correctly. Please click here to view a larger version of this figure.
Figure 7: Example of incorrect erosion identification. (A) Example of a seed point placed where no erosion exists. (B) Example of an error prompt when a seed point is placed at a location that has no erosion. Please click here to view a larger version of this figure.
The locations of all cysts and vascular channels on the training images provided have been identified by an expert. Therefore, it is possible to detect when a user attempts to incorrectly identify a cyst or a vascular channel. Figure 8A illustrates an attempt to identify a cyst by placing a seed point on it. Figure 8B is the subsequent displayed error prompt.
Figure 8: Example of cyst identification. (A) Example of a seed point placed on a cyst. (B) Example of an error prompt when a seed point is placed on a cyst. Please click here to view a larger version of this figure.
The module will also inform the user if they have the right amount of seed points. If the user placed an incorrect number of seed points, the module would inform the user of the exact amount of seed points that are missing or extra to identify all erosions on the image. The module also gives feedback for each placed seed point. Therefore, the user knows what actions to take for each individual seed point. Figure 9 demonstrates an example where a user only placed one seed point when two were expected.
Figure 9: Example of erosions computed while missing one seed point. The example demonstrates an example where the user only placed one seed point when two were expected. Please click here to view a larger version of this figure.
If a user has trouble finding any or all the erosions, they have an option to reveal the expertly annotated locations by pressing a button labeled Reveal Correct Seed Points (Figure 5D). When pressed, this button will load the correct seed points on to the current 3D Slicer window.
In summary, this shows that the software module can assess the correctness of the user's attempt to identify erosions in the select images by comparing the computed erosion with expertly annotated erosions. In addition, the module provides feedback based on each user-placed seed point to guide them towards the expected seed point location and input parameters.
Scan ID | Cortical Interruption | Bone | Label | Centroid Location | Volume (mm3) | Surface Area (mm2) | Roundness | Number of voxels (voxels) |
3_Training.nii | Erosion | Metacarpal | SEEDS_28-1 | 210, 108, 242 | 3.321668853 | 14.46818378 | 0.74411491 | 14853 |
3_Training.nii | Erosion | Metacarpal | SEEDS_28-3 | 179, 100, 241 | 1.100739562 | 7.121231239 | 0.7239659 | 4922 |
Table 1: Example of a generated output file (csv format) describing computed erosions and their statistics.
This training tool provides an opportunity to learn to identify erosions using the bone analysis module. Further use of this erosion analysis tool beyond training requires access to good quality images, with little or no motion artifact. The HR-pQCT erosion definition based on the literature describes anatomical features associated with pathological erosions that can be reported with reasonable reproducibility11,20. However, this definition does not account for common anatomical locations of vascular channels, potentially resulting in their misclassification as bone erosions10.
The critical steps in this protocol are the generation of the bone mask, the placement of the seed points, and the generation of the erosion volume. While automated methods to generate the masks and erosion volume are implemented, the masks often require manual correction to ensure satisfactory results. A comprehensive description of the tools available to perform the manual corrections is provided. The placement of seed points is guided by the training examples provided by the BAM-Training module.
Based on the data utilized to date, this protocol provides suggestions for troubleshooting when the erosion analysis module does not produce the expected results. In future work, access to additional training data will be provided. A previous study showed that the erosion volumes assessed with this method are comparable with existing methods14,21,22. Provision of training data will allow comparison to newer erosion analysis tools as they are developed23.
The training tool introduced here primarily aids with erosion identification; however, the method is currently limited by the lack of a consensus on defining the extent of an erosion in the trabecular bone. Nonetheless, the BAM modules are open source, thus, as future definitions of erosion extent change, other researchers have access to modify the modules to meet their needs.
As the use of HR-pQCT in rheumatological research expands, the training tool provides inexperienced users with guidance on identifying pathological cortical interruptions on HR-pQCT images for erosion analysis. This tool will be applicable to researchers regardless of the method selected for erosion analysis. While completely automated erosion identification is desirable to improve reproducibility and speed of analysis, large reference/benchmark datasets with accurate annotations are required to train machine learning models. As an open-source tool, this module provides an opportunity to collectively develop large, annotated datasets for future use in machine learning. The use of this training tool will enable more researchers to include erosion analysis in their HR-pQCT research.
The authors have nothing to disclose.
The authors would like to acknowledge the following funding agencies that supported this work. SLM is funded through The Arthritis Society (STAR-18-0189) and Canadian Institutes of Health Research Planning and Dissemination Grant. JJT holds a CIHR Fellowship award.
3DSlicer | Open Source | N/A | Download at https://www.slicer.org/ |
BAM Erosion Analysis Modules | Open Source | N/A | Version used in manuscript: download at https://doi.org/10.5281/zenodo.7943007 |
XtremeCTII | Scanco Medical | N/A |