Lesion Explorer (LE) is a semi-automatic, image-processing pipeline developed to obtain regional brain tissue and subcortical hyperintensity lesion volumetrics from structural MRI of Alzheimer’s disease and normal elderly. To ensure a high level of accuracy and reliability, the following is a video-guided, standardized protocol for LE’s manual procedures.
Obtaining in vivo human brain tissue volumetrics from MRI is often complicated by various technical and biological issues. These challenges are exacerbated when significant brain atrophy and age-related white matter changes (e.g. Leukoaraiosis) are present. Lesion Explorer (LE) is an accurate and reliable neuroimaging pipeline specifically developed to address such issues commonly observed on MRI of Alzheimer’s disease and normal elderly. The pipeline is a complex set of semi-automatic procedures which has been previously validated in a series of internal and external reliability tests1,2. However, LE’s accuracy and reliability is highly dependent on properly trained manual operators to execute commands, identify distinct anatomical landmarks, and manually edit/verify various computer-generated segmentation outputs.
LE can be divided into 3 main components, each requiring a set of commands and manual operations: 1) Brain-Sizer, 2) SABRE, and 3) Lesion-Seg. Brain-Sizer’s manual operations involve editing of the automatic skull-stripped total intracranial vault (TIV) extraction mask, designation of ventricular cerebrospinal fluid (vCSF), and removal of subtentorial structures. The SABRE component requires checking of image alignment along the anterior and posterior commissure (ACPC) plane, and identification of several anatomical landmarks required for regional parcellation. Finally, the Lesion-Seg component involves manual checking of the automatic lesion segmentation of subcortical hyperintensities (SH) for false positive errors.
While on-site training of the LE pipeline is preferable, readily available visual teaching tools with interactive training images are a viable alternative. Developed to ensure a high degree of accuracy and reliability, the following is a step-by-step, video-guided, standardized protocol for LE’s manual procedures.
Brain image analysis is an emerging field of neuroscience requiring skilled operators with a high degree of computational and neuroanatomical competency. In order to obtain quantitative information from magnetic resonance imaging (MRI), a trained operator is often required to implement, monitor, and edit, computer-generated imaging outputs generated from raw MRIs. While many ‘fully automatic’ imaging tools are freely available via the internet, accuracy, and reliability is questionable when applied by a novice operator lacking knowledge, training and familiarity with the downloaded tool. Although on-site training is the most preferable teaching approach, the presentation of a video-guided, standardized protocol is a viable alternative, particularly if accompanied by a training set of images. Additionally, the training set of images may be used for quality control measures, such as an off-site inter-rater reliability test.
The challenges of developing an image processing pipeline, particularly when studying aging and Alzheimer’s disease (AD), include a wide range of technical and biological issues. Although some technical issues are addressed with post-processing correction algorithms3, variability due to individual differences and pathological processes introduce more complex obstacles. Brain atrophy and ventricular enlargement can reduce the viability of registration warping and template-matching approaches. The presence of age-related white matter changes4 and small vessel disease5,6, observed as subcortical hyperintensities (SH)7,8, cystic fluid-filled lacunar-like infarcts9,10, and dilated perivascular spaces11,12, further complicate segmentation algorithms. In cases of significant white matter disease, a single T1 segmentation could result in overestimation of gray matter (GM)13, which can only be corrected with an additional segmentation using proton density (PD), T2-weighted (T2), or fluid-attenuated inversion recovery (FLAIR) imaging. In light of these challenges, the Lesion Explorer (LE) image processing pipeline implements a semi-automatic tri-feature (T1, PD, T2) approach, utilizing trained operators at particular stages when human intervention is preferable1,2.
Brain extraction (or skull stripping) is typically one of the first operations performed in neuroimaging. Given this, the accuracy of the total intracranial vault (TIV) extraction process greatly influences subsequent operations further down the pipeline. Significant over-erosion, resulting in loss of brain, may lead to over-estimation of brain atrophy. Alternatively, significant under-erosion, resulting in inclusion of dura and other nonbrain matter, may lead to inflation of brain volumes. LE’s Brain-Sizer component addresses many of these issues by using a tri-feature (T1, T2, and PD) approach to generate a TIV mask, which yields superior results compared to single-feature methods1. Additionally, the automatically generated TIV mask is manually checked and edited using standardized protocol which identifies regions susceptible to skull stripping errors. After brain extraction, segmentation is performed on the skull-stripped T1, where each brain voxel is assigned to 1 of 3 labels: GM, white matter (WM), or cerebrospinal fluid (CSF). Segmentation is accomplished automatically using a robust curve-fitting algorithm applied to global and local intensity histograms; a technique developed to address intensity nonuniformity artifact and a decreased separation between GM and WM intensity amplitude in AD cases14.
The Brain-Sizer component also includes procedures for manual designation of ventricles and removal of subtentorial structures. Segmentation of ventricular CSF (vCSF) is particularly important as ventricle size is a commonly used biomarker for AD dementia15. Additionally, delineation of ventricles and choroid plexus is imperative for proper identification of periventricular hyperintensities (pvSH), which are believed to reflect a form of small vessel disease characterized by venous collagenosis5,16,17. Using T1 for reference, manual relabeling of CSF voxels to vCSF is accomplished with manual floodfill operations on the segmented image. Typically, the lateral ventricles are easier to differentiate from sulcal CSF. For this reason, it is recommended to begin floodfilling in axial view, starting from superior slices and moving inferiorly. The medial parts of the ventricular system, particularly the 3rd ventricle, is more difficult to delineate and is given special anatomy-based rules which are outlined in the manual. Brain-Sizer’s final step includes removal of the brain stem, cerebellum, and other subtentorial structures, using manual tracing procedures described in an additional set of anatomy-based standardized protocols.
The Semi-Automated Brain Region Extraction (SABRE) component is the pipeline’s parcellation procedure. This stage requires trained operators to identify the following anatomical landmarks: anterior and posterior commissure (AC, PC); posterior brain edge; central canal; mid-sagittal plane; preoccipital notch; occipito-parietal sulcus; central sulcus, and; Sylvian fissure. Based on these landmark coordinates, a Talairach-like18 grid is automatically generated and regional parcellation is accomplished19. Landmarks are easily identified on ACPC aligned images, which are generated automatically and manually checked prior to SABRE landmarking procedures.
The Lesion-Seg component is the final stage of the pipeline where SH identification and quantification is accomplished. The initial automatic SH segmentation implements a complex algorithm which includes PD/T2-based SH segmentation, fuzzy c-means masking, and ventricular dilatation. These operations result in an automatically generated lesion segmentation mask that is manually checked and edited for false positives and other errors. As hyperintense signal on MRI may result from nonpathological sources (e.g. motion artifact, normal biology), proper training is required for accurate identification of relevant SH.
The final result of the LE pipeline is a comprehensive volumetric profile containing 8 different tissue and lesion volumetrics which are parcellated into 26 SABRE brain regions. To obtain an individual operator’s inter-rater reliability test off-site, it is recommended to execute the full LE pipeline on the training set provided with the software (http://sabre.brainlab.ca). Using the volumetric results, inter-class correlation coefficient (ICC)20 statistics can be calculated for each tissue class (GM/WM/CSF) in each SABRE region. Using the segmentation images, Similarity Index (SI)21 statistics can be calculated to evaluate the degree of spatial congruence. Additionally, intra-rater reliability can be assessed on the same operator’s results, after a brief period of time has passed between the operator’s 1st and 2nd segmentation edits. Provided that the off-site operator adheres to the file naming conventions outlined in the LE manual, reliability statistics can be calculated off-site using most basic statistical software packages. Given these quality control and video-guided standardized protocol, off-site operators can have greater confidence that the LE pipeline is applied accurately and reliably.
1. Brain-Sizer Component
1.1 Total Intracranial Vault Extraction (TIV-E)
1.2 Ventricular Reassignment
1.3 Removal of Brain Stem, Cerebellum, and Subtentorial Structures
2. SABRE Component
2.1 ACPC Alignment
2.2 SABRE Landmark Identification
Part 1 – Grid File Coordinates
Part 2 – Object Map Creation
Part 3 – Surface Rendered Tracings
3. Lesion-Seg Component
3.1 For Scans with PD/T2 (no FLAIR)
NOTE: Label 2 (default color is RED) is used to signify lesion.
3.2 For Scans with FLAIR Imaging
NOTE: Label 2 (default color is RED) is used to signify lesion.
Inter-rater reliability can be assessed using several metrics. Using the training set provided online (http://sabre.brainlab.ca), the following steps are recommended to assess inter-rater reliability for each of the processing stages after completion of LE.
Brain-Sizer:
To assess inter-rater reliability of the brain extraction procedures, generate volumetrics for each TIV-E masks, <name>_TIVedit, using the <img_count> command. Enter these volumetrics into a statistical software package (e.g. SPSS), along with the TIVedit volumetrics provided for each of the training set (see Excel/csv file provided online) and calculate the inter-rater correlation coefficient (ICC). Whole brain volumetrics for in-house trained raters obtain reported ICC=0.99, p<0.0001 1,2. Additionally, evaluation of the spatial agreement for the TIV masking can be assessed using the SI21. MATLAB code is provided online to calculate SI values between two raters.
To assess ventricular reassignment, generate vCSF volumes using the <img_count> command for each of the segmentation files with the vCSF voxels reassigned, i.e. <name>_ seg_vcsf. The vCSF volume is the value beside row ‘7’ under the column titled ‘volume’. Using the same procedures to evaluate TIV inter-rater reliability, calculate ICC and SI for vCSF.
Removal of brain stem, cerebellum and subtentorial structures can be assessed similarly by running the <img_count> command on <name>_seg_vcsf_st. The volumes used for this segmentation mask are shown at the second last row titled ‘total count of nonzero voxels:’ under ‘volume’ (the last column on the right). Using the same procedures to evaluate TIV and vCSF, calculate ICC and SI for this masking procedure using the volumetrics in the excel file provided and the <name>_seg_vcsf_st files.
SABRE:
While Brain-Sizer’s manual procedures can easily be assessed using standard metrics, ACPC alignment is slightly more difficult. For this reason, matrix files are provided to compare visually for training of off-site operators. After completion of ACPC alignment, open a new ITK-SNAP_sb window, load the T1 image, then load the matrix for the training case provided online, <name>_T1_IHCpre_toACPC.mat, and visually compare the pitch, roll, yaw, and ACPC slice between the two images.
To evaluate SABRE landmarking procedures, run <img_count> on the parcellated mask, <name>_SABREparcel_inACPC for each training case. Enter the volumetrics for each region (3-28). SABRE region codes are provided online. Using the same procedures to evaluate TIV and vCSF, calculate ICC for each SABRE brain region. SABRE parcellated regional volumetrics for in-house trained raters obtain reported mean ICCs=0.98, p<0.01, with ICC values ranging from 0.91-0.99 1,2.
Lesion-Seg:
As this component is the final stage of the LE pipeline, reliability and accuracy will depend on the prior stages.
Inter-rater reliability of SH segmentation is accomplished using regional ICC of SH volumes and spatial agreement of the SH masks. To evaluate regional SH volumes, run <SH_volumetrics>, entering both the lobmask file in T1-acquisition space, <name>_SABREparcel and the final edited lesion segmentation file, <name>_LEedit. Using the same procedures to evaluate SABRE volumetrics, calculate ICC for lesion volumes within each SABRE brain region. Using the same procedures to evaluate spatial agreement of the TIV masking process, calculate SI for the final edited lesion masks, <name>_LEedit (or FLEXedit). The same reliability tests can be performed on both PD/T2-based segmentation and FLAIR-based segmentation.
3D T1 | PD/T2 | |
Imaging Parameters | Axial Volume SAT (S1) SPGR | Axial Spin Echo FC VEMP VB (interleave) |
Pulse Timing | ||
TE (msec) | 5 | 30/80 |
TR (msec) | 35 | 3,000 |
Flip Angle (°) | 35 | 90 |
TI (msec) | N/A | N/A |
Scan Range | ||
FOV (cm) | 22 | 20 |
Slice thickness (mm) | 1.2/0 | 3/0 |
No. Slices | 124 | 62 |
Acquisition | ||
Matrix size | 256 x 192 | 256 x 192 |
Voxel size (mm) | 0.86 x 0.86 x 1.4 | 0.78 x 0.78 x 3 |
NEX | 1 | 0.5 |
Total Time (min) | 11:00 | 12:00 |
Table 1. General Electric 1.5T Structural MRI Acquisition Parameters.
3D T1 | PD/T2 | FLAIR | |
Imaging Parameters | Axial 3D FSPGR EDR IR Prep | Axial 2D FSE-XL, EDR, FAST, fat sat | Axial T2Flair, EDR, FAST |
Pulse Timing | |||
TE (ms) |
3.2 | 11.1 / 90 | 140 |
TR (msec) | 8.1 | 2,500 | 9,700 |
Flip Angle (°) | 8° | 90° | 90° |
TI (msec) | 650 | N/A | 2,200 |
Scan Range | |||
FOV (cm) | 22 | 22 | 22 |
Slice thickness (mm) | 1 | 3 | 3 |
No. Slices | 186 | 48 | 48 |
Acquisition | |||
Matrix size | 256 x 192 | 256 x 192 | 256 x 192 |
Voxel size (mm) | 0.86 x 0.86 x 1 | 0.86 x 0.86 x 3 | 0.86 x 0.86 x 3 |
NEX | 1 | 1 | 1 |
Total Time (min) | 7:20 | 6:10 | 7:20 |
Table 2. General Electric 3T Structural MRI Acquisition Parameters.
Figure 1. Axial T1 with unedited total intracranial vault (TIV) mask overlay (green). This is an example of the use of the closed polygon tool in ITK-SNAP_sb to remove nonbrain tissue as part of the manual editing procedure of the Brain-Sizer’s TIV extraction procedure.
Figure 2. Axial T1 with tissue segmentation overlay. Note that label colors are arbitrary and can be modified using the Label tool. Left image shows default colors. Middle image shows how CSF (5=purple) is reassigned to vCSF (7=magenta). Right image shows how the WM color can be modified without changing the tissue class label, i.e. Label 3=WM remains but color can be modified to blue.
Figure 3. Axial T1 with tissue segmentation overlay (left image, GM=yellow, WM=orange, CSF =purple) (left). Depicted is an example of manual removal of subtentorial structures using the closed polygon tool in ITK-SNAP_sb (middle) and final tissue segmentation after removal (right). As in Figure 2, right image shows how the WM color can be modified without changing the tissue class label, i.e. Label 3=WM remains but color can be modified to blue.
Figure 4. Axial T1 in acquisition space before (left), and after (right) AC-PC alignment is performed.
Figure 5. Two examples showing SABRE landmarking procedures. Axial AC-PC aligned T1 with AC (yellow), PC (blue), and posterior edge (pink) landmark placements (left). A 3D surface-rendered T1 (right) with Sylvian fissure (purple) and central sulcus (pink) delineation.
Figure 6. Axial PD (left) with automatically generated lesion overlay (center), and manually edited lesion (red) overlay (right).
Figure 7. Axial FLAIR (left), with automatically generated lesion overlay (center), and manually edited lesion (red) overlay (right).
The LE segmentation and parcellation procedure was developed specifically to obtain regional volumetrics from MRI of AD and normal elderly. While there are numerous fully automatic pipelines which apply complex computational algorithms to perform these operations, these tools tend to lack the individualized accuracy and precision that LE’s semi-automatic pipeline produces. The trade-off with semi-automatic processes are the resources required to properly train operators with the anatomical knowledge and computational skills needed to apply such a comprehensive pipeline. However, one of the primary benefits of an individualized imaging pipeline is the ability to obtain quantitative volumetrics from moderate to severe cases of neurodegeneration when automatic pipelines fail.
As the LE pipeline has been previously evaluated and applied to various elderly and demented populations1,2,13,14,19,22,23, the main issues that are typically encountered by trained operators have been well-documented and are summarized below.
The manual checking and editing required with the Brain-Sizer component includes the TIV extraction masking procedure, vCSF reassignment and manual removal of the brain stem, cerebellum and other subtentorial structures. For brain extraction, the automatic TIV output is generally a decent mask provided that the original PD/T2 images are good quality. However, due to the relative intensity values of vascular and nerve tissue medial to the inferior temporal poles, proximal to the carotid arteries, this region typically requires some editing. Additionally, mucous in the nasal cavity tends to affect regional intensity histograms, skewing intensity cut-offs values in the anterior frontal regions, which tend to require additional manual editing of the automatic TIVauto mask. Finally, additional manual editing is typically required in the most superior regions, where global atrophy tends to result in an increase in the volume of subarachnoid CSF just below the dura mater. Alternatively, atrophy associated with ventricular enlargement tends to minimize operator interventions required with vCSF reassignment. Another benefit of having a tri-feature coregistration approach is the ability to identify cystic fluid-filled infarcts proximal to the ventricles, potentially due to periventricular venous vasculopathy5,24-26, which are identifiable due to their relative intensity on PD and T1 (hyperintense on PD, hypointense on T1). These hypointensities can be delineated from vCSF using manual limits drawn in ITK-SNAP_sb prior to floodfilling operations. Since vCSF reassignment is performed in T1-acquisition space, in cases where alignment deviates far from the ACPC plane, a limit may be required for the 3rd ventricle and the quadrigeminal cistern, if the PC is not fully visible. Although the tentorium is a relatively easy structure to differentiate, several anatomy-based rules assist in guiding manual removal of the brain stem and subtentorial structures, particularly when locating the separation of the cerebral peduncles from the medial temporal lobe.
SABRE landmarking is a stereotaxic-based procedure performed in standard ACPC aligned images, allowing for moderately predictable localization of particular anatomical landmarks. Exceptions to this are cases with extreme atrophy and normal variability due to individual differences in neuroanatomy. Brain atrophy results in an overall loss of parenchyma, increasing CSF along the midline surrounding the falx cerebri, which increases the difficulty of choosing appropriate points to place landmarks. Rule-based protocols are required, identifying cases where exceptions to the general rule are required. Normal variations in anatomy, particularly in the relative location of the central sulcus and the parieto-occipital sulcus, also increase the difficulty of manual delineation of these structures. However, the graphical user interface used by SABRE allows for real-time rotation of surface rendered images, which significantly assists in the decision-making process for visualization of these particular landmarks. Finally, some rule-based protocol have been integrated programmatically into the software to prevent operator violation e.g. central sulcus delineation is forced to move posteriorly (line tracing is prevented from going back onto itself).
The Lesion-Seg component’s manual checking procedure requires expertise in visual identification of relevant hyperintensities, a visual perception skill that is only acquired after exposure to scans with varying degrees of SH. False-positive minimization algorithms assist with the removal of most errors in the initial segmentation. However, differentiation between dilated perivascular spaces (Virchow-Robin spaces: VRS) in the lentiform nucleus and relevant SH in the external capsule, claustrum, extreme capsule, and subinsular regions can be difficult. This is particularly difficult in cases with VRS in the basal ganglia. A recent paper outlining STandards for ReportIng Vascular changes on nEuroimaging (STRIVE), recommended a size criterion to differentiate VRS from lacunes, and describe VRS to be more linear and CSF intensity on MRI. To address these issues with VRS identification, LE has adopted: a) an anatomy-based rule which prevents operators from selecting any hyperintensity that falls within the lentiform nucleus, b) a size criterion to exclude hyperintensities less than 5mm in diameter, and c) a relative intensity rule for additional exclusion due to the relative CSF intensity on PD, T2 and T127. Additionally, normal hyperintense signal can be found along the midline and falx cerebri, particularly on FLAIR imaging, which can be difficult to differentiate between relevant SH along the corpus callosum. In cases of such overlap, anatomy-based rules are implemented where only SH which extend out into the periventricular regions are accepted.
In conclusion, it is important to appreciate that this written component is meant to supplement a video-guided, standardized protocol publication in JoVE (http://www.jove.com). While traditional static figures assist in explaining some concepts, video-based tutorials are more efficient at communicating the complex methodological processes involved with a comprehensive neuroimaging pipeline such as Lesion Explorer.
The authors have nothing to disclose.
The authors gratefully acknowledge financial support from the following sources. The development and testing of various neuroimaging analyses was supported by several grants, most notably from the Canadian Institutes of Health Research (MOP#13129), the Alzheimer Society of Canada and Alzheimer Association (US), the Heart and Stroke Foundation Canadian Partnership for Stroke Recovery (HSFCPSR), and the LC Campbell Foundation. JR receives salary support from the Alzheimer Society of Canada; SEB from the Sunnybrook Research Institute and the Departments of Medicine at Sunnybrook and U of T, including the Brill Chair in Neurology. Authors also receive salary support from the HSFCPSR.
Magnetic resonance imaging machine (1.5 Tesla) | General Electric | See Table 1 for acquisition parameters | |
Magnetic resonance imaging machine (3 Tesla) | General Electric | See Table 2 for acquisition parameters |