Automatic Image Processing to Determine the Community Size Structure of Riverine Macroinvertebrates

NOTE: The protocol described here is based on the system developed by Gorsky et al.²⁷ for marine mesozooplankton. A specific description of the scanner (ZooSCAN), scanning software (VueScan 9×64 [9.5.09]), image processing software (Zooprocess, ImageJ), and automatic identification software (Plankton Identifier) steps can be found in previous references^32,33. To best adjust the sizes of the benthic macroinvertebrates with respect to the mesozooplankton, once the project is created following the original protocol^32,33, change the parameter of minimum size (minsizeesd_mm) to 0.3 mm and the parameter of maximum size (maxsizeesd_mm) to 100 mm in the configuration file. To help follow the protocol, this is summarized in a work chart (Figure 1). The created project is stored in the computer's C folder and is organized in the following folders: PID_process, Zooscan_back, Zooscan_check, Zooscan_config, Zooscan_meta, Zooscan_results, and Zooscan_scan. Each folder is composed of several subfolders that the different software applications use in the following steps of the protocol.

1. Acquisition of digital images for macroinvertebrate samples

1. Scanning and processing the blank
   NOTE: Create two blank images daily before scanning to extract the background scans while processing the scanned images on the same day.
   1. Turn on the scanner and switch on the light in the dual position to project white light from the top and from the bottom.
      NOTE: When scanning mesozooplankton samples, the upward light direction is used, but because macroinvertebrates are more opaque, it is recommended to switch the light to a dual position.
   2. Clean and rinse the scan tray with tap water.
   3. Pour 110 mL of tap water stored at room temperature (RT) into the scan tray until the glass is covered. Place the large frame (24.5 cm x 15.8 cm) on the scan tray in the correct position (with the corner at the top-left part of the scan tray), and fill it with tap water until the step of the frame is covered to avoid a meniscus effect, which would alter the scanned image. Close the scanner lid.
      NOTE: Use water at RT to avoid condensation and bubble formation. Clean the frame without marks or droplets to avoid light reflection.
   4. Go to the image processing software, select the working project, and click on Scan (Convert) Background Image.
   5. Go to the scanning software and click on Anteprima. Ensure to preview the scanned image, check that there are no lines or spots, and wait for at least 30 s before starting another scan. Click on Scan and press OK in the instructions window before the second scan to send the data from the scanning software to the image-processing software.
      NOTE: Scan twice to obtain the two background scans that will comprise the blank. This step is done once every day before starting the sample processing, and the images are stored in the Zooscan_back folder.
   6. Close the scanning software after finishing the scan.
2. Sample preparation and scanning
   CAUTION: Ethanol is a flammable liquid and could cause serious eye damage/irritation.
   1. Fill in the sample metadata. Go to the image processing software and select Fill in Sample Metadata. Enter the sample identity, click on OK, and fill in the metadata.
      NOTE: The metafile is specifically created for mesozooplankton samples, so it does not fit the benthic macroinvertebrate sampling methodology, yet all fields of the file need to be filled in before the scan, or an error flag will pop up.
   2. Pour 110 mL of 70% ethanol into the scan tray until the glass is covered and place the large frame (24.5 cm x 15.8 cm) with the corner at the top-left part of the scan tray.
      NOTE: Work with ethanol instead of water, as the macroinvertebrates are preserved in ethanol. In water, they float and drift in the scan tray, preventing a sharp image and, thus, reliable size measurements. Ethanol should be preserved at RT to avoid condensation and bubble formation.
   3. Pour the macroinvertebrates sample into the scan tray edged by the frame, and cover the frame step with more ethanol if needed.
      NOTE: Refrain from adding too much ethanol to avoid the organisms floating and drifting.
   4. Homogenize the sample throughout the frame area, placing the largest individuals in the center of the tray for proper image processing, and sink the floating organisms using a wooden needle.
      NOTE: If a subsample numerically contains more than 1,000 individuals, divide the subsample into two or more fractions to minimize touching organisms in the scanned image, and scan the fractions separately.
   5. Separate the touching organisms and the organisms touching the frame edges using the wooden needle.
      ​NOTE: This step requires 5-20 min. Touching organisms are considered a single object by the software; thus, in those cases, the calculated individual sizes do not correspond to actual single organisms and can bias the estimate of the community size structure. There is the possibility of editing the image with the image processing software to separate them, but this additional step involves at least 1.5 h of reprocessing; thus, manual separation is highly recommended.
   6. To scan the sample, close the scanner lid, go to the image processing software, select the working project, and click on SCAN Sample with Zooscan (For Archive, No Process).
   7. Select the sample and follow the instructions.
   8. Go to the scanning software and click on Anteprima. Ensure to preview the scanned image, check that there are no lines or spots, and wait at least 30 s before starting another scan.
   9. After at least 30 s, click on the Scan button in the scanning software.
      NOTE: Press OK in the image processing software after pressing Scan in the scanning software. Do not press any key on the computer keyboard, and avoid vibrations of the scan during the scanning. Three files are generated in the Zooscan_scan > _raw folder: (i) a tagged image file format (.tif) (16 bit); (ii) a standard text document named LOG (.txt) that records information on the scanning parameters; and (iii) a standard text document named META (.txt) with information on the sampling methods.
   10. Verify that the raw scan is correct.
       NOTE: If the scan has light stripes or other visible issues, consider repeating the scan to avoid problems in the following steps.
3. Sample recovery
   1. Remove the frame and rinse it above the scan tray using a squeeze bottle filled with 70% ethanol to recover any attached macroinvertebrates.
   2. Lift the upper part of the scanner to retrieve all the organisms and ethanol from the tray through the scan retrieval funnel into a beaker. With the upper part of the scanner still lifted, rinse the tray with the squeeze bottle to sweep along any remaining organisms.
   3. Pass the specimens and ethanol from the beaker through a 500 µm mesh to retain the invertebrates in the mesh, and store them back in a vial with 70% ethanol.
   4. Once all the specimens are recovered in the vial, clean the tray with tap water.
      NOTE: Wash the tray with tap water between samples to minimize ethanol precipitation, which alters the image processing. Rinse the frame with tap water to avoid potential damages related with ethanol use. At the end of the day, clean the tray using tap water and dry it gently with paper to avoid scratches.
4. Image processing
   1. Go to the image processing software and select CONVERT & PROCESS Images and Organisms in Batch Mode and then Convert AND Process Image AND Particles (Image in RAW Folder). Keep the default settings and click on OK. NORMAL END will appear at the end of the process.
      NOTE: A PID file and the vignettes corresponding to all the detected objects in the scanned image (in a Joint Photographic Group file [.jpg]) will be created in the Zooscan_scan > _work folder. A PID file is a single file that stores all the metadata (metafile), the technical data associated with the log file, and a table with 36 measured variables of all the objects detected in the image. The measured variables correspond to different estimates of grey level, fractal dimension, shape, and size. The variables that can be used for size estimation are the area and the major and minor axes of an ellipse with an equal area to the object (see section 3 of the protocol). The processing time depends on the image density and the computer characteristics, and can be launched between samples while recovering and preparing the next sample. Otherwise, it is recommended to launch the processing of the samples scanned each day in batch mode during the night and check for proper image processing the next morning.
   2. Check if the background in the processed image is appropriately subtracted from the sample image using the image processing software or by checking the mask images (terminated in msk1.gif) located in Zooscan_scan > _work. If the background contains saturated areas or many dots, consider repeating the scan to ensure high-quality images.
      NOTE: To avoid saturated areas in the background, the scan tray should be rinsed with tap water after every scan with ethanol. It is also important to (1) reduce the number of scanned individuals (by fractionating the sample and scanning in different folds); (2) ensure that big organisms are placed in the center of the scan tray; (3) use clean, filtered ethanol; (4) reduce dirtiness on the samples; (5) ensure that the volume of ethanol for the scanning is adequate; and (6) ensure that the delay between the preview of the sample and the scan is at least 30 s.
5. Separation of touching organisms
   NOTE: When there are several vignettes with touching organisms, it is necessary to separate the images of the touching organisms from other organisms and/or from fibers/debris to ensure a proper estimate of the community size structure.
   1. Go to the image processing software to detect the vignettes with multiple objects. Select SEPARATION Using Vignettes and press on OK. In the configuration selection window, keep the default settings and click on OK.
   2. In the SEPARATION from VIGNETTES window, keep the default settings, additionally select ADD Outlines on Vignettes, and then select the sample to edit.
   3. Separate the touching organisms in each vignette that pops up by drawing a line with the mouse (press the roll button to draw). Once the separation in a vignette is complete, click the X button in the upper-right corner of the window, and press YES to process the next one. Press NO to end and save the changes. At the end of the process, NORMAL END will appear if everything is correct.
   4. After separation, reprocess the image to obtain the updated object data. Go to the image processing software, click on PROCESS (Converted) Image (Process One), and select Process Again Particles from Processed Images in WORK Sub-Folders. Select the sample, and in the Single Image Process window, keep the default settings, check Work with Separation Mask (CREATE-MODIFY-INCLUDE), and then click on OK. At the end of the process, NORMAL END will appear if everything is correct.
   5. In the Separation Control window, press OK to save the image with the contours before the processing; if a previous image exists, it will be replaced.
   6. In the Separation Control Mask window, if needed, select EDIT to add separation lines to the mask using the mouse to separate touching organisms that have not appeared before in the separation using vignettes step. When finished, end the process, and in the Separation Mask Control window, select YES to accept the mask. At the end of the process, NORMAL END will appear if everything is correct.
      NOTE: Reprocessing a sample with a separation mask is time-consuming (this could take more than 1.5 h per sample). It is preferable to dedicate the required time in step 1.2.5 to avoid this additional step.

2. Automatic recognition of the objects

NOTE: Create a learning set to automatically predict the identity of the detected objects, thus separating the organisms from the debris in the sample.

1. Learning set creation
   1. Copy the images and the .pid files associated with the images that will be used to create the categories of the learning set from Zooscan_scan > _work a PID_process > Unsorted_vignettes_pid.
      NOTE: Select a subset of samples with high taxa diversity and different sampling sites and/or sampling seasons to ensure maximum representativeness of organisms in the samples.
   2. In the PID_process > Learning set folder, create a subfolder with the name of the new learning set (i.e., yyyymmdd_raw_LS), and inside it, create the subfolders that will correspond to each category of the learning set (i.e., macroinvertebrates, debris, other invertebrates).
      NOTE: To efficiently obtain the community size structure of river macroinvertebrate samples, it is recommended to use a learning set based on just three categories: macroinvertebrates, other invertebrates, and debris. This learning set basically separates the vignettes of objects corresponding to organisms from those corresponding to debris (e.g., fibers, particles, or filamentous algae).
   3. Go to the image processing software (Advanced mode only) and choose EXTRACT Vignettes for PLANKTON IDENTIFIER (unsorted vignettes for training). Keep the default options and check the Add Outlines box.
   4. Go to the automatic identification software, click on Learning, select from PID_process > Learning_set the created subfolder for the new learning set (step 2.1.2), and press OK.
   5. In the left section (Unsorted Thumbs) of the open window, select the folder Unsorted vignettes_pid. Select the vignettes and drag them with the mouse from the unsorted thumbs to the folder of their corresponding category in the right section, Sorted Thumbs, to classify each object into the defined categories. The moved vignettes will be marked with a red X.
      NOTE: Define the categories manually by creating subfolders in the sorted thumbs folder or create them by clicking on the folders icon in the software. Do not move more than 50 vignettes at the same time.
   6. Once all the categories are completed with the selected objects (about 300 objects per category), click on Create Learning File and save it with the desired name.
      NOTE: The learning set will be saved as a .pid file in the PID_process > Learning set folder of the project. It is recommended to create and test several learning sets with different levels of categories (from coarse to fine forms) and with a different balance of the number of objects within each category. Start with a coarse learning set with a low number of categories and at least 50 objects per category, and then increase the number of objects in each category and/or create finer learning sets. A category should be representative of its variability in the set of samples.
2. Evaluation of the learning set
   NOTE: Perform cross-validation with two folds and five trials using the Random Forest method with the automatic identification software to obtain a confusion matrix of the resulting classification of the objects.
   1. Go to the automatic classification software and click on Data Analysis.
   2. In Select learning file, select the created learning set file from PID_process > Learning set.
   3. In Select a method, choose the Cross-Validation Random Forest method. In Original Variables, untick the position variables (X, Y, XM, YM, BX, BY, and Height). In Customized Variables, tick only ESD.
      NOTE: This method uses one random part of the learning set to recognize the other part (two folds), and this is repeated five times to ensure it is statistically robust.
   4. Click on Start Analysis, and save the results as Analysis_name.txt in the PID_process > Prediction folder. When the analysis has been successfully completed, quit the data analysis.
   5. Go to the PID_process > Prediction folder and click on the cross-validation file. A window will pop up with the confusion matrix of the true classification (rows) versus the automatic classification (columns).
      NOTE: The recall is the percentage of organisms belonging to a group that was automatically well recognized, whereas 1-precision is the percentage of organisms classified by the algorithm as a group that is not recognized (contamination in a group). The recall should be above 70%, and the contamination (1-precision) should be lower than 20%.
   6. Repeat steps 2.1-2.5 if several learning sets were created and the recall and 1-precision of each one need to be obtained.
      NOTE: If several learning sets have been created, choose the one with the greatest recall (good recognition) and precision (low contamination) of the group of interest (i.e., macroinvertebrates) to test the automatic prediction of a set of samples in the next step.
3. Prediction of the identification of macroinvertebrates
   NOTE: Use the selected learning set to predict the identity of all the objects in a subset of samples using the automatic identification software with a random forest algorithm.
   1. Go to the automatic identification software and click on Data Analysis.
   2. In Select Learning file, select the learning set file from PID_process > Learning set that must be used for the prediction.
   3. In Select Sample File(s), select from the PID_results folder the samples (PID files) that are going to be predicted.
      NOTE: Process a maximum of 20 .pid files at the same time to avoid errors related to memory problems. If too many .pid files are processed at the same time, the process will show a correct end but may not be processed well, and an error may occur in the next steps when processing with the image processing software.
   4. In Select a Method, choose the Random Forest method. Tick Save Detailed Results for Each Sample. In Original Variables, untick the position variables (X, Y, XM, YM, BX, BY and Height). In Customized Variables, tick only ESD.
   5. Click on Start Analysis, and save the results as Analysis_name.txt in the PID_process > Prediction folder.
4. Manual validation
   NOTE: An expert manually validates the prediction from the previous step to reclassify misclassified objects into the correct category.
   1. Copy the Analysis_sample_dat1.txt files to be validated from the PID_process > Prediction folder to the PID_process > Pid_results folder.
   2. Go to the image processing software and select EXTRACT Vignettes in Folders According to PREDICTION or VALIDATION. Then, select Use PREDICTED Files From "pid_results" Folder. Keep the default settings and press OK.
   3. The software creates a folder called sample_yyyymmdd_hhmm_to_validate with the predicted objects in the PID_process > Sorted vignettes folder.
   4. Go to the PID_process > Sorted vignettes folder, and copy the folder sample_yyyymmdd_ hhmm_to_validate. Replace the folder name _to validate with _validated.
   5. To manually validate the automatic classification, open the folder sample_yyyymmdd_ hhmm_validated, and review all the vignettes from each subfolder (category) in order to identify if there are misclassified objects. When one object is misclassified, drag the vignette using the mouse to the correct folder (category).
   6. Go to the image processing software and select LOAD Identifications from Sorted Vignettes. Keep the default settings and select yyyymmdd_hhmm_name_validated to be processed.
   7. Go to PID_process > Pid_results > Dat1_validated, where a file named Id_from_sorted_vignettes_yyyymmdd_hhmm.txt and one .txt file for each one of the validated samples (sample_tot_1_dat1.txt) have been created.
      NOTE: These .txt files contain a new column that presents the prediction, called pred_valid_Id_yyyymmdd_hhmm, which specifies the expert classification of each object (i.e., the validated classification). New categories (e.g., finer taxonomic categories) could be created at this point, during validation. However, keep the name of the original category in the new name (e.g., macroinvertebrate_chironomidae). This allows for retracing the original category when calculating the recall and precision and for easily grouping all the macroinvertebrates to calculate the community size structure parameters (i.e., the size spectrum and size diversity). The text file provides the data associated with each object, including the minor and major axes that are used to obtain the ellipsoidal volume of each organism as a measure of individual body size. Moreover, the last two columns of the table contain the predicted and validated categories of each object (row), which allow for calculating, by category, the recall and precision of the learning set on the subset of samples.

Figure 1: Work chart representing section 1 and section 2 of the protocol. The times are illustrative and could change depending on the computer, the abundance of vignettes to process, and the number of categories of the learning set. This case corresponds to the validation of a learning set of three categories on a set of 42 subsamples (in total, 47,473 vignettes). Please click here to view a larger version of this figure.

3. Calculating the individual size distribution, size spectra, and size metrics

NOTE: The calculations mentioned in this section were performed using Matlab (see script as Supplementary File 1).

1. Individual size distribution
   1. The last column of the Id_from_sorted_vignettes_YYYYMMDD_HHHH.txt file contains the validated classification of the objects. Select only the objects classified as macroinvertebrates to depict their individual size distribution in the sample.
      NOTE: Individual body size corresponds to the ellipsoidal volume of the macroinvertebrate organisms. The system provides measurements in pixels.
   2. Concatenate the vectors with the size measurements from both scans, because each fraction has a different subsampling exponent. Before concatenation, correct for the fractionation by replicating the size vectors as many times as the corresponding subsample has been fractionated.
      NOTE: This step is needed if a scan corresponds to a fraction of a sample (i.e., coarse or fine).
   3. Calculate the ellipsoidal volume from the major (M) and minor (m) axes of prolate ellipsoids with the same pixel areas as the organisms. Before computing the ellipsoidal volume, convert the major (M) and minor (m) axes from pixels to millimeters (mm) with the following conversion factor (cf):
      1 pixel = 2,400 dpi
      1 in = 25.4 mm
      cf = 25.4/2400
      The ellipsoidal volume (ellipVol with units in mm³) corresponds to:
      
   4. Depict the probability density function of the individual size distribution on the log₂ scale.
2. Size diversity
   1. Calculate the size diversity (Sd) following Quintana et al. (2008)⁸, as in García-Comas et al. (2016)³⁵:
      
      where p_x(x) is the probability density function of size x, and x represents log₂ (ellipVol). This measure is, therefore, the Shannon diversity index adapted to a continuous measure, such as the individual size distribution in a community.
3. Normalized biovolume size spectrum (NBSS)
   1. Define the size classes of the NBSS, establishing the lower bound of the spectrum as the 0.01 quantile of the macroinvertebrate size distribution in the samples and creating size classes by a geometrical scale of base 2 until the largest organism in the samples is encompassed.
      NOTE: The size class width increases with size to account for the greater variability associated with greater sizes. The NBSS of the macroinvertebrate communities analyzed here had 14 size classes (Table 1).
   2. Obtain the normalized biovolume by dividing the total biovolume in each size class by the size class width.
4. Size spectrum slope
   1. Calculate the linear slope of the NBSS.
      NOTE: The slope (µ) is calculated based on the relationship between the log₂ (size class mid-point) and the log₂ (normalized biomass) in the size classes greater than the mode, ignoring any empty ones (in this study, the size classes from 3 to 14).

Table 1: Size classes of the normalized biomass size spectrum (NBSS). The table also shows the 15 size class limits and the size class mid-points of the organisms.