We report detailed procedures for an invasive plant biomass estimation method that utilizes data obtained from unmanned aerial vehicle (UAV) remote sensing to assess biomass and capture the spatial distribution of invasive species. This approach proves highly beneficial for conducting hazard assessment and early warning of invasive plants.
We report on the detailed steps of a method to estimate the biomass of invasive plants based on UAV remote sensing and computer vision. To collect samples from the study area, we prepared a sample square assembly to randomize the sampling points. An unmanned aerial camera system was constructed using a drone and camera to acquire continuous RGB images of the study area through automated navigation. After completing the shooting, the aboveground biomass in the sample frame was collected, and all correspondences were labeled and packaged. The sample data was processed, and the aerial images were segmented into small images of 280 x 280 pixels to create an image dataset. A deep convolutional neural network was used to map the distribution of Mikania micrantha in the study area, and its vegetation index was obtained. The organisms collected were dried, and the dry weight was recorded as the ground truth biomass. The invasive plant biomass regression model was constructed using the K-nearest neighbor regression (KNNR) by extracting the vegetation index from the sample images as an independent variable and integrating it with the ground truth biomass as a dependent variable. The results showed that it was possible to predict the biomass of invasive plants accurately. An accurate spatial distribution map of invasive plant biomass was generated by image traversal, allowing precise identification of high-risk areas affected by invasive plants. In summary, this study demonstrates the potential of combining unmanned aerial vehicle remote sensing with machine learning techniques to estimate invasive plant biomass. It contributes significantly to the research of new technologies and methods for real-time monitoring of invasive plants and provides technical support for intelligent monitoring and hazard assessment at the regional scale.
In this protocol, the proposed method of invasive biomass estimation based on UAV remote sensing and computer vision can reflect the distribution of invasive organisms and predict the degree of invasive biohazard. Estimates of the distribution and biomass of invasive organisms are critical to the prevention and control of these organisms. Once invasive plants invade, they can damage the ecosystem and cause huge economic losses. Quickly and accurately identifying invasive plants and estimating key invasive plant biomass are major challenges in invasive plant monitoring and control. In this protocol, we take Mikania micrantha as an example to explore an invasive plant biomass estimation method based on unmanned aerial remote sensing and computer vision, which provides a new approach and method for the ecological research of invasive plants and promotes the ecological research and management of invasive plants.
At present, the biomass measurement of Mikania micrantha is mainly done by manual sampling1. The traditional methods of biomass measurement need a lot of workforce and material resources, which are inefficient and limited by the terrain; it is difficult to meet the needs of regional biomass estimation of Mikania micrantha. The major advantage of using this protocol is that it provides a method for quantifying regional invasive plant biomass and spatial distribution of invasive plants in a way that does not take into account the sampling limitations of the area and eliminates the need for manual surveys.
UAV remote sensing technology has achieved certain results in plant biomass estimation and has been widely used in agriculture2,3,4,5,6,7, forestry8,9,10,11, and grassland12,13,14. UAV remote sensing technology has the advantages of low cost, high efficiency, high precision, and flexible operation15,16, which can efficiently obtain remote sensing image data in the study area; then, the texture feature and vegetation index of remote sensing image are extracted to provide data support for the estimation of plant biomass in large area. Current plant biomass estimation methods are mainly categorized into parametric and nonparametric models17. With the development of machine learning algorithms, nonparametric machine learning models with higher accuracy have been widely used in remote sensing estimation of plant biomass. Chen et al.18 utilized mixed logistic regression (MLR), KNNR, and random forest regression (RFR) to estimate the aboveground biomass of forests in Yunnan Province. They concluded that the machine learning models, specifically KNNR and RFR, resulted in superior outcomes compared to MLR. Yan et al.19 employed RFR and extreme gradient boosting (XGBR) regression models to assess the accuracy of estimating subtropical forest biomass using various sets of variables. Tian et al.20 utilized eleven machine-learning models to estimate the aboveground biomass of varying mangrove forest species in Beibuwan Bay. The researchers discovered that the XGBR method was more effective in determining the aboveground biomass of mangrove forests. Plant biomass estimation using man-machine remote sensing is a well-established practice, however, the use of UAV for biomass estimation of the invasive plant Mikania micrantha has yet to be reported both domestically and internationally. This approach is fundamentally different from all previous methods of biomass estimation for invasive plants, especially Mikania micrantha.
To sum up, UAV remote sensing has the advantages of high resolution, high efficiency, and low cost. In the feature variable extraction of remote sensing images, texture features combined with vegetation indexes can obtain better regression prediction performance. Nonparametric models can obtain more accurate regression models than parametric ones in plant biomass estimation. Therefore, to calculate the null distribution of invasive plants and their biomass precisely, we suggest the following outlined procedures for the invasive plant biomass experiment that relies on remote sensing using UAVs and computer vision.
1. Preparation of datasets
2. Identification of Mikania micrantha
3. Estimation of invasive plant biomass
We show representative results of a computer vision-based method for the estimation of invasive plants, which is implemented in a programmatic way on a computer. In this experiment, we evaluated the spatial distribution and estimated the biomass of invasive plants in their natural habitats, using Mikania micrantha as a research subject. We utilized a drone camera system to acquire images of the research site, a portion of which is exhibited in Figure 3. We utilized the ResNet101 convolutional neural network to identify the plants present within the study area. Subsequently, we mapped the spatial distribution of invasive plants and illustrated some of our findings in Figure 4. In Figure 3, Mikania micrantha can be observed climbing atop the plant adorned with white flowers. The other plants, as well as the road and accompanying elements, are uniformly depicted in the background. In Figure 4, the model recognizes the red part as Mikania micrantha. Comparing the two sets of images, it is evident that ResNet101 demonstrates robust detection of Mikania micrantha in complex backgrounds. Furthermore, it accurately maps the distribution of Mikania micrantha in the study area with high precision.
The biomass of invasive plants in the study area was estimated by truncating all Mikania micrantha sample plot images from orthophotos at 280 × 280 pixels and extracting the vegetation indices RBRI, GBRI, GRRI, RGRI, NGBDI, and NGBDI. Regression analysis was conducted using the KNNR regression model, with the six indexes as inputs to the estimation model and biomass as the model's output. Figure 5 presents the results: the graph's horizontal coordinates represent the values of the field-measured biomasses, the vertical coordinates represent the values of the model-predicted biomasses, and the gray areas represent the confidence intervals. The results demonstrate strong predictive performance, with an R² value of 0.62 and an RMSE of 10.56 g/m2. The model enhances the accuracy of Mikania micrantha biomass estimation, and the spatial distribution map in Figure 6 effectively captures the distribution of Mikania micrantha biomass.
Figure 1: UAV remote sensing systems. Some examples of RGB image data captured by UAV. Please click here to view a larger version of this figure.
Figure 2: Route planning. Study on regional route planning Please click here to view a larger version of this figure.
Figure 3: Invasive plant identification results within the study area. The figure displays the findings of identifying Mikania micrantha in the study zone via ResNet101 convolutional neural network. The red region in the picture indicates the area detected by ResNet101 as Mikania micrantha, while the other background signifies the rest of the study area. These outcomes correspond to the sample images portrayed in Figure 1. They represent the recognition results of Figure 1, respectively. Please click here to view a larger version of this figure.
Figure 4: Spatial distribution of invasive plants. The model recognizes the red part as Mikania micrantha. Please click here to view a larger version of this figure.
Figure 5: Biomass prediction regression results. The horizontal axis displays biomass values observed in the field, while the vertical axis portrays biomass values estimated by the model. The gray-shaded regions denote confidence intervals. The KNNR model attained an R2 of 0.65 on the test set, while the lowest root mean square error amounted to 30.59 g/m2. In the regression scatter plot of the model, many Mikania micrantha biomass estimations were within the confidence interval, indicating the validity of the biomass prediction. Please click here to view a larger version of this figure.
Figure 6: Spatial distribution of Mikania micrantha biomass. The figure illustrates the estimation of Mikania micrantha biomass throughout the research area utilizing KNNR as the predictive model, along with the extracted Mikania micrantha biomass distribution map. Darker shaded regions represent higher quantities of Mikania micrantha biomass. Please click here to view a larger version of this figure.
Figure 7: Schematic diagram of the main development of this protocol. The figure illustrates the main steps of the protocol presented. Please click here to view a larger version of this figure.
Vegetation Index Name | Calculation Formula |
Green Blue Ratio Index | GBRI = DNG/DNB |
Green Red Ratio Index | GRRI = DNG/DNR |
Red Blue Ratio Index | RBRI = DNR/DNB |
Red Green Ratio Index | RGRI = DNR/DNG |
Normalized Green Blue Difference Index | NGBDI = (DNG – DNB)/(DNG + DNB) |
Normalized Green Red Difference Index | NGRDI = (DNG – DNR)/(DNG + DNR) |
Table 1: Vegetation index calculation formula. The vegetation indices used in this protocol and their respective calculation formulas.
Supplementary Figure 1: Cropping an image to 280 x 280 pixels via Python script using the OpenCV library. Please click here to download this File.
Supplementary Figure 2: Partitioning the dataset into a training set, validation set, and test set. Please click here to download this File.
Supplementary Figure 3: Feature extraction and reduction of dimensionality. (A) Initial feature extraction. (B) Convolution operation. (C) Reduction of dimensionality. Please click here to download this File.
Supplementary Figure 4: Conveying the features to the FC layer in ResNet architecture. Please click here to download this File.
Supplementary Figure 5: Setting the parameters. Please click here to download this File.
Supplemnetary Figure 6: The specific implementation code for generating comprehensive distribution map. Please click here to download this File.
Supplementary Figure 7: RandomResizedCrop and RandomHorizontalFlip functions. Please click here to download this File.
We present the detailed steps of an experiment on estimating the biomass of invasive plants using UAV remote sensing and computer vision. The main process and steps of this agreement are shown in Figure 7. Proper sample quality is one of the most crucial and challenging aspects of the program. This importance holds true for all invasive plants as well as any other plant biomass estimation experiments24.
To identify the distribution of invasive plants in the study area, we must first acquire visible and continuous photogrammetry of the study area using remote sensing from UAV. To achieve this, appropriate UAV photographic altitude and camera resolution are necessary. This will ensure that ResNet101, a convolutional neural network, can obtain features such as vegetation index and texture from the image. These features help to identify invasive plants even in complex environmental backgrounds and ultimately map their distribution in the study area.
To estimate the biomass of invasive plants in the study area, it is necessary to create square sample frames25 with 25-50 random sampling points. Once the unmanned aerial system completes the process of taking photographs, it is necessary to gather the invasive plants from the sample frames. It is crucial to note, as stated in the initial section, that collecting the biomass from the frames should be done without moving them and with precision by assigning unique numbers. Additionally, the biomass should be delicately dried and weighed in the laboratory to obtain an accurate measurement. The KNNR regression model18 was utilized to obtain the spatial distribution of invasive plants, using six vegetation indices extracted from images acquired by the UAV camera system as input to the estimation model and biomass as the model's output.
The methods described in this paper for estimating invasive plant populations are not exhaustive. Additional tools may be utilized to obtain comparable data, and numerous practical or innovative modifications may be employed. A range of drones may be utilized, of which the designated model in the protocol serves as only one option and can be substituted with any drone possessing auto-navigation integration, the capability to adapt to a gimbal, the ability to fly up to 50 meters, a payload capacity exceeding that of the selected camera, and a range covering the entire study area. When choosing a camera for UAV, it is important to consider factors such as maximum resolution, effective image pixels, maximum pixels, and other imaging parameters, as well as the size and weight restrictions of the UAV load. Additionally, adequate range is necessary for the successful completion of the filming process. Furthermore, the model for identification is not the sole method and can be adjusted for multiple invasive plant species, such as woody or diverse types of plants, to enhance the suggested method. This strategy can also be progressed into a regression algorithm that employs fewer samples, acquires additional vegetation indices, and attains a heightened degree of precision. Alternatively, a reduced identification network could be implemented to identify invasive plants, allowing for the practical deployment of intelligent identification methods.
The presented method's primary constraint is the model's dependence on lighting conditions and background. Improvements towards higher accuracy can only come with further incorporation of multispectral26, LiDAR12, and meteorological data27. Acquiring this sort of data can be challenging and may necessitate pricey equipment, but we can boost the accuracy of our data by avoiding direct sunlight and isolating the backgrounds of plants as we sample the area via our UAV filming system.
The benefit of utilizing this methodology, as opposed to others, lies in its ability to furnish the spatial distribution of invasive plant species in the designated study area. The distribution is determined through regression analysis, utilizing six vegetation indices extracted from images captured via the UAV camera system as inputs, with biomass as the output of the model. This approach utilizes a portable method for estimating biomass in multiple study areas simultaneously, which is fundamentally distinct from prior manual biomass collection methods. The conventional method involves manually conducting surveys and collecting significant quantities, which is inefficient and subjective28, thus failing to provide a practical estimation of biomass given complex conditions.
Estimation of invasive plant biomass using the aforementioned methodology allows for quantification of regional distribution. Furthermore, it offers technical assistance for intelligent monitoring and hazard assessment at the regional level.
The authors have nothing to disclose.
The author thanks the Chinese Academy of Agricultural Sciences and Guangxi University for supporting this work. The work was supported by the National Key R&D Program of China (2022YFC2601500 & 2022YFC2601504), the National Natural Science Foundation of China (32272633), Shenzhen Science and Technology Program (KCXFZ20230731093259009)
DSLR camera | Nikon | D850 | Sensor type: CMOS; Maximum number of pixels: 46.89 million; Effective number of pixels: 45.75 million; Maximum resolution 8256 x 5504. |
GPU – Graphics Processing Unit | NVIDIA | RTX3090 | |
Hexacopter | DJI | M600PRO | Horizontal flight: 65 km/h (no wind environment); Maximum flight load: 6000 g |
PyCharm | Python IDE | 2023.1 | |
Python | Python | 3.8.0 | |
Pytorch | Pytorch | 1.8.1 |