- Feb 2017
Comparison of ITCD algorithms is challenging when there are differences in study focus, study area, data applied, and accuracy assessment method used. Before 2005, the few studies that compared methods generally tested approaches on a common dataset.
This difficulty in comparing algorithms (due to differences in forest type, location, and assessment strategy used for different algorithms) indicates a clear need for set of open data and centralized assessment to allow different methods to be competed against one another to determine the best routes forward.
This kind of approach has been very successful in other image analysis problems (e.g., ImageNET).
The National Ecological Observatory Network data seems ideal for doing something like this. Data is/will be available for a variety of different systems and with LiDAR, Hyperspectral, RGB, and field data for large numbers of plots.
Additionally, it is often challenging to apply an algorithm developed in one forest type to another area.
This difficulty of applying across forest types is central to the challenges of developing approaches that can be applied to continental scale data collection like that being conducted by NEON. Overcoming this challenge will likely require incorporating ecological information into models, not just the remote sensing, and determining how to choose and adjust different approaches to get the best delineations possible based on information about the forest type/location.
The most useful information that can be incorporated into ITCD studies is the expected crown size and stand density [25,67].
This kind of data is available for NEON plots and so these methods could potentially be well leveraged with NEON data. This would be particularly true if the NEON plot data could be used to develop a spatial model for these features that could be used to predict their values across space.
Only 23 studies actually integrated both active and passive data sources into the ITCD procedure since 2000 (Figure 2).
Only a small fraction of studies combine LiDAR and Hyperspectral data for the crown delineation phase of analysis.
Another limitation is that few approaches take full advantage of the information contained within remotely sensed data, e.g., using only one band of multispectral imagery  or only the canopy height model derived from LiDAR data . Significant amounts of information are dismissed or neglected during data preparation or processing. The integration of multispectral data and discrete LiDAR data is commonly used to improve tree species classification  and fusion of passive and active remotely sensed data may reduce commission and omission errors in ITCD results .
Excellent point about the importance of integrating all available data to make the best possible crown delineations. In the case of the National Ecological Observatory Network Airborne Observation Platform methods that leverage the LiDAR, hyperspectral, and high-resolution RGB photographic data should have the potential to outperform methods that ignore components of this data.
Comparison of methods continues to be complicated by both choice of reference data and assessment metric; it is imperative to establish a standardized two-level assessment framework to evaluate and compare ITCD algorithms in order to provide specific recommendations about suitable applications of particular algorithms.
An important call for standardized reference data and assessment for moving individual tree crown delineation forward.