OpenForest: a data catalog for machine learning in forest monitoring

Abstract

Forests play a crucial role in the Earth’s system processes and provide a suite of social and economic ecosystem services, but are significantly impacted by human activities, leading to a pronounced disruption of the equilibrium within ecosystems. Advancing forest monitoring worldwide offers advantages in mitigating human impacts and enhancing our comprehension of forest composition, alongside the effects of climate change. While statistical modeling has traditionally found applications in forest biology, recent strides in machine learning and computer vision have reached important milestones using remote sensing data, such as tree species identification, tree crown segmentation, and forest biomass assessments. For this, the significance of open-access data remains essential in enhancing such data-driven algorithms and methodologies. Here, we provide a comprehensive and extensive overview of 86 open-access forest datasets across spatial scales, encompassing inventories, ground-based, aerial-based, satellite-based recordings, and country or world maps. These datasets are grouped in OpenForest, a dynamic catalog open to contributions that strives to reference all available open-access forest datasets. Moreover, in the context of these datasets, we aim to inspire research in machine learning applied to forest biology by establishing connections between contemporary topics, perspectives, and challenges inherent in both domains. We hope to encourage collaborations among scientists, fostering the sharing and exploration of diverse datasets through the application of machine learning methods for large-scale forest monitoring. OpenForest is available at the following url: https://github.com/RolnickLab/OpenForest.

Impact Statement

OpenForest establishes a constantly evolving catalog of open-access forest datasets. This catalog is open for contributions and aims to provide a single central hub for such datasets within the open-source community. In addition to introducing the OpenForest catalog, we provide in this article a detailed overview of complementary research topics and challenges in forest monitoring and machine learning, so as to better enable the impactful use of these datasets in interdisciplinary research. We hope this work will ultimately contribute substantially to enhancing our comprehension of global forest composition as well the development of innovative machine learning algorithms.

1. Introduction

Forests cover one third of the Earth’s land surface (The Food and Agriculture Organization of the United Nations, 2020). They provide a range of valuable ecosystem goods and services to humanity, including timber provision, water and climate regulation, and atmospheric carbon sequestration (Millennium Ecosystem Assessment, 2001; Bonan, 2008). They also serve as habitat for a myriad of plant, animal, and microbial species. However, human activities have had, and continue to have, a major impact on forests worldwide.

More than 3,000 ha of forests disappear every hour from deforestation (Hansen et al., 2013; The Food and Agriculture Organization of the United Nations, 2020). Yet forests are also increasingly recognized as natural solutions to the joint climate and biodiversity crises (Griscom et al., 2017, 2020; Drever et al., 2021). Forest-based adaptation through avoided forest conversion, improved forest management, and forest restoration could mitigate over 2 Gt CO₂-eq emissions per year by 2030 (Intergovernmental Panel On Climate Change (IPCC), 2023), with variations observed in different regions worldwide (Griscom et al., 2017, 2020; Bastin et al., 2019; Busch et al., 2019; Drever et al., 2021), while being limited by the climatic effects we are witnessing on forests (Zhu et al., 2018).

Due to their significant economic and ecological importance, monitoring forests has attracted considerable attention. It includes the assessment of ecosystem functional properties, as well as the evaluation of forest health, vitality, stress, and diseases (see Section 2.1). However, monitoring forests presents significant challenges, especially using field-based approaches (see Section 2.2). Forests cover huge areas and can be difficult to access. Consequently, remote sensing has and continues to play an important role in forest monitoring worldwide. Nowadays, a wide array of remote sensing platforms and sensors to monitor forests are available and being used. This includes platforms such as drones (also referred to as unoccupied aerial vehicles [UAVs]), airplanes, or satellites, and sensors ranging from passive optical imagery, to active methods such as light detection and ranging (LiDAR) or synthetic aperture RADAR (SAR) (Verrelst et al., 2015; White et al., 2016).

In recent years, the applications of remote sensing data for the Earth-related purposes (Ma et al., 2019; Camps-Valls et al., 2021), such as forest monitoring (Fassnacht et al., 2016; Diez et al., 2021; Kattenborn et al., 2021; Michalowska and Rapinski, 2021), have gained momentum due to the adoption of machine learning methods and algorithms. It has been inspired by continuous improvements in the performance of deep learning models used in computer vision challenges in the past decade (Deng et al., 2009; Lin et al., 2014; Everingham et al., 2015). Recent advances in deep learning model architectures have enabled the integration of remote sensing data from various sensors and resolutions—spatial, temporal, or spectral—which presents promising opportunities to enhance forest monitoring practices (Cong et al., 2022; Rahaman et al., 2022; Reed et al., 2022; Tseng et al., 2023).

Numerous machine learning challenges related to forest monitoring have yet to be explored (see Figure 1), and addressing them will require diverse and large forest datasets (Liang and Gamarra, 2020). While there is a wealth of remote sensing data that is freely available, these data can be difficult to access because they involve a wide variety of sensory modalities, geographies, and tasks and are spread out across many repositories. To the best of our knowledge, no comprehensive, central repository of open-access forest datasets currently exists, a gap which we fill here with OpenForest (https://github.com/RolnickLab/OpenForest). The OpenForest catalog is designed to simplify the process of accessing and highlight forest monitoring datasets for researchers in the field of machine learning and forest biology, thereby accelerating progress in these domains.

Figure 1. Overview of forest monitoring topics and challenges associated with machine learning perspectives and challenges. Note: Each forest monitoring topic and its challenges are detailed with their corresponding section number (in red). They are associated with the three main machine learning perspectives and challenge categories, namely generalization, limited data, and domain-specific objectives, along with their corresponding section number (in red).

In this article, we present the existing biological topics and challenges related to forest monitoring that scientists are currently investigating (Section 2) and which could be of interest of machine learning practitioners. Additionally, we briefly introduce several machine learning research topics, exploring their potential applications in addressing biology-related challenges (Section 3). These applications hold promise in assisting ecologists and biologists in their work. Moreover, we conduct a thorough review of open-access forest datasets across different spatial scales (Figure 2) to support both machine learning and biology research communities in their work (Section 4). Finally, we provide perspectives on the space of machine learning applications with forest datasets (Section 5).

Figure 2. Illustration of forest monitoring datasets at different scales. Note: Inventories are in situ measurements realized at the tree level. Ground-based datasets are recorded within or below the canopy of the trees. Aerial datasets are composed of recordings from sensors mounted on unoccupied (drones) or occupied aircrafts. Satellite datasets are collected from sensors mounted on satellites orbiting the Earth. Map datasets are generated at the country or world level using datasets at the aerial or satellite scales.

2. Forest monitoring: current topics and challenges

Forest monitoring is an empirical science that is increasingly based on data-driven machine learning methods and, as such, benefits by improved data access through open data (Wulder et al., 2012; De Lima et al., 2022). In particular, deep learning algorithms are widely recognized for their strong performance in diverse tasks, but their successful application often relies on large datasets to unleash their performance and enhance their generalization potential. This section seeks to emphasize the importance of open-access forest datasets for two primary purposes: first, to facilitate a more comprehensive exploration of current topics in the context of forest monitoring (Section 2.1); and second, to better assess forest monitoring-related challenges (Section 2.2), particularly for machine learning practitioners.

2.1. Forest monitoring topics

Considering the significance of forests both economically and ecologically, forest monitoring encompasses a range of trackable forest attributes. Each of them can be sensed with different sensors, platforms, and across different scales. The forest attribute itself (such as the regression of a biochemical property or the detection of tree individuals), together with the structure of the remotely sensed signals, collectively determines the appropriate machine learning algorithms to be employed. These aspects are succinctly discussed in the following sections.

2.1.1. Forest extent and forest type mapping

Tracking the extent of forests is crucial to understand the spatial distribution of forest resources, ecosystem services, and assess the role of forest in land surface dynamics (Keenan et al., 2015). Thereby, forest can be classified to different management, functional, or ecosystems types (e.g., coniferous, deciduous forests) (Buchhorn et al., 2020; Zhang et al., 2021). In this regard, Earth observation data from long-term satellite missions (e.g., Landsat or Sentinel described in Section 4.4) enable to track forest extent dynamics across past decades (Hansen et al., 2013), enabling to assess conservation efforts and anthropogenic land cover change, such as deforestation for agricultural expansion (Curtis et al., 2018).

2.1.2. Tree species mapping

A fine-scaled representation of forest stands in terms of their species composition is relevant for forestry (e.g., species-specific timber supply), biogeographical assessments (e.g., climate change-induced shifts in species distributions), or biodiversity monitoring (Fassnacht et al., 2016; Wang and Gamon, 2019; Cavender-Bares et al., 2020). Recent developments in machine learning greatly advanced the identification of tree species in high-resolution data (e.g., imagery or LiDAR point clouds from drones and airplanes) using semantic and instance segmentation methods (Schiefer et al., 2020; Li et al., 2022a; Cloutier et al., 2023). At large spatial scales, Earth observation satellite data, providing coarser spatial but high temporal and spectral resolutions, enable accurate assessments of tree species distributions using spatiotemporal machine learning methods (Ienco et al., 2019; Bolyn et al., 2022).

2.1.3. Biomass quantification

Forests provide cardinal ecosystem services through their provision of timber and their role as sinks in the terrestrial carbon cycle (Regnier et al., 2022). Tree biomass is primarily a product of the wood volume and density, while both properties are challenging to obtain from remote sensing data. Accurate biomass estimates of individuals trees can be obtained from close-range 3D representations acquired from terrestrial or drone-based LiDAR systems (Brede et al., 2022). More indirectly related information on crown and canopy structure derived from airborne or spaceborne LiDAR and SAR data can be used to estimate biomass at the stand scale (Le Toan et al., 2011; Lu et al., 2016). Some studies have indicated the value of passive optical data from satellites, since forest biomass is partially correlated with foliage density (Besnard et al., 2021; Potapov et al., 2021). Given that precise large-scale biomass distributions cannot be directly revealed through a single remote sensing modality alone, deep learning may play a crucial role to simultaneously exploit the suite and high dimensionality of available data modalities (Yang et al., 2020).

2.1.4. Forest health, disturbance, and mortality

In many regions, forest ecosystems are under pressure as globalization facilitates the introduction of exotic pests and pathogens, climate change exceeds the resilience and resistance of trees (Hartmann et al., 2022), while nutrient and water cycles are affected by anthropogenic activities (Steffen et al., 2005; Trumbore et al., 2015). A decline in tree health, for example, due to pathogen infestations or shortages of water and nutrients, can lead to a variety of symptoms, such as changing concentrations of multiple biochemical tissue properties (e.g., pigments, carbohydrates, and water content), which in turn can be sensed through multispectral or hyperspectral reflectance (Zarco-Tejada et al., 2018; Zarco-Tejada et al., 2019). In this context, deep learning algorithms are very promising, due to their capability to exploit high-dimensional data (e.g., hyperspectral) and to translate it into a suite of foliage properties relevant to vegetation health (Cherif et al., 2023). An interconnected topic are globally increased rates of forest mortality (Allen et al., 2010; Hartmann et al., 2022). In this context, a wealth of approaches was successfully employed at local scales, such as the detection of dead trees via semantic or instance segmentation techniques (Sani-Mohammed et al., 2022; Cloutier et al., 2023), or at large scales, such as the regression of annual cover of dead tree crowns in satellite image pixels with deep learning-based time series analysis (Schiefer et al., 2023).

2.1.5. Biophysical traits and functional ecosystem properties

With accelerated biodiversity decline and environmental change, understanding functional properties, their diversity across stands and landscapes, as well as their phenology (temporal dynamics), is essential to assess the resilience and resistance of ecosystems (Thompson et al., 2009; Sakschewski et al., 2016). Given that trees through evolution developed different strategies to interact with light, their appearance studied with optical remote sensing signals can inform on a variety of functional traits, such as the foliage density, date of green up, or contents of different pigments, and carbohydrates (Schneider et al., 2017; Cherif et al., 2023). Such functional traits determine the configuration of an ecosystem and thereby modulate functional ecosystem processes (Migliavacca et al., 2021; Gomarasca et al., 2023), that is, fluxes of energy and matter between the terrestrial biosphere, pedosphere, hydrosphere, and atmosphere, including carbon, evapotranspiration, latent, and sensible heat. Due to the cardinal importance of these fluxes in the Earth system, considerable efforts have been made to monitor them using a ground-based sensor network of flux towers (e.g., FLUXNET) (Baldocchi et al., 2001). Given the complexity of these ecosystem processes, deep learning is assumed to greatly enlarge our capabilities to exploit local flux towers and globally available remote sensing data to spatially and temporally extrapolate and understand forest ecosystem process (Jung et al., 2019; Reichstein et al., 2019; Camps-Valls et al., 2021; ElGhawi et al., 2023).

2.2. Forest monitoring challenges

Forests are complex ecosystems dominated by trees. As living organisms, trees are affected by various abiotic and biotic factors, which influence their remotely sensed signal via their foliage properties and crown architecture (Kulawardhana, 2011). Machine learning researchers wishing to develop algorithms to monitor forests using remote sensing data must be aware of these sources of biological variation and their origin. Because some of this biological variation is largely unpredictable but potentially clustered in space and time (e.g., insect outbreaks affecting tree health and random genetic variation within tree species populations), it can be seen as a challenge as it might lead to systematic errors for prediction tasks. On the other hand, part of this variation is deterministic (e.g., changes in leaf color and other phenological changes driven by seasonal fluctuations that occur every year) and could be leveraged to improve model performances (Cloutier et al., 2023). Another major, pervasive challenge in forest monitoring are the difficulties associated with the acquisition of ground data to train or validate machine learning models using remote sensing data. Below we summarize these primary challenges.

2.2.1. Tree species

There are an estimated 73,000 tree species on Earth (Cazzolla Gatti et al., 2022), the majority of which are found in the tropics. While tree species show many similarities (e.g., the presence of woody stems and branches), every tree species differs from one another in their chemical and structural make-up and how they will reflect solar radiation (Asner et al., 2014). For example, tree foliage of different species comes in various shades of green that reflect the concentrations of pigments (e.g., chlorophylls and carotenoids) (Gates et al., 1965). Likewise, tree species differ from each other in their leaf form crown structure (Verbeeck et al., 2019), which will affect the remotely sensed signal. Such foliar biophysical and crown structural variation among tree species is the result of millions of years of evolution and of adaptations to various environmental conditions (Meireles et al., 2020).

From a machine learning perspective, the biggest challenge associated with tree species diversity is that models trained on data from a given set of tree species might transfer poorly to other regions that host different species. However, the phylogeny and evolutionary distances of tree species are fairly well known (Zanne et al., 2014), and tree species that are closer phylogenetically tend to be more similar in their traits (Ackerly, 2009). As such, phylogenetic correlations and distances among tree species can potentially be leveraged to improve model transferability. Another interconnected challenge, elaborated upon in the following sections, pertains to the dynamic nature of tree species in relation to their leaf biophysical and crown structural characteristics. Instead, each individual differs according to their abiotic (e.g., microclimate and soil) and biotic environment (competition and herbivory), and as such, the expression of foliage and crown properties can overlap between species (Fassnacht et al., 2016).

2.2.2. Seasons and phenology

Trees are sessile organisms, but they still respond dynamically to fluctuating seasons. In some cases, phenological properties, such as leaf onset, flowers, or seeds, might be of direct interest to monitor ecological phenomena or species (Wagner, 2021), in which case high-frequency multitemporal imagery might be required. Indeed, phenological changes among species, for example, changes in leaf color during autumn senescence, can help to distinguish tree species based on color, which can be used to improve species classification models (Cloutier et al., 2023). However, phenological properties may also hinder the transferability of models across time (Kattenborn et al., 2022b). For instance, the information learnt by a machine learning model using data acquired in summer may not transfer to the same location in fall as trees may have changed in their leaf biochemical properties or the fraction of flowers and seeds in the canopy (Schiefer et al., 2021). In such cases, the temporal representativeness of data on individual tree species can be key (Kattenborn et al., 2022b).

2.2.3. Forest dynamics

The structure and composition of forests is strongly influenced by abiotic factors such as climate, geology, and soils, as well as water availability. For example, declining temperatures and/or growing season lengths with increasing latitude and/or elevation can filter out tree species that cannot tolerate low temperatures (e.g., low frost resistance) or that do not have enough time to produce mature tissue once the growing season becomes too short (Körner et al., 2016). In addition, changes in soil nutrient availability driven by geomorphological processes can influence forest canopy biochemistry (Chadwick and Asner, 2018). Water supply is also important: too much water favors trees that can tolerate waterlogging, whereas too little water favors trees that can resist or recover from xylem cavitation (Choat et al., 2012). Much of these environmental influences on forest composition express themselves via tree species turnover, that is, changes in tree species composition across these spatial environmental gradients or discontinuities. However, changes in forest composition and structure can also arise through intraspecific variation. Applications of machine learning methods to forest monitoring should integrate these sources of variation. In particular, incorporating environmental drivers of forest composition and structure as model inputs may help to transfer forest monitoring models from one region to the other.

Tree monitoring can also be affected by biotic factors—that is, by other organisms. First, pests and pathogens can impact tree health, foliage chemistry, and/or water content, which in turn can affect the remotely sensed signal of forest canopies (Sapes et al., 2022). The health status of trees is often directly expressed via their foliage properties and crown architecture and therefore can cause a large variability in remote sensing signals (Zarco-Tejada et al., 2018; Kattenborn et al., 2022a). In addition, the remotely sensed signals of trees can also be “overshadowed” by other organisms that live in their crowns (epiphytes), particularly in tropical environments (Baldeck et al., 2015). Prominent examples are lianas or mistletoes.

Forest management activities as part of forest dynamics, such as harvesting, thinning, and pruning, can challenge the accurate mapping of forest attributes with remote sensing, as this crucial information is often unavailable but significantly impacts forest structure and composition. This lack of data therefore introduces uncertainty into remote sensing analyses and models.

2.2.4. Data collection

As previously mentioned, forests can exhibit significant diversity in terms of their composition and structure across different locations and time periods due to a variety of factors. This heterogeneity poses a particular difficulty in creating machine learning models for forest monitoring. Models developed for one region may not easily generalize to other regions that lie beyond the scope of the training data distribution. One solution for this issue would involve training these models using extensive datasets that encompass the complete spectrum of conditions present in diverse forest environments. Forest remote sensing data worldwide, especially acquired from sensors on satellites, are abundant and generally easily obtainable. In sharp contrast, there is a scarcity of ground-based data, including labels or annotations.

In contrast to other disciplines, annotating remote sensing data in the context of vegetation is often time-consuming, costly, and complex. This phenomenon arises due to the fact that vegetation of various species or conditions frequently exhibits striking similarities, often referred to as “greenery.” Moreover, vegetation communities often show smooth transitions across species or growth forms along environmental gradients. This aspect adds another layer of complexity to the task of distinguishing between individual plants, species, or growth forms (Kattenborn et al., 2021). Often, field inventories become essential to validate annotations, such as the identification of tree species (Kattenborn et al., 2020; Cloutier et al., 2023), or when the properties of interest, such as stem diameters, cannot be directly extracted from remote sensing data and require on-site measurements conducted by human observers. Gathering such field data is typically a time-intensive, expensive, and gradual process, leading to significant constraints on its accessibility. Field data such as stem diameters are very important to estimate aboveground tree biomass because most published allometric equations use stem diameter as its primary predictor (Gonzalez-Akre et al., 2022). Moreover, spatial coordinates frequently serve as the sole means of connecting field data to remote sensing data. However, GPS or GNSS geolocation in forest settings often introduces substantial uncertainties (ranging from meters to tens of meters), thereby posing challenges in accurately establishing a posteriori links between field observations and remote sensing data (Kattenborn et al., 2021).

Therefore, integrating various datasets is a strategy aimed at addressing the scarcity of annotated data, cutting down annotation expenses, and lessening the dependency on field-based ground truthing. Nonetheless, this may result heterogenous datasets: In numerous cases, annotations vary (such as boxes, polygons, and points), as well as their quality, across different applications. Annotations are frequently customized to match particular remote sensing data characteristics, such as spatial resolution. Hence, directly merging labels from different datasets is often not feasible, or at the very least, alternative approaches are necessary. One such approach is weakly supervised learning, where the potential lack of label quality is counteracted by leveraging a substantial quantity of data (see Section 3.2.2).

The key takeaway from this section is that the development of machine learning models for forest monitoring will consistently involve a substantial surplus of unlabeled remote sensing data in comparison to labeled ground-truth data. This disparity arises due to the inherent challenges in obtaining labeled data. This scenario is not exclusive to forest monitoring; rather, it is a prevalent aspect shared with other geospatial applications using remote sensing data (Rahaman et al., 2022; Mai et al., 2023a) (see Section 3.1.2). This has two main implications for machine learning research aimed at improving forest monitoring. First, there is a need to develop machine learning methods to forest monitoring that that can effectively utilize limited labeled data. One approach involves leveraging self-supervised learning techniques to extract valuable representations from the available data (see Section 3.2.1). Second, there exists a necessity for novel machine learning strategies, including active learning or alternative forms of model-assisted labeling. These approaches aim to expedite the process of label collection by human observers and reduce associated costs (see Section 3.2.3).

3. Machine learning perspectives and challenges

Machine learning algorithms in computer vision have gained significant capabilities over the past decade, for example, in image classification (Krizhevsky et al., 2012; Simonyan and Zisserman, 2015; Szegedy et al., 2015, 2016, 2017; He et al., 2016; Hu et al., 2018; Dosovitskiy et al., 2021; Liu et al., 2021; Touvron et al., 2021), object detection (Ren et al., 2015; Liu et al., 2016; Redmon et al., 2016; He et al., 2017; Redmon and Farhadi, 2018; Li et al., 2023a), and segmentation (Long et al., 2015; Ronneberger et al., 2015; He et al., 2017; Lin et al., 2017; Chen et al., 2018; Cheng et al., 2022; Kirillov et al., 2023). While many successful algorithmic paradigms have been established, different applications differ widely across sensory modalities and domain-specific constraints, necessitating the adaptation of algorithms to fit specific needs. For instance, detecting, localizing, and segmenting objects in a scene have been explored for LiDAR point cloud (Yang et al., 2018), automotive RADAR (Ouaknine et al., 2021a), and medical echocardiography (Leclerc et al., 2019).

Machine learning algorithms for remote sensing have been the subject of extensive innovation and application (Ma et al., 2019; Camps-Valls et al., 2021) for problems involving classification (Maxwell et al., 2018; Cheng et al., 2020), object detection (Cheng and Han, 2016; Li et al., 2020), and segmentation (Hoeser and Kuenzer, 2020; Yuan et al., 2021). In recent times, there has been a growing exploration of such techniques for forest monitoring purposes, aiming to enhance our understanding of forest composition, with a specific focus on tree species mapping (see Sections 2.2.1 and 2.1.2), that is, tree classification, tree detection, and tree segmentation (Fassnacht et al., 2016; Diez et al., 2021; Kattenborn et al., 2021; Michalowska and Rapinski, 2021). These tasks are accomplished using modalities from diverse sensors to gather complementary information.

Nevertheless, the study of machine learning for forest monitoring has not received as much attention as autonomous driving or medical imagery, despite the importance of forest conservation, restoration, and management as natural solutions to the joint climate and biodiversity crises (IPCC, 2023). Consequently, there are numerous unexplored machine learning challenges that need to be addressed in order to tackle climate change (Rolnick et al., 2023), including improving forest monitoring practices. Can the challenges encountered in adapting machine learning for forest monitoring be beneficial in exploring the challenges in the field of biology and ecology? This section will outline the current challenges in machine learning linked to forest monitoring, as described in Figure 1, and discuss the diverse strategies employed to tackle them.

3.1. Generalization

Generalization in machine learning refers to the ability of an algorithm to continue to perform well when evaluated on data different from that it was trained on (Zhang et al., 2017). One may speak of both in-distribution generalization (performance on data relatively similar to training data) and out-of-distribution generalization (performance on highly different data). Out-of-distribution generalization can be especially relevant to forest monitoring, since, as mentioned in Section 4, forest datasets have a wide range of variations in terms of geographical locations, species composition, sensors and scale (see Figure 2). Such variations introduce distinct data distribution shifts that need to be considered and addressed in forest monitoring tasks. For example, the effects of geographic variability of data have been examined in the context of tree species distributions (Dormann et al., 2007) and biomass estimation (Ploton et al., 2020). Simple algorithmic approaches to improve generalization include various forms of regularization (Zou and Hastie, 2005), data augmentation (Shorten and Khoshgoftaar, 2019), dropout (Srivastava et al., 2014), and batch normalization (Ioffe and Szegedy, 2015), while improving the breadth of training data, where possible, is also almost always beneficial in practice. However, generalization remains a very active field of research in machine learning. We consider two areas of work that may be of especial interest in forest monitoring.

3.1.1. Domain adaptation

Transfer learning refers to transferring information learnt by a model on one set of problems to different set of problems (Weiss et al., 2016). For example, one may pretrain a model on a large, commonly used dataset and then fine-tune it on a smaller dataset representing the specific problem in question. Transfer learning can boost generalization when there is a significant distribution shift between training and inference (Csurka, 2017). One particularly notable approach to transfer learning is domain adaptation, where a model must be applied to target domains that are unknown or lacking labeled data (Soltani et al., 2022). Some domain adaptation approaches have already been applied in plant identification (Ganin and Lempitsky, 2015). Autonomous driving has witnessed significant exploration in the realm of unsupervised domain adaptation (UDA), where a model is trained on labeled data from the source domain and unlabeled data from the target domain, with the objective of improving its performance specifically on the target domain. It has been explored in the context of unlabeled or unseen source or target domains (Wilson and Cook, 2020) using generative (Hoffman et al., 2018) or adversarial (Vu et al., 2019) methods. The UDA framework has also been explored for cross-modal learning considering domains from different sensor modalities (Jaritz et al., 2020). Within the context of forest monitoring, this framework could prove particularly valuable for adapting a model from one forest to another, regardless of whether they belong to the same biome or not, to identify similar species across both the source and target domains (see Sections 2.1.2 and 2.2.1). Additionally, this framework would be beneficial for adapting the model to address distribution shifts that occur between tree signature distributions (see Sections 2.1.4, 2.2.2, and 2.2.3) as well as between different sensors (see Section 2.2.4). Domain adaptation has been investigated in the field of remote sensing mostly in the context of extrapolation across time or geographical region, including approaches for both aerial and satellite data (Shi et al., 2022; Wang et al., 2022; Arnaudo et al., 2023; Ma et al., 2023; Xu et al., 2023). Such work holds potential for training generalizable algorithms for forest monitoring, such as adapting models from PhenoCams to satellite images (Kosmala et al., 2018).

3.1.2. Foundation models

Foundation models are models that can operate on diverse sets of input modalities, scales, data regimes, and downstream tasks. They refer to large-scale multimodal and multitask models, which have opened up research in generalization capacities such as increasing performances in applications unseen during training (Bommasani et al., 2021). Most of the discussed machine learning strategies can be further explored by training foundation models with diverse datasets, thereby enhancing their generalization capabilities. Forest datasets encompass a wide range of scales, ranging from field measurements to estimated world maps (refer to Figure 2), as well as varying resolutions and modalities for different tasks (as outlined in Section 4). The data diversity necessitates the utilization of generalized deep learning architectures. Influenced by the success of large language models (Devlin et al., 2019; Radford et al., 2019, 2021; Brown et al., 2020; Chowdhery et al., 2022; Hoffmann et al., 2022; Driess et al., 2023; Touvron et al., 2023), recent advancements in computer vision have led to the development of models that incorporate multiple modalities and can perform multiple tasks simultaneously. In recent studies, researchers have explored the concept of multitask vision by utilizing natural images (Cheng et al., 2021, 2022; Kirillov et al., 2023; Li et al., 2023a) or by employing text to enhance performance in vision-based tasks (Dancette and Cord, 2022; Xu et al., 2022; Jain et al., 2023a). In the realm of integrating image and text for performing tasks on both modalities, alternative approaches have been developed to improve performances by benefiting from their combination (Zhu et al., 2022; Li et al., 2023b). Additionally, generalist models have been constructed to be agnostic to specific modalities and tasks (Jaegle et al., 2021, 2022), enabling them to handle diverse modalities and tasks with a unified approach. In the field of computer vision, for example, the segment anything model (Kirillov et al., 2023) has demonstrated the capability to perform instance segmentation by leveraging prompts in conjunction with input images. These architecture frameworks hold a significant value for forest monitoring tasks, enabling the detection, segmentation, and estimation of tree properties over large geographical areas, such as their canopy surface or their aboveground biomass (Tolan et al., 2023; Tucker et al., 2023).

The utilization of foundation models with remote sensing data is still in its infancy. However, promising advances have been made in developing multimodal architectures (Zhang et al., 2023) and temporal-based approaches (Garnot et al., 2021; Garnot and Landrieu, 2021; Tarasiou et al., 2023) specifically tailored for precise tasks in remote sensing applications. Based on the masked autoencoder pretraining method (He et al., 2022), multimodal and multitask architectures have been developed for Earth observation applications, particularly for land use and land cover (LULC) estimation (Cong et al., 2022; Reed et al., 2022; Sun et al., 2022; Tseng et al., 2023). These architectures address the challenges posed by data collected from sensors that record diverse physical measurements, such as multispectral or SAR data in remote sensing (Reed et al., 2022; Pan et al., 2023; Yamazaki et al., 2023), as well as in natural images (Themyr et al., 2023). While different spectral, spatial, and temporal resolutions have been considered in previous works, there remains a lack of exploration regarding the resolution gap between datasets captured by aerial and satellite sensors. The integration of multimodal, multitask, and multiscale architectures is expected to significantly enhance the generalization capabilities of models for forest monitoring tasks at global scale (see Section 2.2.4). By training these algorithms with various types of datasets and specializing them for forest monitoring tasks, they could effectively deliver improved performance across different geographical regions such as for species cover estimation or aboveground biomass estimation (see Section 2.1.3).

3.2. Learning from limited data

There are a growing number of massive datasets and algorithms leveraging them, including across remote sensing (Sumbul et al., 2021; Bastani et al., 2023; Rahaman et al., 2022; Mai et al., 2023a). However, many of the most powerful machine learning approaches are supervised and, therefore, require labels, which can be challenging, time-consuming, and costly to obtain. There has been considerable attention given to the problem of learning from limited labeled data; we here present several families of approaches and their relevance to forest monitoring.

3.2.1. Self-supervised learning

Situated in-between supervised and unsupervised learning, the self-supervised learning paradigm involves training a model to reconstruct certain known relationships between or within the datapoints themselves. The resulting model can then be fine-tuned with actual labeled data or directly applied to solve the downstream task. Self-supervised approaches in computer vision include discriminative approaches that distinguish between positive and negative samples while separating their representation (e.g., contrastive learning) (Gidaris et al., 2018; Chen et al., 2020; He et al., 2020; Caron et al., 2021; Oquab et al., 2023) and generative approaches that learn representations through reconstruction (Lehtinen et al., 2018; He et al., 2022). The utilization of self-supervised learning in remote sensing has experienced significant growth due to the abundance of unlabeled open-access data (Tao et al., 2023). For instance, geolocation of satellite images has been exploited with a contrastive approach (Ayush et al., 2021; Mai et al., 2023b). Multispectral and SAR data have been reconstructed based on the temporal information (Cong et al., 2022; Yadav et al., 2022), for multiscale reconstruction (Reed et al., 2022) and for denoising (Dalsasso et al., 2021, 2022; Meraoumia et al., 2023). Emerging cross-modal approaches, encompassing both discriminative (Jain et al., 2022) and generative (Jain et al., 2023b) techniques, are being developed to harness the complementary nature of aligned samples. Self-supervised learning will greatly unleash the potential of remote sensing data in the area of forests (Ge et al., 2023) by learning textural and geometrical structures of forests and trees without labels (see Sections 2.2.1 and 2.2.4).

3.2.2. Weakly supervised learning

Obtaining precise and detailed annotations, for example, for tree crown instance segmentation, can be both costly and time-consuming. Although self-supervised learning aims to learn representations from pretext tasks, it still necessitates precise annotations for fine-tuning the model in a downstream task. In cases where precise annotations are not available, coarse-grained and potentially inaccurate annotations, or even single point locations, can be utilized as weak labels in weakly supervised learning approaches (Zhou, 2018). Given their cost-effectiveness and efficiency, computer vision methods have been developed to leverage weak annotations while addressing their inherent inaccuracies (Zhou, 2018). Weakly supervised learning has been explored in the realm of object location (Oquab et al., 2015), object relationship estimation (Peyre et al., 2017), instance segmentation (Ahn et al., 2019), and contrastive learning (Zheng et al., 2021). Obtaining high-quality annotations for remote sensing data is particularly difficult due to their poor spatial resolution or the physics of the sensors used. Weakly supervised learning has therefore been investigated for Earth observation tasks including object detection (Han et al., 2015; Zhang et al., 2015; Yao et al., 2021), LULC semantic segmentation (Yao et al., 2016; Wang et al., 2020b), and plant traits regression (Schiller et al., 2021; Cherif et al., 2023). Recently, weakly supervised methods have been investigated in the areas of tree classification (Illarionova et al., 2021), tree counting (Amirkolaee et al., 2023), tree detection (Aygunes et al., 2021), and tree segmentation (Gazzea et al., 2022) using multispectral data (see Section 2.2.4).

3.2.3. Active learning

Even highly precise and fine-grained annotations are generally less useful if present in only small quantities. To address this limitation, active learning strategies have been developed to identify and select the optimal way to select a small set of training datapoints to label (Cohn et al., 1996). These strategies for sample selection often rely on estimating the uncertainty of a model (Gal et al., 2017), for instance, using variational approaches (Sinha et al., 2019) or estimated with a loss function (Yoo and Kweon, 2019). They have demonstrated their effectiveness in scenarios where the amount of labeled data is limited, particularly in the context of image classification (Gal et al., 2017; Sinha et al., 2019; Yoo and Kweon, 2019), object detection (Roy et al., 2019), and semantic segmentation (Siddiqui et al., 2020). Active learning has also been investigated for remote sensing applications, including classification (Tuia et al., 2011), object detection (Qu et al., 2020), and LULC semantic segmentation with hyperspectral data (Li et al., 2010; Li et al., 2011; Zhang et al., 2016). Its application would be helpful for forest monitoring to optimize and create relevant human annotations (see Section 2.2.4).

3.2.4. Few-shot learning

Yet another approach to limited data availability is few-shot learning, which refers to efficient fine-tuning of a pretrained model using only a few annotated datapoints. Few-shot learning has been approached from different perspectives, considering the comparison between the small annotated dataset and the data used for pretraining the model—for example, by quantifying the similarities between these datasets (Vinyals et al., 2016), constructing mixtures of feature embeddings (Snell et al., 2017), or adapting the optimization scheme through meta-learning (Finn et al., 2017). Motivated by the limited availability of annotations, applications of few-shot learning in remote sensing tasks have been investigated. For instance, methods based on feature similarity (Alajaji et al., 2020; Zhang et al., 2020; Alosaimi et al., 2023) and metric learning (Liu et al., 2019), aiming at separating representations in an embedding space, have been explored for LULC classification with either multispectral or hyperspectral data. Objects have also been detected by learning meta features (Deng et al., 2022). Metric learning techniques have also been utilized in the context of few-shot learning for semantic segmentation tasks (Jiang et al., 2022) or meta learning with multispectral and SAR data (Rußwurm et al., 2020). Few-shot learning has been explored for tree species classification using feature similarity (Chen et al., 2021) and would be beneficial to recognize a species or estimate the characteristics of a tree with minimal manual annotations (see Sections 2.2.2 and 2.2.4).

3.2.5. Zero-shot learning

The machine learning community has also shown interest in developing methods for training algorithms to differentiate unseen classes without any explicitly annotated samples at all, which is known as zero-shot learning (Xian et al., 2018). In order to categorize unseen classes, the task of zero-shot learning has been accomplished by projecting image and word embeddings (Socher et al., 2013) or known semantic attributes (Lampert et al., 2014) into a shared space. Zero-shot learning has also been investigated by incorporating a mixture of embeddings from the source domain before computing similarities with the target domain, which includes the unseen classes (Zhang and Saligrama, 2015). Generative approaches have been developed to create visual feature embeddings of unseen classes from word embeddings for zero-shot semantic segmentation (Bucher et al., 2019). Zero-shot learning has also garnered attention in remote sensing applications, including combining multispectral data and word embeddings for classification tasks (Li et al., 2017, 2021) and initial exploration of applying zero-shot learning to classify hyperspectral data (Freitas et al., 2022). Generative approaches have also been used with remote sensing data to create visual embeddings from word embeddings (Li et al., 2022b). Zero-shot learning presents a promising approach for forest monitoring, enabling the adaptation of models in regions where previously unseen species are encountered (see Sections 2.2.2 and 2.2.4). By leveraging tree taxonomy hierarchy and meta characteristics to align with visual embeddings (Sumbul et al., 2018), a vast research potential emerges. Notably, utilizing foundation models that have demonstrated strong zero-shot learning capabilities (Brown et al., 2020; Radford et al., 2021) further enhances this potential. In the following paragraph, we will explore methods concerning domain-specific objectives, focusing on the consideration of physical and biological constraints and their applications.

3.3. Domain-specific objectives

Machine learning methods commonly use a fairly limited set of metrics to evaluate success, such as (macro or micro) accuracy of labels and cross-entropy loss for classification tasks, mean squared error or mean average error for regression tasks, and so forth. However, these uniform metrics do not necessarily reflect the realities of real-world use cases (Birhane et al., 2022), where criteria for success may be much more nuanced or domain-specific. In this section, we consider two other families of objectives that may frequently be of relevance in forest monitoring.

3.3.1. Constraints on data

Depending on the domain of application, the outputs of a machine learning pipeline may have specific constraints that must be satisfied if the answer is to be useful or even possible. For example, climate variables may need to obey physical laws such as conservation of energy, engineered systems may need to obey the laws of mechanics, and so forth. Machine learning models to work with such variables have increasingly been designed with soft constraints (Ouaknine et al., 2021a; Harder et al., 2022), which impose penalties for constraint violation, or hard constraints (Donti et al., 2021; Geiss and Hardin, 2021; Harder et al., 2023), where the constraints are strictly enforced by the design of the algorithm. Compared to physics- and engineered-based constraints, fewer authors have to date integrated biological constraints into ML algorithms. Dynamics of biological systems have been included in a deep learning optimization scheme as hard constraints from ordinary differential equations (Yazdani et al., 2020). There are potential opportunities for incorporating biological constraints in forest monitoring by considering phenological (Richardson et al., 2018) or biophysical traits, or ecosystem properties (see Section 2.1.5), for example, by considering the ratio of tree height and canopy size. These constraints could be particularly valuable in tasks such as semantic segmentation or biomass estimation.

Domain-specific constraints on data may also pose opportunities for improving the design of machine learning models. The design of deep learning model architectures can incorporate considerations for, or reconstruction of, physical properties. For instance, a physics-informed architecture has been developed for super-resolution in turbulent flows, incorporating partial differential equations as a form of regularization (Jiang et al., 2020). Similarly, RADAR-based architectures have been created to reconstruct physical properties for scene understanding in the context of autonomous driving (Ouaknine et al., 2021a; Rebut et al., 2022). Leveraging the properties of multiple sensors has also been employed to fuse their representations (Ouaknine, 2022) or to generate annotations from one modality to another (Ouaknine et al., 2021b; Schiefer et al., 2023). In remote sensing, self-supervised learning has benefited from SAR physical properties by considering a pretext denoising task (Dalsasso et al., 2021; Meraoumia et al., 2023), or by separating and reconstructing the real from the imaginary part of the signal (Dalsasso et al., 2022). Such methods could also be explored by exploiting various sensors to learn representations of forests and trees (see Section 2.2.4).

3.3.2. Uncertainty quantification

Biological phenomena adhere to intricate rules that are challenging to estimate and often exhibit inherent uncertainties. The estimation of prediction uncertainty aids in obtaining a better understanding of the strengths and limitations of a machine learning model. The overall uncertainty of these models comprises both aleatoric and epistemic uncertainties (Gal, 2016). They both can be distinguished based on their origins. Aleatoric uncertainty arises from the inherent noise present in the data and label distributions, while epistemic uncertainty is associated with the model itself, encompassing its estimated parameters and structural characteristics. Approaches have been devised to estimate the uncertainties of deep neural networks, for example, by using a Bayesian approach such as Monte Carlo dropout (Gal and Ghahramani, 2016), by using adversarial training combined with model ensembles (Lakshminarayanan et al., 2017), by predicting the uncertainty distribution (Malinin and Gales, 2018), or by learning an auxiliary confidence score from the data (Corbière et al., 2019; Corbière, 2022). Similar methods have been applied to estimate uncertainty in remote sensing data for crop yield estimation (Ma et al., 2021b) or road segmentation (Haas and Rabus, 2021). The quantification of uncertainties in forest monitoring methods has been carried out to assess both aleatoric and epistemic uncertainties (see Section 2.2). This is commonly performed to evaluate the uncertainty of predictions on large-scale maps, utilizing low-resolution satellite data. The uncertainty of plant functional type has been studied for classification in Siberia (Ottlé et al., 2013). Estimating the uncertainty of aboveground biomass has also been conducted to establish a range of estimated values in carbon stock maps (Patterson et al., 2019; Santoro et al., 2021) (see Section 2.1.3). To quantify uncertainty, these methods utilize standard deviations or output probabilities of the model. Recent studies have taken a step further in estimating tree carbon stocks in semi-arid sub-Saharan Africa north of the Equator by combining uncertainty from both allometric equations and predicted crown segmentation, utilizing field measurements (Tucker et al., 2023). There has been limited application of advanced uncertainty quantification methods, whether associated with the data or the predictive model, in the context of forest monitoring.

Despite the extensive application of the presented machine learning techniques in remote sensing, their utilization for forest monitoring has been relatively limited. This presents numerous opportunities to gain deeper insights into the composition of forests while achieving generalization at a large scale. However, it is crucial to have access to high-quality, diverse, and sufficient datasets in order to effectively explore machine learning strategies. In the following section, we will review open-access forest datasets, providing information on their size, tasks, scale, and modalities.

4. Review of open-access forest datasets

Open-access datasets are essential to drive the scientific community in general to exploring forest biology challenges, particularly by using machine learning strategies (see Section 3). Deep learning algorithms have demonstrated strong performance in various forest monitoring tasks, such as tree classification or segmentation (Kattenborn et al., 2021). The availability of open-access datasets has played a significant role in enhancing the algorithm performances and expanding their applications on a larger scale. In this particular field, the use of data, from the tree to the country level (see Figure 2), distributed in the entire globe, must be taken into consideration. Algorithms have been trained for forest monitoring by leveraging datasets that encompass different scales, modalities, and tasks (Guimaraes et al., 2020; Kattenborn et al., 2021; Michalowska and Rapinski, 2021). However, the limited availability of data sources often restricts public access, thereby impeding the progress of extended research projects. While the scientific community emphasizes the importance of reproducible experiments, it is worth noting that some datasets do not fully adhere to the fair principles (https://www.go-fair.org/fair-principles/), which encompass aspects like documentation and findability.

While there is still a considerable quantity of publicly available datasets, it is important to acknowledge that they may have certain limitations that restrict their impact in machine learning applications for forest composition analysis. These limitations can include factors such as the size of the dataset or the specific type of data that is released. This section aims to review forest monitoring datasets considering the following criteria:

1. 1. The dataset should be open-access, that is, without any request requirement.

2. 2. The dataset should be related to at least one published article; exceptions have been made for datasets that are available as preprints, but are considered to be must-see datasets.

3. 3. The dataset should be focused on the composition of the forest, excluding event-based specific ones (i.e., wildfire detection).

4. 4. An LULC dataset should contain more than a single plant functional type (i.e., conifers or deciduous) since a focus is made on better understanding the composition of the forest.

5. 5. The dataset should be at the tree level at least, excluding datasets at the organ or cellular level considered as out of the scope of this review (e.g., leaf spectra or root scans).

6. 6. The dataset should contained at least $ O\left(1,000\right) $ trees.

Based on these criteria, 86 datasets have been identified representing a wide range of geographical locations and spanning from 1974 to 2022. The datasets are associated with publications from 2005 to 2023, as depicted in Figure 3.

Figure 3. Distribution of the reviewed open-access forest datasets. Note: (Left) World map of the location of the reviewed datasets at the country level. Most of the datasets are regional and do not reflect the entire associated country. The datasets categorized with a “Worldwide” location or at the continent level have been excluded for visualization purposes. (Right) Distributions of the publication years and recording years used and/or released in the associated datasets.

The scope of the presented review is broad; it is likely that other datasets meeting these requirements have been missed. Based on this motivation, the study is supported by OpenForest, a dynamic catalog integrating the reviewed datasets and open to updates from the community. (The catalog contains all URLs to access the datasets that are not included in this article to ensure a temporal consistency. OpenForest is available at https://github.com/RolnickLab/OpenForest.) Updates on OpenForest will be restricted with the criteria detailed above. We hope to motivate researchers by grouping our efforts to create the largest database of open-access forest datasets and thus create synergies on forest monitoring applications.

This section will review open-access forest datasets grouped at different scales as presented in Figure 2: inventories (Section 4.1), ground-based recordings (Section 4.2), aerial recordings (Section 4.3), satellite recordings (Section 4.4), and country or world maps (Section 4.5). Datasets composed of mixed scales are finally presented (Section 4.6).

Each section will detail the overall scope of the presented datasets with the specificity of the sensors used to record the data, the information related to each dataset, and their applications. In each section, the reviewed datasets will be categorized in tables, respectively, to the scale of the released data. In these tables, the publication and recording years are differentiated to better understand the temporal scope of the datasets. The recordings years are distinguished with a new line, while time series are represented by an upper dash. Each table will relate the available modalities in the “Data” column. This one is separated with the “Spatial resolution” or “Spatial precision” column (except for inventories) with a dashed line to associate a resolution to the corresponding modality. Each section will also discuss the limits of current open-access datasets to motivate our perspectives presented in Section 5. The following section will review inventory datasets as the smallest scale of recordings that have been taken into account.

4.1. Inventories

Historically, forests have been mostly locally or regionally inventoried based on stratified plot samples acquired in the field (Jucker et al., 2022). Digitized and open-access inventories generally cover small areas, consisting of dozens or a few hundred trees, which limits their impact on the machine learning community (Section 3). As defined in Section 4, this section is focused on medium- to large-scale inventories with at least $ O\left(1,000\right) $ trees. A significant part of reviewed inventory datasets are mixed with modalities at different scales, which will be detailed in Section 4.6.

Inventory datasets are summarized in Table 1; the size of the datasets is quantified by the number of trees. Inventory datasets are composed of various measurements. They commonly contain tree height, canopy diameter, diameter at breast height (DBH), or diameter at soil height (DSH) (Gastauer et al., 2015; Jucker et al., 2022; National Ecological Observatory Network (NEON), 2023; Oliveira et al., 2017;Pérez-Luque et al., 2015; Pérez-Luque et al., 2021). In specific cases, wood density, bark density, and bark thickness are also measured (Schepaschenko et al., 2019; Farias et al., 2020; Kindermann et al., 2022). These information are particularly useful to estimate the tree density, the aboveground biomass (AGB), or the tree carbon stock at large scale (Tucker et al., 2023) even if the inventories have not been released with the estimated maps (Patterson et al., 2019; Dionizio et al., 2020).

Table 1. Review of open-access forest inventories datasets

Note: The dataset size measured in K is $ O\left({10}^3\right) $ . AGB = aboveground biomass; Classif. = classification; DBH = diameter at breast height; DSH = diameter at soil height; N/A = non-applicable; OL = object localization; Reg. = regression; Unknown = non-provided by the authors.

Species, genus, and family of the trees are generally provided. This hierarchy of labels coming alongside with the tree geo-location make inventories a very accurate datasets for understanding forest composition. However, they are geographically sparse and centered in a specific location to reduce measurement efforts (Laar and Akça, 2007; Motz et al., 2010). As an exception, Tallo (Jucker et al., 2022) groups inventories from all around the world, with an unprecedented number of species reported. The latter could have an impact on estimating tree species distribution at large scale.

Considering that inventories contain annotations of trees or tree clusters, they open possibilities to segment tree canopies according to their taxonomic levels, regress continuous metrics (i.e., height and biomass), or even locate tree individuals by predicting their coordinates or crown perimeter (Tucker et al., 2023). Another example could be to estimate the wood density of a tree or its carbon stock using allometric equations with information on taxonomy and height measured on the field (Zianis et al., 2005). Inventories could also be combined with other modalities and used as annotations for larger-scale tasks. As an example, remote sensing datasets presented in the following sections in the same geographic locations could be associated with inventories to enhance the precision of their annotations. While establishing this connection between ground measurements and remote sensing data presents its own set of challenges. In the following section, we will review datasets of ground-based recordings.

4.2. Ground-based recordings

The fine-scaled composition of forests can be understood by visualizing the trees within or under their canopy. Ground-based datasets are composed of recordings inside the forests, under the tree canopy. Trunks and small trees, invisible from a bird’s eye view, can be captured with cameras recording red–green–blue (RGB) images per example. These data are sometimes recorded in time series, for example, PhenoCams (Klosterman et al., 2014; Brown et al., 2016). The use of data recorded by sensors by machine learning algorithms help to have a broader context and more tree information in the samples compared to inventories.

Ground-based datasets are reviewed in Table 2. The dataset size has been measured in hectares (ha) corresponding to the studied surface, in the number of trees in the area, or in the number of samples, which may differ between synthetic and real samples (Grondin et al., 2022).

Table 2. Review of open-access ground-based forest datasets

Note: The dataset size measured in K is $ O\left({10}^3\right) $ . DBH = diameter at breast height; ha = hectares; IMU = inertial measurement unit; IS = instance segmentation; KD = key-point detection; N/A = non-applicable; OD = object detection; PC = point cloud; Reg. = regression; RGB = red–green–blue images; Unknown = non-provided by the authors.

Stereo cameras are parameterized to estimate the depth of a scene differentiating trees and objects from the background in the forest (Grondin et al., 2022). Thermal cameras have also been used to record trees’ signature (Still et al., 2019) and distinguish them from other objects (Reis et al., 2020; da Silva et al., 2021a,b, 2022). In specific cases, camera images have been annotated with bounding boxes around trees to detect them (Tremblay et al., 2020; Grondin et al., 2022). Only two reviewed datasets located in Canada have been annotated with several species classes to combine detection and classification of trees (Tremblay et al., 2020; Grondin et al., 2022). Since these datasets also provide inertial measurement unit (IMU), a potential task could be to predict the next move of an automated agent in a forest.

Forest geometry is also being intensively studied from the ground by using LiDAR—typically referred to as terrestrial laser scanning (TLS). This active sensor records three-dimensional scenes with photon reflections and can be applied from tripods or be combined with IMUs to enable mobile laser scanning. It is not impacted by sun lighting conditions and well suited to understand the structure of forests and trees such as measuring, gap fraction, stand density, tree height, DBH, volume, or biomass (Hackenberg et al., 2015; Liang et al., 2016; Tremblay et al., 2020). The spatial resolution of ground-based LiDAR recordings is either expressed in the average number of points per meter squared, or in the precision of localization of each point, based on information provided by the authors. The generated LiDAR point clouds have been used for instance segmentation (Burt et al., 2018; Tremblay et al., 2020; Grondin et al., 2022), that is, segment each tree independently and associate them an identification number, or key-point detection, that is, localizing points of interest for each tree.

Ground-based datasets are useful to understand the composition of forests under the tree canopy, and recordings were difficult to automatize until recently (Calders et al., 2023). Literature lacks large-scale annotated datasets, although they can provide information at high spatial and temporal resolution and from perspectives that aerial and satellite recordings cannot. Providing both ground-based and aerial-based recordings (Soltani et al., 2022), informing both above and below tree canopy would facilitate transfer and bridging machine learning applications between different modality scales (for details, see Section 5). The next section will review aerial recordings datasets.

4.3. Aerial recordings

Aerial datasets consist of recordings of sensors mounted on unoccupied (drones) or occupied aircrafts flying above the tree canopy, offering a broader perspective of the forest without the hindrance of obstacles impeding the automatic recording process. The diversity in aerial datasets has increased in the past few years since they are used for diverse applications such as vegetation segmentation, disease detection, fire detection, and numerous others (Guimaraes et al., 2020). This is in part also boosted as governmental organizations are increasingly making the imagery of repeated official aerial campaigns openly available (e.g., for entire countries). Furthermore, the decreasing costs of UAVs and the miniaturization of high-quality sensors have served as strong incentives for their adoption within the community.

Aerial-based recordings are reviewed in Table 3. The dataset size is expressed in kilometer squared ( $ {\mathrm{km}}^2 $ ), or in hectares (ha) if the studied area is small. It is also quantified by the number of samples or the number of trees if applicable.

Table 3. Review of open-access aerial forest datasets

Note: The dataset size measured in K is $ O\left({10}^3\right) $ . CHM = canopy height model; Classif. = classification; DBH = diameter at breast height; DSM = digital surface model; DTM = digital terrain model (spatial or vertical); ha = hectares; MC = multi-classification; N/A = non-applicable; OD = object detection; PC = point cloud; Reg. = regression; RGB = red–green–blue images; Seg. = semantic segmentation; Unknown = non-provided by the authors.

Multiple sensors can be carried by UAVs, including RGB and thermal cameras, multispectral sensors, hyperspectral sensors, and LiDAR, which collectively contribute to a captivating array of recorded data, offering diverse perspectives and insights. Cameras mounted on UAVs facilitate the acquisition of overlapping images with a spatial resolution of a few millimeters to centimeters. Such high-resolution image datasets can be applied in concert with photogrammetric workflows, which enable a triangulation of common features found in overlapping images, enabling to precisely reconstruct camera parameters and orientations in hundreds of images automatically. Such workflows enable to reconstruct digital surface models and reprojections of the imagery to generate geocoded image mosaics with orthographic projection (Guimaraes et al., 2020; Diez et al., 2021). Most of the recently publicly released aerial datasets contain RGB images generated by photogrammetry since they are relatively simple and cheap to collect (Morales et al., 2018; Kattenborn et al., 2019a, 2020; Kentsch et al., 2020; Schiefer et al., 2020; Nguyen et al., 2021; Galuszynski et al., 2022; Reiersen et al., 2022). But the original RGB point cloud carrying the height information used to generate the DSM is generally not provided with some exceptions (Brieger et al., 2019; van Geffen et al., 2022). This is unfortunate because there would be opportunities for new multimodal models to leverage both the RGB and point cloud modalities to improve model performance.

An alternative method for studying the topography of both the ground and canopies, depending on the structure of the forest, involves the utilization of airborne LiDAR acquisitions (Ferraz et al., 2018; Kalinicheva et al., 2022). In contrast to terrestrial LiDAR, these measurements commonly have lower point densities, but cover large areas. Airborne LiDAR sensors are operated with IMU sensors, which enables geo-referenced flights transect across large spatial extents. These sensors typically can record multiple returns per LiDAR pulse so that acquisitions can resemble the vertical structure of forest stands, including multiple overlapping tree layers, the understory, and even the ground topography (Kalinicheva et al., 2022). The spatial resolution of airborne LiDAR products is estimated by the average number of points per square meter.

Multispectral and hyperspectral sensors are passive, capturing reflected or emitted photons (Mavrovic et al., 2023) from the sun across wavelength bands that extend beyond the visible spectrum, allowing for comprehensive recording of electromagnetic radiation throughout the near up to the shortwave infrared region. They are especially valuable in assessing the composition of forest canopies, enabling the differentiation of species or retrieving biochemical and structural properties based on the spectral characteristics across spectral bands (Fassnacht et al., 2016; Cherif et al., 2023). A trade-off is usually required between acquiring information with a high spectral and a low spatial resolution (Paz-Kagan et al., 2017), or with a low spectral and a high spatial resolution (Garioud et al., 2022), given that the radiation reflected by plant canopies does not suffice the acquisition at high spectral and high spatial resolution simultaneously.

Forest monitoring can be explored in different ways using aerial datasets relying on the sensors employed and the annotations provided alongside the data. For instance, semantic segmentation is a prevalent method employed to classify forest canopies into tree species (Morales et al., 2018; Kattenborn et al., 2019a, 2020; Kentsch et al., 2020; Schiefer et al., 2020; Galuszynski et al., 2022). Depending on the canopy structural complexity and data quality, the classification might combined with a delineation of individual tree crowns using instance segmentation approaches. Thereby, instance segmentation captures the intricate shapes of tree crowns, unlike object detection, which typically predicts rectangular bounding boxes or centroids for individual objects (Reiersen et al., 2022). Some of the reviewed datasets include a DSM, which can be utilized with tree localization to estimate canopy height. This application using deep learning algorithms and aerial data is an actual active field of research (Yue et al., 2019; Moradi et al., 2022; Reiersen et al., 2022; Wagner et al., 2023).

Due to the high spatial resolution, datasets of aerial recordings enable a granular understanding of forests at the individual tree level. There are still many open challenges which could be explored at the tree level such as segmenting individual tree crowns in dense forests, classifying them between a wide range of species, or adapting algorithms from a forest to another (see Section 2). Nevertheless, the scale of aerial datasets, especially for drones, is constrained by limited battery life and recording capacities, making it challenging to regularly assess and thus monitor large forest areas. Consequently, the next section will explore satellite datasets, which are better suited for capturing a broader scope of forest landscapes at high frequencies.

4.4. Satellite recordings

Satellite imagery has been consistently recorded across the globe for many years, enabling extensive research in the field of temporal remote sensing. This abundance of data has opened up research in machine learning applied to Earth observation, in particular deep learning approaches (Camps-Valls et al., 2021), in the past few years. The datasets generated by diverse satellite missions encompass a wide range of resolutions and employ various sensors, enabling studies of diverse phenomena over both space and time (Swain et al., 2023).

The Landsat missions (https://www.usgs.gov/landsat-missions/landsat-satellite-missions), a collaborative endeavor started in the seventies involving the U.S. Geological Survey, U.S. Department of the Interior, National Aeronautics and Space Administration (NASA), and the U.S. Department of Agriculture, represent the earliest and pioneering attempt to utilize multispectral cameras for Earth observation (Wulder et al., 2022). Landsat missions 4 and 5 capture images with between four and seven spectral bands, offering spatial resolutions ranging from 30 to 120 meters. The more recent Landsat missions, namely Landsat 7 and Landsat 8, record images with eight and nine spectral bands, respectively. These missions provide spatial resolutions ranging from 15 to 60 meters for Landsat 7 and 15 to 30 meters for Landsat 8. All of the Landsat missions have a 16-day repeat cycle. Most of the reviewed datasets used 30-meter-resolution spectral bands to ensure a consistency between the bands used for their final application (Potapov et al., 2019, 2022; Robinson et al., 2019; Irvin et al., 2020; De Almeida Pereira et al., 2021; Feng et al., 2022; Lee and Choi, 2022).

The Sentinel missions (https://sentinel.esa.int/web/sentinel/missions), managed by the European Space Agency, have been designed to comprehensively monitor the Earth’s various domains, encompassing air, land, ocean, and atmospheric measurements. These missions employ multiple sensors, enabling a wide range of Earth observation capabilities. Sentinel-1 includes a SAR generating electromagnetic waves with wavelengths not impacted by clouds. The reviewed datasets provide or use Level-1 Ground Range Detected (GRD) products at a $ 10\times 10 $ meters resolution (Schmitt et al., 2019; Sumbul et al., 2021; Lee and Choi, 2022). Since two satellites (Sentinel-1A and Sentinel-1B) are recording data on the same orbit, the mission has a 6-day exact repeat cycle with less than a day of revisit frequency at high latitudes.

Sentinel-2 utilizes multispectral sensors to scan photon reflectance across multiple spectral bands. The spatial resolution of the recorded data depends on the spectral bands: four bands at 10 meters, six bands at 20 meters, and three bands at 60 meters. Released datasets kept either 10-m-resolution bands (Schmitt et al., 2019; Bastani et al., 2023; Lee and Choi, 2022) or all the bands (Sumbul et al., 2021). The revisit frequency of the combined constellation of Sentinel-2A and B is 5 days on most of the globe.

Data from the Landsat and Sentinel missions are the most commonly provided in the reviewed datasets, but other interesting satellite sources are also explored. For instance, the Moderate Resolution Imaging Spectroradiometer (MODIS) (https://modis.gsfc.nasa.gov) instrument, introduced by NASA and integrated into the Terra and Aqua missions, generates data that are also used for large-scale forest monitoring purposes. The MODIS instrument offers recordings from 36 spectral bands, each defined for diverse observations, including atmospheric gases, ocean components, land boundaries, and properties (Schmitt et al., 2019; Levin et al., 2021). Another example is the Visible Infrared Imaging Radiometer Suite (VIIRS) instrument (https://www.earthdata.nasa.gov/learn/find-data/near-real-time/viirs), part of the NOAA-20 missions, which also have generated data contained in a forest dataset for land and atmospheric observations (Levin et al., 2021). It should be noted that researchers have used recordings from PlanetLabs (https://www.planet.com/), PlanetScope (https://earth.esa.int/eogateway/missions/planetscope), or Maxar (https://www.maxar.com/) missions (e.g., GeoEye, WorldView, or QuickBird), which provide multispectral data with submeter spatial resolution (Brandt et al., 2020). However, these datasets are not publicly accessible due to the associated licensing restrictions.

The datasets that have been reviewed encompass satellite data obtained from various missions and products, originating from different locations, and exhibiting diverse spatial and temporal resolutions. The details of datasets published before 2020 and included, or after 2020, are provided, respectively, in Tables 4 and 5. The dataset size is expressed in kilometer squared ( $ {\mathrm{km}}^2 $ ), or in hectares (ha) if the studied area is small. It is also quantified by the number of samples, trees, or events if applicable.

Table 4. Review of open-access satellite forest datasets before 2020 (included)

Note: The dataset size measured in K is $ O\left({10}^3\right) $ and in M is $ O\left({10}^6\right) $ . CHM = canopy height model; Classif. = classification; ha = hectares; LMFC = live fuel moisture content; LULC = land use and/or land cover; N/A = non-applicable; Reg. = regression; SAR = synthetic-aperture RADAR; Seg. = semantic segmentation.

Table 5. Review of satellite recording datasets after 2021 (included)

Note: The dataset size measured in K is $ O\left({10}^3\right) $ and in M is $ O\left({10}^6\right) $ . CD = change detection; Classif. = classification; ha = hectares; LULC = land use and/or land cover; MC = multi-classification; N/A = non-applicable; NDVI = normalized difference vegetation index; Reg. = regression; SAR = synthetic-aperture RADAR; Seg. = semantic segmentation; Unknown = non-provided by the authors.

^a The list of countries is detailed in the OpenForest catalog.

Satellite datasets are frequently used for classification, multiclassification, or segmentation of satellite tiles, including LULC and tree species distribution. Other tasks include regression applications for forest cover estimation (Bastani et al., 2023; Feng et al., 2022), canopy height (Forkuor et al., 2020; Lang et al., 2022a), or live fuel moisture content estimation (Rao et al., 2020). An intriguing application involves utilizing satellite time series data to evaluate change detection of forest covers at a large scale (Wang et al., 2020a). This approach enables the estimation of deforestation, afforestation, and reforestation activities (Potapov et al., 2022).

Satellite recordings play a crucial role in Earth observation on a large scale, as they are manually or automatically processed to estimate global maps of forest cover, among other applications. Additionally, world maps depicting aboveground biomass, land use, and land cover have been estimated and made publicly available. In the following section, datasets containing maps at the country or global level will be reviewed.

4.5. Country or world maps

Earth observation applications have been resumed into maps at the country, continent, or global level. They are estimated using machine learning algorithms that incorporate manual expert features derived from satellite data such as statistics of the data distribution or vegetation indexes. These features encompass various aspects, ranging from multispectral information (Friedl et al., 2010; Pflugmacher et al., 2019) to climatic and elevation data (Chaves et al., 2020). The majority of the released maps have been estimated using machine learning algorithms trained on satellite data, as these algorithms demonstrate excellent scalability for predicting at a large scale and low resolution. The results obtained from these algorithms have been validated using field inventories. However, it is important to note that the field inventories themselves have not been included in the reviewed datasets discussed in this section. (Open-access datasets releasing both maps and inventories are reviewed in Table 8 and in Section 4.6.) Nonetheless, the reviewed map datasets are notable for their global coverage, which adds to their significance. Despite containing inherent uncertainties in their estimations, these maps have the potential to offer valuable meta-knowledge to downstream applications in the realm of forest monitoring.

Large-scale map datasets before 2019 (included) are reviewed in Table 6, while maps datasets released after 2020 (included) are reviewed in Table 7. The dataset size is expressed in kilometer squared ( $ {\mathrm{km}}^2 $ ), or in hectares (ha) if the studied area is small. It is also quantified by the number of samples or points if applicable.

Table 6. Review of open-access map forest datasets before 2019 (included)

Note: The dataset size measured in K is $ O\left({10}^3\right) $ and in M is $ O\left({10}^6\right) $ . AGB = aboveground biomass; Classif. = classification; IFL = intact forest landscape; LULC = land use and/or land cover; N/A = non-applicable; PFT = plant functional type; Reg. = regression; Seg. = semantic segmentation; Unknown = non-provided by the authors.

^a The list of recording years is detailed in the OpenForest catalog.

^b The list of countries is detailed in the OpenForest catalog.

Table 7. Review of open-access map forest datasets after 2020 (included)

Note: The dataset size measured in K is $ O\left({10}^3\right) $ and in M is $ O\left({10}^6\right) $ . AGB = aboveground biomass; BGB = belowground biomass; CD = change detection; CH = canopy height; Classif. = classification; GSV = growing stock volume; LULC = land use and/or land cover; MC = multi-classification; N/A = non-applicable; Reg. = regression; SCS = soil carbon stock; Seg. = semantic segmentation; Unknown = non-provided by the authors.

^a The list of recording years is detailed in the OpenForest catalog.

A significant portion of map datasets focuses on providing information about LULC (see Section 4 for the proposed definition), which plays a crucial role in distinguishing different types of forests at a large scale (Bartholomé and Belward, 2005; Friedl et al., 2010; Griffiths et al., 2014; Pflugmacher et al., 2019; Thonfeld et al., 2020; Bonannella et al., 2022). Within in LULC maps, the reviewed works have also estimated the extend of forest cover, including time series data. These time series are particularly valuable for quantifying forest loss, that is, deforestation detection, as well as forest gain, that is, afforestation, reforestation monitoring, or deadwood maps (Hansen et al., 2013; Curtis et al., 2018; Bunting et al., 2022; Verhegghen et al., 2022; Schiefer et al., 2023). Another category of maps is specifically designed to differentiate between different plant functional types, particularly distinguishing between broad-leaved and needle-leaf forests, as well as identifying summer-green and evergreen forests across tropical, boreal, and temperate regions (Ottlé et al., 2013).

As mentioned in Section 2.2, accurate estimation of aboveground biomass is crucial for a comprehensive quantification of the carbon stocks that forests worldwide hold. World maps of aboveground biomass have been estimated at different resolutions (Patterson et al., 2019; Santoro et al., 2021; Tang et al., 2021; Ma et al., 2021a). Quantifying the uncertainty of aboveground biomass maps is also important as it depends on multiple factors, including tree species, tree height, canopy size, and reference data distribution (Patterson et al., 2019; Ploton et al., 2020; Santoro et al., 2021). Canopy height maps have also been quantified in sparse boreal forests (Bartsch et al., 2016, 2020). These canopy height maps, both at the country (Tolan et al., 2023) and world levels (Lang et al., 2022a,b), have been estimated using LiDAR sensors as a ground truth. Accurate estimation of canopy height is crucial for evaluating aboveground biomass, which is why a few studies have utilized data from the Global Ecosystem Dynamics Investigation (GEDI) mission (https://gedi.umd.edu/) (Patterson et al., 2019; Tang et al., 2021; Ma et al., 2021a; Lang et al., 2022b; Tolan et al., 2023). GEDI records LiDAR data from the International Space Station, allowing for the estimation of a DSM that serves as a valuable reference for canopy height estimation. The overall biomass estimation of forests also includes belowground biomass (Chen et al., 2023) and soil carbon stock (Dionizio et al., 2020), which have been quantified using SAR satellite data penetrating dense canopies.

Although open-access map datasets are subject to the limitations and uncertainties inherent in the estimation methods employed by the authors, they remain a valuable source of data for obtaining a broad-scale understanding of forests or integrating meta-knowledge into future analyses. These datasets could be helpful for conducting further research and expanding our knowledge of forest ecosystems.

In the preceding sections, the reviewed datasets were presented with a focus on different scales. However, in the forthcoming section, datasets that offer a combination of data at various scales will be discussed in detail.

4.6. Datasets mixed at different scales

Datasets that offer data at various scales play an important role in establishing a bridge between different modalities recorded by diverse sensors. By integrating information from multiple sources, these datasets facilitate a comprehensive understanding of forests and enable cross-modal analysis. Inventories, ground-based, and aerial-based datasets are available at small scale but usually come alongside precise annotations at all tree levels. Conversely, satellite and map datasets are available at a larger scale but often lack precise annotations due to their lower resolution. Integrating data from different scales can be advantageous in generalizing and extrapolating local knowledge to a larger scale, bridging the gap between detailed annotations and broader coverage (Kattenborn et al., 2019b; Schiefer et al., 2023).

The size of each dataset is expressed in kilometer squared ( $ {\mathrm{km}}^2 $ ), or in hectares (ha) if the studied area is small. It is also quantified by the number of samples, points, or trees if applicable.

Mixed datasets composed of inventories and aerial-based recordings (IA); inventories, aerial-based, and satellite-based recordings (IAS); and inventories and maps (IM) are reviewed in Table 8. Inventories provide an additional value to imagery recordings by providing geo-located annotations, depending on the level of precision they offer. These inventories enhance the spatial context and accuracy of the annotations. Combining inventories with aerial-based recordings would be highly beneficial for accurately aligning tree measurements with aerial data, especially with LiDAR (Weiser et al., 2022) or RGB (Brieger et al., 2019; van Geffen et al., 2022) recordings. This integration enables improved estimation of carbon stocks at the aerial scale, among other applications. Field measurements have also been used to validate country or world maps; they are often released together to enable reproducibility of the results. This integration of field measurements and map data enhances the accuracy and reliability of the generated maps. Similarly to datasets presented in Section 4.5, maps of forest age (Besnard et al., 2021), carbon stocks (Tucker et al., 2023), and LULC (Koskinen et al., 2019; Bendini et al., 2020; Shevtsova et al., 2020; European Commission. Statistical Office of the European Union, 2021) have been estimated by machine learning algorithms while being calibrated and validated with inventories.

Table 8. Review of open-access mixed forest datasets, including inventories and aerial-based (IA); inventories, aerial-based, and satellite-based (IAS); and inventories and maps (IM)

Note: The dataset size measured in K is $ O\left({10}^3\right) $ , in M is $ O\left({10}^6\right) $ , and in B is $ O\left({10}^9\right) $ . AGB = aboveground biomass; CBH = crown base height; Classif. = classification; DBH = diameter at breast height; EVI = enhanced vegetation index; IA = inventories and aerial; IAS = inventories, aerial and satellite; IM = inventories and maps; IS = instance segmentation; LULC = land use and/or land cover; MC = multi-classification; N/A = non-applicable; OL = object localization; PC = point cloud; Reg. = regression; RGB = red–green–blue; Seg. = semantic segmentation; Unknown = non-provided by the authors.

^a The dataset includes three LiDAR with different resolutions, which are ALS: 72.5 pts-m²; ULS: 1,029.2 pts-m²; and TLS: Unknown.

^b The list of countries is detailed in the OpenForest catalog.

^c Aerial recordings (3-cm resolution) are aerial RGB, SfM PC, RGB PC, RGN images, DEM, CHM, DSM, and DTM.

^d The list of recording years is detailed in the OpenForest catalog.

^e Field measurements (50-cm resolution) are location, crown area, wood mass, mass, root dry mass, count density, coverage density, and area density.

Mixed datasets composed of ground-based and aerial-based recordings (GA); aerial-based and satellite-based recordings (AS); aerial-based recordings and maps (AM); and satellite-based recordings and maps (SM) are reviewed in Table 9. Aligning ground-based and aerial-based imagery recordings is valuable in integrating information from both above and below the canopies of a forest. For instance, models can be trained using ground recordings sourced from citizen science-based photographs, and then effectively transferred to aerial data (Soltani et al., 2022). This alignment could enable a comprehensive understanding of the forest ecosystem by bridging the gap between ground-level and aerial-level observations.

Table 9. Review of open-access mixed forest datasets, including ground-based and aerial-based (GA); aerial-based and satellite-based (AS); aerial-based and maps (AM); and satellite-based and maps (SM)

Note: The dataset size measured in K is $ O\left({10}^3\right) $ and in M is $ O\left({10}^6\right) $ . AGB = aboveground biomass; Align. = alignment; AM = aerial and maps; AS = aerial and satellite; CHM = canopy height model; Classif. = classification; GA = ground and aerial; LULC = land use and/or land cover; MC = multi-classification; N/A = non-applicable; NDVI = normalized difference vegetation index; OD = object detection; PC = point cloud; Reg. = regression; RGB = red–green–blue; SAR = synthetic-aperture RADAR; Seg. = semantic segmentation; SM = satellite and maps; Unknown = non-provided by the authors.

Mapping aerial-based and satellite-based recordings is helpful for extrapolating high-resolution information at a small scale to a lower resolution at a larger scale. This process allows for the transfer of detailed information captured through aerial LiDAR, for example, to validate canopy height models derived from satellite imagery (Marconi et al., 2019; Weinstein et al., 2021b; Lang et al., 2022a). The integration of these datasets facilitates a more comprehensive and accurate representation of forest characteristics across different spatial scales. Integrating SAR and multispectral satellite data with aerial imagery can potentially enhance model performances (Schmitt and Zhu, 2016), particularly by leveraging the varying reflection and absorption characteristics of different tree species (Ahlswede et al., 2022). Aerial LiDAR metrics have also been used as validation points to estimate the aboveground biomass at large scale (Hudak et al., 2020). At a larger scale, satellite data analyzed with multiclassification algorithms have also been useful to monitor and detect forest loss (Turubanova et al., 2018).

Open-access datasets featuring modalities at different scales have been made available to enable result reproducibility and promote heterogeneity in the way of observing forests. These datasets incorporate various modalities aligned at different scales, which could aim to enhance the generalization capabilities of machine learning algorithms at a larger scale. This not only facilitates research in solving tasks at different scales depending on the modality but also fosters a comprehensive understanding of forests through multimodal analysis. To date, the publications related to the reviewed datasets have not extensively explored multimodal (e.g., point clouds with raster data and point observations with spatially continuous data), multiscale, and multitask approaches. However, it is our hope that the machine learning and computer vision communities will venture into forest monitoring along this path, as it holds great potential for advancing our understanding of the composition of forests worldwide. By embracing these comprehensive approaches, we can enhance our comprehension of forests and contribute to more effective and efficient forest management strategies. The upcoming section will explore perspectives on forest datasets, shedding light on the potential challenges that researchers could prioritize and address in their work.

5. Perspectives

The enthusiasm for forest monitoring is on the rise, serving as a safeguard to protect forests and their ecological and societal significance. Proper monitoring is essential for avoided forest conversion, supporting forest management initiatives, and ensuring successful reforestation and afforestation projects by enhancing survival rates and preventing diseases (van Lierop et al., 2015; Martin et al., 2021). Additionally, the effects of climate change on forest dynamics (Fassnacht et al., 2023) imply a growing need for heightened surveillance of these ecosystems.

As a data-driven and empirical science, respectively, forest monitoring benefits from open-access, diverse, and large datasets, coupled with advancements in machine learning research (De Lima et al., 2022). This endeavor seeks to address existing challenges and research strategies while extensively reviewing open-access forest datasets, with the ultimate goal of encouraging the research community to further investigate this field.

As evident from Section 2.2, forest monitoring remains an active area of research. Numerous ongoing inquiries delve into various aspects, such as tree species identification, phenology, abiotic factors, exogenous influences, and many more. Machine learning already greatly advanced our capabilities to monitor forests through novel analytical tools and capacities. This involves sensing past, current, and dynamic forest states through predictive modeling. Such models and information, build to be explainable by design, will greatly advance our understanding of forests, including insights into how diverse environmental and anthropogenic drivers impact forest dynamics, as well as the operational mechanisms of forest ecosystems. A related and cardinal interest lies in the projection of future forest dynamics to guide decision-makers, to improve management and anticipate consequences (Requena-Mesa et al., 2018). In this context, it is important to consider that ongoing and accelerated changes induced by global warming and climate change reshape the dynamics of the Earth system and its respective data (making data through time nonstationary). Therefore, it becomes essential to explore solutions that streamline the adaptability and transferability of data-driven machine learning methods, ensuring their efficiency in extrapolating from existing and historical data to future circumstances.

Machine learning and computer vision are exerting a progressively increasing influence across various domains, including forest monitoring, as elaborated in Section 3. Strategies related to model generalization, learning schemes, and forestry-based metrics are valuable for delving further into the challenges presented by forest biology.

Enhancing the generalization capabilities of models involves better adaptation to diverse spatial and temporal domains, encompassing different forests, sensors, and resolutions. To achieve this, machine learning strategies will be explored, focusing on leveraging existing datasets through weakly supervised (see Section 3.2.2) or few-shot (see Sections 3.2.4 and 3.2.5) learning approaches. Moreover, hybrid models, which integrate physical knowledge (see Section 3.3), or space-for-time substitutions, which enable to learn temporal dynamics from spatial dynamics, may greatly advance our capabilities to design robust data-driven machine learning applications for monitoring and forecasting in a nonstationary world. In line with this perspective, the OpenForest dynamic catalog could serve as a suitable reference to consistently enhance and refine models using the latest data.

Implementing active learning methods (see Section 3.2.3) can significantly optimize the process of generating annotations for future datasets. As datasets continue to grow in size, self-supervised learning methods (see Section 3.2.1) offer a valuable perspective to learn meaningful representations in deep learning algorithms for forest monitoring without relying heavily on manual annotations.

Incorporating multimodal and multitask computer vision architectures into forest monitoring presents an intriguing opportunity to capitalize on task complementarity. For instance, by predicting multiple foliage traits from hyperspectral data, a model can learn the covariance among different traits and, hence, provide more robust estimates for challenging traits based on their relation with more accurately predicted ones (Schiller et al., 2021; Cherif et al., 2023). An additional area of potential research could involve enhancing carbon stock estimation by simultaneously predicting both tree species and height.

Foundation models (see Section 3.1.2) have demonstrated remarkable capabilities in managing various modalities, such as LiDAR, RADAR, and hyperspectral data, with varying spatial and temporal resolutions. These models remain task-agnostic and can achieve high zero-shot performances, making their pretraining a challenging yet promising endeavor for forest monitoring. Once pretrained, they can be adapted to multiple other tasks in this domain. While relying on the complementarity of large-scale multimodal and multitask datasets, research on foundation models for forest monitoring worldwide would benefit from the OpenForest catalog dynamically enriched by the community.

Section 4 provides a comprehensive review of open-source forest datasets, categorized according to specific criteria and identified scales. These datasets are grouped in OpenForest, a dynamic catalog open for updates from the community. The aim is to foster communication, inspire new applications of machine learning in forest monitoring, and motivate advancements in this field.

Datasets, as prerequisite of machine learning applications, commonly exhibit a lack of geographical representativeness, particularly noticeable in African and Asian regions, as depicted in Figure 3. Whenever feasible, ideal datasets would preferably align multimodal recordings based on their temporal and spatial resolutions while offering annotations in the highest available resolution. As demonstrated in remote sensing (Lacoste et al., 2023), aggregating numerous diverse forest datasets could foster research in developing specialized foundation models for effective forest monitoring.

The OpenForest catalog, in addition to providing a list of open-access datasets, will also curate information about data providers (see related information at https://github.com/RolnickLab/OpenForest), a crucial resource for generating well-structured datasets that cater to specific needs. Citizen-generated data, such as curated on OpenAerialMap (https://openaerialmap.org/) or GBIF (https://www.gbif.org/), hold significant values since they integrate information from across the globe, making them ideal for self-supervised learning. The data provider list will also be frequently updated to integrate most recent initiatives. For instance, incorporating data from the Biomass mission (https://www.esa.int/Applications/Observing_the_Earth/FutureEO/Biomass) as soon as possible into future datasets is essential. The mission’s provision of P-band SAR data will greatly benefit worldwide forest tomography (Berenger et al., 2023), advancing our comprehension of forest carbon stock and its dynamics. Exploring its potential can lead to the creation of valuable and structured datasets.

Recordings from aerial data, especially UAVs, gain momentum by offering promising prospects, with more affordable and easier-to-pilot vehicles equipped with higher-resolution sensors. Leveraging UAV technology allows for high-resolution forest analysis, even in remote or inaccessible areas, and can moreover advance large-scale assessments by its integration with Earth observation satellite missions (Schiefer et al., 2023).

As more and more datasets are released at various scales, the OpenForest catalog offers the opportunity to centralize this information with details. It will help to motivate research in bridging the gap between scales, sensors, and resolutions while hopefully motivating collaborations between researchers.