Conference Agenda

Session
318R: Uncertainty assessment of land system science products
Time:
Friday, 26/Apr/2019:
3:00pm - 4:15pm

Session Chair: Robert Gilmore Pontius Jr
Session Chair: Letícia Santos Lima
Location: UniS-A 022
UniS Building, room A 022, ground floor, 72 seats
Session Topics:
How do we support transformation?

Session Abstract

Land System Science products have gained increasing importance in addressing global environmental issues. In particular, spatially explicit models and maps have a central role in supporting science and policy debates. However, the development and use of models and maps are permeated with uncertainties, most prominently: data uncertainty, inter- and extrapolation uncertainty, model parameter uncertainty, model structural uncertainty. Due to the relevance of models and maps when it comes to debates and decisions regarding land use and its impacts, it is of strategic importance for the scientific community to engage in more open and deeper dialogues about the sources, magnitudes and impacts of uncertainty in such products. These dialogues should address not only advances in uncertainty analysis techniques but also issues of transparency and liability.

To move the uncertainty dialogue in Land System Science forward, we will hold a research presentation session focusing on uncertainty analysis of spatially explicit products and exploring the following questions: (1) What is the state-of-the-art in uncertainty assessment related to spatially explicit products? (2) What are typical uncertainty magnitudes of those products? (3) Are there best practices that should be promoted to increase the transparency of Land System Science? (4) How to develop a strategic communication of scientific uncertainty to peers and to non-academic audiences?

Session Organizers: Letícia Santos Lima, Tobias Krueger, Ana María Sanchez Cuervo, and Reinhard Prestele


Presentations
Full talk
ID: 379 / 318R: 1
318R Uncertainty assessment of land system science products
Keywords: Land-use allocation, uncertainty, stochastic disturbance, Monte Carlo simulation, cellular automata

A Time Monte Carlo method to consider uncertainty in land change models

Ahmed Mustafa, Ismaïl Saadi, Mario Cools, Jacques Teller

Liege University, Belgium

One of the main objectives of land change models is to explore future land-use patterns. Therefore, the issue of addressing uncertainty in land-use forecasting has received an increasing attention in recent years. Many current models consider uncertainty by including a randomness component in their structure. In this study, we present a novel approach for tuning uncertainty over time, which we refer to as the Time Monte Carlo (TMC) method. The TMC uses a specific range of randomness to allocate new land uses. This range is associated with the transition probabilities from one land use to another. The range of randomness is increased over time so that the degree of uncertainty increases over time. We compare the TMC to the randomness components used in previous models, through a coupled logistic regression-cellular automata model applied for Wallonia (Belgium) as a case study. Our analysis reveals that the TMC method produces results comparable with the existing methods over the short-term validation period (2000–2010). Furthermore, the proposed method is capable of tuning uncertainty on longer-term horizons. Controlling the degree of randomness over time is an important feature of the TMC method as the distant future is characterized by more uncertainty than the near future.



Full talk
ID: 614 / 318R: 2
318R Uncertainty assessment of land system science products
Keywords: land-use change modelling, remote sensing, uncertainty assessment

Assessing uncertainties in global land-use modelling

Ruediger Schaldach, Jan Schuengel, Benjamin Stuch

University of Kassel, Germany

Land-use and land-cover change are major drivers of climate and environmental change. For investigating these processes on the global scale, different spatially explicit simulation models have been developed in recent years. These models integrate socio-economic and environmental processes to reproduce observed changes and to assess future development pathways in form of scenarios. Thereby, they can help to understand complex interactions of spatiotemporal processes (e.g. expansion vs. intensification of agriculture) and can provide valuable information for policy and decision makers (e.g. nature conservation). Nevertheless, the outcomes of such modelling exercises are influenced by uncertainties of input data, model parameters and model structure that need to be taken into account when interpreting the generated information.

An important factor of uncertainty is the remote sensing product determining the land-cover distribution at the starting point of a simulation. Available products show large discrepancies due to different satellite sensors, processing methods and classification systems. Our study investigates the influence of the land-cover product used for model initialization on the estimation of model parameter values and the simulated spatial extent and location of land-use change. In addition, we propose a mechanism to assess and communicate these uncertainties in form of an ensemble analysis. The study is conducted with the spatially explicit land-system model LandSHIFT that operates on a global 5 arc-min raster. The model is initialized with the global land-cover datasets CCI, MODIS and GlobeLand30. Based on this input data, the model parameters used for evaluating cell suitability for different land-use types were estimated with two different approaches. In the following, the six resulting model configurations (= ensemble members) were applied to calculate land-use change between 2000 and 2010. Different methods are discussed to quantify and visualize the variability of simulation results between the ensemble members in order to communicate the inherent uncertainties to model users.



Flash talk
ID: 342 / 318R: 3
318R Uncertainty assessment of land system science products
Keywords: land cover, accuracy assessment, confusion matrix, sampling design, remote sensing

Trends in land cover mapping accuracy assessment approaches

Lucia Morales-Barquero1, Mitchell B. Lyons1,2, Stuart Phinn1, Chris Roelfsema1

1Remote Sensing Research Centre, School of Earth and Environmental Sciences, The University of Queensland, Australia; 2Centre for Ecosystem Science, School of Biological, Earth and Environmental Sciences, University of New South Wales, Australia

The utility of land cover maps for natural resources management relies on knowing the associated map uncertainty. Despite being a well-established practice, assessing land cover map accuracy is not without problems. Recent advances in remote sensing, including the increasing availability of higher spatial and temporal resolution satellite data, and data analysis capabilities create opportunities and challenges for improving the application of accuracy assessment. Consequently, revisiting how map error and accuracy levels has been carried out and reported in the last two decades is timely, to highlight areas where there is scope for taking advantage of these emerging opportunities. We conducted a quantitative literature review on accuracy assessment practices of land cover maps produced through classification of remote sensing images. We performed a structured search for land cover mapping that reported accuracy assessments, limiting our search to journals within the remote sensing field and papers published between 1998-2017. After an initial screening process, we selected a random sample of 225 papers, and extracted and standardized information on various components of their accuracy assessments. We discovered that only 63 % of the papers explicitly included an error matrix, and a very limited number (21%) reported overall accuracy with confidence intervals. The use of kappa continues to be standard practice, being reported in 54% of the literature published on or after 2012. The sampling scheme used to obtain the reference data could not be determined for approximately 20% of the studies; from those that reported the sampling design, 86% applied a probability sampling. No trend was found between classification complexity (i.e. number of classes) and measured accuracy, independent from the size of the study area. Our findings indicate that considerable work remains to identify and adopt more robust accuracy assessment practices to achieve fully transparent and comparable land cover maps.



Flash talk
ID: 442 / 318R: 4
318R Uncertainty assessment of land system science products
Keywords: FRA, sequestration, tropics, biomass, uncertainties

It's a numbers game? Uncertainties in tropical forest carbon stocks driven by differing global assessments

Manan Bhan, Karlheinz Erb, Simone Gingrich

Institute of Social Ecology, University of Natural Resources and Life Sciences (BOKU), Schottenfeldgasse 29, 1070 Vienna, Austria

Tropical forest carbon stocks are a key determinant of forest change and its assessment is acknowledged as being central to evolving strategies for climate change mitigation. Yet, estimates of the size and distribution of tropical forest carbon stocks differ widely due to multiplicity of approaches and inadequate data. The 5-yearly FAO Forest Resource Assessments (FRAs) show a consistent decline of tropical forest carbon stocks, but its estimates are in disagreement with other published studies. However, the extent of divergence at pan-tropical and continental levels remains unquantified. While differences in spatial patterns have been analysed in existing studies, the uncertainty related to carbon stocks at landscape and higher spatial levels, and its relevance, is not well-explored. Here, I quantify the disparities for corresponding years between FAO trends and other studies focussed on tropical forest carbon stocks to further the understanding of uncertainties among these assessments. Extrapolating FAO assessments over all years between 1990 and 2015, I find that there are marked contrasts between carbon stocks estimates at pan-tropical and continental levels. The mapping of absolute and relative differences at the pixel level reveal pronounced regional variations. Further, an analysis of the provenance of above-ground biomass displays inconsistencies. However, a spatially-explicit judgement over the viability of particular estimates remains elusive and no single approach persists as more precise than others. My results provide insights on the disparities among tropical assessments of forest carbon stocks, thereby furthering the study of forest change in the tropics. This analysis is another effort towards reconciling differences in carbon stocks measurement approaches and their impacts on on-ground estimates, which is an important step towards establishing more robust estimates of global ecosystem carbon stocks. This assumes added significance in light of the stock-taking exercises envisaged as part of the Paris Agreement, the implementation of mitigation policies like REDD+ as well as emerging, novel remote sensing products.



Flash talk
ID: 345 / 318R: 5
318R Uncertainty assessment of land system science products
Keywords: validation, LUCC models, futures

Uncertainties in LUCC modelling: a contributive review and implications for validation techniques

Thomas HOUET

CNRS, France

When using LUCC models for projecting future possible land changes, four type of uncertainties can be distinguished: (1) the data uncertainty (map accuracy), (2) the inherent model uncertainty, (3) ensemble multi-model uncertainty and (4) the future ensemble uncertainty.

This contribution propose a definition of these types of uncertainties, illustrated with examples from the literature or from various models, and the related implications for validating models outcomes.

The data uncertainty relies for instance to LULC maps produced thanks to inter/extrapolation climate data or remotely sensed data which accuracy is usually assessed using Overall Accuracy or Kappa of Agreement Indices.

The inherent model uncertainty highlights the variability of locations or quantity of futures LUCC changes due to the model functioning or data representation (continuous or discrete for instance).

The multi-model ensemble uncertainty illustrate the previous uncertainty but inheriting from the use of various models with a unique dataset (or scenario).

The future ensemble uncertainty describes the wide range of future LUCC changes. In others words, it encompasses all of the previous uncertainties by considering a set of various prospective scenarios. It means that it considers only consistent and plausible combination of scenarios’ hypothesis (and their related datasets), which is more limited compared to assessing the combination of all possible assumptions (assimilated here to sensitivity analysis).

Validation techniques that can be used for assessing the three latter uncertainty are more sensitive as validating the future is impossible. Validation techniques and scenarios rely to two contrary paradigms: ‘prediction’ vs. ‘multiple possible future’. But, when replaced in the latter paradigm, combined validation techniques can be useful to improve the confidence users can have in model outcomes.



Full talk
ID: 238 / 318R: 6
307R Large-scale behavioural models of land use change
Keywords: carbon, deforestation, model, simulation, REDD

Criteria to confirm models that simulate deforestation and carbon disturbance

Robert Gilmore Pontius Jr

Clark University, United States of America

The Verified Carbon Standard (VCS) recommends the Figure of Merit (FOM) as a possible metric to confirm models that simulate deforestation baselines for Reducing Emissions from Deforestation and forest Degradation (REDD). The FOM ranges from 0% to 100%, where larger FOMs indicate more-accurate simulations. VCS requires that simulation models achieve a FOM greater than or equal to the percentage deforestation during the calibration period. This article analyses FOM’s mathematical properties and illustrates FOM’s empirical behavior by comparing various models that simulate deforestation and the resulting carbon disturbance in Bolivia during 2010–2014. The Total Operating Characteristic frames FOM’s mathematical properties as a function of the quantity and allocation of simulated deforestation. A leaf graph shows how deforestation’s quantity can be more influential than its allocation when simulating carbon disturbance. Results expose how current versions of the VCS methodologies could conceivably permit models that are less accurate than a random allocation of deforestation, while simultaneously prohibit models that are accurate concerning carbon disturbance. Conclusions give specific recommendations to improve the next version of the VCS methodology concerning three concepts: the simulated deforestation quantity, the required minimum FOM, and the simulated carbon disturbance. These concepts apply generally to a wide array of large-scale behavioural models that simulate land use change.