Journal cover Journal topic
Geoscientific Model Development An interactive open-access journal of the European Geosciences Union
https://doi.org/10.5194/gmd-2017-271
© Author(s) 2017. This work is distributed under
the Creative Commons Attribution 4.0 License.
Model evaluation paper
13 Nov 2017
Review status
This discussion paper is a preprint. It is a manuscript under review for the journal Geoscientific Model Development (GMD).
Fast sensitivity analysis methods for computationally expensive models with multidimensional output
Edmund Ryan1, Oliver Wild1, Fiona O'Connor2, Apostolos Voulgarakis3, and Lindsay Lee4 1Lancaster Environment Centre, Lancaster University, Lancaster, UK
2UK Met Office Hadley Centre, Exeter, UK
3Department of Physics, Imperial College London, London, UK
4School of Earth and Environment, University of Leeds, UK
Abstract. Global sensitivity analysis (GSA) is a critical approach in identifying which inputs or parameters of a model most affect model output. This determines which inputs to include when performing model calibration or uncertainty analysis. GSA allows quantification of the sensitivity index (SI) of a particular input – the percentage of the total variability in the output attributed to the changes in that input – by averaging over the other inputs rather than fixing them at specific values. Traditional methods of computing the SIs (e.g. Sobol) involve running a model thousands of times, but this may not be feasible for computationally expensive earth system models. GSA methods that use a statistical emulator in place of the expensive model are popular as they require far fewer model runs. Here, we perform an eight-input GSA on two computationally expensive atmospheric chemistry transport models using emulators that were trained with 80 runs of the models. We consider two methods to further reduce the computational cost of GSA: (1) a dimension reduction approach and (2) an emulator-free approach. When the output of a model is multi-dimensional, it is common practice to build a separate emulator for each dimension of the output space. Here, we use principal component analysis (PCA) to reduce the output dimension and build an emulator for each of the transformed outputs. We consider the global distribution of the annual column mean lifetime of atmospheric methane, which requires ~ 2000 emulators without PCA, but only 5–40 emulators with PCA. As an alternative, we apply an emulator-free method using a generalised additive model (GAM) to estimate the SIs using only the training runs. Compared to the emulator-only method, the hybrid PCA-emulator and GAM methods are 6 and 30 times quicker, respectively, at computing the SIs for the ~ 2000 methane lifetime outputs. The SIs computed using the two computationally faster methods are almost identical to those computed using the standard emulator-only method.

Citation: Ryan, E., Wild, O., O'Connor, F., Voulgarakis, A., and Lee, L.: Fast sensitivity analysis methods for computationally expensive models with multidimensional output, Geosci. Model Dev. Discuss., https://doi.org/10.5194/gmd-2017-271, in review, 2017.
Edmund Ryan et al.
Edmund Ryan et al.

Data sets

The inputs and outputs of the FRSGC chemistry model that was used to train the emulators in this paper
O. Wild and E. M. Ryan
https://doi.org/10.5281/zenodo.1038670

Model code and software

R code to carry out the global sensitivity analysis using the methods described in this paper.
E. M. Ryan
https://doi.org/10.5281/zenodo.1038667
Edmund Ryan et al.

Viewed

Total article views: 154 (including HTML, PDF, and XML)

HTML PDF XML Total Supplement BibTeX EndNote
110 40 4 154 11 1 1

Views and downloads (calculated since 13 Nov 2017)

Cumulative views and downloads (calculated since 13 Nov 2017)

Viewed (geographical distribution)

Total article views: 154 (including HTML, PDF, and XML)

Thereof 149 with geography defined and 5 with unknown origin.

Country # Views %
  • 1

Saved

Discussed

Latest update: 20 Nov 2017
Publications Copernicus
Download
Short summary
Global sensitivity analysis (GSA) identifies which parameters of a model most affect its output. We performed GSA using statistical emulators as surrogates of two slow-running atmospheric chemistry transport models. Due to the high dimension of the model outputs we considered two alternative methods: one that reduced the output dimension and one that did not require an emulator. The alternative methods accurately performed the GSA but were significantly faster than the emulator-only method.
Global sensitivity analysis (GSA) identifies which parameters of a model most affect its output....
Share