Journal cover Journal topic
Geoscientific Model Development An interactive open-access journal of the European Geosciences Union
Journal topic

Journal metrics

Journal metrics

  • IF value: 4.252 IF 4.252
  • IF 5-year value: 4.890 IF 5-year
    4.890
  • CiteScore value: 4.49 CiteScore
    4.49
  • SNIP value: 1.539 SNIP 1.539
  • SJR value: 2.404 SJR 2.404
  • IPP value: 4.28 IPP 4.28
  • h5-index value: 40 h5-index 40
  • Scimago H <br class='hide-on-tablet hide-on-mobile'>index value: 51 Scimago H
    index 51
Discussion papers
https://doi.org/10.5194/gmd-2019-17
© Author(s) 2019. This work is distributed under
the Creative Commons Attribution 4.0 License.
https://doi.org/10.5194/gmd-2019-17
© Author(s) 2019. This work is distributed under
the Creative Commons Attribution 4.0 License.

Methods for assessment of models 04 Feb 2019

Methods for assessment of models | 04 Feb 2019

Review status
This discussion paper is a preprint. It is a manuscript under review for the journal Geoscientific Model Development (GMD).

What do we do with model simulation crashes? Recommendations for global sensitivity analysis of earth and environmental systems models

Razi Sheikholeslami1,2, Saman Razavi1,2,3, and Amin Haghnegahdar1,2 Razi Sheikholeslami et al.
  • 1School of Environment and Sustainability, University of Saskatchewan, Saskatoon, Canada
  • 2Global Institute for Water Security, University of Saskatchewan, Saskatoon, Canada
  • 3Department of Civil, Geological, and Environmental Engineering, University of Saskatchewan, Saskatoon, Canada

Abstract. Complex, software-intensive, technically advanced, and computationally demanding models, presumably with ever-growing realism and fidelity, have been widely used to simulate and predict the dynamics of the Earth and environmental systems. The parameter-induced simulation crash (failure) problem is typical across most of these models, despite considerable efforts that modellers have directed at model development and implementation over the last few decades. A simulation failure mainly occurs due to the violation of the numerical stability conditions, non-robust numerical implementations, or errors in programming. However, the existing sampling-based analysis techniques such as global sensitivity analysis (GSA) methods, which require running these models under many configurations of parameter values, are ill-equipped to effectively deal with model failures. To tackle this problem, we propose a novel approach that allows users to cope with failed designs (samples) during the GSA, without knowing where they took place and without re-running the entire experiment. This approach deems model crashes as missing data and uses strategies such as median substitution, single nearest neighbour, or response surface modelling to fill in for model crashes. We test the proposed approach on a 10-paramter HBV-SASK rainfall-runoff model and a 111-parameter MESH land surface-hydrology model. Our results show that response surface modelling is a superior strategy, out of the data filling strategies tested, and can scale well to the dimensionality of the model, sample size, and the ratio of number of failures to the sample size. Further, we conduct a "failure analysis" and discuss some possible causes of the MESH model failure.

Razi Sheikholeslami et al.
Interactive discussion
Status: open (until 01 Apr 2019)
Status: open (until 01 Apr 2019)
AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment
[Subscribe to comment alert] Printer-friendly Version - Printer-friendly version Supplement - Supplement
Razi Sheikholeslami et al.
Razi Sheikholeslami et al.
Viewed  
Total article views: 306 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
245 58 3 306 4 3
  • HTML: 245
  • PDF: 58
  • XML: 3
  • Total: 306
  • BibTeX: 4
  • EndNote: 3
Views and downloads (calculated since 04 Feb 2019)
Cumulative views and downloads (calculated since 04 Feb 2019)
Viewed (geographical distribution)  
Total article views: 95 (including HTML, PDF, and XML) Thereof 95 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
1
 
 
 
 
Cited  
Saved  
No saved metrics found.
Discussed  
No discussed metrics found.
Latest update: 21 Feb 2019
Publications Copernicus
Download
Short summary
The ever-growing complexity of the Earth and Environmental Systems Models will pose many types of software development and implementation issues such as parameter-induced simulation crashes which are mainly caused by violation of the numerical stability conditions. Here, we present a new strategy to cope with crashed simulations when performing sensitivity analysis. Our results show that this strategy can scale well to the dimensionality of the model, sample size, and the number crashes.
The ever-growing complexity of the Earth and Environmental Systems Models will pose many types...
Citation