Journal cover Journal topic
Geoscientific Model Development An interactive open-access journal of the European Geosciences Union
Journal topic

Journal metrics

Journal metrics

  • IF value: 5.154 IF 5.154
  • IF 5-year value: 5.697 IF 5-year
  • CiteScore value: 5.56 CiteScore
  • SNIP value: 1.761 SNIP 1.761
  • IPP value: 5.30 IPP 5.30
  • SJR value: 3.164 SJR 3.164
  • Scimago H <br class='hide-on-tablet hide-on-mobile'>index value: 59 Scimago H
    index 59
  • h5-index value: 49 h5-index 49
Discussion papers
© Author(s) 2019. This work is distributed under
the Creative Commons Attribution 4.0 License.
© Author(s) 2019. This work is distributed under
the Creative Commons Attribution 4.0 License.

Submitted as: development and technical paper 13 Jun 2019

Submitted as: development and technical paper | 13 Jun 2019

Review status
This discussion paper is a preprint. It is a manuscript under review for the journal Geoscientific Model Development (GMD).

Replicability of the EC-Earth3 Earth System Model under a change in computing environment

François Massonnet1,2, Martin Ménégoz2,3, Mario Acosta2, Xavier Yepes-Arbós2, Eleftheria Exarchou2, and Francisco J. Doblas-Reyes2,4 François Massonnet et al.
  • 1Georges Lemaître Centre for Earth and Climate Research, Earth and Life Institute, Université catholique de Louvain, 3 Place Louis Pasteur, 1348 Louvain-la-Neuve, Belgium
  • 2Barcelona Supercomputing Center (Centro Nacional de Supercomputación), Nexus II-Planta 1 C/ Jordi Girona, 29, 08034 Barcelona, Spain
  • 3Institut des Géosciences de l’Environnement, Université Grenoble-Alpes CNRS, CS 40700 38 058 Grenoble CEDEX 9, France
  • 4Institució Catalana de Recerca i Estudis Avançats (ICREA), Pg. Lluís Companys 23, 08010 Barcelona, Spain

Abstract. Most Earth System Models (ESMs) are running under different high-performance computing (HPC) environments. This has several advantages, from allowing different groups to work with the same tool in parallel to leveraging the burden of ensemble climate simulations but also offering alternative solutions in case of shutdown (expected or not) of any of the environments. However, for obvious scientific reasons, it is critical to ensure that ESMs provide identical results under changes in computing environment. While strict bit-for-bit reproducibility is not always guaranteed with ESMs, it is desirable that results obtained under one computing environment are at least statistically indistinguishable from those obtained under another environment, which we term a replicability condition following the metrology nomenclature. Here, we develop a protocol to assess the replicability of the EC-Earth ESM. Using two versions of EC-Earth, we present one case of non-replicability and one case of replicability. The non-replicable case occurs with the older version of the model and likely finds its origin in the treatment of river runoffs along Antarctic coasts. By contrast, the more recent version of the model provides replicable results. The methodology presented here has been adopted as a standard test by the EC-Earth consortium (27 institutions in Europe) to evaluate the replicability of any new model version across platforms, including for CMIP6 experiments. To a larger extent, it can be used to assess whether other ESMs can safely be ported from one HPC environment to another for studying climate-related questions. Our results and experience with this work suggest that the default assumption should be that ESMs are not replicable under changes in the HPC environment, until proven otherwise.

François Massonnet et al.
Interactive discussion
Status: final response (author comments only)
Status: final response (author comments only)
AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment
[Login for Authors/Topical Editors] [Subscribe to comment alert] Printer-friendly Version - Printer-friendly version Supplement - Supplement
François Massonnet et al.
François Massonnet et al.
Total article views: 561 (including HTML, PDF, and XML)
HTML PDF XML Total BibTeX EndNote
406 153 2 561 3 2
  • HTML: 406
  • PDF: 153
  • XML: 2
  • Total: 561
  • BibTeX: 3
  • EndNote: 2
Views and downloads (calculated since 13 Jun 2019)
Cumulative views and downloads (calculated since 13 Jun 2019)
Viewed (geographical distribution)  
Total article views: 428 (including HTML, PDF, and XML) Thereof 428 with geography defined and 0 with unknown origin.
Country # Views %
  • 1
No saved metrics found.
No discussed metrics found.
Latest update: 21 Oct 2019
Publications Copernicus
Short summary
Earth System Models (ESM) are one of the cornerstones of modern climate science. They are usually run on high-performance computers (HPC). Whether the choice of HPC can affect the model results is a question of importance for optimizing the design of scientific studies. Here, we introduce a protocol for testing the replicability of the EC-Earth3 ESM across different HPC. We find the simulation results to be replicable only if specific precautions are taken at the compilation stage.
Earth System Models (ESM) are one of the cornerstones of modern climate science. They are...