Importance of bitwise identical reproducibility in earth system modeling and status report
L. Liu1,2, S. Peng1, C. Zhang3, R. Li3, B. Wang1,2,4, C. Sun1, Q. Liu1, L. Dong4, L. Li4, Y. Shi1, Y. He1, W. Zhao1, and G. Yang1,2,31Ministry of Education Key Laboratory for Earth system modeling, Center for Earth System Science (CESS), Tsinghua University, Beijing, China 2Joint Center for Global Change Studies (JCGCS), Beijing, China 3Department of Computer Science and Technology, Tsinghua University, Beijing, China 4State Key Laboratory of Numerical Modelling for Atmospheric Sciences and Geophysical Fluid Dynamics (LASG), Institute of Atmospheric Physics, Chinese Academy of Sciences, Beijing, China
Received: 30 Apr 2015 – Accepted for review: 25 May 2015 – Discussion started: 12 Jun 2015
Abstract. Reproducibility is a fundamental principle of scientific research. Bitwise identical reproducibility, i.e., bitwise computational results can be reproduced, guarantees the reproduction of exactly the same results. Here we show the importance of bitwise identical reproducibility to Earth system modeling but the importance has not yet been widely recognized. Modeled mean climate states, variability and trends at different scales may be significantly changed or even lead to opposing results due to a slight change in the original simulation setting during a reproduction. Out of the large body of Earth system modeling publications, few thoroughly describe the whole original simulation setting. As a result, the reproduction of a particular simulation experiment by fellow scientists heavily depends on the interaction with the original authors, which is often inconvenient or even impossible. We anticipate bitwise identical reproducibility to be promoted as a worldwide standard, to guarantee the independent reproduction of simulation results and to further improve model development and scientific research.
Liu, L., Peng, S., Zhang, C., Li, R., Wang, B., Sun, C., Liu, Q., Dong, L., Li, L., Shi, Y., He, Y., Zhao, W., and Yang, G.: Importance of bitwise identical reproducibility in earth system modeling and status report, Geosci. Model Dev. Discuss., 8, 4375-4400, doi:10.5194/gmdd-8-4375-2015, 2015.