Journal cover Journal topic
Geoscientific Model Development An interactive open-access journal of the European Geosciences Union
© Author(s) 2016. This work is distributed
under the Creative Commons Attribution 3.0 License.
Model evaluation paper
20 Sep 2016
Review status
A revision of this discussion paper was accepted for the journal Geoscientific Model Development (GMD) and is expected to appear here in due course.
Global Gridded Crop Model evaluation: benchmarking, skills, deficiencies and implications
Christoph Müller1, Joshua Elliott2,3,24, James Chryssanthacopoulos3,24, Almut Arneth4, Juraj Balkovic5,6, Philippe Ciais7, Delphine Deryng2,3,24, Christian Folberth5,8, Michael Glotter9, Steven Hoek10, Toshichika Iizumi11, Roberto C. Izaurralde12,13, Curtis Jones12, Nikolay Khabarov5, Peter Lawrence14, Wenfeng Liu15, Stefan Olin16, Thomas A. M. Pugh4,17, Deepak Ray18, Ashwan Reddy12, Cynthia Rosenzweig3,19,24, Alexander C. Ruane3,19,24, Gen Sakurai11, Erwin Schmid20, Rastislav Skalsky5, Carol X. Song21, Xuhui Wang7,22, Allard de Wit10, and Hong Yang15,23 1Potsdam Institute for Climate Impact Research, 14473 Potsdam, Germany
2University of Chicago and ANL Computation Institute, Chicago, IL 60637, USA
3Columbia University Center for Climate Systems Research, New York, NY 10025, USA
4Karlsruhe Institute of Technology, IMK-IFU, 82467 Garmisch-Partenkirchen, Germany
5International Institute for Applied Systems Analysis, Ecosystem Services and Management Program, 2361 Laxenburg, Austria
6Comenius University in Bratislava, Department of Soil Science, 842 15 Bratislava, Slovak Republic
7Laboratoire des Sciences du Climat et de l'Environnement, CEA CNRS UVSQ Orme des Merisiers, F-91191 Gif-sur-Yvette, France
8Department of Geography, Ludwig Maximilian University, 80333 Munich, Germany
9University of Chicago, Department of the Geophysical Sciences, Chicago, IL 60637, USA
10Alterra Wageningen University and Research Centre, Earth Observation and Environmental Informatics, 6708PB Wageningen, Netherlands
11National Agriculture and Research Organization, National Institute for Agro-Environmental Sciences, Agro-Meteorology Division, Tsukuba, 305-8604, Japan
12University of Maryland, Department of Geographical Sciences, College Park, MD 20742, USA
13Texas A&M University, Texas AgriLife Research and Extension, Temple, TX 76502, USA
14National Center for Atmospheric Research, Earth System Laboratory, Boulder, CO 80307, USA
15Eawag, Swiss Federal Institute of Aquatic Science and Technology, CH-8600 Duebendorf, Switzerland
16Department of Physical Geography and Ecosystem Science, Lund University, 223 62 Lund, Sweden
17School of Geography, Earth & Environmental Science and Birmingham Institute of Forest Research, University of Birmingham, Edgbaston, Birmingham, B15 2TT, United Kingdom
18Institute on the Environment, University of Minnesota, Saint Paul, USA
19National Aeronautics and Space Administration Goddard Institute for Space Studies, New York, NY 10025, USA
20University of Natural Resources and Life Sciences, Institute for Sustainable Economic Development, 1180 Vienna, Austria
21Rosen Center for Advanced Computing, Purdue University, West Lafayette, Indiana, USA
22Peking University, Sino-French Institute of Earth System Sciences, 100871 Beijing, China
23Department of Environmental Sciences, University of Basel, Petersplatz 1, CH-4003 Basel, Switzerland
24NASA Goddard Institute for Space Studies, New York, NY 10025, USA
Abstract. Crop models are increasingly used to simulate crop yields at the global scale, but there so far is no general framework on how to assess model performance. We here evaluate the simulation results of 14 global gridded crop modeling groups that have contributed historic crop yield simulations for maize, wheat, rice and soybean to the Global Gridded Crop Model Intercomparison (GGCMI) of the Agricultural Model Intercomparison and Improvement Project (AgMIP). Simulation results are compared to reference data at global, national and grid cell scales and we evaluate model performance with respect to time series correlation, spatial correlation and mean bias. We find that GGCMs show mixed skill in reproducing time-series correlations or spatial patterns at the different spatial scales. Generally, maize, wheat and soybean simulations of many GGCMs are capable of reproducing larger parts of observed temporal variability (time series correlation coefficients (r) of up to 0.888 for maize, 0.673 for wheat and 0.643 for soybean at the global scale) but rice yield variability cannot be well reproduced by most models. Yield variability can be well reproduced for most major producer countries by many GGCMS and for all countries by at least some. A comparison with gridded yield data and a statistical analysis of the effects of weather variability on yield variability shows that the ensemble of GGCMs can explain more of the yield variability than an ensemble of regression models for maize and soybean, but not for wheat and rice. We identify future research needs in global gridded crop modeling and for all individual crop modeling groups. In the absence of a purely observation-based benchmark for model evaluation, we propose that the best performing crop model per crop and region establishes the benchmark for all others, and modelers are encouraged to investigate how crop model performance can be increased. We make our evaluation system accessible to all crop modelers so that also other modeling groups can test their model performance against the reference data and the GGCMI benchmark.

Citation: Müller, C., Elliott, J., Chryssanthacopoulos, J., Arneth, A., Balkovic, J., Ciais, P., Deryng, D., Folberth, C., Glotter, M., Hoek, S., Iizumi, T., Izaurralde, R. C., Jones, C., Khabarov, N., Lawrence, P., Liu, W., Olin, S., Pugh, T. A. M., Ray, D., Reddy, A., Rosenzweig, C., Ruane, A. C., Sakurai, G., Schmid, E., Skalsky, R., Song, C. X., Wang, X., de Wit, A., and Yang, H.: Global Gridded Crop Model evaluation: benchmarking, skills, deficiencies and implications, Geosci. Model Dev. Discuss., doi:10.5194/gmd-2016-207, in review, 2016.
Christoph Müller et al.
Interactive discussionStatus: closed
AC: Author comment | RC: Referee comment | SC: Short comment | EC: Editor comment
Printer-friendly Version - Printer-friendly version      Supplement - Supplement
RC1: 'review comments', Anonymous Referee #1, 19 Oct 2016 Printer-friendly Version 
RC2: 'review', Anonymous Referee #2, 21 Oct 2016 Printer-friendly Version 
AC1: 'Author response to the two reviewer comments', Christoph Müller, 14 Dec 2016 Printer-friendly Version 
Christoph Müller et al.
Christoph Müller et al.


Total article views: 366 (including HTML, PDF, and XML)

HTML PDF XML Total Supplement BibTeX EndNote
258 99 9 366 24 6 16

Views and downloads (calculated since 20 Sep 2016)

Cumulative views and downloads (calculated since 20 Sep 2016)

Viewed (geographical distribution)

Total article views: 366 (including HTML, PDF, and XML)

Thereof 366 with geography defined and 0 with unknown origin.

Country # Views %
  • 1



Latest update: 29 Mar 2017
Publications Copernicus
Short summary
Crop models are increasingly used in climate change impact research and integrated assessments. For the Agricultural Model Intercomparison and Improvement Project (AgMIP), 14 global gridded crop models (GGCMs) have supplied crop yield simulations (1980–2010) for maize, wheat, rice and soybean. We evaluate the performance of these models against observational data at global, national and grid cell level. We propose an open access benchmark system against which future model versions can be tested.
Crop models are increasingly used in climate change impact research and integrated assessments....