Journal cover Journal topic
Geoscientific Model Development An interactive open-access journal of the European Geosciences Union
https://doi.org/10.5194/gmd-2017-301
© Author(s) 2018. This work is distributed under
the Creative Commons Attribution 4.0 License.
Methods for assessment of models
22 Jan 2018
Review status
This discussion paper is a preprint. It is a manuscript under review for the journal Geoscientific Model Development (GMD).
Simulation of the Performance and Scalability of MPI Communications of Atmospheric Models running on Exascale Supercomputers
Yongjun Zheng and Philippe Marguinaud Centre National de Recherches Météorologiques, Météo France, Toulouse 31057, France
Abstract. In this study, we identify the key MPI operations required in atmospheric modelling; then, we use a skeleton program and a simulation framework (based on SST/macro simulation package) to simulate these MPI operations (transposition, halo exchange, and allreduce), with the perspective of future exascale machines in mind. The experimental results show that the choice of the collective algorithm has a great impact on the performance of communications, in particular we find that the generalized ring-k algorithm for the alltoallv operation and the generalized recursive-k algorithm for the allreduce operation perform the best. In addition, we observe that the impacts of interconnect topologies and routing algorithms on the performance and scalability of transpositions, halo exchange, and allreduce operations are significant, however, that the routing algorithm has a negligible impact on the performance of allreduce operations because of its small message size. It is impossible to infinitely grow bandwidth and reduce latency due to hardware limitations, thus, congestion may occur and limit the continuous improvement of the performance of communications. The experiments show that the performance of communications can be improved when congestion is mitigated by a proper configuration of the topology and routing algorithm, which uniformly distribute the congestion over the interconnect network to avoid the hotspots and bottlenecks caused by congestion. It is generally believed that the transpositions seriously limit the scalability of the spectral models. The experiments show that although the communication time of the transposition is larger than those of the wide halo exchange for the Semi-Lagrangian method and the allreduce in the GCR iterative solver for the Semi-Implicit method below 200 000 MPI processes, the transposition whose communication time decreases quickly as the number of MPI processes increases demonstrates strong scalability in the case of very large grids and moderate latencies; the halo exchange whose communication time decreases more slowly than that of transposition as the number of MPI processes increases reveals its weak scalability; in contrast, the allreduce whose communication time increases as the number of MPI processes increases does not scale well. From this point of view, the scalability of the spectral models could still be acceptable, therefore it seems to be premature to conclude that the scalability of the grid-point models is better than that of spectral models at exascale, unless innovative methods are exploited to mitigate the problem of the scalability presented in the grid-point models.

Citation: Zheng, Y. and Marguinaud, P.: Simulation of the Performance and Scalability of MPI Communications of Atmospheric Models running on Exascale Supercomputers, Geosci. Model Dev. Discuss., https://doi.org/10.5194/gmd-2017-301, in review, 2018.
Yongjun Zheng and Philippe Marguinaud
Yongjun Zheng and Philippe Marguinaud
Yongjun Zheng and Philippe Marguinaud

Viewed

Total article views: 386 (including HTML, PDF, and XML)

HTML PDF XML Total BibTeX EndNote
279 84 23 386 4 22

Views and downloads (calculated since 22 Jan 2018)

Cumulative views and downloads (calculated since 22 Jan 2018)

Viewed (geographical distribution)

Total article views: 377 (including HTML, PDF, and XML)

Thereof 376 with geography defined and 1 with unknown origin.

Country # Views %
  • 1

Saved

Discussed

Latest update: 24 Feb 2018
Publications Copernicus
Download
Short summary
The impacts of collective algorithms, interconnect topologies, and routing algorithms on the performance and scalability of transpositions, halo exchange, and allreduce operations are significant. The performance of communications can be improved when the congestion is mitigated by a proper configuration of the topology and routing algorithm. The scalability of the spectral models could be acceptable for exascale supercomputers.
The impacts of collective algorithms, interconnect topologies, and routing algorithms on the...
Share