Balancing CPU Load for Irregular MPI Applications
2012 (English)In: Advances in Parallel Computing, vol. 22: Applications, Tools and Techniques on the Road to Exascale Computing / [ed] Koen De Bosschere, Erik H. D'Hollander, Gerhard R. Joubert, David Padua, Frans Peters, Mark Sawyer, IOS Press, 2012, 307-316 p.Conference paper (Refereed)
MPI applications typically are designed to be run on a parallel machine with one process per core. If processes exhibit different computational load, either the code must be rewritten for load balancing, with negative side-effects on readability and maintainability, or the one-process-per-core philosophy leads to a low utilization of many processor cores. If several processes are mapped per core to increase CPU utilization, the load might still be unevenly distributed among the cores if the mapping is unaware of the process characteristics.
Therefore, similarly to the MPI_Graph_create() function where the program gives hints on communication patterns so that MPI processes can be placed favorably, we propose a MPI_Load_create() function where the program supplies information on the relative loads of the MPI processes, such that processes can be favorably grouped and mapped onto processor cores. In order to account for scalability and restricted knowledge of individual MPI processes, we also propose an extension MPI_Dist_load_create() similar to MPI_Dist_graph_create(), where each individual MPI process only knows the loads of a subset of the MPI processes.
We detail how to implement both variants on top of MPI, and provide experimental performance results both for synthetic and numeric example applications. The results indicate that load balancing is favorable in both cases.
Place, publisher, year, edition, pages
IOS Press, 2012. 307-316 p.
Parallel computing, load balancing, task mapping, parallel computer system, message passing interface (MPI), cluster computing, MPI_Graph_create, MPI_Load_create
IdentifiersURN: urn:nbn:se:liu:diva-93372DOI: 10.3233/978-1-61499-041-3-307ISBN: 978-1-61499-040-6OAI: oai:DiVA.org:liu-93372DiVA: diva2:624359
International Conference on Parallel Computing (ParCo-2011), 30 August - 2 September 2011, Ghent, Belgium