|
Published Articles >> Table of Contents >> Abstract
International Parallel and Distributed Processing Symposium (IPDPS'03)
p. 68b
Improving MPI-IO Output Performance with Active Buffering Plus Threads
Xiaosong Ma, University of Illinois at Urbana-Champaign
Marianne Winslett, University of Illinois at Urbana-Champaign
Jonghyun Lee, University of Illinois at Urbana-Champaign
Shengke Yu, University of Illinois at Urbana-Champaign
Full Article Text:
  
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/IPDPS.2003.1213165
Send link to a friend
| Abstract |
|
Efficient collective output of intermediate results to secondary storage becomes more and more important for scientific simulations as the gap between processing power/interconnection bandwidth and the I/O system bandwidth enlarges. Dedicated servers can offload I/O from compute processors and shorten the execution time, but it is not always possible or easy for an application to use them. We propose the use of active buffering with threads (ABT) for overlapping I/O with computation efficiently and flexibly without dedicated I/O servers. We show that the implementation of ABT in ROMIO, a popular implementation of MPI-IO, greatly reduces the application-visible cost of ROMIO’s collective write calls, and improves an application’s overall performance by hiding I/O cost and saving implicit synchronization overhead from collective write operations. Further, ABT is high-level, platform-independent, and transparent to users, giving users the benefit of overlapping I/O with other processing tasks even when the file system or parallel I/O library does not support asynchronous I/O.
|
 Additional Information
|
Index Terms- Collective I/O, MPI-IO, Parallel Simulation, Clusters
Citation:
Xiaosong Ma, Marianne Winslett, Jonghyun Lee, Shengke Yu,
"Improving MPI-IO Output Performance with Active Buffering Plus Threads,"
ipdps,
p. 68b,
International Parallel and Distributed Processing Symposium (IPDPS'03),
2003
|
|
