Abstract
Many scientific applications require high-performance concurrent I/O accesses to a file by multiple processes. Those applications rely indirectly on atomic I/O capabilities in order to perform updates to structured datasets, such as those stored in HDF5 format files. Current support for atomicity in MPI-IO is provided by locking around the operations, imposing lock overhead in all situations, even though in many cases these operations are non-overlapping in the file. We propose to isolate non-overlapping accesses from overlapping ones in independent I/O cases, allowing the non-overlapping ones to proceed without imposing lock overhead. To enable this, we have implemented an efficient conflict detection algorithm in MPI-IO using MPI file views and datatypes. We show that our conflict detection scheme incurs minimal overhead on I/O operations, making it an effective mechanism for avoiding locks when they are not needed.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Lustre filesystem, http://www.lustre.org/
Parallel virtual file system version 2, http://www.pvfs.org/
IEEE/ANSI std.1003.1. portable operating system interface (POSIX)-part 1: System application program interface (API)[C language] (1996)
Aarestad, P.M., Ching, A., Thiruvathukal, G.K., Choudhary, A.N.: Scalable approaches for supporting MPI-IO atomicity. In: CCGRID 2006: Proceedings of the Sixth IEEE International Symposium on Cluster Computing and the Grid, Washington, DC, USA, pp. 35–42. IEEE Computer Society Press, Los Alamitos (2006)
Carns, P.H., Ligon III, W.B., Ross, R.B., Thakur, R.: PVFS: a parallel file system for linux clusters. In: ALS 2000: Proceedings of the 4th annual Linux Showcase & Conference, Berkeley, CA, USA, p. 28. USENIX Association (2000)
Ching, A., Feng, W., Lin, H., Ma, X., Choudhary, A.: Exploring I/O strategies for parallel sequence-search tools with s3asim. hpdc, 229–240 (2006)
Ching, A., Choudhary, A., Coloma, K., Liao, W.k., Ross, R., Gropp, W.: Noncontiguous I/O accesses through MPI-IO. In: CCGRID 2003: Proceedings of the 3rd International Symposium on Cluster Computing and the Grid, Washington, DC, USA, p. 104. IEEE Computer Society Press, Los Alamitos (2003)
Ching, A., Choudhary, A., Liao, W., Ross, R., Gropp, W.: Noncontiguous I/O through PVFS. In: CLUSTER 2002: Proceedings of the IEEE International Conference on Cluster Computing, Washington, DC, USA, p. 405. IEEE Computer Society Press, Los Alamitos (2002)
Ching, A., Liao, W.k., Choudhary, A., Ross, R., Ward, L.: Noncontiguous locking techniques for parallel file systems. In: SC 2007: Proceedings of the ACM/IEEE conference on Supercomputing, pp. 1–12. ACM, New York (2007)
Crandall, P.E., Aydt, R.A., Chien, A.A., Reed, D.A.: Input/output characteristics of scalable parallel applications. In: Proceedings of Supercomputing 1995. Press (1995)
Liao, W.k., Choudhary, A., Coloma, K., Thiruvathukal, G.K., Ward, L., Russell, E., Pundit, N.: Scalable implementations of MPI atomicity for concurrent overlapping I/O. In: International Conference on Parallel Processing, p. 239 (2003)
Nieuwejaar, N., Kotz, D., Purakayastha, A., Ellis, C.S., Best, M.L.: File-access characteristics of parallel scientific workloads. IEEE Trans. Parallel Distrib. Syst. 7(10), 1075–1089 (1996)
Pervez, S., Gopalakrishnan, G.C., Kirby, R.M., Thakur, R., Gropp, W.D.: Formal verification of programs that use MPI one-sided communication. In: Mohr, B., Träff, J.L., Worringen, J., Dongarra, J. (eds.) PVM/MPI 2006. LNCS, vol. 4192, pp. 30–39. Springer, Heidelberg (2006)
Ross, R.B., Latham, R., Gropp, W., Thakur, R., Toonen, B.R.: Implementing MPI-IO atomic mode without file system support. In: CCGRID, pp. 1135–1142 (2005)
Schmuck, F., Haskin, R.: GPFS: A shared-disk file system for large computing clusters. In: FAST 2002: Proceedings of the 1st USENIX Conference on File and Storage Technologies, Berkeley, CA, USA, vol. 19. USENIX Association (2002)
Thakur, R., Ross, R., Latham, R.: Implementing byte-range locks using MPI one-sided communication. In: Di Martino, B., Kranzlmüller, D., Dongarra, J. (eds.) EuroPVM/MPI 2005. LNCS, vol. 3666, pp. 119–128. Springer, Heidelberg (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sehrish, S., Wang, J., Thakur, R. (2009). Conflict Detection Algorithm to Minimize Locking for MPI-IO Atomicity. In: Ropo, M., Westerholm, J., Dongarra, J. (eds) Recent Advances in Parallel Virtual Machine and Message Passing Interface. EuroPVM/MPI 2009. Lecture Notes in Computer Science, vol 5759. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03770-2_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-03770-2_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03769-6
Online ISBN: 978-3-642-03770-2
eBook Packages: Computer ScienceComputer Science (R0)