I/O constitutes a significant portion of most of the application run-time. Spawning many such applications concurrently on an HPC system leads to severe I/O contention. Thus, understanding and subsequently reducing I/O contention induced by such multi-tenancy is critical for the efficient and reliable performance of the HPC system. In this study, we demonstrate that an application’s performance is influenced by the command line arguments passed to the job submission. We model an application’s I/O behavior based on two factors: past I/O behavior within a time window and user-configured I/O settings via command-line arguments. We conclude that I/O patterns for well-known HPC applications like E3SM and LAMMP are predictable, with an average uncertainty below 0.25 (A probability of 80%) and near zero (A probability of 100%) within a day. However, I/O pattern variance increases as the study time window lengthens. Additionally, we show that for 38 users and at least 50 applications constituting approximately 93000 job submissions, there is a high correlation between a submitted command line and the past command lines made within 1 to 10 days submitted by the user. We claim the length of this time window is unique per user.
@inproceedings{10.1145/3700838.3700865,author={Yazdani, Ahmad Hossein and Paul, Arnab K. and Karimi, Ahmad Maroof and Wang, Feiyi and Butt, Ali},title={User-based I/O Profiling for Leadership Scale HPC Workloads},year={2025},isbn={9798400710629},publisher={Association for Computing Machinery},address={New York, NY, USA},url={https://doi.org/10.1145/3700838.3700865},doi={10.1145/3700838.3700865},booktitle={Proceedings of the 26th International Conference on Distributed Computing and Networking},pages={181–190},numpages={10},keywords={High Performance Computing, I/O characterization, I/O scheduler, Darshan, I/O profiling},location={Heydarabad, India},series={ICDCN '25},month=jan,}
FAST’23
SHADE: Enable Fundamental Cacheability for Distributed Deep Learning Training
Redwan Ibne Seraj Khan, Ahmad Hossein Yazdani, Yuqi Fu, and 5 more authors
In 21st USENIX Conference on File and Storage Technologies (FAST 23), Feb 2023
@inproceedings{285774,author={Khan, Redwan Ibne Seraj and Yazdani, Ahmad Hossein and Fu, Yuqi and Paul, Arnab K. and Ji, Bo and Jian, Xun and Cheng, Yue and Butt, Ali R.},title={{SHADE}: Enable Fundamental Cacheability for Distributed Deep Learning Training},booktitle={21st USENIX Conference on File and Storage Technologies (FAST 23)},year={2023},isbn={978-1-939133-32-8},address={Santa Clara, CA},pages={135--152},url={https://www.usenix.org/conference/fast23/presentation/khan},publisher={USENIX Association},month=feb,}