High-end parallel applications that store and analyze large scientific datasets demand significant I/O capacity. Clusters of storage servers, managed by parallel file systems and integrated with MPI programs using MPI-IO, promise to provide scalable I/O performance for high-end applications. These systems are complex in the interaction between multiple levels of application/system software and hardware components, and in the wide ranges of system features and configuration settings. This project investigates system-level techniques to better manage parallel and concurrent I/O for high-end computing. To characterize I/O performance behaviors and pinpoint anomalies, we devised a model-driven performance debugging approach (FAST'05), studied multi-level I/O tracing (ICPP'07), and explored the use of references in I/O performance characterization (SIGMETRICS'09). To address the disk seek/rotation during concurrent I/O, we proposed a new I/O prefetching approach that achieves 2-competitive performance (EuroSys'07). Our recent work also examined the performance effects of scientific I/O workloads on the emerging flash-based solid state storage (IASDS'09).
Project Members:
Chuanpeng Li, graduated with Ph.D., now at Bloomberg
Pin Lu, graduated with M.S., now at Microsoft
Stan Park, University of Rochester
Jonathan Pearson, graduated with M.S., now at SIXNET
Kai Shen, University of Rochester
Ming Zhong, graduated with Ph.D., now at Google
Contact: Kai Shen (kshen@cs.rochester.edu)
Most Relevant Publications:
Support: This project is supported by the National Science Foundation under Grants No. 0937571 and No. 0621472.
Note: Any opinions, findings, and
conclusions or recommendations expressed in this material are those of the
author(s) and do not necessarily reflect the views of the National Science Foundation.