Trace-Driven Scientific I/O Benchmarks

Introduction

Traces of scientific I/O workloads are being made available to enable computing-related research. Examples include traces from Sandia National Laboratories and Los Alamos National Laboratory. Useful statistics can be extracted from such traces. However, it is sometimes desirable to run applications represented by the traces so as to evaluate the performance and other behaviors of I/O and storage systems. For this purpose, we have created a trace player, TracePlay/Control, written in C, to recreate some of the trace conditions in the form of runnable benchmarks. Our trace player uses formatted traces that are derived from original scientific I/O traces. Our player is less of a full-blown utility but more of a benchmark using traces extracted from actual scientific applications. The benchmark itself is largely just a shell or wrapper for a parsed trace.

Design

In order to successfully replay a trace, the file system context must be recreated. Our trace player extracts directory and file names accessed throughout the trace and recreates the hierarchy. Files are created with the maximum estimated size inferred from I/O system calls (read, write, seek). Our trace player is capable of running in two modes:

Download

Download source: traceplayer_v0.9.zip

Traces below originated from those released by Sandia National Laboratories and Los Alamos National Laboratory. We sanitized the original traces and converted them into the format suitable for our trace player.

Sanitized and formatted Sandia traces:
sandia.alegra.2744.zip
sandia.cth.zip
sandia.s3d_fortIO_031709.zip

Sanitized and formatted LANL traces:
32PE_N-1_nonstrided_64K_MPI.zip
32PE_N-1_nonstrided_1024K_MPI.zip

Installation notes (also in the README)

Usage (also in README)

All the above benchmark runs may take an optional -nofs and -drop flags. If -nofs is set, the benchmark run will not try to reconstruct the filesystem in the trace. Because file system reconstruction is time-consuming, it can be helpful to utilize the -nofs flag on subsequent runs. If -drop is set, the file cache will be sync'd and dropped before each replay. This feature is available in Linux kernels 2.6.16 and newer. You must have root privileges to utilize the -drop flag.

Supported research

Contact: Stan Park
email: park _at_ cs.rochester.edu

Support: This project is supported by the National Science Foundation under Grants No. 0937571 and No. 0621472.
Note: Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.