Monday, March 04, 2002
11:00 AM
CSB 209
Yanyong Zhang
Pennsylvania State University
Scheduling and Resource Management for Next-Generation Clusters
Clusters of workstations built with off-the-shelf hardware, are playing an important role in mainstream high performance computing. A wide range of applications are running on clusters, ranging from traditional scientific applications, to more recent commercial ones like databases, web services and multimedia. All these applications pose tremendous processing and/or storage demands on the underlying system, making resource management an important issue for cluster design and deployment. At the same time, many of the environments where clusters are deployed, need to accommodate several such applications/users at the same time. Efficient scheduling and resource management is essential to make clusters more suitable for next generation applications.

In this talk, I will present three main contributions to cluster scheduling and resource management. I will first show how we can boost system utilization to as high as 95% for workloads running at supercomputing centers (primarily scientific applications). Next, I will present a novel suite of light-weight cluster scheduling mechanisms for providing better response times to interactive workloads, which are better suited to commercial applications (databases and e-commerce) and web services. The third contribution is in scheduling support for applications having different Quality-of-Service (QoS) demands on the underlying system. Finally, I will summarize our ongoing work in providing systems support for a commercial database server on our Linux cluster.

These studies have required extensive simulations and experimental evaluations. I will also present details on our work in developing the first set of analytical models for these complicated systems. There has been extensive collaboration with IBM Research in this thesis, and the research has been supported in part by an IBM fellowship.