Friday, March 29, 2002
11:00 AM
CSB 209
Kai Shen
U. California, Santa Barbara
Clustering, Resource Management, and Replication Support for Large-Scale Internet Services
With the increasing demand of providing highly scalable, available and easy-to-manage Internet services, the deployment of large-scale complex server clusters has been rapidly emerging in which service components are usually partitioned, replicated, and aggregated. In this talk, I will present my research in building a software infrastructure (Neptune) that provides clustering, resource management, and replication support for large-scale Internet services. Neptune has been deployed at Internet search engine Ask Jeeves in a commercial system serving 6-7 million search queries per day.

Neptune's clustering architecture is based on the design principles of symmetry, decentralization, and non-scaling overhead to achieve high scalability, availability, and manageability. I will first talk about Neptune's overall clustering architecture, followed by a brief discussion on the load balancing support for fine-grain services. In the second part of the talk, I will present an integrated resource management framework combining resource utilization efficiency, consumer-perceived service response time, and service differentiation support. In the end, I will talk about my work on service replication.

Replication of persistent service data is crucial to achieving high availability. My work is focused on providing the desired level of replica consistency with the emphasis on performance scalability.