Speaker: Kai Shen , Computer Science, University of California - Santa Barbara
Clustering, Resource Management, and Replication Support for Large-scale Internet Services
Abstract: With the increasing demand of providing highly scalable, available and easy-to-manage Internet services, the deployment of large-scale complex server clusters has been rapidly emerging in which service components are usually partitioned, replicated, and aggregated. In this talk, I will present my research in building a software infrastructure (Neptune) that provides clustering, resource management, and replication support for large-scale Internet services. Neptune has been deployed at Internet search engine Teoma and Ask Jeeves in a commercial system serving 6-7 million search queries per day.
Neptune's clustering architecture is based on the design principles of symmetry, decentralization, non-scaling overhead, and soft-state to achieve high scalability, availability, and manageability. I will first talk about Neptune's overall clustering architecture, followed by a brief discussion on the load balancing support for fine-grain services. In the second part of the talk, I will present an integrated resource management framework combining resource utilization efficiency, consumer-perceived service response time, and service differentiation support. In the end, I will talk about my work on service replication. Replication of persistent service data is crucial to achieving high availability. My work is focused on providing the desired level of replica consistency with the emphasis on performance scalability.
Short Bio: Kai Shen received his B.S. degree in Computer Science and Engineering from Shanghair Jiaotong University in China in 1996. He currently is enrolled in the Ph.D. program in the Department of Computer Science at University of California at Santa Barbara. The title of his dissertation is "Supporting Replication, Load Balancing, and QoS for Cluster-based Network Services". Mr. Shen's research interest include parallel and distributed systems, scalable network services, Internet search, high performance cluster computing, and scientific computing.
Host: R. Fornaro and F. Mueller , Computer Science, NCSU