NC State University

Department of Computer Science Colloquia 2000-2001

Date: Friday, December 8, 2000
Time: 3: 30 PM (talk)
Place: 402A Withers, NCSU Historical Campus (click for courtesy parking request)

Speaker: Xiaolan (Catherine) Zhang, Computer Science, Harvard University

Application-Specific Benchmarking

Abstract: In this talk I will introduce a novel approach to performance evaluation, called application-specific benchmarking, and present techniques for designing and constructing meaningful benchmarks.

Conventional benchmarks (such as SPEC) assume a workload consisting of a certain mix of programs.  They are very useful in predicting the performance of applications that exhibit similar behaviors to those of the benchmark programs.  However, if the intended application does not behave the same way as the benchmark programs, the benchmark scores are often uninformative, and sometimes can be misleading.

The application-specific benchmarking approach incorporates characteristics of the application of interest into the benchmarking process, yielding performance metrics that reflect the expected behavior of a particular application across a range of different platforms.  It also allows benchmarks to evolve with applications and consequently the benchmarks are always up-to-date.  In this talk I will present this methodology and describe how we applied it to evaluating the performance of Java Virtual Machines (HBench:Java) and garbage collectors (HBench:JGC) in general.  Our experimental results demonstrate that HBench:Java is able to correctly predict the rank of running times of three commercial applications on a variety of Java Virtual Machine implementations. In the realm of garbage collection, the predicted garbage collection times for a stop-the-world, mark-sweep garbage collector closely match the actual times.

At the end of my talk, I will also briefly discuss two other systems projects that I carried out before: 1) HACC, a clustered web-server  architecture that achieves high performance by improving the total cache efficiency of the cluster; and 2) Morph, a framework for continuous profiling and automatic optimization to attain better performance by adapting applications to individual usages and/or specific machine configurations.
Short Bio: Xiaolan (Catherine) Zhang  is a Ph.D candidate in Computer Science at the Division of Engineering and Applied Sciences of Harvard University.  Catherine graduated with a B.S. from Shanghai Jiao Tong University, P.R.China in 1993, and a M.S. from Northeastern University in 1995, both in Computer Science.  She has worked at GTE Laboratories, Digital's System Research Center (SRC) and Sun Microsystem Labs East during past summers.  Her thesis research concerns novel approaches to system performance evaluation.  She has also worked on a few projects covering a broad range of systems topics, including profiling, compiler optimization, and high-performance web server architectures..

Hosts:  C. Healey, Computer Science, NCSU

Colloquia Home Page.