Abstract: Heterogeneous systems are now widespread, with nearly all systems containing CPUs and GPUs. While synchronization on such devices has traditionally been coarse-grained, many workloads can be optimized using finer-grained idioms, e.g. mutexes. While such constructs have been studied on traditional CPU systems, heterogeneous systems often have under-specified programming models, and thus, many critical underlying assumptions, e.g., memory orderings, need to be re-evaluated. This talk will discuss work at UC Santa Cruz that (1) provides rigorous testing for GPU memory consistency: a key building block for synchronization, and (2) identifies pragmatic application classes that can benefit from fine-grain heterogeneous synchronization. Our novel testing method applies classic mutation testing techniques to axiomatic consistency formalizations to more precisely target interesting areas of GPU frameworks. We evaluate our methods on over 100 different GPUs and have identified several memory ordering bugs on popular devices, such as the Google Pixel. For applications, we show that tree-based algorithms, used in resource-constrained classification, can achieve up to a 13x speedup using fine-grained heterogeneous synchronization. Combined, we hope to provide more reliable heterogeneous synchronization that can be used to further optimize important applications on widespread devices.

Short Bio: Tyler Sorensen is an Assistant Professor at UC Santa Cruz and works on programming languages, especially related to concurrency on heterogeneous processors, e.g., GPUs. As an invited member of the Khronos group, his work has been influential in GPU programming standards, both in specifications and conformance test suites. His research has been published in top PL, SE, and architecture venues, including FSE, PLDI, and ASPLOS, and has been recognized with several distinguished paper awards. He plans to continue this work with the support of an NSF CAREER grant for exploring heterogeneous memory consistency models. Tyler graduated with a Ph.D. from Imperial College London and was a postdoc at Princeton University before joining UC Santa Cruz.