Center for Applied Scientific Computing. Lawrence Livermore National Laboratory. US.
“Collaborative continuous benchmarking for HPC”
18 February 2025
Abstract
Benchmarking is integral to procurement of HPC systems, communicating HPC center workloads to HPC vendors, and verifying performance of the delivered HPC systems. Currently, HPC benchmarking is manual and challenging at every step, posing a high barrier to entry, and hampering reproducibility of the benchmarks across different HPC systems.
In this talk, we describe collaborative continuous benchmarking which enables functional reproducibility, automation, and community collaboration in HPC benchmarking. We develop a common language to streamline the interactions between HPC centers, vendors, and researchers, further enabling the previously unimaginable large-scale improvements to the HPC ecosystem.
We introduce an open source continuous benchmarking repository, Benchpark, for community collaboration. We believe collaborative continuous benchmarking will help overcome the human bottleneck in HPC benchmarking, enabling better evaluation of our systems and enabling a more productive collaboration within the HPC community.
Keywords: HPC, benchmarking, collaborative continuous benchmarking.
Bio
Dr. Olga Pearce is a computer scientist in the Center for Applied Scientific Computing at Lawrence Livermore National Laboratory. She created Benchpark, an open collaborative repository for reproducible specifications of HPC benchmarks and cross site benchmarking environments, and Thicket, an open source python-based toolkit for Exploratory Data Analysis (EDA) of parallel performance data. Olga leads benchmarking for Advanced Technology Systems, the Performance Analysis and Visualization for Exascale Project, and Performance Modeling in the Fractale SI. Her research interests include HPC architectures and simulations, parallel algorithms and programing models, system software, performance analysis and optimization.
Olga has been at LLNL since 2007. She received the NSF graduate fellowship in 2006, Lawrence Scholar Fellowship in 2009, and joined CASC as technical staff in 2014. Olga received the LLNL Deputy Director’s Science & Technology award in 2015, and LLNL awards for developing RAJA performance portability model (2018), porting and optimization of codes on LLNL’s first accelerated supercomputer (2019), developing GPU capabilities of the Next-Gen Multiphysics code (2021), response to the National Academies of Sciences RFI (2023), and acceptance of El Capitan (2022, 2024).
Olga helped create the SC Student Cluster Challenge in 2008, started a joint appointment at Texas A&M University as the Associate Professor of Practice in the Computer Science and Engineering in 2021, and serves as a co-chair of the Salishan Conference on High-Speed Computing. Olga received her Ph.D. in Computer Science from Texas A&M University in 2014, and her B.S. in Computer Science and Mathematics from Western Oregon University in 2004.
multicore.world 