Thicket: Seeing the Performance Experiment Forest for the Individual Run Trees

Stephanie Brink, Michael McKinsey, David Boehme, Connor Scully-Allison, Ian Lumsden, Daryl Hawkins, Treece Burgess, Vanessa Lama, Jakob Lüttgau, Katherine E. Isaacs, Michela Taufer, Olga Pearce

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Scopus citations

Abstract

Thicket is an open-source Python toolkit for Exploratory Data Analysis (EDA) of multi-run performance experiments. It enables an understanding of optimal performance configuration for large-scale application codes. Most performance tools focus on a single execution (e.g., single platform, single measurement tool, single scale). Thicket bridges the gap to convenient analysis in multi-dimensional, multi-scale, multi-architecture, and multi-tool performance datasets by providing an interface for interacting with the performance data. Thicket has a modular structure composed of three components. The first component is a data structure for multi-dimensional performance data, which is composed automatically on the portable basis of call trees, and accommodates any subset of dimensions present in the dataset. The second is the metadata, enabling distinction and sub-selection of dimensions in performance data. The third is a dimensionality reduction mechanism, enabling analysis such as computing aggregated statistics on a given data dimension. Extensible mechanisms are available for applying analyses (e.g., top-down on Intel CPUs), data science techniques (e.g., K-means clustering from scikit-learn), modeling performance (e.g., Extra-P), and interactive visualization. We demonstrate the power and flexibility of Thicket through two case studies, first with the open-source RAJA Performance Suite on CPU and GPU clusters and another with a large physics simulation run on both a traditional HPC cluster and an AWS Parallel Cluster instance.

Original languageEnglish (US)
Title of host publicationHPDC 2023 - Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing
PublisherAssociation for Computing Machinery, Inc
Pages281-293
Number of pages13
ISBN (Electronic)9798400701559
DOIs
StatePublished - Aug 7 2023
Externally publishedYes
Event32nd International Symposium on High-Performance Parallel and Distributed Computing, HPDC 2023 - Orlando, United States
Duration: Jun 16 2023Jun 23 2023

Publication series

NameHPDC 2023 - Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing

Conference

Conference32nd International Symposium on High-Performance Parallel and Distributed Computing, HPDC 2023
Country/TerritoryUnited States
CityOrlando
Period6/16/236/23/23

Keywords

  • exploratory data analysis
  • hpc
  • multi-dimensional
  • parallel profile
  • performance analysis

ASJC Scopus subject areas

  • Information Systems
  • Software
  • Safety, Risk, Reliability and Quality
  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Science Applications
  • Hardware and Architecture

Fingerprint

Dive into the research topics of 'Thicket: Seeing the Performance Experiment Forest for the Individual Run Trees'. Together they form a unique fingerprint.

Cite this