Foundational principles for a portable HPC validation framework.
HPCPerf is an exploratory effort to rethink how HPC benchmarking and system validation are structured.
This project does not attempt to define a standard.
It attempts to define clean boundaries.
The goal is to explore whether a small set of disciplined structural principles can make HPC validation more portable, reproducible, and maintainable across sites and architectures.
Workloads define what is built and executed.
Site configuration defines how it is executed in a specific environment.
Hardware description (if introduced) defines what architecture is being exercised.
These concerns must remain structurally independent.
Portability begins with separation of responsibility.
Workloads must not encode:
- scheduler assumptions
- module names
- filesystem paths
- node topology
- device counts
If a workload cannot move across sites without modification, the boundary has failed.
HPCPerf is built on ReFrame, an established and widely adopted framework for HPC system testing and validation.
ReFrame provides:
- scheduler abstraction
- environment management
- dependency handling
- sanity and performance checks
We deliberately align with ReFrame as a community standard rather than inventing new orchestration machinery.
At the same time, benchmark meaning remains conceptually separate from execution mechanics.
Workloads define what is being tested.
ReFrame defines how it is executed.
ReFrame is the execution foundation - not the semantic definition of the benchmark itself.
The framework favors:
- explicit configuration
- declarative structure where possible
- minimal hidden state
- predictable behavior
Magic reduces portability.
Clarity scales.
This repository is intentionally small.
Features such as:
- formal hardware targets
- tag taxonomies
- declarative benchmark specifications
- result schemas
may emerge later - but only after structural boundaries are validated.
Premature formalization creates rigidity.
HPCPerf exists to:
- test structural ideas
- encourage discussion
- reduce cross-site friction
- provide a concrete reference model
It does not claim authority.
If the ideas prove useful, they can evolve collaboratively.
If better ideas emerge, that is success too.
The repository is early-stage and experimental.
Structure and APIs may change.
No stability guarantees are provided.
ORNL
- Nick Hagerty
AMD
- Jakub Kurzak
- Ian Bogle