Skip to content

Latest commit

 

History

History
46 lines (33 loc) · 2.73 KB

render-bench.md

File metadata and controls

46 lines (33 loc) · 2.73 KB
type Attested Computation
title The render bench
description Proves that a fidelity tier holds its frame-time budget on a machine, and records what was drawn.
status draft
tags
oracle
rendering
benchmark
runtime shell
parameters
name type required
tier
string
true
executor
resource receipt
bench
tier
camera
without
commit
machine
resolution
rendered
assets
frames
frame_ms
budget_ms
over_budget_percent
screenshots
attester
resource
generated
by at
claude-code/claude-opus-5-5
2026-11-12T02:00:00Z

Computation

cargo run --release -p osage-kart-native -- --bench --tier {tier}

The run writes report.json and four screenshots; report.json is the receipt. How the bench makes every machine draw the same frames is described in timedemo bench.

Tiers T3 and T4 also run on a cloud GPU for the changes merged into main, at most once an hour (D-042, D-043): the workflow cloud-bench builds the bench, runs it with --headless on a Google Cloud machine with an NVIDIA L4, and attests the receipts it brings back (tools/cloud-bench). That machine has no checkout, so the commit is compiled into the build (OSAGE_COMMIT) instead of asked of git.

What it proves

A claim such as "T2 holds 60 frames a second on an integrated GPU" is accepted with a receipt, not on a statement. The attester reads the receipt and answers pass only if:

  • it comes from the sanctioned bench (bench is kart-lap/3: version 3 races AI drivers with temperaments and contacts with momentum; version 2 let karts overtake instead of running into each other; frames differ between versions) run for the claimed tier;
  • the camera followed the race (camera is race), not held at a place;
  • no technique was left out and at least 2000 frames were timed;
  • it was run from a committed state of the code (commit does not end in -dirty);
  • the mean frame time is within the tier's budget.

It also reports the share of frames over budget, which the mean hides.

python3 tools/attest-bench.py bench/out/<run>/report.json

What it does not prove

That the picture is right: compare the screenshots, which are taken at the same frames in every run — the report's race is the same on every run that raced the same frames, on any machine and at any tier (D-116). That another machine does as well: a receipt names its GPU and driver. Times taken while other programs run are pessimistic.

Used by

Every technique of osage-render, for its cost and for its tier's budget.