fix(circuits): bench each protocol circuit under the scheme its VK uses - #25507
Merged
Merged
Conversation
AztecBot
force-pushed
the
cb/bench-scheme-classification
branch
from
September 18, 2026 13:51
67bc356 to
8783d32
Compare
bench_cmds selected the bb scheme for each artifact by grepping its name against ivc_patterns,
an array that no longer exists: the chonk patterns are read into ivc_kernel_patterns and
ivc_app_patterns. An unset array expands to one empty line, and an empty pattern in a grep -f
file matches every input, so every artifact was benched with --scheme chonk. bb rejects that
for the rollup circuits ("Rollup Honk proof type not supported on MegaBuilder"), and run_bench.sh
piped the failure into jq without pipefail, so each rollup bench passed in zero seconds and
wrote an empty bench file. The rollup gate series have been absent from the bench since.
VK generation and the bench now classify a circuit through one function, circuit_kind, and each
turns the kind into its own bb flags. run_bench.sh fails on a bb error instead of hiding it, and
uses the bb the build keys the circuits with, so the public tx-base circuit, which verifies an
AVM proof, measures under bb-avm rather than failing under the non-AVM binary.
The classifier only runs in the build for a VK that misses the cache, so a test pins the flavor
each circuit family lands on, keyed on the script and the pattern files it reads.
AztecBot
force-pushed
the
cb/bench-scheme-classification
branch
from
September 18, 2026 14:12
8783d32 to
8165b2c
Compare
iAmMichaelConnor
marked this pull request as ready for review
September 18, 2026 14:15
iAmMichaelConnor
requested review from
IlyasRidhuan,
LeilaWang,
iAmMichaelConnor,
iakovenkos and
ledwards2225
as code owners
September 18, 2026 14:15
iAmMichaelConnor
approved these changes
Sep 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The rollup circuits have not been benched since the per-kind VK change in May.
bench_cmdsinnoir-projects/fnd/noir-protocol-circuits/bootstrap.shpicks thebb gatesscheme for each artifact by grepping its name againstivc_patterns. That array does not exist: the chonk patterns are read fromchonk_circuits.jsonintoivc_kernel_patternsandivc_app_patterns, andwrite_vkmatches them through the split regexes, whilebench_cmdskept the old name. An unset array expands to one empty line, and an empty pattern in agrep -ffile matches every input, so the first branch always wins and every artifact is benched with--scheme chonk.bb rejects that for the rollup circuits, and
run_bench.shpipes the failure intojqwithoutpipefail, so the command exits 0 and writes an empty bench file thatbench_mergethen drops. In the last merge-queue bench run (engine log) everyrun_bench.sh ./target/rollup_*.json --scheme chonkline isPASSED (0s); the rollup_tx_merge sub-log is justRollup Honk proof type not supported on MegaBuilder. The chonk circuits happen to land on the right scheme, so the kernel series were unaffected.Surfaced while reviewing #25506, which relies on the bench to show reset-variant cost changes.
Change
circuit_kindmaps an artifact name tohiding,kernel,app,rollup_honk,rollup_rootorultra_honk, in the orderwrite_vkalready applied.write_vkandbench_cmdseach turn the kind into their ownbbflags:write_vkkeeps--circuit_kindfor the chonk kinds and--oracle_hash keccakfor the root, neither of whichbb gatestakes. The combinedivc_regexexport had no remaining caller and is removed.run_bench.shruns underset -euo pipefail, so abberror fails the bench command and, through the bench engine's fail-fast, the bench run. It also resolvesbbthroughfind-bb, the AVM-enabled binary the build keys the circuits with, instead of the hard-coded non-AVMbuild/bin/bb:rollup_tx_base_publicverifies an AVM proof and the non-AVM binary refuses it.Effect on the bench
The rollup circuits'
_gatesand_opcodesseries reappear. The kernel, app and hiding series are unchanged.Tests
scripts/circuit_kind.test.shpins the flavor each circuit family lands on, including the mock circuits' names, which share this script. It is wired intotest_cmds, keyed on the bootstrap script, the test and the two pattern files it reads, so CI runs it undernoir-protocol-circuits-tests. It exists because the build only runs the classifier for a VK that misses the cache: on a warm cache (example) every VK is downloaded andwrite_vknever executes, so a misclassification would otherwise surface on the next cold-cache build.6.0.0-nightly.20260902artifacts with the releasebbof the same version: every non-simulated artifact measured under the flags the fixedbench_cmdsassigns it. 49 of 50 succeed; the one failure isrollup_tx_base_publicunder the non-AVM binary ("AVM recursion is not supported in this build"), which is what thefind-bbchange addresses. Thebench-foundationtarget buildsbb-cpp-native, which producesbb-avmalongsidebb.run_bench.shexercised directly: a correct invocation writes the two-entry bench file; a wrong-scheme invocation now exits 1../bootstrap.sh bench_cmdsagainst placeholder artifacts covering every class plus a_simulatedone: simulated skipped, chonk kinds get--scheme chonk, rollup honk gets--scheme ultra_honk --ipa_accumulation, root and the rest get--scheme ultra_honk.bash -non both scripts.Created by claudebox · group:
slackbot· requested by Mike (@iAmMichaelConnor) · Slack thread