Repository navigation
Run CI tests with cargo-nextest - #203
Conversation
Each test gets its own process and a per-test timeout, so a stuck or swapping test fails on its own instead of taking the job to its limit. The production-circuit test group keeps the two ~12 GB builds serial. The tests crate's Once-based artifact fixtures stay on cargo test, as do doctests, which nextest does not support. Co-authored-by: Cursor <cursoragent@cursor.com>
n13
left a comment
There was a problem hiding this comment.
Verdict: Request changes
Reviewer: GPT-6.1 Sol
Reviewed head e7e95abc165fcaf8768a0d56b67c784f322b1357 against base 62459294753db7a669562aab488f09163321a2dd.
[P2] Preserve the feature-enabled GPU doctests — .github/workflows/ci.yml:78.
The previous cargo test -p qp-wormhole-gpu --features constraint-export also ran the two compile_fail doctests guarding ProofEncoder::upload and ProofEncoder::copy against overwriting FixedFieldSlice (runtime.rs:963, runtime.rs:1010). Nextest does not run doctests, and the retained workspace --doc command does not enable wgpu/constraint-export: the GPU crate has no default features and gates its runtime module on wgpu. Neither the new aggregator doctest command nor the analysis jobs restores this coverage. Consequently CI would stop detecting an API regression that permits writes to fixed circuit buffers.
Reproduced locally: cargo test --locked -p qp-wormhole-gpu --doc -- --list finds 0 tests, while cargo test --locked -p qp-wormhole-gpu --features constraint-export --doc runs and passes both compile-fail tests. Add cargo test --release --locked -p qp-wormhole-gpu --features constraint-export --doc alongside the converted GPU invocation to preserve the existing checks.
Validation:
git diff --check, repository Taplo checks, CI YAML parsing and locked Cargo metadata passed.- With pinned nextest 0.9.151, the workspace filterset and previous libtest skip selection match exactly: 275 selected tests.
show-config test-groups --profile ci --run-ignored allplaces both production guards inproduction-circuitwithmax-threads = 1. - Focused debug runs under nextest's
ciprofile: GPU 27 passed, 23 skipped; encoding safety/spec differential 19 passed. The feature-enabled GPU doctest run above passed 2 tests. - Full release and production-circuit runs were not repeated locally. GitHub's workspace jobs on Ubuntu/macOS and slow-test job were still in progress; the completed checks were successful.
The compile_fail doctests on ProofEncoder::upload/copy only exist with the gpu crate's features enabled, which the workspace --doc run does not do. CodeQL flags the unpinned taiki-e/install-action@v2 tag. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Addressed in c396c38:
First CI run on e7e95ab was otherwise fully green: macOS test-workspace 17m32s, ubuntu 17m05s, slow-tests 23m27s. |
n13
left a comment
There was a problem hiding this comment.
Verdict: Approve
Reviewer: GPT-6.1 Sol
Reviewed head c396c386680b5a96d1ce84411531e58c9a6e3fc7 against base 62459294753db7a669562aab488f09163321a2dd, including the full diff and previous review discussion. No blocking findings remain.
The previous GPU doctest coverage finding is resolved: the explicit feature-enabled cargo test --doc command runs both ProofEncoder::upload/copy compile-fail tests. Both install-action uses now pin the verified v2.90.3 commit. The other converted invocations preserve their target and feature selection, while the artifact-sharing tests remain on libtest.
Validation in an isolated detached worktree:
git diff --check, repository Taplo formatting, CI YAML parsing, and locked Cargo metadata passed.- With pinned nextest 0.9.151, compared the workspace filterset with the previous libtest
--skipselection by enumerating the actual test binaries: exactly the same 275 tests are selected. show-config test-groups --profile ci --run-ignored allconfirms both production guards inheritproduction-circuitwithmax-threads = 1.- Focused debug runs using nextest's
ciprofile: GPU 27 passed, 23 skipped; encoding safety/spec differential 19 passed. cargo test --locked -p qp-wormhole-gpu --features constraint-export --doc: 2 passed.
Full release and production-circuit runs were not repeated locally. At posting, the current head's workspace jobs on Ubuntu/macOS, slow tests, and security audit were still running; format, Clippy/doc, formal spec/bridge, and CodeQL checks had passed. The earlier head's complete CI run also passed, but that is separate from the current head's pending results.
Summary
Switches the CI test runs to cargo-nextest. Motivation: on #202 a single memory-heavy test swapped on the macOS runner and, because
cargo testruns all tests in one process, dragged every other test with it until the job's 30-minute limit killed the whole step with no per-test attribution.With nextest:
.config/nextest.tomlgives a per-testslow-timeout(flagged at 60 s, killed at 15 min in theciprofile), so a hang fails as a named test rather than a job timeout;production-circuittest group (max-threads = 1) pins the two#[ignore]d ~12 GB production-circuit builds so they never run concurrently, replacing the hand-written--test-threads=1inslow-tests;--skip a --skip b …lists become one-Efilterset.What changed
.config/nextest.toml—defaultandciprofiles, theproduction-circuitgroup and its override..github/workflows/ci.yml—taiki-e/install-action@v2installscargo-nextest@${NEXTEST_VERSION}(pinned0.9.151inenv) intest-workspaceandslow-tests; thecargo testinvocations becomecargo nextest run --profile ci …with equivalent target/filter selection.README.md— one paragraph noting CI uses nextest.Kept on
cargo test:--docruns (nextest does not support doctests; the aggregator run inslow-testsgained an explicit--docstep so no coverage is lost);cargo test -p tests -- aggregator_tests …inslow-tests: that crate generates its circuit artifacts once per process withstd::sync::Once, which under process-per-test would be regenerated for every test.Testing (local, M4)
Every converted invocation run under nextest with the same pass counts as under
cargo test:--lib --binswith the skip filterset-p tests --test encoding_safety --test spec_differential-p qp-wormhole-circuit --test heap_zeroization --test formal_trace-p qp-wormhole-gpu --features constraint-export-p qp-wormhole-aggregator --features gpu --lib -E 'test(gpu)'--features gpu --example public_batch_compare--run-ignored onlyproduction guards (serial via group)Back-to-back on the aggregator lib (the dominant binary):
cargo test95.0 s vs nextest 97.0 s.cargo nextest show-config test-groups --run-ignored allconfirms both production tests land inproduction-circuit.taplo format --checkpasses on the new TOML.