CI runs the convention checks, the Rust gates, and the tests on every push and
pull request. How the workflows are laid out, and why security.yml is
separate, is CONTRIBUTING.md §3.
These are the gates, as CI runs them:
./ci/lint.sh # repo conventions; no toolchain needed
cargo fmt --all -- --check
cargo deny check licenses advisories bans sources # needs cargo-deny 0.19+
cargo machete --with-metadata # needs cargo-machete
cargo clippy --workspace --all-targets --locked
cargo clippy -p pipette-plan-types --all-targets --locked
cargo doc --workspace --no-deps --locked
cargo nextest run --workspace --locked # or: cargo test --workspace --locked
cargo nextest run -p pipette-plan-types --locked
cargo test --workspace --doc --locked
(cd ui && npm ci && npm run check && npm test && npm run build)
cargo run -p pipette-mgmt --locked -- openapi --site target/api-docs # the API reference pageRun these exact invocations locally. The flags are not cosmetic: without
--all rustfmt skips other crate targets, without --all-targets clippy
skips tests, and without --locked the build is free to resolve
dependencies differently than CI does. --workspace names the member set
explicitly. The root manifest is virtual and declares no default-members, so
a gate run from a crate directory would cover that crate alone. A looser local
command can pass while CI fails.
Two test sets need more than the command:
-
The tests that run on PostgreSQL need a server at
TEST_DATABASE_URL. These arepipette-store-sql's tests, the tests built onseeded_sql_stateorseeded_queue_stateinpipette-web-testkit, and every test that runsqueue-maintenance. An example address ispostgres://postgres@127.0.0.1:5432/postgres. Unset, they skip locally and fail in CI. Each test makes and drops a schema of its own in apipette_testdatabase, which the first test creates and later runs reuse.scripts/dev-postgres.shstarts the server that CI uses, in a Docker container, and prints the line that setsTEST_DATABASE_URL. Runexport_line=$(scripts/dev-postgres.sh) && eval "$export_line"to start it and set the variable in one step. SetPORTto use a port other than 5432. A second run reuses the container if its settings match CI. If its authentication, port binding, or initialized locale differs, the script tells you to remove it. To remove it, rundocker rm -f pipette-test-postgres. -
The
pipette-store-s3counter tests read process-wide counters, so run them under nextest or withcargo test -p pipette-store-s3 -- --test-threads=1; plaincargo testfails them.
The opt-in recalculation load test needs PostgreSQL and several GB of free RAM. The test uses these settings:
- An in-memory S3 submission store.
- A pool of 32 SQL connections, with the production acquire timeout of 10 seconds.
- 256 results of 30 MiB each.
cargo test -p pipette-pipeline --locked large_recalculation_pool_reproduction -- --ignored --nocaptureTo run a smaller test, set PIP_RECALC_JOBS (the number of jobs) and
PIP_RECALC_MB (the size of each result in MiB). The test prints the
recalculation duration apart from the seed time.
To measure the peak memory:
- Build the test with
--no-run. - Run the test executable under
/usr/bin/time -lon macOS, or under/usr/bin/time -von Linux. - Give the same test filter,
--ignored, and--nocapture.
Run the same test on the baseline revision and on the changed revision. Report the pool errors, the elapsed time, and the peak memory.
What each gate is for:
./ci/lint.shruns every check inci/checks/: the repo conventions that rustfmt, clippy and rustdoc cannot express (import grouping, the anyhow import rule, theErrorsuffix on error types, redacting secret newtypes, Rust paths cited in the docs, and the ban on em dashes in prose and user-visible copy). It needs no Rust toolchain and runs on every PR, including a docs-only one. See ci/README.md, which also covers adding a check.- The
ui/commands type-check, unit-test and build the Svelte app (§3). The release binary embeds that build. - The two
-p pipette-plan-typesruns buildpipette-plan-typeswithout itsschemarsfeature, which derives a JSON Schema forClientRunSpecand the types it reaches.pipette-pipelineturns it on for the app's generated TypeScript, so every--workspacerun builds it on, and these two cover the default a client builds. - No
-D warningsanywhere. Lint levels live in[workspace.lints]in the rootCargo.toml, which each crate opts into with[lints] workspace = true, so a localcargo clippyenforces exactly what CI does. Those tables also ban the panicking constructs:unwrap,expect,panic!,todo!andunreachable!are denied in every crate here, tests included. A reachable failure returns a typed error, and a test is written-> anyhow::Result<()>and propagates with?. Where a value really is valid by construction,types::id_from_validis the one sanctioned place that says so. - The HTTP tier (
pipette-httpapi) also bansserde_json::json!, through itsclippy.toml: a response is a#[derive(Serialize)]type, so the wire shape is written down once. Tests are exempt, since a test builds the request a client would send, malformed ones included. cargo denyis the dependency policy, all of it indeny.toml. An unlisted license fails the build; so does a RUSTSEC advisory, a crate banned by name, aversion = "*"requirement, and a dependency resolved from anywhere but crates.io. The file records why each ignored advisory is out of reach.cargo docis a gate, not a build step:[workspace.lints.rustdoc]denies broken intra-doc links, which nothing else in the build reads.- Dead code is gated in two halves, because no single tool sees both.
Within a crate,
warnings = "deny"makes rustc'sdead_codea build error, andunreachable_pubkeeps an item from escaping that lint by beingpubwherepub(crate)would do.dead_codestops at a crate'spubsurface, since rustc counts apubitem as used downstream; nothing here is published, so prefer private modules and narrow visibility. Whole dependencies arecargo machete's half: it fails on a crate kept past its last caller, which otherwise costs build time, aTHIRD-PARTY-LICENSES.mdentry and an advisory surface. It reads sources textually, so a dependency reached only through a macro expansion is its blind spot: list such a crate underignoredin[package.metadata.cargo-machete]in its own manifest, with a comment naming what reaches it, ascrates/pipette-plan-types/Cargo.tomldoes forregex.
gitleaks reads commits rather than the working tree, so a credential committed
once and deleted in the next commit is still found. Over the whole history it
takes minutes, so it is not worth running before every push. The push to main
and the daily cron scan everything; a pull request scans base..HEAD:
gitleaks git --redact --verbose --config .gitleaks.toml .
gitleaks git --redact --verbose --config .gitleaks.toml --log-opts="origin/main..HEAD" .--redact keeps the matched text out of the output, so a CI log is never a
second copy of the secret. A finding that is not a credential belongs in
.gitleaks.toml. History cannot be rewritten once it is
pushed, so every entry has to say why the string is not a secret, as deny.toml
says why it ignores an advisory.
The Python examples are linted and formatted with
ruff, scoped to examples/:
uv sync --group dev # install the dev tooling
uv run ruff check examples
uv run ruff format --check examplesCheck the pull request description before you ask for review (CONTRIBUTING.md §5):
./ci/pr-body.py --pr 1234 # or a file path, or - for stdinexamples/benchmarks/ contains the benchmark catalog (TOML definitions).
The ASR entries are not in it: they live in the deployment's catalog, and
benchmarks.md shows their shape for a
local sample_data/benchmarks/. Seed a storage root with it and start the
server:
mkdir -p sample_data # storage root (gitignored)
cp -r examples/benchmarks sample_data/benchmarks # seed the catalog
cargo run --bin pipette-mgmt -- --config examples/config.toml serveThe asr_eval entries are organization data and are not in this repository
(benchmarks.md §2.12): a deployment
puts each into its organization's private catalog with
pipette-mgmt orgs benchmarks put <org> <file>, never into the public one.
The warehouse starts empty and fills as submissions are scored.
examples/notebooks/ holds a ready-made notebook for exploring it.
/app/ serves the Svelte app in ui/; ui/README.md covers
how it is served, how it signs in, and its pages. A checkout that has not run
npm run build still compiles, and /app/ then serves a page saying the app
was not built, so the Rust gates never need Node. PIPETTE_REQUIRE_UI=1 turns
that into a build error; the release build and docker/build.sh set it.
To work on it, run a local pipette-mgmt serve and the Vite dev server, which
proxies the API routes to it (PIPETTE_MGMT_URL, default
http://127.0.0.1:3000):
cd ui
npm ci
npm run dev # http://localhost:5173/app/WebCrypto signs only in a secure context, which localhost is.
npm run check type-checks with TypeScript 7 (svelte-check --tsgo).
svelte-check still loads TypeScript 6 itself, so both are installed, 7 under
the alias @typescript/native.
The app's request and response types are generated, not written.
pipette_httpapi::api_schema derives JSON Schemas from the Rust bodies in
pipette-plan-types into ui/src/lib/api-schema.json (a request from its
deserialize side, so a field with a default is optional), npm run types turns
that into ui/src/lib/api-types.ts, and both files are committed. After
changing a type the app reads:
UPDATE_API_SCHEMA=1 cargo test -p pipette-httpapi --lib api_schema
(cd ui && npm run types)A Rust test fails while the JSON is stale and the console-ui job while the
TypeScript is. The HTTP tests also check the bodies the app reads against the
schemas, which catches a hand-written Serialize that drifts from its declared
shape.
The bundle refuses a name two types share, which schemars would otherwise split
as Foo and Foo2, and a request and a response of one name with different
schemas. Two types are told apart with #[schemars(rename = "…")] on one. One
type the app both sends and reads, whose deserialize side differs (a field with
#[serde(default)]), needs a request-side mirror type in api_schema under its
own name, since a rename would move both sides.
The npm packages the bundle carries are credited in THIRD-PARTY-LICENSES.md,
and CI checks it (third-party-attribution.md).
The operator API is also an OpenAPI 3.1 document
(httpapi.md §1.5), built from the same
body types. Its route table,
crates/pipette-httpapi/src/api_schema/openapi/table.rs, is held to router.rs
by a unit test, and test_app checks every /api/v1 answer the HTTP tests
receive against the document, so an undeclared status (a 5xx aside) or a body
its schema refuses fails the test that met it.