Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/idea/06-performance.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Every release must run, and must not regress on:
4. Swarm boot time, Rails-app baseline.
5. Memory per worker post-boot, Rails-app baseline.

The benchmark job runs in CI on Blacksmith. Results uploaded as artifacts. PR comment shows delta vs main. Greater than 5% regression flags the PR.
The benchmark job runs in CI wherever the `vars.WURK_BENCH_RUNNER` repository variable points (`ubuntu-latest` when unset). Results are published to the job summary and a sticky PR comment showing the delta vs main. Greater than 5% regression flags the PR.

## What actually gates a merge

Expand Down
20 changes: 10 additions & 10 deletions docs/idea/11-testing-ci.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,24 +33,24 @@ For each public class we implement, the equivalent test from upstream Sidekiq is

A dedicated CI job runs the test suites of widely-used Sidekiq ecosystem gems (sidekiq-cron, sidekiq-unique-jobs, sidekiq-scheduler, etc.) against Wurk. See `14-ecosystem-compat.md`. These are the strongest possible drop-in proof.

## CI: GitHub Actions on Blacksmith runners
## CI: GitHub Actions

Blacksmith (https://blacksmith.sh) provides faster runners than stock ubuntu-latest with better caching. We use Blacksmith for:
Runner selection is a repository variable, not a hard-coded label: `vars.WURK_CI_RUNNER` for the detect, test, parity, lint, frontend, and ecosystem jobs, `vars.WURK_BENCH_RUNNER` for the benchmark job. Both fall through to stock `ubuntu-latest` when unset, and fork PRs are pinned to `ubuntu-latest` unconditionally. Release, pages, and dependabot workflows stay on `ubuntu-latest` by design (credential containment). CI runs:

- Test matrix (Ruby × Redis × Rails versions)
- Test suite (one full run on the newest Ruby + newest Rails, coverage gate folded in — no version matrix)
- Ecosystem compat suite
- Benchmark suite
- Docs site build
- Docs site build (pages.yml, `ubuntu-latest`)

The test workflow runs the matrix on a 4-vCPU Blacksmith runner. Each matrix cell:
The test workflow's suite job:

- Checks out the repo.
- Sets up Ruby via Blacksmith's setup action with bundler cache.
- Boots Redis and Postgres service containers.
- Sets up Ruby via `ruby/setup-ruby` with bundler cache.
- Boots a Redis service container.
- Runs the dummy app setup.
- Runs the full Minitest suite in parallel mode.

Benchmark job runs on an 8-vCPU Blacksmith runner and uploads results as artifacts. A bot comments deltas vs main on every PR. Regressions greater than 5% flag the PR.
The benchmark job runs wherever `vars.WURK_BENCH_RUNNER` points and publishes the delta vs main to the job summary and a sticky PR comment, on PRs that touch a bench input (`lib/`, `exe/`, `bench/`, `bin/bench-compare`, the Rakefile, Gemfile/gemspec, or the workflow itself). Regressions greater than 5% flag the PR.

## Coverage

Expand All @@ -60,8 +60,8 @@ SimpleCov with branch coverage. CI fails when branch coverage on the gem's main

Before a tag is cut:

- Full matrix green.
- Ecosystem compat matrix green.
- Test suite green.
- Ecosystem compat suite green.
- Benchmark suite reports no regressions vs the previous tag.
- Precompiled assets bundle is fresh.
- Parity test SHA pin matches the latest Sidekiq main we've reviewed.
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# 08 — Bench harness, CI gate, final verification

> **Executed; one instruction superseded (2026-08-21).** This plan step predates #430, which removed
> hard-coded runner labels from CI. Step 1's `blacksmith-8vcpu-ubuntu-2404` instruction and its
> "Per CI standard (CLAUDE.md)" citation are stale: `CLAUDE.md` now states runner selection is a
> repository variable (`vars.WURK_BENCH_RUNNER` for bench, `ubuntu-latest` fallback) — do not
> hard-code a runner label on the bench job. `bench.yml` already exists on `main`. Kept unedited
> below as an executed-plan record.

> Part of [`overview.md`](overview.md). Depends on: 02–07 (this is the proof). Do the gate restoration (step 1) FIRST, before any perf commits land, so deltas attribute per commit — it's independent of the other slices.

## Files to change
Expand Down