v1.0.0 readiness assessment, and six gates that could report a pass they had not earned

[ChelseaKR]

An audit pass against the v1.0.0 question: read the open issues and PRs, check
the validator's own falsifiability rule by rule, fix the real defects, and name
what is blocked. Every guard added or repaired was broken on purpose, watched
fail, restored, and watched pass; both directions are in the commit messages.

Three things to read first

1. The v1.0.0 readiness assessment is in docs/plans/v1.0.0-readiness.md

Verdict: not ready, and nothing remaining is engineering in this repository.
There was no single v1.0.0 checklist but four overlapping ones that disagree
about the world, so the assessment merges them and marks where a document is
wrong about its own repo. Three items are left:

1. A conformance-only release has not shipped (A2). v0.10.0 disqualified
   itself in its own release note: it added TODS-E207 and changed
   coverage-manifest behavior in three commands. ## Unreleased changes
   behavior again, so the earliest qualifying release is two away. Maintainer.
2. The upstream MobilityData PR 156 is still open (A4; that is MobilityData's PR number, not this repository's v1.0.0 readiness assessment, and six gates that could report a pass they had not earned #156, which is this pull request). Read live 2026-08-28:
   state=open, merged=false, last updated 2026-07-17. Freezing TODS-E204
   vs TODS-W408 today freezes this repo's reading of an issue thread rather
   than published text. Upstream, and the only blocker with no date attached.
3. No PyPI publishing environment exists (C8). gh api repos/ChelseaKR/tods-validate/environments lists exactly one environment,
   github-pages. There is nothing to scope a trusted publisher to.

2. Three documents are wrong about the branch ruleset

Read live on 2026-08-28: ruleset protect-main (id 18752857) is active on
refs/heads/main. DEFINITION_OF_DONE.md ("Not yet enabled as a live GitHub
ruleset") and docs/v1-contract-audit.md ("no ruleset is enabled on the
repository") both say otherwise. And the live one is not the committed
docs/rulesets/main.json:

	Committed docs/rulesets/main.json	Live, read 2026-08-28
Name	main	protect-main
Rules	deletion, non_fast_forward, required_linear_history, pull_request, required_status_checks	deletion, non_fast_forward, required_status_checks
Required checks	15	10
Missing live		accessibility, action-self-test, citation, contract, perf

Two consequences:

• contract is not a required check. The gate that protects the v1 public
  contract can be red on a pull request that merges. For a v1.0.0 story that is
  the one required check that actually matters.
• Applying the committed payload is not a no-op. The names differ, so gh api ... /rulesets with that body creates a second ruleset rather than
  updating the live one, and it replaces the merge requirements wholesale. Its
  pull_request rule sets required_approving_review_count: 1 with
  require_last_push_approval: true and require_code_owner_review: true; on a
  repository whose CODEOWNERS names one person, that is unsatisfiable for a
  solo maintainer.

Recommendation: export the live ruleset and diff it before applying
anything. This is a documentation defect, not a request to change a setting.
No live setting was changed by this pass and none should be changed on the
strength of a file that does not describe what is there.

3. v0 is a lightweight tag pointing at the v0.5.0 commit

v0 resolves to 097427f, the v0.5.0 release commit. Anyone following the
Actions convention and pinning ChelseaKR/tods-validate@v0 today gets software
four releases old, including a fixed companion-GTFS fail-open. Deleting a
published ref is a live action and out of scope here; the gaps ledger records
it and the two commands. Worth doing before v1.0.0 either way, because v1
will invite exactly the same pin.

The falsifiability census: 43/43 clean

Enumerated by AST, not regex -- a [A-Z_]+ pattern misses TODS-E207.

• 43 @rule(...) registrations and 43 distinct literal Finding(rule_id=...)
  values, sets identical, and zero non-literal rule_id construction
  sites, so no rule ID can be assembled at runtime and escape the census.
• All 43 fire on their own tests/fixtures/invalid/<ID>/ fixture; 0 silent.
• The valid fixture, every opt-in category enabled, emits nothing at all.
• For each rule, replacing its check with one that yields nothing made its
  finding disappear -- which is what rules out a finding produced by a
  different rule under the same ID.

Nothing in the rule set is decorative, and it was true for a structural
reason: the fixture-per-rule parity assertion is not something a passing
implementation can satisfy by accident. One residual is stated in the
assessment (B7): the additional rule IDs in each fixture's expected set were
seeded from a run and are reviewed by eye; the load-bearing assertion is not.

The defects found and fixed

Commit	Defect
4647e38	gitleaks --source . scans history, not the working tree. A root-level file holding an AWS key pair, a GitHub PAT and a Slack bot token, never staged, gave "283 commits scanned / no leaks found", exit 0. With --no-git: "leaks found: 1", exit 1. The recipe's own comment claimed "working tree + history".
101cffb	No py.typed. A five-line consumer running mypy --strict against the installed package got Skipping analyzing "tods_validate": ... missing library stubs or py.typed marker, exit 1. Freezing a semver promise on an API every downstream type checker refuses to look at.
897fd0e	mypy files = ["src"] while ruff covered src tests scripts, leaving the gate scripts themselves the least verified code here. 34 files checked becomes 44; one real error in spec_watch.py, where -> str was a promise mypy had never checked.
9041979	The perf gate's failure was unreachable from below. It discarded run(feed)'s result and divided an assumed row count by CPU time, so a validator that had stopped reading the feed would burn no CPU, report an enormous rate, and pass further inside the budget than a correct one. Every existing test stubs measure out, which is why nothing noticed.
fa3d92a	A conformance claim for a flag that does not exist. The README asserted Observability Tier C and named --log-format json; the flag is nowhere in src/, and no module imports logging at all, so there is no stream to format.
2dba719	TODS-I601's 6h bound was never exercised by its 8h fixture. Moving the constant to seven hours left the conformance test passing, exit 0.
8c5c2f3	Two pull_request jobs (perf, and the path-filtered VS Code package job) had no make equivalent and no mention in the header that enumerates CI-only work.
3e4d01c	read.to_dataframe and __version__ are in the v1 contract and were named in none of the 52 test modules. A 90% coverage floor cannot see an export nothing imports.
5899608	docs/api.md described Finding as 7 fields and 2 helpers; it has 10 and 3, and report.schema.json requires the three omitted plus fingerprint(). docs/read-api.md had no currency stamp, so make docs-check had nothing to fail on.

Verified

PATH=.venv/bin:$PATH make verify < /dev/null; echo "EXIT=$?"
EXIT=0

13 of 13 gates PASS, 701 tests (from 664), coverage 91.80% against a
90% floor. The AST rule census and falsifiability harness were re-run after
every change and stayed 43/43.

Note for anyone reproducing this: .venv/bin must be on PATH, which is
what CI does (ci.yml writes it to $GITHUB_PATH). Without it six gates fail
for environment reasons on a laptop. Not a repository defect, but a sharp edge:
CONTRIBUTING.md's local instructions and CI's PATH step describe two
different setups.

Base and merge notes

This branch is cut from a019bbe (v0.10.0), which is 5 commits behind
main (7a25056). a019bbe is an ancestor of main, so the diff here is
exactly these 11 commits. It does not merge cleanly, and the conflicts are
measured rather than guessed -- git merge-tree --write-tree origin/main HEAD
reports exactly three files:

• CHANGELOG.md -- both sides add to ## Unreleased. Concatenate.
• Makefile -- the secrets recipe and the CI-only header paragraph, both
  of which docs+fix: a multiyear phase plan, and phase 1 (three gates could report a pass they had not earned) #151/phase 2: everything that must be true before v1.0.0 (not the release) #152/phase 3: measure the two axes nobody was measuring, audit the pages nobody was auditing #153 also touched. Take both sets of changes; the header
  paragraph is now pinned by tests/test_ci_gate_parity.py, so a wrong
  resolution turns that test red rather than passing quietly.
• scripts/check_doc_currency.py -- the STAMPED list. Union the entries;
  tests/test_doc_currency.py now derives its parametrize from STAMPED, so a
  dropped entry is visible.

scripts/spec_watch.py auto-merges, contrary to the note in the working
plan that expected it to need a hand re-apply. docs/rulesets/main.json, which
arrived with #152, does not exist in this checkout and nothing here edits it,
deliberately, so no conflicting duplicate is created; the divergence is written
up in the readiness assessment instead.

Rebasing was left to the maintainer rather than done here, because every
measurement in this PR (701 tests, 91.80%, 43/43 census) was taken against
a019bbe and a rebase would invalidate the evidence without re-running it.

Every candidate finding was re-checked against git show origin/main:<path>
before being called a defect. Two fail-opens found during the audit turned
out to be already fixed on main by #151 (spec_watch.py treating an
unrecognised document as "in sync"; check_npm_audit.py disarming its own
cross-check on an unparseable report) and are not re-fixed here.

PRs #154 and #155 (stacked) and draft #79 were read and deliberately not
touched; #154/#155 already report CONFLICTING against main, and nothing
here edits a file they add.

Left undone, with the reason

• --log-format json. Adding structlog as a second runtime dependency to
  a tool that deliberately has one, to format log records that do not exist, is
  a product decision. Recorded as a gap with both options written out.
• tests/ under mypy. scripts/ was the half that gates merges.
• Any new or changed TODS rule. None was touched, so no spec text needed
  citing. Add a structure warning rule (TODS-x1xx) for a recognized-but-unexpected file #143 and Add a second advisory rule (TODS-x6xx, opt-in) #144 both need a citation chosen and defended, and inventing
  one to close an issue would be the exact failure this validator exists to
  refuse.

Two issue texts did not survive checking: #145 is already fixed (landed in
#152; close it against that PR), and #146's premise predates #150, which
now boots the live page in a real browser -- what remains is a dated human
record, narrower than the issue says.

🤖 Generated with Claude Code