v1.0.0 readiness assessment, and six gates that could report a pass they had not earned

.github/workflows/ci.yml

@@ -206,8 +206,11 @@ jobs:
         if: ${{ !cancelled() }}
         run: make npm-audit
 
-  # Secret scan over the PR/push diff and full history (SEC-17/18). Installs
-  # the gitleaks CLI directly (checksum-verified) rather than the third-party
+  # Secret scan over the full history and, separately, the checked-out tree
+  # (SEC-17/18). `make secrets` runs both: a commit scan cannot see a file that
+  # was never committed, and the tree scan cannot see a secret that was
+  # committed and later deleted. See the Makefile recipe. Installs the gitleaks
+  # CLI directly (checksum-verified) rather than the third-party
   # `gitleaks/gitleaks-action`, which is license-gated for organizations.
   secrets:
     runs-on: ubuntu-latest

.gitleaks.toml

@@ -0,0 +1,40 @@
+# gitleaks configuration for the `make secrets` gate (SEC-17/18).
+#
+# `make secrets` runs two scans, because they answer different questions and
+# neither answers the other's:
+#
+#   gitleaks detect --source .            -> every commit reachable from HEAD
+#   gitleaks detect --no-git --source .   -> the files on disk right now
+#
+# The history scan is blind to a secret that has been written but not yet
+# committed, which is exactly the state a working tree is in when a gate runs
+# against it. Measured on this repository: a file containing an AWS key pair, a
+# GitHub PAT and a Slack bot token, saved at the repository root and never
+# added to the index, produced "no leaks found" and exit 0 from the history
+# scan and "leaks found: 1" and exit 1 from the --no-git scan.
+#
+# This file exists so the second scan has a stable scope. Without it the
+# working-tree scan also walks .venv/ and node_modules/, which the release
+# verification workflow populates before it runs `make verify`: roughly 29 MB
+# of third-party code whose test fixtures are a standing source of findings
+# this project cannot fix. Paths below are build output and installed
+# dependencies, never project source.
+
+[extend]
+useDefault = true
+
+[allowlist]
+description = "Build output, installed dependencies and tool caches"
+paths = [
+    '''(^|/)\.git/''',
+    '''(^|/)\.venv/''',
+    '''(^|/)node_modules/''',
+    '''(^|/)\.mypy_cache/''',
+    '''(^|/)\.ruff_cache/''',
+    '''(^|/)\.pytest_cache/''',
+    '''(^|/)\.hypothesis/''',
+    '''(^|/)\.worktrees/''',
+    '''(^|/)dist/''',
+    '''(^|/)htmlcov/''',
+    '''(^|/)__pycache__/''',
+]

CHANGELOG.md

@@ -113,6 +113,45 @@ Fixed:
   the honest answer to "the page this repository publishes", and which goes
   green again on the deploy that resolves any real drift.
 
+- The secret-scan gate could not see the working tree. `make secrets` ran
+  `gitleaks detect --source .`, which walks commits, under a comment claiming
+  it covered "working tree + history". Measured: a file at the repository root
+  holding an AWS key pair, a GitHub PAT and a Slack bot token, saved and never
+  added to the index, gave "283 commits scanned / no leaks found" and exit 0;
+  the same tree with `--no-git` gave "leaks found: 1" and exit 1. The gate now
+  runs both scans, each reporting its own result, and neither can
+  short-circuit the other. `.gitleaks.toml` scopes the working-tree scan away
+  from `.venv/` and `node_modules/`, which `verify.yml` populates before it
+  runs `make verify`.
+- The performance budget could only fail for one reason, and doing less work
+  made it greener. `scripts/check_perf_budget.py` divided an assumed row count
+  (`trips * 2`) by CPU time and discarded the timed run's result entirely, so
+  a validator that had stopped reading the feed would have burned almost no
+  CPU, reported an enormous rate, and passed further inside the budget than a
+  correct one. Every repetition now counts the rows the loader actually
+  parsed, refuses to report a rate below the floor the generator writes, and
+  refuses when two repetitions disagree about the count.
+  `maxRegressionFactor` is also bounded: it was read unbounded from the same
+  file as the baseline, where a large enough value retires the gate rather
+  than loosening it.
+- `mypy` did not check the scripts that are the gates. `ruff` covered
+  `src tests scripts`; `mypy` had `files = ["src"]`, leaving
+  `check_public_contract.py`, `check_npm_audit.py`, `generate_rules_doc.py`
+  and `spec_watch.py` unchecked. Now `["src", "scripts"]`, 44 files. One real
+  error surfaced and is fixed: `spec_watch.py`'s spec fetch returned `Any`
+  from a function declared `-> str`.
+- `make verify` could be green on a tree CI rejects, without saying so. Two
+  `pull_request` jobs had no `make` equivalent and no mention in the Makefile
+  header that enumerates CI-only work: `perf`, and the VS Code extension
+  package job (path-filtered to `editor/vscode/**`, which is why it went
+  unnoticed). Both are named now, and `tests/test_ci_gate_parity.py` compares
+  the header to the workflows so the next one cannot go unnamed.
+- `docs/read-api.md` was outside the currency gate. It documents ten of the
+  nineteen names the v1 contract freezes and carried no `Last verified` stamp,
+  so `make docs-check` had nothing to fail on when it drifted. Added to
+  `STAMPED`, and `tests/test_doc_currency.py` now derives its parametrize from
+  that list instead of restating it.
+
 Changed:
 
 - `loader.py` now pools cell values per file: equal cells in one file share one
@@ -206,6 +245,42 @@ Added:
   a pin PyPI did not serve, so `micropip.install` rejected it for every
   visitor while every gate stayed green). Runs in `pages.yml` after each
   deploy and weekly in `playground-deployment.yml` (#146).
+- `src/tods_validate/py.typed`. The package declared no type information, so
+  every downstream type checker treated an installed `tods-validate` as
+  untyped and refused to look inside it: a five-line consumer importing
+  `validate_feed` got `Skipping analyzing "tods_validate": ... missing library
+  stubs or py.typed marker` and exit 1 from `mypy --strict`, and gets
+  "Success" now. `mypy --strict` has run over `src/` on every pull request
+  since 0.1.0 without any of that reaching a caller.
+- Tests for the two public exports nothing exercised.
+  `tods_validate.read.to_dataframe` and `tods_validate.__version__` are both in
+  `docs/v1-contract-candidate.json` and were named in none of the 52 test
+  modules, which a 90% line-coverage floor cannot see. Both are covered, and
+  `tests/test_contract_surface.py` adds the floor that finds the next one.
+- `tests/test_readme_claims.py`: every `--flag` the README names must exist in
+  the CLI, or be attributed to another program, or be recorded as
+  documented-absent with a link to the gap that tracks it.
+
+Documentation:
+
+- The Observability section claimed an opt-in `--log-format json` flag. No
+  such flag exists, and no module under `src/` imports `logging`, so there are
+  no log records for one to format. The section says that now, the Standards
+  Conformance table points at the new
+  `docs/CONFORMANCE-GAPS.md#observability` row, and the row sets out both ways
+  to close it without picking one.
+- `docs/api.md` listed seven `Finding` fields and two helpers. The dataclass
+  has ten fields and three helpers, and `docs/report.schema.json` already
+  required the three it omitted (`data`, `caused_by`, `severity_original`) and
+  `fingerprint()`, which is the identity `--baseline` matches on.
+- `docs/read-api.md` now documents `FeedFile.readable` and `LoadProblem`;
+  `problems` had been documented without its element type.
+- Two smaller README corrections: "16 reference checks" is now "the 16 checks
+  that read GTFS files" (six of the sixteen are field, semantic or coverage
+  rules), and the `ingest-ready` paragraph records that it currently resolves
+  to the same settings as `strict`.
+- `docs/plans/v1.0.0-readiness.md`, an item-by-item readiness assessment with
+  evidence per item, and `docs/plans/improvement-plan.md`, the log behind it.
 
 ## [0.10.0] - 2026-08-21
 

Makefile

@@ -1,9 +1,24 @@
 # make verify runs every merge-blocking gate that can run on a laptop, with the
 # same command CI runs (CICD-27). Run it before opening a PR; the release
 # workflows re-run it at the tagged commit before anything publishes
-# (REL-14/15). CI additionally runs what needs GitHub itself -- the composite
-# action's self-test, CodeQL, Semgrep and zizmor -- so a green `make verify` is
-# a necessary condition for merge, not a sufficient one.
+# (REL-14/15).
+#
+# CI additionally runs five things this file does not, so a green `make verify`
+# is a necessary condition for merge and not a sufficient one:
+#
+#   - the composite action's self-test, CodeQL, Semgrep and zizmor, which need
+#     GitHub itself;
+#   - the `perf` job, which runs `make perf-check` and `make memory-check`
+#     against baselines recorded on the runner's machine class (see those
+#     targets below);
+#   - the VS Code extension package job, which type-checks, audits and builds
+#     a VSIX out of editor/vscode. It is path-filtered to that directory, so it
+#     is absent from most pull requests, which is how it stayed off this list
+#     for as long as it did.
+#
+# This paragraph is checked against the workflows by
+# tests/test_ci_gate_parity.py, so a job added later cannot reject a tree that
+# `make verify` has just called green without saying so here.
 .PHONY: verify lockfile lint format typecheck test docs-check contract-check i18n-check audit npm-audit secrets a11y citation perf-check memory-check
 
 # Every gate `make verify` runs, in reporting order. Each one is independent:
@@ -89,8 +104,36 @@ audit:
 # Secret scan over the working tree + history (SEC-17/18). Requires the
 # gitleaks binary (see https://github.com/gitleaks/gitleaks#installing); the
 # CI job installs it explicitly rather than via the license-gated Action.
+#
+# Two scans, because one of them cannot see what the other is for. `gitleaks
+# detect --source .` walks commits: it answers "was a secret ever committed",
+# and it is blind to a file that exists on disk and has not been committed
+# yet. That is the state every working tree is in while a gate runs against
+# it. Measured on this repository at v0.10.0: a file at the repository root
+# holding an AWS key pair, a GitHub PAT and a Slack bot token, saved and never
+# added to the index, gave "283 commits scanned / no leaks found" and exit 0
+# from the history scan, and "leaks found: 1" and exit 1 from --no-git. The
+# comment above this recipe had said "working tree + history" since the gate
+# was written; only the history half existed.
+#
+# Both run, whatever the other one did, and each reports its own result -- the
+# same reason `verify` does not stop at its first failure. `.gitleaks.toml`
+# scopes the working-tree scan away from installed dependencies; see that file.
 secrets:
-	gitleaks detect --source . --redact --exit-code 1
+	@status=0; \
+	printf '%s\n' '' 'gitleaks scan 1 of 2: committed history'; \
+	if gitleaks detect --source . --redact --exit-code 1; then \
+		printf '%s\n' 'gitleaks history: PASS'; \
+	else \
+		status=1; printf '%s\n' 'gitleaks history: FAIL'; \
+	fi; \
+	printf '%s\n' '' 'gitleaks scan 2 of 2: working tree, uncommitted files included'; \
+	if gitleaks detect --no-git --source . --redact --exit-code 1; then \
+		printf '%s\n' 'gitleaks working tree: PASS'; \
+	else \
+		status=1; printf '%s\n' 'gitleaks working tree: FAIL'; \
+	fi; \
+	exit $$status
 
 # Node dependency vulnerability audit (SEC-11). This used to be the first line
 # of the `a11y` recipe, which meant a HIGH advisory anywhere in the npm

README.md

@@ -207,7 +207,10 @@ be passed with `--config path/to/file.toml`. A config may also `extends =
 A third preset, `ingest-ready`, is for a downstream CAD/AVL system deciding
 whether to import a feed at all: it is at least as strict as `strict` (fails
 on warnings, enables `coverage` and `advisory`) and adds no ignores, so it
-doubles as a go/no-go gate rather than an authoring-time policy.
+doubles as a go/no-go gate rather than an authoring-time policy. Today it
+resolves to exactly the same settings as `strict`; it is a separate name
+because the two answer different questions, and a later change to one should
+not silently move the other.
 
 Some checks are off by default because they surface judgement calls rather than
 spec violations. Turn them on with `--enable coverage` (which GTFS trips have no
@@ -388,8 +391,8 @@ jobs:
 The action runs `--format github`, so the annotations it leaves on the pull
 request include the checks that did not run and why (see
 [Rule-set coverage](#rule-set-coverage)). Leaving `gtfs`
-out is the case worth knowing about: 16 reference checks cannot run, 9 of them
-ERROR-severity, and the job still passes. Add
+out is the case worth knowing about: the 16 checks that read GTFS files cannot
+run, 9 of them ERROR-severity, and the job still passes. Add
 `require-complete-run: "true"` to fail it instead.
 
 The action installs `tods-validate` from a hash-verified
@@ -473,8 +476,18 @@ bug — please report it.
 
 ## Observability
 
-Observability: Tier C — OTel tracing out-of-scope (no network surface). Opt-in
---log-format json only.
+Observability: Tier C. OpenTelemetry tracing is out of scope, because there is
+no network surface to trace.
+
+The tier also asks for an opt-in `--log-format json` flag, and that flag does
+not exist. It is not an oversight that a release would quietly carry: the
+package emits no log records at all (nothing under `src/` imports `logging`),
+so a flag to choose their format would be a claim rather than a capability.
+What is machine-readable here is the report, through `--format json`, `--format
+sarif`, and the schema at [docs/report.schema.json](docs/report.schema.json).
+That is a different thing from a log stream, and this section previously
+conflated them. Tracked in
+[docs/CONFORMANCE-GAPS.md](docs/CONFORMANCE-GAPS.md#observability).
 
 ## Standards Conformance
 
@@ -488,7 +501,7 @@ Applicability and current state:
 | CI-CD | Applies | Applies — gap tracked, see [docs/CONFORMANCE-GAPS.md](docs/CONFORMANCE-GAPS.md#ci-cd) |
 | RELEASE-AND-VERSIONING | Applies (PyPI + GHCR + GitHub Releases + Action) | Applies — gap tracked, see [docs/CONFORMANCE-GAPS.md](docs/CONFORMANCE-GAPS.md#release-and-versioning) |
 | ACCESSIBILITY | Applies (scoped to the `--format html` report and the `web/` playground) | Applies — gap tracked, see [docs/CONFORMANCE-GAPS.md](docs/CONFORMANCE-GAPS.md#accessibility) |
-| OBSERVABILITY | Applies at Tier C (see `## Observability` above) | Applies — Tier C; N/A — tracing has no network surface, as declared above |
+| OBSERVABILITY | Applies at Tier C (see `## Observability` above) | Applies — Tier C; tracing N/A (no network surface); the tier's `--log-format json` is a gap, see [docs/CONFORMANCE-GAPS.md](docs/CONFORMANCE-GAPS.md#observability) |
 | INTERNATIONALIZATION | N/A — no user-facing strings requiring translation | N/A — see [docs/I18N.md](docs/I18N.md) |
 | AI Development Measurement | Applies | Applies — gap tracked, see [docs/CONFORMANCE-GAPS.md](docs/CONFORMANCE-GAPS.md#ai-development-measurement) |
 | AI Evaluation | N/A — no LLM/AI runtime | N/A — no LLM SDK or generative/agentic component anywhere in `src/` or `scripts/`; deterministic rule engine only |

docs/CONFORMANCE-GAPS.md

@@ -34,6 +34,42 @@ validator itself has no model runtime.
 v2.0.0 tiers and add a mechanically checked data-card/source inventory without
 claiming ownership of users' input feeds.
 
+## observability
+
+**Current boundary:** Tier C, per `OBSERVABILITY-STANDARD.md` §0. OTel tracing
+is out of scope and the README's `## Observability` section declares it: there
+is no network surface to trace, and the tool is offline by design.
+
+**Still open (found 2026-08-28):** Tier C also asks for "an opt-in
+`--log-format json` flag backed by `structlog`" (`OBSERVABILITY-STANDARD.md`
+§3, and `QUALITY-AND-METRICS-STANDARD.md` line 190 restates it as a must). The
+flag does not exist anywhere in `src/`. Until today the README reproduced the
+standard's own declaration sentence verbatim, ending "Opt-in `--log-format
+json` only", which reads as a statement that the flag is there; nothing in
+this ledger recorded otherwise, and no gate compared the sentence to the CLI.
+`tests/test_readme_claims.py` now does, so the claim cannot return without the
+flag returning with it.
+
+Two ways to close it, and the choice is a product decision rather than a
+remediation:
+
+1. **Restate the tier.** Nothing under `src/` imports `logging`; the package
+   emits no log records at all, so there is no stream for a format flag to
+   select. The machine-readable surface here is the *report* (`--format json`,
+   `--format sarif`, `docs/report.schema.json`), which is a different artifact
+   from a log. If the standard's intent is "a machine can consume this tool's
+   output", that is already met, and the row should say so in those words
+   rather than by naming a flag.
+2. **Implement it.** `structlog` would be a second runtime dependency for a
+   tool that deliberately has one (`click`), added to satisfy a sentence
+   rather than a user. Weaker unless an operator asks for parseable progress
+   logs on large feeds.
+
+Not on the v1.0.0 critical path either way: `--log-format` does not appear in
+`docs/v1-contract-candidate.json`, so adding it later is an additive minor
+release. What was on the critical path was shipping v1.0.0 with the README
+claiming it.
+
 ## incident-response
 
 **Still open:** security reporting and release recovery exist, but the v2.0.0

docs/api.md

@@ -45,10 +45,24 @@ all.
 
 ## `Finding`
 
-A frozen dataclass: `rule_id`, `severity`, `message`, `file`, `row`, `field`,
-`suggestion`. Helpers: `location()` (human string) and `pointer()` (a stable
-`file.txt#L4/field` identifier). `to_dict()` matches
-[docs/report.schema.json](report.schema.json).
+A frozen dataclass. Fields: `rule_id`, `severity`, `message`, `file`, `row`,
+`field`, `suggestion`, `data` (the rule's own machine context, such as the
+offending value or the ID a reference failed to resolve), `caused_by` (set
+when this finding is a downstream echo of another, carrying that root's
+`pointer()`), and `severity_original` (set when a config `[severity]` remap
+moved the level, so the change is disclosed rather than silent).
+
+Helpers: `location()` (human string), `pointer()` (a stable
+`file.txt#L4/field` identifier), and `fingerprint()` (a content hash over rule
+ID, file, field and `data`, deliberately not over row or message, so inserting
+an unrelated row does not change every later finding's identity; this is what
+`--baseline` matches on). `to_dict()` matches
+[docs/report.schema.json](report.schema.json), which requires every field
+above.
+
+The last three fields and `fingerprint()` were missing from this list while
+the report schema already required them, so a caller reading only this page
+did not know what they were being handed.
 
 ## `suggest_fixes(path, gtfs=None, *, enable=(), encoding=None, spec_version=SPEC_VERSION)`
 
@@ -121,12 +135,16 @@ stable. The lower-level `tods_validate.runner.run` is available too, but
 
 ---
 
-Last verified: 2026-08-14, against tods-validate 0.8.0. Every signature,
+Last verified: 2026-08-28, against tods-validate 0.10.0. Every signature,
 `ValidationResult` and `Finding` member, `Suggestion` field, and test helper on
 this page was called and checked against the implementation, including the
 documented `PackageNotFoundError` and the `SUPPORTED_SPEC_VERSIONS` values.
+The `Finding` list was checked field by field against `findings.py` and
+`report.schema.json` this time, which is how the four missing entries were
+found; the previous stamp said 0.8.0 while the tree shipped 0.10.0, and this
+gate compares content rather than versions, so it had no way to say so.
 Recheck cadence: every release, and whenever this page changes —
 `make docs-check` fails if the page is edited without a fresh verification.
 
-<!-- doc-currency: sha256=603b2ea973ec -->
+<!-- doc-currency: sha256=bd77bb402218 -->