Add changelog registry reconcile and verify commands

[cotti]

Why

Phase 2 of elastic/docs-eng-team#688, stacked on #3738. Once the scrubber Lambda owns the public registry.json (Phase 1), two things are still missing: a way to trigger the cutover heal (every existing public manifest is a legacy pass-through copy that needs one authoritative rebuild — and historical drift needs repairing retroactively), and a way to prove convergence before Phase 3 retires the client-side refresh.

What

• changelog registry reconcile plans the affected groups — a single scope, or the union of groups discovered in both buckets, so orphan public groups (including manifest-only leftovers) are covered — and sends one explicit reconcile message per group to the scrubber queue: {"kind":"reconcile","version":1,"scope":"bundle"|"changelog","group":"…","correlation_id":"…"}, strictly validated through ChangelogKeys on both ends and serialized with the same source-generated context the Lambda parses with (AOT). The CLI never mutates S3 — the Lambda stays the public bucket's single writer. --dry-run prints the plan; the non-dry-run path asks for confirmation (--yes for non-interactive use). Each run stamps one correlation id and prints a ledger line (group, SQS message id, correlation id) for the cutover checkpoint. Attributes: [RequiresAuth], [CommandIntent(Intent.Destructive | Intent.Idempotent)], [MutationScope(MutationScope.Global)].
• On a reconcile message the Lambda performs a full group heal: object-level reconcile over the union of both buckets' group listings — scrub/copy what is live in the private bucket, conditionally delete what isn't — then the group reconcile. This is what makes a lost or DLQ-expired scrub event recoverable by message (it subsumes Add changelog registry state discovery and reconciliation commands #3717's republish). Reads the (already-provisioned) PRIVATE_BUCKET_NAME Lambda env var — S3 events keep working without it since they carry their source bucket. Malformed reconcile messages are rejected to the DLQ, where the Phase 0 alarm makes the sender bug visible — redelivery can't fix a bad message.
• changelog registry verify is the read-only sibling and the cutover gate: for each planned group it compares the public manifest against what a reconcile of the current public listing would write — same listing spec and entry rules by construction (RegistryReconciler.VerifyGroupAsync) — and reports divergence using Add changelog registry state discovery and reconciliation commands #3717's taxonomy: Missing / Stale / Corrupt / ObjectDivergent, with UnsupportedSchema reported distinctly. Non-zero exit on any divergence.
• Manifest ETag wire-format fix: the snake_case naming policy serialized the producer-side ETag as "e_tag", while the consumer contract (ChangelogRegistryBundle) and the documented format read "etag" — every recorded ETag has been invisible to every consumer. [JsonPropertyName("etag")] aligns the writer with the documented format; old "e_tag" manifests now parse as corrupt and get rebuilt, which the cutover forces anyway via the producer mismatch.
• docs/cli-schema.json regenerated; supplemental CLI pages added under docs/cli/changelog/registry/.

Part of elastic/docs-eng-team#688 (Phase 2).

🤖 Generated with Claude Code

[Mpdreamz]

Request changes: reconcile/verify against the wrong registry model

Same architectural pushback as on #3738 — please don't build CLI cutover (registry reconcile / registry verify) around per-product / per-group changelogs/<product>/registry.json.

Why

Automated release discovery always has a list of PRs first (elastic/docs-eng-team#698: prestage query, GitHub Release PR list, serverless query). From that list we probe CloudFront; 404 means no checked-in changelog, and changelog bundle inlines one into the bundle anyway.

S3 changelogs/ is a mirror of repo-checked-in entries, not a required inventory at bundle time. A product-scoped registry would only grow. Drop that contract and the reconcile/verify surface that exists to prove convergence of it.

What to keep / retarget

What we actually want on S3 is the shallow cache-opt-out maps:

• changelogs/registry.json
• bundles/registry.json

{
  "<folder>": "<etag>"
}

ETag = last-touched file under that folder, so caches can skip per-folder work eagerly.

Please reshape (or largely remove) the reconcile/verify commands so they target that folder→etag registry if anything — not an object-level per-product manifest heal. Much of ChangelogGroupDiscovery, group listing, divergence taxonomy (Missing/Stale/Corrupt/ObjectDivergent), and the cutover heal queue flow can go with the old model.

[cotti]

Closing per the re-scope agreed on #3738/#3681: the per-product registry model is dropped, and with it this reconcile/verify surface. If a folder→ETag-level verification turns out to be worth having, it can come with the reshaped #3738.