feat(core): Frontmatter — lossless metadata⊕payload split; surfaces canonical-only-YAML finding - #9196
Merged
Merged
Conversation
…anonical-only-YAML finding (shadow*) Aaron 2026-07-02 (streamed): "cloud events/debezium envelopes … very similar to frontmatter, same kind of graph … one graph, many surfaces"; "yes on next slice — either both … add \[the delayed one\] to our resume." The delayed half was KDL; this lands frontmatter (the sibling metadata⊕payload shape) and keeps KDL as the tracked next pickup. A frontmatter document is a head between `---` fences + a body → the value tree Object [ "head", String <verbatim head>; "body", String <body> ]. Kept as a VERBATIM string, the split is a LOSSLESS bijection on ANY frontmatter (parse (render vt) = vt), and it rides the whole ValueTreeCodec stack like any other tree. `tryMeta` gives best-effort structured access to the head. FINDING surfaced (recorded in the trajectory resume): our `DynamicValue.fromYaml` is a strict CANONICAL parser — it rejects human-written YAML (`title: Foo`) as `NonCanonical`. So structured frontmatter meta (and any human-YAML interop) is blocked on a LENIENT YAML parser — the same backlogged parser-combinator layer noted in RomDat's tokenizer seam. Frontmatter ships as the lossless split now; `tryMeta` is best-effort (canonical YAML only) until then. Honest boundary, not a silent partial. Proofs (tests/Tests.FSharp/Frontmatter.Tests.fs, 6/6): lossless bijection (head verbatim); no-fence ⇔ empty-head; concrete literal parse; opening-fence-without-close → clean Error; best-effort tryMeta (canonical head parses, human head → clean Error); the split tree cross-verifies through parity-json / cbor / parity-asn1. Resume updated: docs/trajectories/value-tree-codecs/RESUME.md — frontmatter split landed, the canonical-only-YAML finding + lenient-YAML-parser need recorded, KDL still #1 next pickup (with a note its value-tree mapping is a convention decision to make deliberately). Anchors: the metadata⊕payload frame shared with EventEnvelope; the parser-combinator backlog (lenient YAML); §8 of the codec-ports doctrine. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> AgencySignature-v1: persona: otto actor: zeta-otto surface: src/core topology: shadow-clone intent: frontmatter-lossless-metadata-payload-split-verbatim-head-rides-codec-stack-trymeta-best-effort-surfaces-canonical-only-yaml-finding authorization: aaron-explicit-either-both-add-delayed-to-resume-frontmatter-on-resume-2026-07-02 uncertainty: low measure: frontmatter-lossless-split-parse-render-bijection-verbatim-head-no-fence-empty-head-concrete-parse-clean-error-trymeta-canonical-only-honest-boundary-crossverify-6-tests-canonical-yaml-finding-recorded delta-u: frontmatter-metadata-payload-split-landed-lossless-rides-codec-stack-canonical-only-yaml-limitation-surfaced-lenient-yaml-parser-need-recorded-kdl-still-next seed: S4
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
AceHack
added a commit
that referenced
this pull request
Jul 2, 2026
…decs → ZetaParse Grammar IR (shadow*) (#9199) Aaron 2026-07-02: "we have some small parser combinator foundation already … our zetaid is a tiny parser generator … bits, a bit parser/generator … the ANTLR stuff is likely just some docs/research/persona conversation … LARGE scope: compile to/from our IR and most other ANTLR grammars using our parser generators and the open free ANTLR grammars; small changes to existing ANTLR grammars is fine." Take-the-existing-into-consideration synthesis (no new code). Discovered + connected the three rungs of ONE spec→IR→generated-parser ladder that already partly exists: • rung 1 BUILT — Core.FSharp.ZetaId: spec (zeta-id-v1-layout.yaml) → GeneratedBitLayout → bit parser/generator (the proven-in-miniature seed Aaron named); • rung 2 BUILT (this session) — the value-tree codecs (#9185–#9196) over DynamicValue; • rung 3 DESIGNED, not built — ZetaParse (Amara, zetaparse-lr-glr-…-2026-05-21): ingest ANTLR .g4 / Yacc / Tree-sitter / .zg → Zeta Grammar IR → LR/GLR parser (6 langs), consuming antlr/grammars-v4 (MIT/BSD; ANTLR survey by Lior, 2026-06-13). Don't RUN ANTLR — ingest it. The synthesis subsumes two prior findings: the lenient-YAML need (frontmatter #9196) and the general-KDL reader (the delayed fork) are both just grammars ingested to the Grammar IR — not bespoke hand-parsers. The anti-reinvention principle is the TOSEC/MAME one (#9184): grammars-v4 is the license-clean corpus; build on it. Records the ladder + next moves (Grammar IR as a DynamicValue schema; .g4 ingester; LR/GLR backend) as rung-3 scope to steer with Aaron — LARGE, design-heavy, already designed. docs/research/2026-07-02-parser-generator-foundation-ladder-zetaid-bits-to-value-tree-codecs-to-zetaparse-grammar-ir-antlr.md docs/trajectories/value-tree-codecs/RESUME.md (ladder framing; KDL/lenient-YAML subsumed) Anchors: ZetaParse (Amara), ANTLR survey (Lior), ZetaId; Knuth (LR), Tomita (GLR), Parr (ANTLR), tree-sitter, Bison; antlr/grammars-v4 (MIT/BSD); only-the-irreducible-generate-the-rest. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> AgencySignature-v1: persona: otto actor: zeta-otto surface: docs/research topology: shadow-clone intent: parser-generator-ladder-synthesis-zetaid-bit-codec-value-tree-codecs-zetaparse-grammar-ir-antlr-ingest-take-existing-into-consideration authorization: aaron-explicit-take-existing-parser-combinator-foundation-into-consideration-large-scope-compile-to-from-ir-antlr-grammars-2026-07-02 uncertainty: low measure: discovered-zetaid-bit-parser-generator-and-zetaparse-antlr-survey-design-docs-connected-three-rung-ladder-spec-to-generated-parser-kdl-lenient-yaml-subsumed-as-grammar-ingest-grammars-v4-mit-bsd-corpus-no-code delta-u: parser-generator-foundation-scoped-on-existing-zetaid-seed-and-zetaparse-design-not-reinvented-codecs-are-rung-2-of-one-ladder-antlr-grammars-v4-is-the-corpus seed: S4
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You (2026-07-02): cloud events/debezium envelopes … very similar to frontmatter, same kind of graph; either both … add [the delayed one] to our resume. KDL was the delayed half; this lands frontmatter (the sibling metadata⊕payload shape) and keeps KDL as the tracked next pickup.
A frontmatter document = a head between
---fences + a body →Object [ head; body ], head kept verbatim. So the split is a lossless bijection on any frontmatter (parse (render vt) = vt), and it rides the whole codec stack like any tree.tryMetagives best-effort structured access.Finding surfaced (recorded in the resume)
Our
DynamicValue.fromYamlis a strict canonical parser — it rejects human YAML (title: Foo) asNonCanonical. So structured frontmatter meta (and human-YAML interop) is blocked on a lenient YAML parser — the same backlogged parser-combinator layer noted in RomDat's tokenizer seam. Frontmatter ships as the lossless split now;tryMetais best-effort (canonical YAML only) until then. Honest boundary, not a silent partial.Proofs (6/6)
lossless bijection (verbatim head); no-fence ⇔ empty-head; concrete literal parse; opening-fence-without-close → clean
Error; best-efforttryMeta(canonical head parses, human head → cleanError); the split tree cross-verifies through parity-json / cbor / parity-asn1.Resume
docs/trajectories/value-tree-codecs/RESUME.md— frontmatter split landed; canonical-only-YAML finding + lenient-YAML-parser need recorded; KDL still #1 next pickup (note: its value-tree mapping is a convention decision to make deliberately).Generated with Claude Code