docs(research): parser/generator ladder — ZetaId bits → value-tree codecs → ZetaParse Grammar IR - #9199
Merged
Merged
Conversation
…decs → ZetaParse Grammar IR (shadow*)
Aaron 2026-07-02: "we have some small parser combinator foundation already … our zetaid is a
tiny parser generator … bits, a bit parser/generator … the ANTLR stuff is likely just some
docs/research/persona conversation … LARGE scope: compile to/from our IR and most other ANTLR
grammars using our parser generators and the open free ANTLR grammars; small changes to
existing ANTLR grammars is fine."
Take-the-existing-into-consideration synthesis (no new code). Discovered + connected the three
rungs of ONE spec→IR→generated-parser ladder that already partly exists:
• rung 1 BUILT — Core.FSharp.ZetaId: spec (zeta-id-v1-layout.yaml) → GeneratedBitLayout →
bit parser/generator (the proven-in-miniature seed Aaron named);
• rung 2 BUILT (this session) — the value-tree codecs (#9185–#9196) over DynamicValue;
• rung 3 DESIGNED, not built — ZetaParse (Amara, zetaparse-lr-glr-…-2026-05-21): ingest
ANTLR .g4 / Yacc / Tree-sitter / .zg → Zeta Grammar IR → LR/GLR parser (6 langs), consuming
antlr/grammars-v4 (MIT/BSD; ANTLR survey by Lior, 2026-06-13). Don't RUN ANTLR — ingest it.
The synthesis subsumes two prior findings: the lenient-YAML need (frontmatter #9196) and the
general-KDL reader (the delayed fork) are both just grammars ingested to the Grammar IR — not
bespoke hand-parsers. The anti-reinvention principle is the TOSEC/MAME one (#9184): grammars-v4
is the license-clean corpus; build on it.
Records the ladder + next moves (Grammar IR as a DynamicValue schema; .g4 ingester; LR/GLR
backend) as rung-3 scope to steer with Aaron — LARGE, design-heavy, already designed.
docs/research/2026-07-02-parser-generator-foundation-ladder-zetaid-bits-to-value-tree-codecs-to-zetaparse-grammar-ir-antlr.md
docs/trajectories/value-tree-codecs/RESUME.md (ladder framing; KDL/lenient-YAML subsumed)
Anchors: ZetaParse (Amara), ANTLR survey (Lior), ZetaId; Knuth (LR), Tomita (GLR), Parr
(ANTLR), tree-sitter, Bison; antlr/grammars-v4 (MIT/BSD); only-the-irreducible-generate-the-rest.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AgencySignature-v1:
persona: otto
actor: zeta-otto
surface: docs/research
topology: shadow-clone
intent: parser-generator-ladder-synthesis-zetaid-bit-codec-value-tree-codecs-zetaparse-grammar-ir-antlr-ingest-take-existing-into-consideration
authorization: aaron-explicit-take-existing-parser-combinator-foundation-into-consideration-large-scope-compile-to-from-ir-antlr-grammars-2026-07-02
uncertainty: low
measure: discovered-zetaid-bit-parser-generator-and-zetaparse-antlr-survey-design-docs-connected-three-rung-ladder-spec-to-generated-parser-kdl-lenient-yaml-subsumed-as-grammar-ingest-grammars-v4-mit-bsd-corpus-no-code
delta-u: parser-generator-foundation-scoped-on-existing-zetaid-seed-and-zetaparse-design-not-reinvented-codecs-are-rung-2-of-one-ladder-antlr-grammars-v4-is-the-corpus
seed: S4
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
AceHack
added a commit
that referenced
this pull request
Jul 2, 2026
…(rung 3, first step) (shadow*) (#9205) Aaron 2026-07-02: "we can do the first steps but i can also answer any design heavy questions too" (green-lighting the bounded first step of the parser/generator ladder rung 3). Lands the Grammar IR's STRUCTURAL core as a value tree — a `Grammar ⇄ DynamicValue` bijection — so a grammar is just DATA: it rides the rung-2 codecs (JSON/CBOR/DER + parity), is byte-lockable and DST-replayable, and schema-evolves like any value tree. Rung 2 serves rung 3. Types mirror the ZetaParse design (docs/research/zetaparse-lr-glr-fsharp-compiler-fork-design- 2026-05-21, Amara): Terminal / NonTerminal / Production / Grammar. This first step is the structural core (terminals, nonterminals, productions, start); the advanced ZetaParse fields — semantic actions, precedence/associativity, attributes, recovery rules, incremental/retraction hooks — are DEFERRED to a v2 schema pass (a design decision for Aaron), addable later as optional fields under the versioned/zero-downtime-rollable discipline. Proofs (tests/Tests.FSharp/GrammarIr.Tests.fs, 3/3): a real arithmetic-expression grammar round-trips Grammar → DynamicValue → Grammar; the grammar-as-value-tree cross-verifies through parity-json / cbor / parity-asn1 (byte-lockable + DST-replayable); malformed input yields a clean Error (total parse, never an exception). Next (resume): the `.g4` → Grammar IR ingester (one grammars-v4 asset, compatible subset, log drops) — where KDL and lenient-YAML are subsumed as grammars ingested, not bespoke hand-parsers. Anchors: ZetaParse (Amara); Knuth (LR), Tomita (GLR), Parr (ANTLR); antlr/grammars-v4 (MIT/BSD corpus — ingest, don't reinvent); the parser/generator ladder synthesis (#9199). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> AgencySignature-v1: persona: otto actor: zeta-otto surface: src/core topology: shadow-clone intent: grammar-ir-as-dynamicvalue-schema-rung-3-first-step-grammar-is-data-rides-codec-stack-byte-lockable-dst-replayable-zetaparse-structural-core authorization: aaron-explicit-we-can-do-the-first-steps-green-light-grammar-ir-dynamicvalue-schema-2026-07-02 uncertainty: low measure: grammarir-terminal-nonterminal-production-grammar-to-from-dynamicvalue-bijection-arith-expr-grammar-round-trips-rides-parity-json-cbor-asn1-malformed-clean-error-3-tests-structural-core-v2-deferred delta-u: zeta-grammar-ir-structural-core-lands-as-dynamicvalue-schema-grammar-is-byte-lockable-dst-replayable-data-rung-2-serves-rung-3-g4-ingester-next seed: S4
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You (2026-07-02): we have some small parser combinator foundation already … our zetaid is a tiny parser generator … bits, a bit parser/generator … the ANTLR stuff is likely just docs/research … LARGE scope: compile to/from our IR and most other ANTLR grammars using our parser generators and the open free ANTLR grammars.
Take-the-existing-into-consideration synthesis (no new code). Found and connected the three rungs of one
spec → IR → generated-parserladder that already partly exists:zeta-id-v1-layout.yamlBitLayoutDynamicValueValueTreeCodec.g4/.y/.zgKey: rung 3 is not a green field — ZetaParse already designed it (ingest ANTLR into a Grammar IR; don't run ANTLR), and
antlr/grammars-v4is MIT/BSD (the license-clean corpus — don't reinvent, same principle as TOSEC/MAME #9184).Subsumes two prior findings: the lenient-YAML need (#9196) and the general-KDL reader (the delayed fork) are both just grammars ingested to the Grammar IR — not bespoke hand-parsers.
Records rung-3 next moves (Grammar IR as a DynamicValue schema →
.g4ingester → LR/GLR backend) as LARGE, design-heavy scope to steer with you, built on the existing foundation.Docs: the synthesis note +
docs/trajectories/value-tree-codecs/RESUME.md(ladder framing).Generated with Claude Code