FE-1322: Generate Petrinaut architecture docs from in-code annotations - #9165
Closed
kube wants to merge 23 commits into
Closed
FE-1322: Generate Petrinaut architecture docs from in-code annotations#9165kube wants to merge 23 commits into
kube wants to merge 23 commits into
Conversation
Architecture docs rot because nothing fails when they stop being true. `@local/petrinaut-arch-docs` extracts the architecture from annotations that live next to the code they describe, and CI fails when a declaration stops matching reality. Two inputs, each in its natural home: - Folder `README.md` frontmatter declares a layer, and the prose below it becomes that layer's page — folder docs that already exist turn into architecture pages for free. - `@boundary`, `@invariant` and `@seam` doc-comment tags attach facts to the specific code that upholds them. Files with no annotation inherit from the nearest declaring ancestor, so a few dozen declarations cover several hundred files. The output is a portable bundle rather than a site: `architecture.json` (the model), `architecture.md` (the whole architecture in one file, which is the cheapest read for an agent), `llms.txt`, `manifest.json` (a page tree so a host can build navigation without crawling), MDX pages, and D2 diagram sources. Generated MDX is YAML frontmatter plus plain CommonMark with no JSX, which is what lets one bundle render in Astro, in hash.dev's Next.js MDX pipeline, and as plain text. `lint:arch-docs` runs in CI and fails on: a source file no declaration covers, a layer id implying an undeclared ancestor, a duplicate layer or two declarations on one folder, a malformed tag, a `@seam` that is no longer an export, a dependency violating a declared rule, and a committed bundle that no longer matches the source. It needs no `d2`, comparing the text artefacts rather than re-rendering SVGs. Edges carry only `crossesPackage` as a boundary fact. Which runtime boundaries an import crosses cannot be read off a static import graph — a module importing into a worker-boundary layer is how you obtain the module, not evidence that a thread hop occurs — so that is the one such fact that is always true when reported. yarn.lock covers this package's dependencies and those of the docs site added in a later commit.
Replaces the hand-maintained path-to-layer mapping with declarations that sit next to the code they describe: 37 layers across petrinaut-core and petrinaut, covering 412 source files. Most declarations are frontmatter added to READMEs that already existed and already explained their folder, so their prose now doubles as the layer's documentation. Where a folder had a barrel entry file and no README, `@layerRoot` on that file does the same job. Fifteen boundaries and 25 invariants are recorded against the specific files that uphold them. Three dependency rules are now enforced against the real import graph. The substantive one is that `react` must not depend on `ui`: state providers stay mountable without rendering the editor. That already held — 0 imports in that direction against 251 the other way — so the rule locks in an existing property rather than asking for new work. Retires `scripts/generate-dependency-diagrams.mjs`, which held the architecture as ~180 lines of `if (path.startsWith(...))` far from the code, with a fallback that silently mis-bucketed anything renamed. It also hard-coded seven of petrinaut-core's ten entry points, so imports through `./ai`, `./optimization` and `./compiled-model` resolved to nothing and were absent from the diagrams entirely; aliases are now derived from the package's `exports`. Its generated `.d2`/`.svg` output and `dependency-diagrams.md` go with it, and `dependency-cruiser` is no longer a petrinaut-core dependency. The hand-written HTML in `docs/architecture/` is deliberately left in place — the content is valuable but unverified, and its custom lane-and-box CSS needs rewriting page by page. A README there records that status and points at the generated docs for facts about the current shape of the system.
The bundle is committed because it is what CI diffs against to detect drift, and what a host embedding these docs consumes. Regenerate with `mise run doc:architecture` after changing annotations or moving code. `@apps/petrinaut-docs` is a Starlight site that owns no content: every page comes from the bundle. That is deliberate — the bundle has to render in a host that did not generate it, so this site is a portability test as much as a way to read the docs, and anything that only works here is a bug in the bundle. It builds its sidebar from `manifest.json` rather than from Starlight-shaped frontmatter, which is what keeps the bundle framework neutral. `trailingSlash: "never"` and `build.format: "file"` are load-bearing. Inter-page links in the bundle are relative and assume a slug maps to a URL with no trailing slash; serving `/architecture/core/` instead would resolve them one level too deep. The bundle is copied into the app rather than loaded in place because `astro dev` resolves an MDX page's relative image paths against the project root, so a diagram referenced from outside the project cannot be found. Copying is also what an embedding host does. `installConfig.hoistingLimits` nests this app's dependencies. Astro's generated prerender entry resolves `cookie` from the app's build output, which would otherwise reach the root-hoisted `cookie@0.7.2` that `express` pins and fail on a missing `parseCookie` export. Nesting keeps Astro on its own `cookie@2.x` without changing hoisting for the rest of the monorepo. There is no `lint:tsc` for the app: everything in it is `.mjs`, so `astro check` would pull in `@astrojs/check` and `typescript` to check almost nothing. `astro build` is the real check.
oxfmt and the architecture generator both claimed ownership of the bundle's MDX, which made the format check and the drift check mutually exclusive: formatting the bundle made a fresh generate look like drift, and regenerating it made the format check fail. The generator owns that output byte-for-byte — CI diffs a fresh build against the committed files to detect drift — so the bundle is now ignored by the formatter, alongside the other autogenerated paths. Authored pages in `content/` are still formatted; they are inputs, and the bundle copies them verbatim.
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
github-actions
Bot
added
area/deps
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| export const GENERATED_ORDER_BASE = 1000; | ||
|
|
||
| const escapeTableCell = (text: string): string => | ||
| text.replace(/\|/gu, "\\|").replace(/\n/gu, " "); |
|
|
||
| const fields = match[1] ?? ""; | ||
| const read = (key: string): string | null => { | ||
| const found = new RegExp(`^${key}\\s*:\\s*(.+)$`, "mu").exec(fields); |
Merging this PR will degrade performance by 15.38%
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing 
Footnotes |
The bundle is derived entirely from the annotations in the source and the authored pages in `content/`, so committing it meant reviewing every change twice and resolving conflicts in generated files. It is now git-ignored build output. This removes the need for the drift check that compared a fresh build against the committed copy: with nothing stored, nothing can be stale. What `lint:arch-docs` still enforces is the part that was always the real value — unannotated files, undeclared ancestors, malformed tags, dead `@seam`s, and dependency rules the import graph violates. It builds the bundle in memory and discards it. The docs site now regenerates the bundle rather than assuming a committed one is present, so `dev` and `build` share one code path and cannot render a stale copy. One consequence worth naming: a reviewer can no longer see the rendered documentation change in a PR diff. The annotations that produce it are still reviewable, and the docs can be regenerated locally in seconds.
The five hand-written HTML pages in `petrinaut-core/docs/architecture/`
carried genuinely useful detail — the binary frame format, the worker
message protocol, the ack contract, the Monte Carlo memory model — but
nothing verified them, and their custom lane-and-box CSS meant they could
only be read by opening a file in a browser.
They are now authored MDX under `content/simulation/`, bundled alongside
the generated pages and reachable from the docs site. Diagrams that relied
on CSS are plain tables or fenced text blocks, so they render anywhere the
rest of the bundle does.
Migrating surfaced staleness, which was the main argument for moving them:
- The frame-format page said UUID support was designed but not implemented
("the layout machinery is ready; the value plumbing is not"). `uuid` is a
real element type now, with parsing, formatting, namespaced generation
and seeded fallback.
- The sandbox section described only global shadowing plus a constructor
guard "for the duration of the call". The guard now swaps `.constructor`
descriptors on the built-in prototypes and freezes the user-facing
argument.
Everything retained was checked against the code rather than copied:
frame version and header size, the play-mode backpressure profiles, the
worker and Monte Carlo batch defaults, and the string-pool design.
Content the generated pages already own — module maps, per-layer file
lists, responsibilities — was dropped rather than duplicated, so these
pages carry only what an import graph cannot express.
The three inbound references to the HTML now point at the docs site, and
the site nests authored pages by slug directory so the five appear as one
"Simulation" group.
The bundle referenced `diagrams/*.svg` unconditionally, but rendering them needs `d2`, and a failure to render only produced a warning. In any environment without `d2` the result was a bundle pointing at images that were never written, which fails the consuming site's build rather than degrading. `d2` is a declared repo tool, so a `mise install` environment is fine. An environment that installs tools individually is not — the Vercel install script for the Petrinaut website names its tools one by one and does not include it, which is exactly the situation a deployment would hit. The generator now probes for the renderer before emitting pages and omits the diagram images when it is absent, so the bundle is internally consistent either way. `check` skips the probe: it writes nothing, so availability cannot affect its result. Verified in both directions — with `d2`, 8 diagrams and 7 pages embedding them; without it, no SVGs, no references, and the site still builds all 47 pages.
Hand-written guides sat in their own sidebar section, separate from the
generated reference for the same code. Someone reading about the Monte
Carlo layer had no reason to discover the page explaining its memory
model, and vice versa.
An authored page can now name the layer it explains:
attachTo: core.simulation.monte-carlo
which moves it beneath that layer's page, adds it to a "Guides" section
there, and — because nesting follows the slug — places it in the sidebar
next to the layer's sub-layers. The five simulation deep-dives now live
inside the architecture tree rather than beside it.
`attachTo` references a layer; it does not declare one. Declaring layers
from `content/` stays forbidden, and naming a layer that does not exist
fails the check.
This required a way to link between pages that does not depend on where a
page ends up, since `attachTo` decides that:
[the engine](layer:core.simulation.engine)
[memory model](doc:simulation/memory-model)
Both resolve to the correct relative path at emit time, fragments
included, and an unresolved target is a build error rather than a link
that 404s for a reader. Ordinary relative and absolute links are
untouched, so pages that will never move can still use them.
The site's sidebar now nests purely by slug rather than by whether a page
was generated, which is what lets an attached guide appear inside a
generated group at all.
Migrating the HTML pages flattened their diagrams into tables and fenced
text. The lane-and-box thread views, the frame memory map and the message
sequence carried real information in their layout — a byte map read as a
table loses the sense of a single contiguous buffer, which is the point of
the format.
They are back as React components in the bundle, imported by authored
pages:
import { ByteMap } from "@diagrams/byte-map";
`@diagrams/` is rewritten to a real relative path at emit time, for the
same reason as `layer:` and `doc:` — a page's depth depends on `attachTo`.
An import naming a component that does not exist fails the check.
Four components cover every diagram the old pages had: `lanes` (parallel
columns of boxes), `pipeline` (a numbered chain), `byte-map` (an offset
gutter with typed sections) and `sequence` (two actors exchanging
messages). They are data-driven, so a page supplies content and the
component owns presentation.
Two constraints keep the bundle portable, both learned the hard way here:
- **Plain React, no dependencies.** Styling is one stylesheet deriving its
colours from the host's `currentColor`, so it works on light and dark
themes it has never seen. No design system, no Astro, no `next/*`.
- **String props, never JSX.** JSX inside MDX is compiled by the host's MDX
renderer, and handing that to a React component fails at render with
"Objects are not valid as a React child". Props are strings, and
backticks render as `<code>`.
This is the one thing the bundle now asks of a host: a React-capable MDX
pipeline for authored pages. Generated pages stay plain CommonMark, and
`architecture.md` — the single-file artefact for agents — has no
components at all.
`starlight-llms-txt` is dropped rather than worked around: it renders MDX
to text in a container with no React renderer, and its `exclude` option is
accepted but never passed to the `/llms-full.txt` route, so components
were a hard build failure. The site now serves the bundle's own
`architecture.md` and `architecture.json`, which is better anyway — the
machine-readable surface is identical to what any other host would serve.
Also fixes a race the components exposed: `build` and `lint:tsc` each
invoked `sync:bundle`, which wipes and recopies the same directories, so
Turborepo running them concurrently failed intermittently. `sync:bundle`
is now a task both depend on.
Contributor
Dependency ReviewThe following issues were found:
|
Nothing off the shelf does this. Every mature architecture-docs tool —
Structurizr, LikeC4, the C4 tooling generally — implements drill-down as
view switching rather than in-place folding, and the two libraries with
first-party collapse-with-aggregation are both ruled out: cytoscape's
expand-collapse plugin declares itself unmaintained, and G6 renders to
canvas, which costs the real `<a href>` into each layer's page that is the
whole point here.
So the renderer is reused and the domain logic is ours. `@xyflow/react` is
already this repo's canvas — Petrinaut's own Petri net editor runs on it,
and hash-frontend pins the same 12.10.1 — and `elkjs` is already a
dependency of petrinaut-core. Zero new vendors.
The split that makes it work: nothing computes a layout in the browser.
Folding a layer makes its descendants' own fold states unobservable, so the
reachable states are enumerable — 30, not 2^6 — and the build lays out every
one with ELK and ships the coordinates. That keeps elkjs a devDependency
that is never distributed, which matters because it is EPL-2.0 rather than
MIT/Apache, and it drops 1.6 MB of elk-worker from the client.
It also renders without JavaScript. The server, and the first client render,
emit a plain inline `<svg>` from the same coordinates, with a real anchor per
layer and ELK's routed edges; only after mount does React Flow take over for
pan, zoom and the fold controls. A host that never hydrates the island still
gets a correct, navigable diagram — which is what makes the new dependency
acceptable in a bundle that has to embed elsewhere.
`src/emit/collapse.ts` holds the re-pointing as pure functions with 14 tests.
Two cases stop being drawable when folded and are reported on the node as
internal instead: an edge between two layers folded into the same box, and an
edge between a layer and something nested inside it. 69 of the 177 edges are
the latter, so drawing them would mean 69 arrows pointing into their own box.
Reciprocal pairs — 33 of them — merge into one edge keeping both counts.
Nothing here invents a dependency: aggregated edges sum real fileDependencies
and internal counts report real imports.
One fix worth knowing about, because it fails silently and confusingly:
React Flow gives its per-edge `<svg>` no size, relying on the SVG default of
`display: inline` plus `overflow: visible`. Starlight's reset says
`svg { display: block }`, which makes it `width: auto` inside a zero-width
parent — and an SVG of zero width is not rendered at all, while still
reporting a correct bounding box to script. Every edge disappeared. The CSS
now pins those dimensions so the diagram does not depend on the host's reset.
Payload: 289 KB of coordinates, 13 KB gzipped.
Verified in the browser: server render carries the static SVG with 6 routed
edges and four `/architecture/*.html` anchors; after hydration, folding
`core` open shows its 15 sub-layers nested in a container with 32 edges
re-routed; no console errors. 72 tests, tsc, oxlint and the architecture
check all pass, and the site builds 47 pages.
The README claimed a React-capable MDX pipeline was "the one thing the bundle asks of a host". That stopped being true in the previous commit, which added a layer map needing `@xyflow/react` and a hydration directive — and said so twenty lines further down, contradicting the earlier sentence. Replaced with the full list, and a statement of what is deliberately absent: the bundle asks for no Markdown or Rehype plugins, so a host renders it with its pipeline exactly as configured. That is a real constraint rather than an accident — anything requiring a plugin would turn "render the bundle" from a small job into a negotiation with every host.
Generating the architecture bundle was reachable two ways, and neither went through Turborepo. `mise run doc:architecture` made it a repo-root task, and `sync-bundle.mjs` spawned the generator itself with `spawnSync` — a cross-package build step hidden inside a shell call, so Turborepo could not order it, report on it, or know it had happened. Both are gone. `@apps/petrinaut-docs#sync:bundle` now declares `@local/petrinaut-arch-docs#doc:architecture` as a dependency and the script only copies, so the whole chain resolves in the graph: build → sync:bundle → doc:architecture `dev` joins `build` and `lint:tsc` in depending on the one shared `sync:bundle`, rather than chaining it in its package script. Running the sync without a bundle now exits 1 with the Turborepo command to use, instead of failing on a missing directory partway through a copy. `doc:architecture` stays uncached and says why: Turborepo hashes a package plus its dependencies' task *outputs*, and the annotations this reads are source comments in petrinaut and petrinaut-core, which are nobody's output. A cached result would survive an annotation change and go quietly stale — the exact rot this package exists to catch. It declares `outputs` so consumers can depend on the task rather than on the directory existing. Also removes `.claude/launch.json`, and two things the branch had left stale: the CI step's comment still said `@seam`, and the app README still claimed it had no `lint:tsc`.
![semgrep-code-hashintel[bot]](https://avatars.githubusercontent.com/u/22116112?s=60&v=4){kind=small}
| @@ -30306,7 +32036,7 @@ __metadata: | |||
| languageName: node | |||
| linkType: hard | |||
|
|
|||
| "js-yaml@npm:4.3.0, js-yaml@npm:^4.1.0, js-yaml@npm:^4.1.1": | |||
| "js-yaml@npm:4.3.0, js-yaml@npm:^4.1.0, js-yaml@npm:^4.1.1, js-yaml@npm:^4.3.0": | |||
There was a problem hiding this comment.
High severity vulnerability may affect your project—review required:
Line 32039 lists a dependency (js-yaml) with a known High severity vulnerability.
ℹ️ Why this matters
Affected versions of js-yaml are vulnerable to Inefficient Algorithmic Complexity. An attacker can supply a YAML document containing a large !!omap sequence, which js-yaml resolves with a linear duplicate-key scan inside its per-element loop. Resolution is therefore quadratic in the number of entries, so a modestly sized document consumes disproportionate CPU inside the load call and blocks the event loop, resulting in a denial of service.
References: GHSA
To resolve this comment:
Check if you are using js-yaml on the CLI.
- If you're affected, upgrade this dependency to at least version 4.3.1 at yarn.lock.
- If you're not affected, comment
/fp we don't use this [condition]
💬 Ignore this finding
To ignore this, reply with:
/fp <comment>for false positive/ar <comment>for acceptable risk/other <comment>for all other reasons
You can view more details on this finding in the Semgrep AppSec Platform here.
| let parsed: unknown; | ||
|
|
||
| try { | ||
| parsed = load(match[1] ?? ""); |
There was a problem hiding this comment.
High severity and reachable issue identified in your code:
Line 75 has a vulnerable usage of js-yaml, introducing a high severity vulnerability.
ℹ️ Why this is reachable
A reachable issue is a real security risk because your project actually executes the vulnerable code. This issue is reachable because your code uses a certain version of js-yaml.
Affected versions of js-yaml are vulnerable to Inefficient Algorithmic Complexity. An attacker can supply a YAML document containing a large !!omap sequence, which js-yaml resolves with a linear duplicate-key scan inside its per-element loop. Resolution is therefore quadratic in the number of entries, so a modestly sized document consumes disproportionate CPU inside the load call and blocks the event loop, resulting in a denial of service.
References: GHSA
To resolve this comment:
Upgrade this dependency to at least version 4.3.1 at yarn.lock.
💬 Ignore this finding
To ignore this, reply with:
/fp <comment>for false positive/ar <comment>for acceptable risk/other <comment>for all other reasons
You can view more details on this finding in the Semgrep AppSec Platform here.
Nothing inside a node responded to a real mouse click — not the fold buttons, not the layer links. React Flow sets `pointer-events: none` on a node that is not selectable, not draggable and not connectable, and this diagram is configured as all three: dragging a layer box would imply the arrangement means something the reader chose, and it does not. Its interaction model assumes a node is interactive *because* you select, drag or connect it, so a read-only node that merely contains links has no supported shape. Restoring pointer events on the node re-enables its contents without re-enabling dragging. `!important` is needed because the value is written as an inline style, which no stylesheet rule can outrank. My earlier verification of this was wrong, not merely incomplete: it called `element.click()` from script, which dispatches straight to the handler and bypasses hit-testing entirely — so it passed against a node that no pointer could ever reach. Re-checked with a real click through the browser, and by hit-testing each button's centre with `elementFromPoint` before clicking.
The map is read-only: there is nothing in it to select, drag or connect, only things to read and click through to. React Flow is a node-editor, and using it as a viewer cost more than it gave. It had already produced one silent failure — a node that is not selectable, draggable or connectable is marked `pointer-events: none`, which is exactly how a read-only diagram is configured, so nothing inside any node could be clicked. By the end it needed four CSS overrides, one of them `!important`, purely to un-editor it. And it meant two renderers for one picture: a server-rendered SVG and a React Flow tree that had to agree, and did not. We already owned both hard parts. ELK computes every fold state's layout at build time, and the SSR renderer already drew it as SVG with real anchors and routed edges. React Flow was supplying pan, zoom and the fold buttons — and its pan/zoom *is* d3-zoom, which is now used directly. So there is one renderer. The markup the server produces is the markup the browser produces, plus a zoom transform and working controls. Fold buttons are our own SVG, so nothing can intercept them. `@xyflow/react` leaves the bundle's dependency surface; `d3-zoom` and `d3-selection` replace it. Layout now emits routed edge points for all 30 states, not just the one the server rendered — the previous version could rely on React Flow routing the rest. That is most of the payload: 34 KB gzipped of the 50 KB island, with the code itself around 16 KB. Verified by clicking with a real mouse, after hit-testing each control with `elementFromPoint`: 4 nodes to 19 on expanding core, 32 edges re-routed, wheel and drag both moving the transform. The no-JavaScript render is unchanged — 4 boxes, 6 routed edges, 4 layer links. Also fixes pointers left stale by the previous commit: AGENTS.md and three package docs still named `mise run doc:architecture` or the yarn dev script that now skips the task graph. My earlier sweep for these reported none, wrongly — `grep "a\|b"` under this system's grep matches the literal string rather than either alternative, so it searched for something impossible.
`extract` skipped any package whose language was not TypeScript with a bare `continue`, and said nothing. The package still reached the model through `build.ts`, so a Python entry produced a package listed as covered, zero layers, zero files, and a clean CI run. `checkEmptyLayers` cannot catch this: it reports layers that exist and hold no files, and a package with no declarations produces no layers at all. That is the same failure as the path-prefix map this package replaced — a silent fallback that quietly mis-describes what it covers — rebuilt inside the thing meant to remove it. It is now an error naming the package and saying to remove it from the config until an extractor exists.
The interactive map answered "what is the shape of the whole system", which is a question you ask once. The question a reader actually arrives with is "what does this layer touch", and the map was the only place to ask it — the static diagrams existed solely for layers *with* sub-layers, so a leaf, which is where people land, had no picture at all. Every layer now gets a neighbourhood diagram: what it depends on, what depends on it, with the focus outlined and only edges incident to it drawn. Edges among the neighbours are real but belong to those layers' own pages; including them rebuilds the tangle the overview already avoids. Layers with sub-layers keep their drill-down, so a parent answers both questions. That is 44 diagrams — 1 overview, 37 neighbourhoods, 6 drill-downs — against 6 before. They are per-page images fetched on demand rather than one module loaded on the home page, which is what the map cost: 440 KB of pre-computed ELK coordinates for all 30 fold states, more than twice the entire model. Neighbour count is bounded at 12. `core.types` has 18, and the remainder is drawn as a dashed "+6 further layers" node carrying its 8 file-level dependencies, in whichever directions those layers actually use. Elided where it would be unreadable, never dropped where it would read as absent. Removed with the map: `emit/layout.ts`, `emit/collapse.ts`, the graph component and its stylesheet — 1,515 lines — plus `elkjs`, `d3-zoom` and `d3-selection`. No layout runs anywhere but `d2` now. Diagram names are namespaced by directory (`around/`, `within/`) rather than by a name prefix: a layer id is unique only among layer ids, so a flat `around-<id>` would collide with a top-level layer named `around-something`. Also clears `components/` between builds. Only `pages/` and `diagrams/` were rewritten wholesale, so the deleted component survived the first rebuild as an orphan — the same stale-artefact problem that comment already describes, in the one directory it did not cover.
Seven tags, of which one was checked. `@role`, `@boundary` and `@invariant` were prose the generator could not verify, so "CI fails when a declaration stops being true" held for about a third of what you could declare — and the gap was invisible from the outside. This version claims less and means it. Two tags remain: `@layerRoot` names the layer a folder and its descendants form, `@role` says what it is for. Between them they place a node in the graph and label it, which is the whole of what the docs assert. Every surviving check is a statement about the graph — an unannotated file, an ancestor nobody declared, a rule the real import graph violates — and none of them rests on prose. `@layerName`, `@boundary`, `@invariant` and `@entryPoint` are no longer read. `@layer` is deleted outright: it was used zero times, so it had never been exercised against `core.types`, the one case it exists for. The 58 annotations already written stay exactly where they are. Unknown tags are ignored by design, so nothing had to be stripped out of the Petrinaut source to satisfy a generator that has simply stopped looking — and README frontmatter is tolerant rather than strict for the same reason, since rejecting `boundaries:` would have forced those facts to be deleted. Reading one again is a schema change here, not a rewrite of the packages. `@entryPoint` is the one that cost something real: it was the only annotation checkable against evidence the docs did not control, validated against each package's `exports`. Nothing it did was structural — `graph.ts` never read it, and the 177 edges resolve through `deriveAliases` — so the graph is unchanged, but CI no longer holds any documentation claim to account. Pass 2 of the extractor no longer scans tags at all. With no per-file annotations left, a file's contents say nothing about the architecture beyond which folder it sits in, so it is read only for its line count. Also folds in the tidying this left obvious: `toPosix` was copied verbatim into three modules and is now one; `slugForLayer` was a pass-through alias over `layerSlug` in the same file; the authored-page slugs, guide grouping and link resolution walked the same list three times with the `attachTo` validity check duplicated across two of them, and now walk it once; and `bundleTextFiles` stripped a `components/` prefix only to re-add it, which was papering over an inconsistency that left with the generated layouts.
Collaborator
Author
Thanks @lunelson, it actually seems to be a good thing to add on top. For this first version I'll keep things simple so we can already provide some value with docs, see graph of dependencies etc... But definitely a good tool to add in a next step. |
Collaborator
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🌟 What is the purpose of this PR?
Petrinaut's architecture documentation had no mechanism keeping it true. This makes the architecture something you declare next to the code it describes, generates the docs from those declarations, and fails CI when a declaration stops matching the code.
The output is a portable bundle, not a website — the same artefact renders locally, embeds into
hash.dev/docs/petrinaut, or gets handed to an AI agent.flowchart LR A["@layerRoot + @role<br/><i>in doc comments and README frontmatter</i>"] C["content/<br/><i>authored MDX, optional</i>"] E{{"Extractor<br/>+ dependency-cruiser"}} D{{"d2"}} F["bundle/<br/><i>architecture.json · architecture.md<br/>pages · diagrams · components</i>"] G["Starlight site"] H["hash.dev"] I["AI agents"] A --> E E -->|"layer model +<br/>real import graph"| F E --> D D -->|"44 SVGs"| F C --> F F --> G F --> H F --> I🔍 What does this change?
1. Two annotations describe the whole architecture
A declaration is two lines — which layer a folder forms, and what it is for:
A folder
README.mdcan declare the same thing in frontmatter, and its prose becomes that layer's page:Files with no annotation inherit from the nearest declaring ancestor, which is what keeps this proportional to the architecture rather than to the file count: 37 declarations cover 412 files. Everything else — layer sizes, 177 dependency edges, the parent/child tree — is derived from the real TypeScript import graph.
The vocabulary is deliberately this small. Both tags are needed to place a node in the graph and label it; anything more would be a claim the generator cannot check.
2. Every layer gets a diagram
D2, rendered to SVG at build time. 44 diagrams, in three kinds that each bound their node count differently:
Aggregation is honest: an edge appears because imports exist, and counts are real
fileDependenciessums. Neighbours are capped at 12 —core.typeshas 18, so the rest becomes a dashed "+6 further layers" node carrying its 8 dependencies rather than being dropped.3. Hand-written content merges into the generated tree
content/is optional (the system works with the directory absent) and carries the reasoning an import graph cannot express. A page names the layer it explains and nests beneath it:The layer's page gains a Guides section. Because
attachTodecides a page's depth, links are written by name —[text](layer:core.simulation.engine),[text](doc:simulation/memory-model)— and resolved at emit time. Unresolved targets fail the build. Authored pages may import diagram components from the bundle (@diagrams/byte-map).4. CI enforces the structure
lint:arch-docsfails on: an unannotated source file, a layer id implying an undeclared ancestor, a duplicate declaration, a malformed tag, anattachToor link target that does not resolve, a configured package whose language has no extractor, and any dependency violating a rule inarchitecture.config.ts.Every check is a statement about the graph, which is the whole of what the docs assert. Four rules are enforced; the substantive one —
reactmust not depend onui— already held (0 imports against 235 the other way), so it locks in an existing property.5. What this replaces
petrinaut-core/scripts/generate-dependency-diagrams.mjsheld the architecture as ~180 lines ofif (path.startsWith(...))far from the code, with a fallback that silently mis-bucketed anything renamed. It also hard-coded 7 ofpetrinaut-core's 10 entry points, so imports through./ai,./optimizationand./compiled-modelwere absent from the diagrams entirely.docs/architecture/are migrated to authored MDX. Doing so surfaced two stale claims: UUID support described as unimplemented (it is fully implemented), and a sandbox description thinner than the current implementation.🔗 Related links
hash.dev/docs/petrinaut. This produces the exportable artefact that work needs; it publishes nothing itself.Pre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
@hashintel/petrinautandpetrinaut-coreare touched only by comments, READMEs, and removal of a private script and itsdependency-cruiserdevDependency. No runtime code, types or exports change.📜 Does this require a change to the docs?
The user-facing guide (
libs/@hashintel/petrinaut/docs/) is untouched — no UI or behaviour changed.AGENTS.mdgains a section on declaring layers and what CI enforces.🕸️ Does this require a change to the Turbo Graph?
turbo.json's have been updated to reflect thisThe docs build lives entirely in the task graph — no repo-root entry point, nothing shelling out across packages:
doc:architectureis deliberately not cached: Turborepo hashes a package plus its dependencies' task outputs, and the annotations this reads are source comments inpetrinautandpetrinaut-core— nobody's output. A cached bundle would survive an annotation change and go quietly stale, which is the rot this package exists to catch. It declaresoutputs, so consumers depend on the task rather than on the directory existing.Removes
doc:dependency-diagramfrompetrinaut-coreand the repo-rootmise run doc:architecturetask.core.typesis the symptom already in the model: four distinct parents across two packages depend on it, so it is filed undercorewhile behaving like a shared foundation under everything.@role. The structure is checked against the import graph; the one-line description of each layer is prose. Declaring it beside the code makes it likelier to be corrected when that code changes, but CI does not hold it to account.@boundary,@invariant,@entryPointand@layerNameremain in the Petrinaut source and are ignored by this version. Unknown tags are skipped by design and README frontmatter is tolerant, so re-reading one is a schema change in the generator, not a rewrite of the packages. Layer names are meanwhile derived from the id's last segment, so a few read plainly (uirenders as "Ui").petrinaut-cli,petrinaut-websiteandpetrinaut-opthave no declarations. A TypeScript package is a config entry plus one root declaration; the Python app needs docstring extraction, which is not written — configuring a package for a language with no extractor is a hard error rather than a silent no-op.architecture.mdhas no components at all.mise run fix:package-jsoncould not run locally (needs a nightly Cargo feature), sopackage.jsonkey ordering was verified by reading the sorter's field list.🐾 Next steps
demo.petrinaut.org/docsis feasible — the SPA has no catch-all rewrite, and an Astrobase: "/docs"build was tested — but it needsd2added to the Vercel install step, and is worth settling against FE-1157 first so there is one canonical URL.🛡 What tests cover this?
48 tests in
@local/petrinaut-arch-docs:tags.test.ts— tag grammar: multi-line continuation, duplicates, typo suggestions, that a tag named in prose is not a declaration, and that an unread annotation is ignored rather than rejected.frontmatter.test.ts— declarations, malformed YAML, half-written declarations, CRLF, unknown keys ignored.extract.test.ts— inheritance through undeclared folders, uncovered files, stable ordering.check.test.ts— each CI check in both directions: fires when broken, silent when not.emit/mdx.test.ts— link resolution at varying depths, fragments, unresolved targets.Existing suites unaffected: 842 (
petrinaut-core), 187 (petrinaut).❓ How to test this?
turbo run dev --filter @apps/petrinaut-docs # http://localhost:4321Open Architecture for the overview, then any layer — including a leaf such as
core.clipboard— and confirm it opens with a diagram of what it depends on and what depends on it.core.typesexercises the neighbour cap: 12 drawn, plus a dashed "+6 further layers" node.To watch the task graph do its job, delete the bundle first — the build regenerates it, syncs it, then renders, as three ordered tasks:
Confirm the checks hold, and hold in either order — formatting and generation both used to claim the bundle's files:
Then break something and confirm it is caught: change a
role:in any layer-declaring README, point anattachToat a layer that does not exist, or referencelayer:core.nonexistent— each failslint:arch-docswith the offending file named.For the AI-facing side, read
libs/@local/petrinaut-arch-docs/bundle/architecture.md— the whole architecture in one file.