Skip to content

docs: publish AI-agent Markdown mirror - #179

Merged
theduke merged 2 commits into
mainfrom
eco-485-publish-an-ai-agent-markdown-mirror-of-docswasmerio
Aug 31, 2026
Merged

theduke merged 2 commits into
mainfrom
eco-485-publish-an-ai-agent-markdown-mirror-of-docswasmerio

Conversation

@theduke

@theduke theduke commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor

Why

Wasmer documentation is currently published for people reading in a browser. AI agents can use those pages, but they have to scrape rendered HTML, filter out navigation and layout markup, and guess what interactive MDX components mean.

This PR gives agents a direct Markdown version at /ai-docs. The goal is not to maintain a second copy of the docs. Authors keep working in the existing content/ tree, and the Markdown mirror is derived during the build so it cannot quietly drift away from the website.

What changed

The new generator reads every Markdown and MDX page, converts the document into a syntax tree, and writes a matching .md file under public/ai-docs. The existing static export then publishes those files under the same subpath.

Custom components have explicit Markdown counterparts beside their React implementations. Tabs become labeled sections, steps become numbered lists, callouts become blockquotes, cards become links, file trees become text blocks, and imported images are copied into the mirror. Reusable MDX fragments are expanded in place.

This conversion happens at the AST level instead of rendering React to HTML and converting the HTML back. That preserves Markdown structure and makes the supported component surface deliberate. If a new component, dynamic attribute, or executable expression has no safe Markdown representation, the generator stops with the source file and location rather than publishing partial content.

Internal documentation links are rewritten to point directly at /ai-docs/*.md files. The generator also validates generated page and asset targets, and two stale documentation routes now resolve to their current destinations.

The browser footer now includes a visible “Markdown version (for AI)” link with explanatory hover text. The Markdown landing page also ends with a generated, product-grouped index of every document, so an agent can discover the entire mirror from one request instead of guessing routes.

Generation runs automatically before pnpm dev and pnpm build. The generated directory is ignored by Git; only the generator and serializers are committed.

Result

The current documentation produces 88 Markdown pages and 26 imported assets. Visiting /ai-docs redirects to /ai-docs/index.md, while direct page requests are served as text/markdown. The root Markdown file now provides a complete directory of all 88 pages.

Verification

  • pnpm test:ai-docs (six tests)
  • targeted ESLint for the footer, generator, serializers, and tests
  • pnpm build
  • checked the exported page and asset counts
  • fetched the exported footer and confirmed the exact link and title text
  • fetched /ai-docs/index.md, confirmed the Markdown content type, and checked the generated product sections

Linear: https://linear.app/wasmer/issue/ECO-485/publish-an-ai-agent-markdown-mirror-of-docswasmerio

@theduke
theduke requested a review from baalimago August 28, 2026 07:59
@theduke

theduke commented Aug 28, 2026

Copy link
Copy Markdown
Contributor Author

@baalimago not sure how useful this is, but i think it could be a good addition .

@baalimago baalimago left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very solid effort, well needed!

Tiny scope-creep, but I recommend adding support for https://acceptmarkdown.com/ in the server somehow. So basically Accept: text/markdown parsing -> parse markdown render of the same docs. Might require rewrite of the server to make it work automatically?

Image

theduke commented Aug 28, 2026

Copy link
Copy Markdown
Contributor Author

The problem is, this would need a real server instead of a static page.

Copy link
Copy Markdown
Collaborator

True, but as we're in no rush, I think we should scale up and go for a refactor which supports it.

Maybe in a phase 2 though, and we start with this PR, no reason to block one for the otehr

@baalimago baalimago left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should have some pipeline step which validates that we have content in both markdown and statically, otherwise they'll drift

@theduke

theduke commented Aug 28, 2026

Copy link
Copy Markdown
Contributor Author

@baalimago they literally can't drift because the markdown pages get auto-generated based on the existing content....

baalimago commented Aug 28, 2026 •

Copy link
Copy Markdown
Collaborator

Ohh so they go to https://docs.wasmer.io/ai-docs/<path> and get a markdown version rendered?

@theduke

theduke commented Aug 29, 2026

Copy link
Copy Markdown
Contributor Author

Yes.

@theduke
theduke merged commit 8a68528 into main Aug 31, 2026
1 check passed
@theduke
theduke deleted the eco-485-publish-an-ai-agent-markdown-mirror-of-docswasmerio branch August 31, 2026 11:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants