Skip to content

Commit b29a8a6

Browse files
Docs pipeline Phase 2.2/2.3 + Site.md content fixes + api-block rollout (#24)
Supersedes #23 (same tooling commits + the content fixes, marker rollout, and a Windows fix). Builds on #21, #22 (merged). ## Tooling (P2.2 / P2.3) - **forge-etch single-page Starlight emitter** (`render_single_page`) — for genuinely-empty modules. - **Marker-hybrid API blocks** (`forge-docs-check::apiblock`): `<!-- forge:api -->` regions hold the SDK's public signatures (re-exports/aliases resolved; hook plumbing excluded); `api-block` rule fails CI when stale; `make docs-api` regenerates in place. Bespoke prose outside the markers is never touched. Re-plan recorded as plan §11 (doc-comment migration proven infeasible on `os_compat`). ## Real Site.md content fixes - **runtime:dock** — documents the M5 events: `info()`, `nextMenuEvent()`, `onMenuItemClick()` + `MenuClickEvent`/`MenuItemClickListener`. - **runtime:shell** — `setCwd()` → `chdir()` (the real export) + new `kill()`. - **crates/forge.md** — adds `forge smelt` and `forge docs`. - Drift baseline 17 → 9. ## Marker rollout `<!-- forge:api -->` blocks added + populated on 14 API pages (clean `## API Reference` anchor). The other ~12 pages use heterogeneous headings (no common anchor) and are left for per-page placement — their methods are already enforced by the bidirectional `api-drift` rule. ## Fixes - **Windows**: normalize CRLF in `find_block` (a CRLF checkout made the api-block gate report phantom drift on windows-latest only). CRLF regression test added. All `forge-docs-check` + forge-etch tests, `fmt`, `clippy -D warnings` pass on Linux/macOS; the CRLF fix addresses the windows-latest failure seen on #23. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
2 parents e78eab3 + e1c0856 commit b29a8a6

27 files changed

Lines changed: 950 additions & 38 deletions

‎Makefile‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,18 @@
11
# Forge developer convenience targets.
22
# CI enforces fmt, clippy (-D warnings), tests, and the documentation drift gate.
33

4-
.PHONY: docs-check docs-report docs-baseline install-hooks fmt clippy test check ci
4+
.PHONY: docs-check docs-report docs-api docs-baseline install-hooks fmt clippy test check ci
55

66
# Documentation drift gate (baseline-aware): fails only on NEW drift beyond the
77
# recorded baseline. Same logic as CI and the pre-commit hook.
88
docs-check:
99
cargo test -q -p forge-docs-check --test docs_sync
1010

11+
# Regenerate every <!-- forge:api --> signature block in place from the SDK
12+
# (marker-hybrid generator). Bespoke prose outside the markers is untouched.
13+
docs-api:
14+
cargo run -q -p forge-docs-check -- --write-api-blocks
15+
1116
# Full drift report: list ALL current drift (fails if any exists at all).
1217
# Useful for seeing the whole backlog; not the burn-down gate.
1318
docs-report:
Lines changed: 326 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,326 @@
1+
//! Marker-hybrid API blocks.
2+
//!
3+
//! The design decision behind this (recorded in
4+
//! `docs/plans/2026-06-06-19-docs-autogeneration-pipeline.md`): the rich API
5+
//! pages are 400–600 lines of bespoke hand-authored prose that cannot live in
6+
//! doc-comments or be reproduced by a templated generator. So instead of
7+
//! generating whole pages, a page opts in by embedding a single marker region
8+
//! under its `## API Reference` heading:
9+
//!
10+
//! - an opening `<!-- forge:api -->` marker,
11+
//! - a generated fenced `typescript` list of the module's public signatures
12+
//! (e.g. `info(): OsInfo` / `pathSep(): string`),
13+
//! - a closing `<!-- /forge:api -->` marker.
14+
//!
15+
//! Everything outside the markers — the bespoke `### info()` prose, examples,
16+
//! tables — is authored and never touched.
17+
//!
18+
//! The block holds the current public signatures of the module's
19+
//! `sdk/runtime.<mod>.ts` (excluding hook plumbing). [`check`] fails when a block
20+
//! is stale; [`write_all`] regenerates every block in place from the SDK.
21+
22+
use crate::checks::{read_optional, HOOK_PLUMBING};
23+
use crate::discovery::Workspace;
24+
use crate::Finding;
25+
use regex::Regex;
26+
use std::collections::{HashMap, HashSet};
27+
28+
pub const BLOCK_OPEN: &str = "<!-- forge:api -->";
29+
pub const BLOCK_CLOSE: &str = "<!-- /forge:api -->";
30+
31+
/// Extract the public function signatures from an SDK module's source, in
32+
/// declaration order, excluding hook/handler plumbing. A signature is the text
33+
/// from the function name up to the body brace, whitespace-collapsed
34+
/// (e.g. `info(): OsInfo`).
35+
pub fn public_signatures(sdk_src: &str) -> Vec<String> {
36+
// 1. Map every declared function name -> the signature tail after the name,
37+
// i.e. "(params): ret". Covers both `export function foo` and bare
38+
// `function foo` (which modules like runtime:timers/webview declare and
39+
// then re-export via `export { foo }`).
40+
let decl_re =
41+
Regex::new(r"(?m)^\s*(?:export\s+)?(?:async\s+)?function\s*\*?\s*([A-Za-z0-9_]+)")
42+
.expect("valid decl regex");
43+
let mut decl_tail: HashMap<String, String> = HashMap::new();
44+
for cap in decl_re.captures_iter(sdk_src) {
45+
let name = cap[1].to_string();
46+
let name_start = cap.get(1).expect("name group").start();
47+
if let Some(brace_rel) = sdk_src[name_start..].find('{') {
48+
let collapsed = sdk_src[name_start..name_start + brace_rel]
49+
.split_whitespace()
50+
.collect::<Vec<_>>()
51+
.join(" ");
52+
let tail = collapsed
53+
.strip_prefix(&name)
54+
.unwrap_or(&collapsed)
55+
.to_string();
56+
decl_tail.entry(name).or_insert(tail);
57+
}
58+
}
59+
60+
// 2. Collect the public surface as (exported_name, source_name) pairs:
61+
// `export function foo` (source == export), and `export { a, b as c }`
62+
// re-export lists (source is the name before `as`, export is after).
63+
let mut public: Vec<(String, String)> = Vec::new();
64+
let exp_fn = Regex::new(r"(?m)^\s*export\s+(?:async\s+)?function\s*\*?\s*([A-Za-z0-9_]+)")
65+
.expect("valid export-fn regex");
66+
for cap in exp_fn.captures_iter(sdk_src) {
67+
let n = cap[1].to_string();
68+
public.push((n.clone(), n));
69+
}
70+
let exp_list = Regex::new(r"(?m)^\s*export\s*\{([^}]*)\}").expect("valid export-list regex");
71+
for cap in exp_list.captures_iter(sdk_src) {
72+
let cleaned: String = cap[1]
73+
.lines()
74+
.map(|l| l.split("//").next().unwrap_or(""))
75+
.collect::<Vec<_>>()
76+
.join("\n");
77+
for item in cleaned.split(',') {
78+
let parts: Vec<&str> = item
79+
.split(" as ")
80+
.map(str::trim)
81+
.filter(|s| !s.is_empty())
82+
.collect();
83+
if parts.is_empty() {
84+
continue;
85+
}
86+
let source = parts[0].trim_end_matches(',').trim().to_string();
87+
let exported = parts
88+
.last()
89+
.unwrap()
90+
.trim_end_matches(',')
91+
.trim()
92+
.to_string();
93+
let ident = |s: &str| s.chars().all(|c| c.is_ascii_alphanumeric() || c == '_');
94+
if source.is_empty() || !ident(&source) || !ident(&exported) {
95+
continue;
96+
}
97+
public.push((exported, source));
98+
}
99+
}
100+
101+
// 3. Render `exportedName(params): ret`, excluding hook plumbing, first
102+
// occurrence wins, only for names that have a known declaration.
103+
let mut seen = HashSet::new();
104+
let mut sigs = Vec::new();
105+
for (exported, source) in public {
106+
if HOOK_PLUMBING.contains(&exported.as_str()) || !seen.insert(exported.clone()) {
107+
continue;
108+
}
109+
if let Some(tail) = decl_tail.get(&source) {
110+
sigs.push(format!("{exported}{tail}"));
111+
}
112+
}
113+
sigs
114+
}
115+
116+
/// Render the body that belongs *between* the markers (a fenced TypeScript
117+
/// signature list with a generated-by notice).
118+
pub fn render_block_body(module: &str, signatures: &[String]) -> String {
119+
let mut out = String::new();
120+
out.push_str(&format!(
121+
"<!-- generated from sdk/runtime.{module}.ts — edit signatures in the SDK, run `make docs-api` to refresh -->\n"
122+
));
123+
out.push_str("```typescript\n");
124+
for sig in signatures {
125+
out.push_str(sig);
126+
out.push('\n');
127+
}
128+
out.push_str("```");
129+
out
130+
}
131+
132+
/// Locate a marker region in `page`. Returns `(open_start, close_end, body)`
133+
/// where `body` is the exact text strictly between the open and close markers
134+
/// (trimmed of the single surrounding newlines).
135+
fn find_block(page: &str) -> Option<(usize, usize, String)> {
136+
let open = page.find(BLOCK_OPEN)?;
137+
let after_open = open + BLOCK_OPEN.len();
138+
let close_rel = page[after_open..].find(BLOCK_CLOSE)?;
139+
let close = after_open + close_rel;
140+
let close_end = close + BLOCK_CLOSE.len();
141+
// Normalize CRLF -> LF so the comparison is line-ending-agnostic: on Windows
142+
// git may check the docs out with CRLF, while the generated `expected` body
143+
// uses LF. Without this the gate reports a phantom "stale" block on Windows.
144+
let body = page[after_open..close]
145+
.replace("\r\n", "\n")
146+
.trim_matches('\n')
147+
.to_string();
148+
Some((open, close_end, body))
149+
}
150+
151+
/// Rule `api-block`: every page that opts in (contains `<!-- forge:api -->`) has
152+
/// a block whose body matches the current SDK signatures.
153+
pub fn check(ws: &Workspace) -> Vec<Finding> {
154+
let api_dir = ws.docs_dir().join("api");
155+
let mut findings = Vec::new();
156+
for module in &ws.sdk_modules {
157+
let page_path = api_dir.join(format!("{}.md", module.api_page_stem()));
158+
let page = match read_optional(&page_path) {
159+
Some(p) => p,
160+
None => continue,
161+
};
162+
if !page.contains(BLOCK_OPEN) {
163+
continue; // not opted in
164+
}
165+
let sdk = match read_optional(&module.path) {
166+
Some(s) => s,
167+
None => continue,
168+
};
169+
let expected = render_block_body(&module.name, &public_signatures(&sdk));
170+
match find_block(&page) {
171+
Some((_, _, body)) if body == expected => {}
172+
Some((_, _, _)) => findings.push(Finding::new(
173+
"api-block",
174+
format!(
175+
"runtime:{}: the <!-- forge:api --> block in {}.md is stale; run `make docs-api` to refresh it",
176+
module.name,
177+
module.api_page_stem()
178+
),
179+
)),
180+
None => findings.push(Finding::new(
181+
"api-block",
182+
format!(
183+
"runtime:{}: {}.md has an opening <!-- forge:api --> with no closing <!-- /forge:api -->",
184+
module.name,
185+
module.api_page_stem()
186+
),
187+
)),
188+
}
189+
}
190+
findings
191+
}
192+
193+
/// Regenerate every opted-in block in place from the SDK. Returns the list of
194+
/// page paths that were rewritten (only pages whose block actually changed).
195+
pub fn write_all(ws: &Workspace) -> std::io::Result<Vec<std::path::PathBuf>> {
196+
let api_dir = ws.docs_dir().join("api");
197+
let mut written = Vec::new();
198+
for module in &ws.sdk_modules {
199+
let page_path = api_dir.join(format!("{}.md", module.api_page_stem()));
200+
let page = match read_optional(&page_path) {
201+
Some(p) => p,
202+
None => continue,
203+
};
204+
if !page.contains(BLOCK_OPEN) {
205+
continue;
206+
}
207+
let sdk = match read_optional(&module.path) {
208+
Some(s) => s,
209+
None => continue,
210+
};
211+
let expected = render_block_body(&module.name, &public_signatures(&sdk));
212+
if let Some((open, close_end, body)) = find_block(&page) {
213+
if body == expected {
214+
continue;
215+
}
216+
let mut updated = String::with_capacity(page.len());
217+
updated.push_str(&page[..open]);
218+
updated.push_str(BLOCK_OPEN);
219+
updated.push('\n');
220+
updated.push_str(&expected);
221+
updated.push('\n');
222+
updated.push_str(BLOCK_CLOSE);
223+
updated.push_str(&page[close_end..]);
224+
std::fs::write(&page_path, updated)?;
225+
written.push(page_path);
226+
}
227+
}
228+
Ok(written)
229+
}
230+
231+
#[cfg(test)]
232+
mod tests {
233+
use super::*;
234+
235+
#[test]
236+
fn signatures_exclude_hook_plumbing() {
237+
let sdk = r#"
238+
export function info(): OsInfo { return ops.info(); }
239+
export function pathSep(): string { return ops.pathSep(); }
240+
export function onBefore<T extends OpName>(x: T): void {}
241+
export async function invokeHandler(name: string): Promise<unknown> {}
242+
"#;
243+
assert_eq!(
244+
public_signatures(sdk),
245+
vec![
246+
"info(): OsInfo".to_string(),
247+
"pathSep(): string".to_string()
248+
]
249+
);
250+
}
251+
252+
#[test]
253+
fn signatures_resolve_reexports_and_aliases() {
254+
// Declared as bare `function` then re-exported (timers/webview pattern),
255+
// including an `as` alias.
256+
let sdk = r#"
257+
function setTimeout(cb: () => void, delay?: number): number { return 0; }
258+
async function execute(cmd: string): Promise<string> { return ""; }
259+
export { setTimeout };
260+
export { execute as exec };
261+
"#;
262+
assert_eq!(
263+
public_signatures(sdk),
264+
vec![
265+
"setTimeout(cb: () => void, delay?: number): number".to_string(),
266+
// alias `exec` rendered with execute's parameters
267+
"exec(cmd: string): Promise<string>".to_string(),
268+
]
269+
);
270+
}
271+
272+
#[test]
273+
fn find_block_extracts_body() {
274+
let page = "intro\n<!-- forge:api -->\nBODY\nLINE2\n<!-- /forge:api -->\noutro\n";
275+
let (_, _, body) = find_block(page).expect("block present");
276+
assert_eq!(body, "BODY\nLINE2");
277+
}
278+
279+
#[test]
280+
fn find_block_normalizes_crlf() {
281+
// Windows git checkout may use CRLF; the extracted body must match the
282+
// LF-generated `expected`, so the gate is line-ending-agnostic.
283+
let page =
284+
"intro\r\n<!-- forge:api -->\r\nBODY\r\nLINE2\r\n<!-- /forge:api -->\r\nouttro\r\n";
285+
let (_, _, body) = find_block(page).expect("block present");
286+
assert_eq!(body, "BODY\nLINE2");
287+
}
288+
289+
#[test]
290+
fn public_signatures_is_line_ending_agnostic() {
291+
// Same source, LF vs CRLF, must yield identical signatures — otherwise a
292+
// Windows checkout would generate a different block than the gate expects.
293+
let lf = "export function info(): OsInfo { return x; }\n\
294+
export function pathSep(): string { return y; }\n";
295+
let crlf = lf.replace('\n', "\r\n");
296+
assert_eq!(public_signatures(lf), public_signatures(&crlf));
297+
assert!(!public_signatures(&crlf).is_empty());
298+
}
299+
300+
#[test]
301+
fn whole_block_round_trips_through_crlf() {
302+
// Render an LF block, simulate a CRLF on-disk copy, and confirm the
303+
// extracted body still equals the freshly-rendered (LF) expected body.
304+
let sigs = vec![
305+
"info(): OsInfo".to_string(),
306+
"pathSep(): string".to_string(),
307+
];
308+
let expected = render_block_body("os_compat", &sigs);
309+
let on_disk = format!(
310+
"# page\r\n\r\n{BLOCK_OPEN}\r\n{}\r\n{BLOCK_CLOSE}\r\n\r\n## next\r\n",
311+
expected.replace('\n', "\r\n")
312+
);
313+
let (_, _, body) = find_block(&on_disk).expect("block present");
314+
assert_eq!(body, expected, "CRLF on-disk block must match LF expected");
315+
}
316+
317+
#[test]
318+
fn render_block_body_is_deterministic() {
319+
let sigs = vec!["info(): OsInfo".to_string()];
320+
let a = render_block_body("os_compat", &sigs);
321+
let b = render_block_body("os_compat", &sigs);
322+
assert_eq!(a, b);
323+
assert!(a.contains("```typescript"));
324+
assert!(a.contains("info(): OsInfo"));
325+
}
326+
}

‎crates/forge-docs-check/src/checks/api_drift.rs‎

Lines changed: 1 addition & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -10,26 +10,11 @@
1010
//! SDK module *should* have a page is a generation/policy concern (Phase 2),
1111
//! not drift — flagging intentionally-internal modules would make CI noisy.
1212
13-
use crate::checks::read_optional;
13+
use crate::checks::{read_optional, HOOK_PLUMBING};
1414
use crate::discovery::Workspace;
1515
use crate::Finding;
1616
use regex::Regex;
1717

18-
/// Generic hook/handler plumbing emitted into every extensibility-enabled SDK
19-
/// module. These are not part of a module's documented surface, so neither
20-
/// direction should require them.
21-
const HOOK_PLUMBING: &[&str] = &[
22-
"invokeHandler",
23-
"hasHandler",
24-
"listHandlers",
25-
"onAfter",
26-
"onBefore",
27-
"onError",
28-
"registerHandler",
29-
"removeAllHooks",
30-
"removeHandler",
31-
];
32-
3318
pub fn check(ws: &Workspace) -> Vec<Finding> {
3419
let api_dir = ws.docs_dir().join("api");
3520
// `export function foo`, `export async function foo`, `export async function* foo`.

‎crates/forge-docs-check/src/checks/counts.rs‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -157,9 +157,15 @@ fn walk_markdown(dir: &std::path::Path) -> Vec<std::path::PathBuf> {
157157
out
158158
}
159159

160+
/// Workspace-relative path as a string with forward slashes on every platform.
161+
/// Findings are compared against a checked-in baseline; using the OS-native
162+
/// separator (`\` on Windows) would make the same drift produce a different
163+
/// message on Windows and spuriously fail the gate there.
160164
fn rel(ws: &Workspace, path: &std::path::Path) -> String {
161165
path.strip_prefix(&ws.root)
162166
.unwrap_or(path)
163-
.display()
164-
.to_string()
167+
.components()
168+
.map(|c| c.as_os_str().to_string_lossy())
169+
.collect::<Vec<_>>()
170+
.join("/")
165171
}

0 commit comments

Comments
 (0)