rustdoc: rename the doc parts metadata params - #159415
Conversation
|
rustbot has assigned @GuillaumeGomez. Use Why was this reviewer chosen?The reviewer was selected based on:
|
rustbot
added
A-rustdoc-js
|
CC @camelid |
|
Looks good to me, thanks! @bors r+ rollup |
rust-bors
Bot
added
S-waiting-on-bors
…, r=GuillaumeGomez rustdoc: rename the doc parts metadata params Written in response to rust-lang#152902 (comment)
…uwer Rollup of 16 pull requests Successful merges: - #150732 (Convert `-Ctarget-cpu` into a target-modifier for AVR, AMDGCN and NVPTX ) - #159301 (Update Enzyme to handle LLVM23) - #159365 (fix: point at method call chain when a return-position `impl Trait` assoc type diverges) - #159402 (Clarify safety requirements for SIMD shl/shr and masked load/store) - #159410 (rustdoc: remove old `--emit` types) - #158398 (Comment about empty run_passes, fixup of #158040) - #158843 (Fix ICE in `write_interface` when the interface file can't be written) - #159302 (Implement `Debug` helpers via `Cell`) - #159332 (Honor field-level lint attributes in non_snake_case) - #159386 (add a fallback for `fmuladdf*`) - #159391 (Update tests for LLVM 23) - #159400 (Update books) - #159401 (Gate `tests/debuginfo/function-call.rs` on min GDB 15.1) - #159404 ([aarch64][win] Pass oversized c-variadic args indirectly on Arm64EC) - #159405 (Manually implement Clone for GrowableBitSet) - #159415 (rustdoc: rename the doc parts metadata params)
…uwer Rollup of 17 pull requests Successful merges: - #159301 (Update Enzyme to handle LLVM23) - #159365 (fix: point at method call chain when a return-position `impl Trait` assoc type diverges) - #159402 (Clarify safety requirements for SIMD shl/shr and masked load/store) - #159408 (rustc_data_structures: Expand documentation for rustc jobserver APIs) - #159410 (rustdoc: remove old `--emit` types) - #158398 (Comment about empty run_passes, fixup of #158040) - #158843 (Fix ICE in `write_interface` when the interface file can't be written) - #159302 (Implement `Debug` helpers via `Cell`) - #159332 (Honor field-level lint attributes in non_snake_case) - #159340 (Rename `errors.rs` file to `diagnostics.rs` (14/N)) - #159386 (add a fallback for `fmuladdf*`) - #159391 (Update tests for LLVM 23) - #159400 (Update books) - #159401 (Gate `tests/debuginfo/function-call.rs` on min GDB 15.1) - #159404 ([aarch64][win] Pass oversized c-variadic args indirectly on Arm64EC) - #159405 (Manually implement Clone for GrowableBitSet) - #159415 (rustdoc: rename the doc parts metadata params)
| When `--write-doc-meta-dir` is supplied, rustdoc will write the crate's metadata to that directory. | ||
| If this parameter is supplied but `--read-doc-meta-dir` isn't, it runs in *intermediate mode*: | ||
| some pages may be written to the output dir, but there is a lot of functionality that won't work | ||
| until rustdoc is run in *finalize mode*. | ||
|
|
||
| When `--read-doc-meta-dir` is supplied, rustdoc runs in *finalize mode*. It will read the data from | ||
| the supplied directory, and will write it to the doc output directory in the form that the web | ||
| frontend will use. | ||
|
|
||
| If both `--write-doc-meta-dir` and `--read-doc-meta-dir` are specified, the crate metadata will be | ||
| written to both the HTML `--out-dir` and to the supplied `--write-doc-meta-dir`. |
There was a problem hiding this comment.
I like these names (better than the previous and what I'd proposed), and I think this is a good interface given the current state of CCI. Long-term, once full metadata is implemented, I don't want rustdoc to emit HTML at all until its last invocation in a workspace (i.e. the last crate, or maybe even a step after that). However, that's obviously a ways off, so we shouldn't block CCI on that. But I would like to be forwards-compatible with it as much as possible.
So, I'm wondering if we should forbid the combination of --write-doc-meta-dir and --read-doc-meta-dir. Right now, there is no use for it as far as I know. Notice that the original --merge flag only had three options, not four, for this reason.
To be clear, my long-term vision for rustdoc usage:
- no flags: legacy shared mode
- just write: produce only metadata for the current crate, no HTML -- but the metadata includes all of
cleanand anything else needed to generate HTML later - read and write: read in metadata for dependencies so anything that needs info about them, e.g., inlining will work. produce only metadata for the current crate, no HTML
- just read: read in metadata for all documented crates in the workspace, and produce a static HTML site for all of them
To implement this in a way that is backwards-compatible with CCI, we'd probably need to add a--defer-htmlflag that switches to the mode I just described. Note that this mode would require feeding in metadata to all intermediate crates, not just the final crate like in CCI.
The reason to delay HTML generation until the end is then we have complete information about the workspace and can remove all of our hacks that use JS to inject downstream implementors, etc. Rustdoc really wants to operate at the workspace level, not the crate level where it actually operators, which is why we have so much pain from cross-crate inlining and other features.
There was a problem hiding this comment.
I don't want rustdoc to emit HTML at all until its last invocation in a workspace (i.e. the last crate
It sounds like you’re expecting rustdoc to perform finalization at the same time that it documents the last crate? That makes sense, and that’s why the current system lets you do read and write-doc-meta-dir at the same time.
It was designed that way because the build system might schedule a different crate to be documented last in a different run. For example, with Cargo, you would supply the -p parameter. To make sure this doesn’t cause problems, you’re expected to write doc metadata for every crate in your workspace, even the one that gets documented last and used for finalization.
One thing that does concern me, though, is that Cargo always does finalization as a separate step, because the finalization step isn’t parallelized. It might not make sense to worry about doing finalization and documentation at the same time, if no builds systems want to do that.
There was a problem hiding this comment.
Yes, exactly. I would prefer that finalization happens as a separate step (like you said Cargo does) after the last crate. In the future, once finalization will do all the HTML generation too, this would also get rid of our weird quirk where the last documented crate is considered the main one.
I suppose supporting --read and --write together is fine for CCI. The future version of rustdoc that defers all HTML generation until the end will need to be gated behind a new flag anyway, so we can change the interface when that flag is passed.
There was a problem hiding this comment.
However, if we don't need to support simultaneous documenting-last-crate and finalizing, it'd be better to save ourselves the trouble and just disallow it. I'm a bit rusty on the details of the CCI implementation, but it sounds like rustdoc already supports finalizing as a separate step like you said?
Sorry if this all seems a bit pedantic, I just want to make sure we don't paint ourselves into a corner.
There was a problem hiding this comment.
Yes, it's possible. I'll do that, then.
There was a problem hiding this comment.
Thanks! And thanks for taking action to make this PR, much appreciated.
Rollup merge of #159415 - notriddle:rename-parts-to-dep-meta, r=GuillaumeGomez rustdoc: rename the doc parts metadata params Written in response to #152902 (comment)
Follow up rust-lang/rust#159415 The rustdoc CLI interface was slightly simplified for CCI, and should be usable even if more data is moved through this metadata system.
…uwer Rollup of 17 pull requests Successful merges: - rust-lang/rust#159301 (Update Enzyme to handle LLVM23) - rust-lang/rust#159365 (fix: point at method call chain when a return-position `impl Trait` assoc type diverges) - rust-lang/rust#159402 (Clarify safety requirements for SIMD shl/shr and masked load/store) - rust-lang/rust#159408 (rustc_data_structures: Expand documentation for rustc jobserver APIs) - rust-lang/rust#159410 (rustdoc: remove old `--emit` types) - rust-lang/rust#158398 (Comment about empty run_passes, fixup of rust-lang/rust#158040) - rust-lang/rust#158843 (Fix ICE in `write_interface` when the interface file can't be written) - rust-lang/rust#159302 (Implement `Debug` helpers via `Cell`) - rust-lang/rust#159332 (Honor field-level lint attributes in non_snake_case) - rust-lang/rust#159340 (Rename `errors.rs` file to `diagnostics.rs` (14/N)) - rust-lang/rust#159386 (add a fallback for `fmuladdf*`) - rust-lang/rust#159391 (Update tests for LLVM 23) - rust-lang/rust#159400 (Update books) - rust-lang/rust#159401 (Gate `tests/debuginfo/function-call.rs` on min GDB 15.1) - rust-lang/rust#159404 ([aarch64][win] Pass oversized c-variadic args indirectly on Arm64EC) - rust-lang/rust#159405 (Manually implement Clone for GrowableBitSet) - rust-lang/rust#159415 (rustdoc: rename the doc parts metadata params)
…, r=camelid Fix, simplify, and document doc meta finalize mode Follow up rust-lang#159415 (comment) Get rid of the mode where you can finalize the CCI and generate more docs at the same time. It isn't used in Cargo, and probably won't be used elsewhere? Fixes a bug where the crate index, settings page, and help page aren’t generated at finalize time. Update documentation. Move CCI tests to run-make, so that we can test the finalize step’s CLI.
Rollup merge of #159473 - notriddle:rename-parts-to-dep-meta, r=camelid Fix, simplify, and document doc meta finalize mode Follow up #159415 (comment) Get rid of the mode where you can finalize the CCI and generate more docs at the same time. It isn't used in Cargo, and probably won't be used elsewhere? Fixes a bug where the crate index, settings page, and help page aren’t generated at finalize time. Update documentation. Move CCI tests to run-make, so that we can test the finalize step’s CLI.
Fix, simplify, and document doc meta finalize mode Follow up rust-lang/rust#159415 (comment) Get rid of the mode where you can finalize the CCI and generate more docs at the same time. It isn't used in Cargo, and probably won't be used elsewhere? Fixes a bug where the crate index, settings page, and help page aren’t generated at finalize time. Update documentation. Move CCI tests to run-make, so that we can test the finalize step’s CLI.
Fix, simplify, and document doc meta finalize mode Follow up rust-lang/rust#159415 (comment) Get rid of the mode where you can finalize the CCI and generate more docs at the same time. It isn't used in Cargo, and probably won't be used elsewhere? Fixes a bug where the crate index, settings page, and help page aren’t generated at finalize time. Update documentation. Move CCI tests to run-make, so that we can test the finalize step’s CLI.
…nto unified docs build_all_docs.sh cross-compiles arch/* and select chips/* crates against their real target triples instead of the host, and merges the results into one unified doc tree using rustdoc's cross-crate-info merging mechanism (RFC 3662, "Mergeable rustdoc cross-crate info": --write-doc-meta-dir/--read-doc-meta-dir, gated behind -Z unstable-options). This lets those crates drop the `doc` half of their cfg gates, since their real (not mocked) implementation gets documented directly. This relies entirely on rustdoc's own merge logic -- the script never parses or rewrites rustdoc's own output format -- but the flags it's built on are still actively changing shape, not close to stabilizing: - rust-lang/rust#130676 is the tracking issue. - rust-lang/rust#152902, the stabilization PR, is S-blocked/needs-fcp. Its FCP was cancelled 2026-07-17 pending interface changes, and T-rustdoc has floated replacing the whole mechanism with info embedded in .rmeta files instead. As of 2026-08-07 the team wants to "dogfood this feature a bit" before reconsidering stabilization. - rust-lang/rust#159415 (merged 2026-07-17) renamed the RFC's original --parts-out-dir / --include-parts-dir / --merge=none|shared|finalize flags to today's --write-doc-meta-dir / --read-doc-meta-dir. - rust-lang/rust#159473 (merged 2026-08-06) removed the mode this script's finalize step would otherwise need (passing both --write-doc-meta-dir and --read-doc-meta-dir to one invocation) -- that combination is a hard error on recent nightlies, and finalize mode (--read-doc-meta-dir) takes no crate source at all. That's why the finalize step below runs bare `rustdoc`, not `cargo rustdoc -p ...`. - Cargo has a separate, less mature automation layer for this (-Z rustdoc-mergeable-info, tracked by rust-lang/cargo#16306) -- not an option here, since it's explicitly same-target-only and this script's whole point is merging docs built for different targets. Expect to revisit this script on future toolchain bumps. CRATE_TARGETS is a plain indexed array of "crate:target-triple" entries (rather than an associative array via `declare -A`) deliberately: macOS ships bash 3.2, which predates associative arrays entirely. Crates whose dependent boards/chips actually build for more than one real target -- where picking one is a deliberate, somewhat arbitrary call rather than a fact about the crate -- are called out separately in SHARED_CRATE_TARGETS and folded into CRATE_TARGETS before use. Verified: run the script locally; diff the set of crates listed in the merged `crates.js` against the previous single-pass build's output -- every crate still appears. Spot-checked that `trait.impl/kernel/platform/mpu/trait.MPU.js` shows implementors from more than one arch crate, confirming cross-crate merging works. Verified against both the pinned nightly and a nightly after rust-lang/rust#159473, installing missing target components as needed, confirming the finalize step works on both sides of that upstream change. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018kKWYpBqL9jafD7rtvt9qH
4 participants
Written in response to #152902 (comment)