Build rustc and codegen backends with -Zembed-metadata=no

[bjorn3]

View all comments

While this won't change much for rustc and rustc-dev, it should make codegen backends a bit smaller by omitting their crate metadata. In addition it should reduce disk usage while compiling.

Follow up to #145343

[rustbot]

r? @jieyouxu

rustbot has assigned @jieyouxu.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[bjorn3]

cc @Kobzol

@bors try

[rust-bors]

☀️ Try build successful (CI)
Build commit: d9668cf (d9668cf85dd8b3cac5d57f7b5b8b1508112db899, parent: b2a322beb29110e22a1782e2ce5ed2a0719b81ed)

[bjorn3]

And the baseline to compare the disk space against for try builds:

@bors try

[Kobzol]

r? @Kobzol

[rust-bors]

☀️ Try build successful (CI)
Build commit: db6fee5 (db6fee556e8bf32d307f378d2acbb52ba25b5a8b, parent: db1484bdee1cb065b322a5bd512b39c49dadc9e5)

[bjorn3]

For the try build it saves 2GB or 4% of the total space the build takes (46GB -> 44GB). Hoped it would be a bit more, but still not bad I think. And maybe it gives a higher relative saving when not collecting PGO profiles?

@rustbot ready

[Kobzol]

Let's do a full try build to see the codegen backend savings (we can compare them with the parent commit, should be enough), and also run perf., just in case.

@bors try jobs=dist-x86_64-linux @rust-timer queue

[bjorn3]

For cg_clif the .rmeta section of the codegen backend dylib when built from the cg_clif repo is almost 1MB out of the 15MB the full dylib uses. The savings from this PR should be near identical, though the full size of the dylib may be different due to different codegen options.

[rust-bors]

☀️ Try build successful (CI)
Build commit: 2cbe7f7 (2cbe7f7eda161d598e47752d03fc34a1539dad7c, parent: 9b81629631b382fd49ee3a20ac47797b1467e52d)

[rust-timer]

Finished benchmarking commit (2cbe7f7): comparison URL.

Overall result: no relevant changes - no action needed

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results (secondary -1.4%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.6%	[2.5%, 2.7%]	2
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-4.1%	[-6.1%, -2.2%]	3
All ❌✅ (primary)	-	-	0

Cycles

Results (secondary 2.7%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.7%	[2.1%, 3.1%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 476.337s -> 475.884s (-0.10%)
Artifact size: 383.08 MiB -> 383.14 MiB (0.02%)

[Kobzol]

Before:

2026-01-13T17:34:05.4564183Z -rw-r--r-- 1 root root  2.5M Jan 13 16:45 rustc-codegen-cranelift-nightly-x86_64-unknown-linux-gnu.tar.xz
2026-01-13T17:34:05.4565056Z -rw-r--r-- 1 root root  1.1M Jan 13 17:19 rustc-codegen-gcc-nightly-x86_64-unknown-linux-gnu.tar.xz

After:

2026-01-13T22:55:02.7071176Z -rw-r--r-- 1 root root  2.3M Jan 13 22:07 rustc-codegen-cranelift-nightly-x86_64-unknown-linux-gnu.tar.xz
2026-01-13T22:55:02.7072077Z -rw-r--r-- 1 root root  827K Jan 13 22:40 rustc-codegen-gcc-nightly-x86_64-unknown-linux-gnu.tar.xz

And no other components seem to have changed their size. Yeah, pretty nice, a solid ~10%!

[bjorn3]

I can't reproduce the CI failure locally. Could it be related to rust-lang/cargo#16503?

[Kobzol]

Possibly, although my first guess is that we are not copying rustc .rmeta files somewhere in bootstrap.

[Kobzol]

Maybe we could retry this after #151225?

[omnivagant]

this looks like what I came up with while upgrading to 1.94.0 in alpinelinux https://git.alpinelinux.org/aports/commit/?id=e688d5dcba35af7386e9b9f24527ea98dad4040d

[bjorn3]

In this PR, the -Zno-embed-metadata is just moved and also enabled by rustc and codegen backends.

without this we would need to keep the lib*.rmeta files in the main rust
package and have it grow more than twice in size and there would be less
reason of keeping the then much smaller rustc-dev subpackage split

You only need to keep the .rmeta files for the standard library. Without -Zno-embed-metadata they are still present, just embedded in the .rlib and .so files. If anything removing -Zno-embed-metadata increases the size of the package you create as now both libstd.rlib and libstd.so contain a separate copy of the crate metadata rather than sharing a single .rmeta file.

[rustbot]

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

[bjorn3]

Managed to reproduce it locally. Seems to be working now. I wasn't able to unconditionally pass -Zembed-metadata=no though because of rust-lang/cargo#17359.

[bjorn3]

@rustbot ready

[Kobzol]

I'm a bit confused by the move from Builder::cargo to run_cargo. So there were some places that use Builder::cargo, but not run_cargo, for which this broke?

View changes since this review

[bjorn3]

So there were some places that use Builder::cargo, but not run_cargo, for which this broke?

The run-make-support build code doesn't use run_cargo. Previously this would be skipped by the Mode::Std check, but with that check gone, run-make-support would be built using -Zembed-metadata=no, which breaks compiletest.

[Kobzol]

Hmm, it's not just compiletest, rustdoc also breaks if we apply this everywhere:

error: only metadata stub found for `rlib` dependency `rustc_public` please provide path to the corresponding .rmeta file with full metadata
  --> src/librustdoc/lib.rs:35:1
   |
35 | extern crate rustc_driver;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^

error: could not compile `rustdoc` (lib) due to 1 previous error

[bjorn3]

Looks like someone decided to add a dependency on rustc_public to rustc_driver. That would have to be the other way around for rustc_public to function without manually linking in rustc_driver as user of rustc_public. That said, I don't understand how that error you got would be triggered. Both librustc_public.rmeta and librustc_public.rlib should be included in the sysroot.

[Kobzol]

Sorry, I tried this in a fresh checkout and didn't port your rustc_public allowlist changed over. Nevermind.