Skip to content

move mir-opt miri tests to CI logic - #160422

Open
RalfJung wants to merge 1 commit into
rust-lang:mainfrom
RalfJung:miri-mir-opt-tests
Open

move mir-opt miri tests to CI logic#160422
RalfJung wants to merge 1 commit into
rust-lang:mainfrom
RalfJung:miri-mir-opt-tests

Conversation

@RalfJung

@RalfJung RalfJung commented Aug 3, 2026

Copy link
Copy Markdown
Member

It is useful to run Miri tests with mir-opts as that sometimes finds miscompilations.

However, doing so on every ./x test miri is annoying as it makes that take a lot longer, and it is inconsistent with ./miri test in the Miri repo. So remove this logic there, and instead have the check-miri script in CI invoke the test suite a 2nd time with mir-opts enabled.

Cc @rust-lang/miri

@rustbot rustbot added A-CI Area: Our Github Actions CI A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Aug 3, 2026
@rustbot

rustbot commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

  • Owners of files modified in this PR: bootstrap
  • bootstrap expanded to 6 candidates
  • Random selection from Mark-Simulacrum, clubby789, jieyouxu

@Kobzol

Kobzol commented Aug 4, 2026

Copy link
Copy Markdown
Member

In general, I like to make bootstrap steps more composable and remove similar special cases from them. On the other hand, adding bespoke test commands into the CI bash scripts runs into the risk of making it less obvious to people what failed on CI, and how to reproduce it locally. Especially since this is configured via an environment variable, and bootstrap will not print it after a failure, unlike the command-line args.

That being said, we already have a bunch of similar commands on CI, so no big deal. Just a remark.

@RalfJung

RalfJung commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

Hm actually strangely this does not seem to run any mir-opt tests any more. In the logs I just see

2026-08-03T09:44:14.6157327Z �[1m�[92m     Running�[0m tests/ui.rs (obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/build/miri/855274d73082e308/out/ui-855274d73082e308)
2026-08-03T09:44:14.6364854Z ## Running pass ui tests in tests/pass for x86_64-unknown-linux-gnu
2026-08-03T09:44:14.6383112Z    Compiler: "MIRI_ENV_VAR_TEST"="0" "MIRI_TEMP"="/tmp/miri-uitest-YbtdTO" "RUST_BACKTRACE"="1" /checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/miri "--error-format=json" "-Dwarnings" "-Dunused" "-Ainternal_features" "-Zui-testing" "--sysroot=/checkout/obj/build/x86_64-unknown-linux-gnu/miri-sysroot" "-O" "-Zmir-opt-level=4" "-Cdebug-assertions=yes" "--out-dir" OUT_DIR
2026-08-03T09:44:14.7388489Z 
2026-08-03T09:44:14.7389319Z test result: ok. 375 filtered out
2026-08-03T09:44:14.7389840Z 
2026-08-03T09:44:14.7390360Z ## Running pass ui tests in tests/pass-dep for x86_64-unknown-linux-gnu
2026-08-03T09:44:14.7393389Z    Compiler: "MIRI_ENV_VAR_TEST"="0" "MIRI_TEMP"="/tmp/miri-uitest-YbtdTO" "RUST_BACKTRACE"="1" /checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/miri "--error-format=json" "-Dwarnings" "-Dunused" "-Ainternal_features" "-Zui-testing" "--sysroot=/checkout/obj/build/x86_64-unknown-linux-gnu/miri-sysroot" "-O" "-Zmir-opt-level=4" "-Cdebug-assertions=yes" "--out-dir" OUT_DIR
2026-08-03T09:44:14.8393102Z 
2026-08-03T09:44:14.8393588Z test result: ok. 68 filtered out
2026-08-03T09:44:14.8393883Z 
2026-08-03T09:44:14.8394180Z ## Running pass-native ui tests in tests/pass for x86_64-unknown-linux-gnu
2026-08-03T09:44:14.8396350Z    Compiler: "MIRI_BE_RUSTC"="host" "MIRI_ENV_VAR_TEST"="0" "MIRI_TEMP"="/tmp/miri-uitest-YbtdTO" "RUST_BACKTRACE"="1" /checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/miri "--error-format=json" "-Dwarnings" "-Dunused" "-Ainternal_features" "-Zui-testing" "--out-dir" OUT_DIR
2026-08-03T09:44:14.9397002Z 
2026-08-03T09:44:14.9397568Z test result: ok. 375 filtered out
2026-08-03T09:44:14.9398021Z 
2026-08-03T09:44:14.9398752Z ## Running pass-native ui tests in tests/pass-dep for x86_64-unknown-linux-gnu
2026-08-03T09:44:14.9400581Z    Compiler: "MIRI_BE_RUSTC"="host" "MIRI_ENV_VAR_TEST"="0" "MIRI_TEMP"="/tmp/miri-uitest-YbtdTO" "RUST_BACKTRACE"="1" /checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/miri "--error-format=json" "-Dwarnings" "-Dunused" "-Ainternal_features" "-Zui-testing" "--out-dir" OUT_DIR
2026-08-03T09:44:15.0402720Z 
2026-08-03T09:44:15.0403285Z test result: ok. 68 filtered out
2026-08-03T09:44:15.0403776Z 
2026-08-03T09:44:15.0404144Z ## Running panic ui tests in tests/panic for x86_64-unknown-linux-gnu
2026-08-03T09:44:15.0407080Z    Compiler: "MIRI_ENV_VAR_TEST"="0" "MIRI_TEMP"="/tmp/miri-uitest-YbtdTO" "RUST_BACKTRACE"="1" /checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/miri "--error-format=json" "-Dwarnings" "-Dunused" "-Ainternal_features" "-Zui-testing" "--sysroot=/checkout/obj/build/x86_64-unknown-linux-gnu/miri-sysroot" "-O" "-Zmir-opt-level=4" "-Cdebug-assertions=yes" "--out-dir" OUT_DIR
2026-08-03T09:44:15.1407746Z 
2026-08-03T09:44:15.1408257Z test result: ok. 13 filtered out
2026-08-03T09:44:15.1408716Z 
2026-08-03T09:44:15.1409124Z ## Running fail ui tests in tests/fail for x86_64-unknown-linux-gnu
2026-08-03T09:44:15.1411729Z    Compiler: "MIRI_ENV_VAR_TEST"="0" "MIRI_TEMP"="/tmp/miri-uitest-YbtdTO" "RUST_BACKTRACE"="1" /checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/miri "--error-format=json" "-Dwarnings" "-Dunused" "-Ainternal_features" "-Zui-testing" "--sysroot=/checkout/obj/build/x86_64-unknown-linux-gnu/miri-sysroot" "-O" "-Zmir-opt-level=4" "-Cdebug-assertions=yes" "--out-dir" OUT_DIR
2026-08-03T09:44:15.2412037Z 
2026-08-03T09:44:15.2412648Z test result: ok. 533 filtered out
2026-08-03T09:44:15.2413045Z 
2026-08-03T09:44:15.2413400Z ## Running fail ui tests in tests/fail-dep for x86_64-unknown-linux-gnu
2026-08-03T09:44:15.2416208Z    Compiler: "MIRI_ENV_VAR_TEST"="0" "MIRI_TEMP"="/tmp/miri-uitest-YbtdTO" "RUST_BACKTRACE"="1" /checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/miri "--error-format=json" "-Dwarnings" "-Dunused" "-Ainternal_features" "-Zui-testing" "--sysroot=/checkout/obj/build/x86_64-unknown-linux-gnu/miri-sysroot" "-O" "-Zmir-opt-level=4" "-Cdebug-assertions=yes" "--out-dir" OUT_DIR
2026-08-03T09:44:15.3417692Z 
2026-08-03T09:44:15.3418204Z test result: ok. 91 filtered out
2026-08-03T09:44:15.3418496Z 
2026-08-03T09:44:15.3418802Z ## Running pass ui tests in tests/native-lib/pass for x86_64-unknown-linux-gnu
2026-08-03T09:44:15.4409738Z    Compiler: "MIRI_ENV_VAR_TEST"="0" "MIRI_TEMP"="/tmp/miri-uitest-YbtdTO" "RUST_BACKTRACE"="1" /checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/miri "--error-format=json" "-Dwarnings" "-Dunused" "-Ainternal_features" "-Zui-testing" "--sysroot=/checkout/obj/build/x86_64-unknown-linux-gnu/miri-sysroot" "-O" "-Zmir-opt-level=4" "-Cdebug-assertions=yes" "-Zmiri-native-lib=/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/tmp/miri-native-lib/native-lib-tests.so" "--out-dir" OUT_DIR
2026-08-03T09:44:15.5408923Z 
2026-08-03T09:44:15.5409454Z test result: ok. 4 filtered out
2026-08-03T09:44:15.5409828Z 
2026-08-03T09:44:15.5410219Z ## Running fail ui tests in tests/native-lib/fail for x86_64-unknown-linux-gnu
2026-08-03T09:44:15.6352522Z    Compiler: "MIRI_ENV_VAR_TEST"="0" "MIRI_TEMP"="/tmp/miri-uitest-YbtdTO" "RUST_BACKTRACE"="1" /checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/release/miri "--error-format=json" "-Dwarnings" "-Dunused" "-Ainternal_features" "-Zui-testing" "--sysroot=/checkout/obj/build/x86_64-unknown-linux-gnu/miri-sysroot" "-O" "-Zmir-opt-level=4" "-Cdebug-assertions=yes" "-Zmiri-native-lib=/checkout/obj/build/x86_64-unknown-linux-gnu/stage2-tools/x86_64-unknown-linux-gnu/tmp/miri-native-lib/native-lib-tests.so" "--out-dir" OUT_DIR
2026-08-03T09:44:15.7353534Z 
2026-08-03T09:44:15.7354159Z test result: ok. 11 filtered out
2026-08-03T09:44:15.7354866Z 
2026-08-03T09:44:15.7433790Z 	finished in 1.345 seconds

@RalfJung

RalfJung commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

In general, I like to make bootstrap steps more composable and remove similar special cases from them. On the other hand, adding bespoke test commands into the CI bash scripts runs into the risk of making it less obvious to people what failed on CI, and how to reproduce it locally. Especially since this is configured via an environment variable, and bootstrap will not print it after a failure, unlike the command-line args.

That being said, we already have a bunch of similar commands on CI, so no big deal. Just a remark.

Yeah that is a fair point. I think ideally we'd also print these env vars if the command fails but I am not sure how to best do that. Is there a shell mode like -x but that only prints failed commands?
Or maybe we can have ./x test --set-env so that ./x knows to print these?

That said I think in this case the bespoke special case in ./x test miri is not clearly better than a bespoke extra test invocation in the bash script.

@Kobzol

Kobzol commented Aug 4, 2026

Copy link
Copy Markdown
Member

We already use set -x or equivalent in this job anyway, so the command is printed. I think it's fine to keep it as it is.

That said I think in this case the bespoke special case in ./x test miri is not clearly better than a bespoke extra test invocation in the bash script.

👍

Hm actually strangely this does not seem to run any mir-opt tests any more. In the logs I just see

Hmm, strange indeed. It works for me locally.

@RalfJung

RalfJung commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

We already use set -x or equivalent in this job anyway, so the command is printed. I think it's fine to keep it as it is.

But that prints the command before it runs it. Ideally we'd have a mode that prints the command after it got run, but only if it failed.

That's a separate discussion though.

@RalfJung

RalfJung commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

It looks like it's not actually doing expansion of the {...} pattern:

MIRIFLAGS=-O -Zmir-opt-level=4 -Cdebug-assertions=yes MIRI_SKIP_UI_CHECKS=1 python3 ../x.py test --stage 2 src/tools/miri -- tests/{pass,panic}

Oh, this uses /bin/sh which I guess is dash...

@RalfJung
RalfJung force-pushed the miri-mir-opt-tests branch 2 times, most recently from 891d7c1 to d60de5c Compare August 4, 2026 11:57
@RalfJung

RalfJung commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

Now it finished in 52min?^^

@RalfJung

RalfJung commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

Well anyway the main point is it actually ran the mir-opt tests this time, so
@rustbot ready

@RalfJung
RalfJung force-pushed the miri-mir-opt-tests branch 2 times, most recently from 075edf5 to 548894b Compare August 4, 2026 12:58
MIRIFLAGS=-Zmiri-provenance-gc=1 python3 "$X_PY" test --stage 2 miri miri/cargo-miri
else
python3 "$X_PY" test --stage 2 src/tools/miri src/tools/miri/cargo-miri
python3 "$X_PY" test --stage 2 miri miri/cargo-miri

@Kobzol Kobzol Aug 4, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you inline #160502 into this one? Btw it should be just cargo-miri, rather than miri/cargo-miri.

View changes since the review

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I kind of it, as I was editing the file anyway I wanted tog et rid of the cargo-miri runs for looking at CI times here.

Btw it should be just cargo-miri, rather than miri/cargo-miri.

👍

@RalfJung
RalfJung force-pushed the miri-mir-opt-tests branch from 548894b to 2d91ed5 Compare August 4, 2026 12:59
also refactor check-miri a bit to make it easier to read
@RalfJung
RalfJung force-pushed the miri-mir-opt-tests branch from 2d91ed5 to c8e4034 Compare August 4, 2026 13:09
@Kobzol

Kobzol commented Aug 4, 2026

Copy link
Copy Markdown
Member

@bors r+

Let's land this one then, and I'll close #160502.

@rust-bors

rust-bors Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

📌 Commit c8e4034 has been approved by Kobzol

It is now in the queue for this repository.

🌲 The tree is currently closed for pull requests below priority 10. This pull request will be tested once the tree is reopened.

Reason for tree closure: manually handling queue due to backlog

@rust-bors rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 4, 2026
@Kobzol

Kobzol commented Aug 4, 2026

Copy link
Copy Markdown
Member

Ah, nevermind, #160502 is already in a rollup.

@bors r-

@rust-bors rust-bors Bot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 4, 2026
@rust-bors

rust-bors Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

This pull request was unapproved.

View changes since this unapproval

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-CI Area: Our Github Actions CI A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants