Skip to content

event_loop: post to a MiniEventLoop through raw pointers and publish last - #37883

Open
robobun wants to merge 2 commits into
mainfrom
farm/a735700a/mini-loop-publish-last
Open

event_loop: post to a MiniEventLoop through raw pointers and publish last#37883
robobun wants to merge 2 commits into
mainfrom
farm/a735700a/mini-loop-publish-last

Conversation

@robobun

@robobun robobun commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Problem

  • Other threads hand work to a MiniEventLoop with enqueue_task_concurrent / enqueue_task_concurrent_with_extra_ctx (src/event_loop/MiniEventLoop.rs): push the task onto the loop's queue, then wake the loop. Bun.build makes one loop per pass and frees it when the pass returns, and the task being posted can be the last thing the pass was waiting for, so the owner can run it and free the loop while the poster is still inside the call.
  • On main the body then read self.loop_ after the push (MiniEventLoop.rs:389 / :416 before this change): a read of the freed struct. Never observed; the window is the gap between two adjacent instructions on the poster, so there is no runtime repro.
  • The receivers are the underlying problem: &mut self (or &self) asserts the struct for the whole call, and Miri rejects both as soon as the owner drains the queue, free or no free (matrix below). The same holds one level down for UnboundedQueue::push(&self): the queue's alignment padding is not interior mutable, so a &self on the queue is rejected once the owner frees it mid-push.

Fix

  • The two entry points take this: *const Self, read loop_, push, and wake through the copy; nothing on *this after the push. The uws loop they wake belongs to the owning thread and outlives the struct (loop_ptr invariant).
  • The push goes through a new UnboundedQueue::push_raw, and push_batch (crate-private, one caller) carries the body on a raw pointer, documented as: the next / front store publishes, nothing after it may touch the queue. push(&self) is unchanged for every other user.
  • Every existing caller changes spelling only: BackRef::as_const_ptr() in ConcurrentPoster::post_mini, the process waiter thread and the shell yes builtin; &raw const **mini at the five bundler sites. No bundler frame is rewritten here; the frames above these entry points are bundler: route every cross-thread event to the bundle thread through one post() #37997's post() (see scope).
  • Test: bun_threading joins MIRI_CRATES (and the Miri workflow's path filter), and unbounded_queue.rs gets the two-thread reduction as a #[test]: the consumer frees the queue the moment the node is visible, for both publishing stores. Under Miri any access to the queue after the publishing store is reported as a race with that free on the first iteration (checked by adding one back: Data race detected between (1) atomic load ... and (2) deallocation); natively the test checks delivery. It cannot see a regression of the receiver shape itself (that would need the free to land inside the call, which Miri's scheduler does not produce in practice); that part is the signature.
  • Joining the Miri set surfaced that WaitGroup's own hand-off test fails under Tree Borrows: wait() can return while finish(&self) / Mutex::unlock(&self) are still on the finisher's stack, so the waiter's free is rejected. That is the same class, separate code; it is ignored under Miri with a note and reported separately so the crate can join now. The rest of the crate's tests pass, and the full default set still passes (bun run rust:miri, the crate adds about 5 s).
  • Also run on a debug build: bundler_plugin.test.ts (55 pass), bundler_defer.test.ts (10), 200 Bun.build passes that fail after parsing (the exposed worker path), bun exec with cp -R / rm -rf (post_mini), the waiter-thread half of bun-install-lifecycle-scripts.test.ts (54 pass; the one local failure is the pre-existing bun-on-PATH assumption, identical on the released binary); cargo check for windows-msvc and aarch64-darwin, clippy and rustfmt on the touched crates.

Scope: this is step 1 of the plan in the review thread. The frames above these entry points on the Bun.build chains (on_load_async and friends, run_from_thread_pool) are replaced by #37997's single post(), which is where the same discipline goes next, once; hoisting the per-pass loop onto the bundle thread is the third step. #37701 rebases onto this. Earlier revisions of this PR carried the bundler rewrites; they are in the history below and superseded.

Miri matrix (same result under Tree Borrows and Stacked Borrows)

One poster thread, one owner thread; the owner pops the node and frees the Box<Mini> as soon as it has it, and the poster parks at the point the variant names until that has happened. The queue is unbounded_queue.rs copied verbatim, QueuePadded included; Mini has the field shape of MiniEventLoop (a plain fifo, the queue, loop_).

receiver body owner frees verdict
&mut self (main) push, then self.loop_ after the push UB: the owner's pop is a foreign read of a protected &mut (reported before the free is even reached)
&mut self -> publish(&self) (this PR's first revision) loop_ read first after the push UB: same as above
&self loop_ read first after the push UB: the owner's write to its own task fifo hits the protected &self
*const Self loop_ read first, push(&self) after push returns ok
*const Self loop_ read first, push(&self) inside push, after the publishing store UB: deallocation while push's &self protects the padding byte at offset 8 of QueuePadded; passes with the repr(align) removed
*const Self loop_ read first, push_raw (shipped) after the push ok
*const Self loop_ read first, push_raw (shipped) inside the push, after the publishing store ok
*const Self push, then (*this).loop_ after the push UB: read of freed memory (the original bug, in raw form)

Background

  • A MiniEventLoop is Bun's event loop for threads that run no JavaScript (bundler, bun install, bun exec): a task queue plus a pointer to the thread's uws loop. The uws loop is the OS poller, one per thread, alive for the whole thread; waking it makes a sleeping loop look at its queue. An awake loop drains its queue on its own, which is how the owner can be done with a task before the poster's wake runs.
  • UnboundedQueue is the intrusive lock-free queue under the task queue. One store makes a pushed node visible (front when the queue was empty, the previous tail's next otherwise); from then on the consumer can pop and run it. That store is "the publish".
  • A protector is a rule in Rust's aliasing models (Miri checks them; bun run rust:miri uses Tree Borrows, which the Miri workflow runs in CI for the crates in MIRI_CRATES): a reference passed as an argument stays valid, and for &mut exclusive, until the function returns, even if the function never uses it again. Another thread freeing or writing that memory during the call is undefined behaviour. A raw pointer argument carries no such rule. Atomics are exempt; padding bytes are not, which is what catches push(&self).

@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 1b7bf585-4786-48c2-9c21-76e4141633cc

📥 Commits

Reviewing files that changed from the base of the PR and between 9a543cc and 5a87a34.

📒 Files selected for processing (11)
  • src/bundler/LinkerContext.rs
  • src/bundler/ParseTask.rs
  • src/bundler/ServerComponentParseTask.rs
  • src/bundler/bundle_v2.rs
  • src/event_loop/MiniEventLoop.rs
  • src/jsc/VmHandle.rs
  • src/runtime/api/JSBundler.rs
  • src/runtime/shell/builtin/yes.rs
  • src/spawn/process.rs
  • src/threading/unbounded_queue.rs
  • test/internal/source-lints/mini-loop-post.test.ts

Walkthrough

Changes

The PR replaces mutable mini event-loop access with shared access and raw-pointer enqueue APIs. Queue publication and wake-up ordering remain explicit. Bundler, VM, shell, process, and deferred-load paths now use the updated APIs. Source-lint tests pin the required implementation ordering.

Mini event-loop enqueueing

Layer / File(s) Summary
Raw queue publication
src/threading/unbounded_queue.rs
UnboundedQueue now provides raw-pointer enqueue operations. Atomic tail and front publication use the raw pointer.
Mini event-loop enqueue APIs
src/event_loop/MiniEventLoop.rs, test/internal/source-lints/mini-loop-post.test.ts
Concurrent enqueue methods accept raw loop pointers and forward task publication through UnboundedQueue. Source-lint tests verify receiver types, publication ordering, task forwarding, and wake-up behavior.
Shared loop routing
src/bundler/LinkerContext.rs, src/bundler/ParseTask.rs, src/bundler/ServerComponentParseTask.rs, src/bundler/bundle_v2.rs, src/runtime/api/JSBundler.rs
Bundler accessors now return shared event-loop references. Scheduling, cancellation, wake-up, load, resolve, and deferred-notification paths use the updated mini-loop APIs.
Posting call-site updates
src/jsc/VmHandle.rs, src/runtime/shell/builtin/yes.rs, src/spawn/process.rs
VM, shell, and process posting paths pass const mini-loop pointers to the updated enqueue methods.

Suggested reviewers: dylan-conway, jarred-sumner

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: raw-pointer posting to MiniEventLoop and publishing the task before waking the loop.
Description check ✅ Passed The description explains the problem, fix, scope, and extensive verification, including Miri, tests, clippy, and formatting.

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author

(Superseded; the current status is in the comment below.)

@robobun

robobun commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 10:42 PM PT - Aug 13th, 2026

@robobun, your commit 0caaf9b has 2 failures in Build #95175 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 37883

That installs a local version of the PR into your bun-37883 executable, so you can run:

bun-37883 --bun

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs. The reordering in publish looks correct and the source-lint test pins it. Because this is a cross-thread memory-safety change in the event loop whose correctness rests on lifetime arguments (uws loop outlives the boxed MiniEventLoop; UnboundedQueue::push's last access to self is the publishing store) and it intersects with #37691/#37701 on the same lines, a human look would still be worthwhile.

What was reviewed:

  • publish reads loop_ before concurrent_tasks.push and wakes via bun_uws::us_wakeup_loop(loop_) — matches the existing pattern in MiniEventLoop::wakeup and js_bundle_completion_task.rs:622.
  • UnboundedQueue::push takes &self and its final access is the AcqRel swap on back / Release store on front, so the consumer cannot observe the task before the poster's last read of the queue.
  • The new lint follows the conventions of the sibling test/internal/source-lints/ tests (uses globAllSources, git-tracked filter, self-pinning fixtures) and its non-vacuous guard checks both public entry points are still recognized as publishers.
Extended reasoning...

Overview

The PR reorders two statements in MiniEventLoop::enqueue_task_concurrent and enqueue_task_concurrent_with_extra_ctx: instead of push(task) then (*self.loop_ptr()).wakeup(), both now go through a shared publish(&self, task) that copies self.loop_ into a local, pushes, and calls bun_uws::us_wakeup_loop on the copy. This closes a use-after-free window where a per-build boxed MiniEventLoop (created in JSBundleCompletionTask::init_and_run) can be dropped by the bundle thread between the poster's push and its subsequent read of self.loop_. A source-lint test enforces that no function under src/event_loop/ that pushes to concurrent_tasks (or calls one that does through self) touches self after the publishing call.

Security risks

None. This is a data-race / UAF fix in internal event-loop plumbing; no user-controlled input, no auth, no crypto.

Level of scrutiny

High. This is native cross-thread memory-safety code — the review guide's most-blocked category. The correctness of the fix rests on two lifetime claims: (1) the uws loop pointer is thread-lifetime and outlives any boxed MiniEventLoop, and (2) UnboundedQueue::push_batch's final access to self is the release store the consumer synchronizes on. I checked (2) against src/threading/unbounded_queue.rs:215-240 and it holds: the last touch of self is the AcqRel swap on back or the Release store on front, both of which are what the consumer's Acquire load observes. Claim (1) matches the field's documented invariant and the existing precedent in stop_for_vm_teardown. But these are exactly the kind of subtle arguments a maintainer who owns the bundle thread lifecycle should confirm, especially given the PR notes #35060's overflow-thread design would need additional care.

Other factors

The PR description explicitly flags that #37691 and #37701 touch the same lines with a different ordering, so whichever lands second needs a rebase and the lint here is what keeps the invariant. That coordination is a human call. There is no runtime regression test (the race is not deterministically reproducible, per the description); the source-lint substitutes for it and is well-constructed — it self-tests against the old and new bodies, a pre-read-but-still-wakes-through-self shape, a caller that touches self after, an unattributable push, and the owner's own drain — but a maintainer should confirm this style of test-as-lint is acceptable coverage for a UAF fix here. publish taking &self while the callers keep &mut self is fine (UnboundedQueue::push is &self); the residual &self being notionally live past the free is the same pre-existing concern as the &mut self receiver and is not worsened by this change.

Comment thread src/event_loop/MiniEventLoop.rs Outdated

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and the automated bug-hunting pass found no issues. The reordering in publish looks correct — loop_ptr() is copied before the push, UnboundedQueue::push takes &self, and us_wakeup_loop on the copied *mut Loop matches what MiniEventLoop::wakeup already does. Because this is a cross-thread memory-safety change in core event-loop code whose correctness rests on the uws-loop-outlives-the-struct argument, and because comment-cop has flagged the doc comment on publish (line ~400) which the author hasn't yet addressed or dismissed, a human look is still worthwhile.

What was reviewed:

  • Confirmed bun_uws::us_wakeup_loop is re-exported and takes *mut Loop, and UnboundedQueue::push(&self, ...) — so publish(&self, ...) compiles and the wakeup path is behaviorally identical.
  • Checked the new source-lint follows the existing test/internal/source-lints/ pattern (same globAllSources().rust + git ls-tree guard as dead-code-escapes.test.ts); the self-check test pins the matcher on both the old and new bodies.
  • Noted the stated overlaps with #37691 / #37701 touch the same lines — landing order is a maintainer call.
Extended reasoning...

Overview

The PR reorders two statements in MiniEventLoop::enqueue_task_concurrent / enqueue_task_concurrent_with_extra_ctx so the uws loop pointer is read out of self before the task is published to the owning thread, closing a narrow cross-thread UAF window when the owning thread frees a per-build MiniEventLoop (the Bun.build case) between the poster's push and its wakeup. Both call sites are folded into a single publish(&self, ...) helper. A new source-lint test (test/internal/source-lints/mini-loop-publish-last.test.ts, ~290 lines) enforces that no function under src/event_loop/ touches self after a concurrent_tasks.push.

Security risks

None user-facing. This is an internal memory-safety hardening; no new inputs, no parsing, no auth surface.

Level of scrutiny

High. This is exactly the category REVIEW.md calls out as most-blocked: raw-pointer cross-thread lifetime reasoning in native runtime code. The fix's soundness depends on the invariant that the uws loop (a thread-local obtained via UwsLoop::get()) outlives the MiniEventLoop struct — the PR argues this holds for every mini loop today and stop_for_vm_teardown already relies on it, but a maintainer familiar with #35060's overflow-thread design and the two overlapping in-flight PRs (#37691, #37701) should confirm the argument and decide landing order.

Other factors

  • Outstanding comment: the comment-cop bot flagged the 8-line doc comment on publish at MiniEventLoop.rs:~400 ("paragraph-long comment to justify a workaround"). The author hasn't shortened it or replied. Whether that's a merge blocker or a false positive (the comment documents a real cross-thread invariant, not a workaround) is a maintainer judgment.
  • No runtime repro: the guard is a source-level lint plus a Miri reduction in the description; the PR is explicit that the race window is too small to reproduce deterministically. That's a reasonable trade-off for this bug class but again a maintainer call.
  • Test conventions: the new lint mirrors existing files in test/internal/source-lints/ (same import shape, same tracked-file guard), and includes a self-test that pins the regex matcher on the exact old and new bodies plus adversarial variants, so it isn't vacuous.
  • I verified bun_uws::us_wakeup_loop(*mut Loop) is the re-exported C binding already used by MiniEventLoop::wakeup and HTTPThread, and UnboundedQueue::push takes &self, so the refactor to publish(&self, ...) type-checks and is behavior-preserving apart from the read ordering.

@alii alii left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The two-line reorder is fine and this is the right layer to fix it at: a join at the owner cannot cover the plugin thread, and the WorkPool is process-wide, so publish-last in the poster is the only shape that works. Requesting changes because the PR's account of why it is correct does not hold up, and the lint that ships as the regression guard enforces that account rather than the real rule. Details inline; summary here.

1. The receiver, not the statement order, is what has to change

The description and the new SAFETY comment say the hand-off is sound because nothing on self is touched after the push. In the scenario the PR describes, the owner frees the Box while enqueue_task_concurrent(&mut self) and publish(&self) are still on the poster's stack. A reference argument is protected for the duration of the call, and freeing protected memory is UB under Tree Borrows (what bun run rust:miri runs) and Stacked Borrows, independent of whether self is read again. The repo already documents this rule in test/internal/source-lints/self-receiver-reclaim.test.ts:19-28, and this file already follows it for file_polls_raw(this: *mut Self).

I ran a reduction with this PR's exact statement order under Miri with tree borrows, freeing the struct between the queue's publishing store and the poster's return. &mut self at the entry point is rejected; &self at the entry point (the #37691 shape) is rejected, on the loop_ field this PR pre-reads; this: *const Self at the entry point, still calling UnboundedQueue::push(&self) on the field, is accepted. The Miri reduction in the description passes because it was written with this: *const Mini, which is not what the diff ships.

So: make enqueue_task_concurrent, enqueue_task_concurrent_with_extra_ctx and publish take this: *const Self, and pass the pointer the callers already hold (LinkerContext::any_loop_mut, the vtable at the bottom of this file) instead of a reborrow. The queue does not need to change. #37691 touches the same receivers; one PR should own the conversion and the other rebase.

2. The load-bearing invariant is in a file the PR does not touch

The fix also depends on UnboundedQueue::push_batch not touching the queue struct after the node is visible. That is true today (src/threading/unbounded_queue.rs: back.swap and front.store are each the last access to self in their branch) but is written down only in this PR's description. A debug_assert! after the swap, or a -> bool was_empty, reintroduces the UAF with the lint green. Please put a one-paragraph contract on push/push_batch naming the rule and this caller, and make the publish comment cite it.

3. The lint does not see the fix from (1)

mini-loop-publish-last.test.ts matches the token self. Running its check() unchanged against the description's own enqueue_buggy(this: *const Mini) reduction, or against both entry points converted to a raw receiver with the original buggy order, gives zero offenders while the second test still passes. After the receiver conversion the file reports coverage it does not have. Either drop it, pin the two bodies as literal snapshots the way #37691 does, or make it capture the receiver expression and add the description's reduction as a must-fail fixture. Any of those is much smaller than what is here.

4. The description puts the window on a path that is already joined (minor)

On the successful Bun.build path the parse workers are joined before init_and_run returns (wait group released after the callback in src/threading/ThreadPool.rs, and generate_chunks_in_parallel waits on the pool), so "the shared WorkPool is not waited on" is not true for the case narrated. The exposed cases are the error and cancellation returns after wait_for_parse, scan_module_graph_from_cli's early return, and the plugin JS thread on every path. Worth correcting because the ASAN evidence listed exercises the joined path, and because it steers a reader toward adding a join, which cannot work here.

Checked and not raising

Fixing at the owner instead (not viable, above); the #37701 rewrite to self.wakeup() (merge ordering only); the uws loop being freed at thread exit (pre-existing, this PR narrows it); push_batch front/back ordering, a late wakeup landing in the next build on the same uws loop, the uv loop living in the owner's TLS on Windows, the other per-scope mini loop owners under --watch and in build_command.rs. None of these change the verdict.

Comment thread src/event_loop/MiniEventLoop.rs Outdated
Comment thread src/event_loop/MiniEventLoop.rs Outdated
Comment thread src/event_loop/MiniEventLoop.rs Outdated
Comment thread src/event_loop/MiniEventLoop.rs Outdated
Comment thread test/internal/source-lints/mini-loop-publish-last.test.ts Outdated
Comment thread test/internal/source-lints/mini-loop-publish-last.test.ts Outdated
@alii

alii commented Aug 12, 2026

Copy link
Copy Markdown
Member

@robobun adopt

@robobun

robobun commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author

Status: approved in the leaf shape (head 0caaf9b: mini loop entry points on raw pointers, UnboundedQueue::push_raw, spelling-only callers, bun_threading in the Miri set with the reduction as a test). CI on this head finished: 177 of 179 jobs green; the two red jobs are both darwin 14 aarch64 test shards, failing only on astro-post and vite-build, which abort at startup on main as well (marked pre-existing by the CI tooling, reported for main triage) and do not involve this diff. Ready to merge. Bundler frames follow in #37997; the WaitGroup Miri finding is reported separately.

Comment thread src/bundler/LinkerContext.rs Outdated
Comment thread src/bundler/ParseTask.rs Outdated
Comment thread src/bundler/ServerComponentParseTask.rs Outdated
Comment thread src/bundler/bundle_v2.rs Outdated
Comment thread src/bundler/bundle_v2.rs Outdated
Comment thread src/bundler/bundle_v2.rs Outdated
Comment thread src/bundler/bundle_v2.rs Outdated
Comment thread src/event_loop/MiniEventLoop.rs
Comment thread src/event_loop/MiniEventLoop.rs
Comment thread src/jsc/VmHandle.rs Outdated
Comment thread src/threading/unbounded_queue.rs Outdated
Comment thread src/threading/unbounded_queue.rs
Comment thread src/bundler/LinkerContext.rs Outdated
Comment thread src/bundler/bundle_v2.rs Outdated
Comment thread src/threading/unbounded_queue.rs
@robobun robobun changed the title event_loop: read the mini loop's uws pointer before publishing a task event_loop: post to a MiniEventLoop through raw pointers and publish last Aug 12, 2026
@robobun

robobun commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author

@alii reworked along the lines of your review, in 5a87a34 (code) and 0d08ae5 (comment trims); title and description rewritten to match.

  1. Receivers: enqueue_task_concurrent and enqueue_task_concurrent_with_extra_ctx take this: *const Self, publish is gone, and the callers pass the pointer they hold (BackRef::as_const_ptr(), or the Box's address out of LinkerContext::any_loop, which is shared now; is_done derefs linker.r#loop for its own &mut).
  2. Queue: the contract is written on push_raw / push_batch in unbounded_queue.rs. One departure from "the queue can stay as it is": re-running the reduction with the real queue copied in, push(&self) is also rejected if the owner frees between the publishing store and push's return, because QueuePadded's padding is not interior mutable (it passes with the repr(align) removed). So there is a raw push_raw, and push_batch carries the body on a raw pointer; push(&self) is unchanged for its other users. Full matrix, identical under both models, is in the description. It also shows &mut self and &self failing on an ordinary post (owner pops, or writes its fifo), without any free.
  3. The lint is replaced by a roughly 100-line file that pins the four bodies as inline snapshots and says so; it fails against main at all four.
  4. The window description is corrected as you stated it, and the ASAN runs now include failing builds and CLI runs on that path.

Not changed here, and stated in the description: the frames above these entry points (run_from_thread_pool(&mut ParseTask), on_load_async(&mut self, &mut Load), the onLoadAsync thunk's &mut Load, Load::on_defer(&mut self)) have the same exposure for the pass's arena and BundleV2 on the same paths. That is the layer #37691 touches; it needs reworking on top of this rather than the other way round, since its &self receivers are one of the rejected rows. #37701 still needs its one-line rebase here.

CI: the build-bun steps on several lanes are failing to download vendored dependencies (c-ares, mimalloc, WebKit, lol-html); main's current build has the same failure, so those are not from this diff.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional findings (outside current diff — PR may have been updated during review):

  • 🔴 src/bundler/bundle_v2.rs:4346-4359on_load_async(&mut self, load: &mut Load) / on_resolve_async(&mut self, ...) still hold a protected &mut BundleV2 (and &mut Load/&mut Resolve) across the Mini-arm publish, so the same protector UB alii demonstrated for enqueue_task_concurrent(&self) remains one frame up on the Box<BundleV2> allocation. The PR modified both bodies and its raw-receiver rework stops one frame short; per REVIEW.md ("fix the whole class") and self-receiver-reclaim.test.ts:19-28, these receivers (and their C-ABI callers in JSBundler.rs) need to become raw pointers as well.

    Extended reasoning...

    What the bug is

    The PR was reworked per alii's review so that MiniEventLoop::enqueue_task_concurrent* and UnboundedQueue::push_batch take this: *const Self instead of &self/&mut self, because under Tree Borrows (and Stacked Borrows) a reference argument is protected for the whole call, and freeing the pointee while that protector is live is UB regardless of whether the body reads self again. The PR's own doc comment on enqueue_task_concurrent now states this: "A reference argument would assert *this for the whole call."

    But one frame up, BundleV2::on_load_async(&mut self, load: &mut jsc_api::JSBundler::Load) (bundle_v2.rs:4324) and on_resolve_async(&mut self, resolve: &mut ...) (bundle_v2.rs:4366) — both modified in this PR — still take &mut self. In the Mini arm (the Bun.build cross-thread case) they call MiniEventLoop::enqueue_task_concurrent_with_extra_ctx(&raw const **mini, ...). Once that call publishes the task, the bundle thread can drain it, satisfy is_done(), return from init_and_run, and drop bv2: Box<BundleV2> (js_bundle_completion_task.rs:1236) — while the JS plugin thread's on_load_async frame, with its protected &mut BundleV2 argument, is still live. That is exactly the shape alii's Miri reduction rejected ("deallocation ... is forbidden ... protected tag"), on the BundleV2 heap allocation instead of the MiniEventLoop one.

    The specific code path

    1. JS plugin thread: C++ calls JSBundlerPlugin__onLoadAsyncbv2_mut(this.bv2).on_load_async(this) (JSBundler.rs:1625), which reborrows the raw *mut BundleV2 backref to &mut BundleV2 and passes it as the &mut self receiver. Under Tree Borrows this creates a protector on the BundleV2 allocation for the whole call.
    2. on_load_async matches self.any_loop()Mini(mini) and calls enqueue_task_concurrent_with_extra_ctx(&raw const **mini, ptr::from_mut(load), ...) (bundle_v2.rs:4352-4360). The task is published to the bundle thread's queue.
    3. Bundle thread: tick_once drains the concurrent queue, runs on_load_mini (which forms &mut *load — see below), the pass finishes, init_and_run returns, and its local bv2: Box<BundleV2> drops, deallocating the BundleV2.
    4. JS plugin thread: on_load_async is still on the stack (in the match/fn epilogue after the enqueue returns). Its &mut self protector on the now-freed BundleV2 allocation is still live → Tree Borrows rejects the deallocation.

    The &mut Load / &mut Resolve argument has the identical problem: it is enqueued as the task ctx, and on_load_mini/on_resolve_mini on the bundle thread form &mut *load while the JS thread's protected &mut Load is still on the stack.

    Why existing code doesn't prevent it

    alii's third review comment explicitly named the plugin thread (on_load_async, on_resolve_async, on_defer) as "exposed on every path" — unlike the parse workers, nothing joins the plugin thread before Box<BundleV2> drops. The PR's original analysis of these two functions ("return straight out to C++, which does not touch the request again") uses the statement-order framing that alii's review corrected for enqueue_task_concurrent: the protector applies whether or not self is read after the enqueue. The rework applied the raw-receiver conversion to MiniEventLoop and UnboundedQueue but stopped at the frame above them.

    The repo already states this rule: test/internal/source-lints/self-receiver-reclaim.test.ts:19-28 — "a reference argument is protected for the duration of the call, and deallocating protected memory is rejected by both Stacked Borrows ... and Tree Borrows ... even if self is never touched again. The free has to go through the raw pointer the owner actually holds, which is why the tree's functions that end in a free take this: *mut Self."

    The ParseTask / ServerComponentParseTask completions do not have this problem: they reach any_loop() through worker.ctx.linker.any_loop() where worker.ctx is a BackRef<BundleV2> — the &BundleV2 there is a temporary formed by the Deref chain inside the match scrutinee, not a protected fn argument spanning the enqueue.

    Impact

    Same as the bug the PR fixes: Tree-Borrows-rejected UB on the Bun.build plugin path. The runtime window is one instruction wide (fn epilogue on the poster while the bundle thread finishes), so it has never been observed, but it is the same class REVIEW.md requires be closed together: "Fix the whole class in the same PR (same-class sites are ONE concern, not scope creep)." The PR modified both function bodies (the any_loop_mutany_loop and raw-pointer-enqueue changes) and its post-review soundness argument ("a reference argument would assert *this for the whole call") is exactly what applies here.

    How to fix

    Convert the receivers to raw pointers, mirroring what the PR did for MiniEventLoop:

    • pub unsafe fn on_load_async(this: *mut Self, load: *mut jsc_api::JSBundler::Load) and likewise for on_resolve_async. Read any_loop() via (*this) before the enqueue (or via the raw linker.r#loop NonNull, as is_done now does).
    • Callers in JSBundler.rs already hold the raw pointers: JSBundlerPlugin__onLoadAsync has this.bv2: *mut BundleV2 and this: &mut Load from C++ — pass this.bv2 and ptr::from_mut(this) (or make the FFI signature take *mut Load) directly instead of routing through bv2_mut(). Same for JSBundlerPlugin__onResolveAsync and the two JSBundlerPlugin__addError arms.
    • The &mut Load/&mut Resolve argument becomes *mut for the same reason (it is handed to the other thread as the task ctx while the reference protector on this thread is still live).

    Step-by-step proof

    Using alii's own reduction shape with -Zmiri-tree-borrows:

    struct BundleV2 { mini: Mini }                       // Box'd; init_and_run drops it
    fn on_load_async(bv2: &mut BundleV2, freed: &Receiver<()>) {
        unsafe { Mini::enqueue(&raw const bv2.mini) };   // publish (raw, as in this PR)
        freed.recv().unwrap();                           // bundle thread ran task, dropped Box<BundleV2>
        // fn epilogue: &mut BundleV2 protector still live → deallocation rejected
    }

    Miri rejects this with "deallocation ... is forbidden ... protected tag (currently Unique)" on the BundleV2 allocation — the same output alii posted for enqueue_task_concurrent(&mut self). Changing the signature to fn on_load_async(bv2: *mut BundleV2, ...) and reading &raw const (*bv2).mini before the publish is accepted.

    Concretely in this PR: bundle_v2.rs:4324 on_load_async(&mut self, ...) runs on the JS plugin thread via bv2_mut(this.bv2) (JSBundler.rs:1625). At bundle_v2.rs:4352-4360 it publishes to &raw const **mini. If that task is the last thing the pass was waiting for, the bundle thread's init_and_run returns and bv2: Box<BundleV2> (js_bundle_completion_task.rs:1236) drops, freeing the allocation the JS thread's &mut self protector still covers. on_resolve_async at 4366/4391-4399 is identical.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the reworked revision and the bug-hunting pass found no issues. Given this changes the receiver shape and safety contract of UnboundedQueue::push_batch and the two MiniEventLoop::enqueue_task_concurrent* entry points — a rework alii specifically requested based on the Miri reduction — their sign-off on the final shape is the right next step.

Checked: every any_loop_mut caller is converted (no stragglers in-tree); push_batch is pub(crate) with push/push_raw as its only callers, so the signature change is contained; BackRef::as_const_ptr exists for the three EventLoopHandle::Mini call sites; the &raw const **mini at the four bundler sites takes the address of the boxed MiniEventLoop without forming a reference through the enum arm's &Box.
Also checked the source-lint: fnSource's brace-at-header-indent close matcher relies on rustfmt, which the header comment states, and the self-test covers the \b word boundary and the missing-fn error path.

Extended reasoning...

Overview

This PR converts MiniEventLoop::enqueue_task_concurrent / enqueue_task_concurrent_with_extra_ctx and UnboundedQueue::push_raw / push_batch from &mut self / &self receivers to this: *const Self, and reorders the body so loop_ is read before the publishing store and nothing on *this is touched after it. All eight call sites (four bundler posting paths via LinkerContext::any_loop — now shared — plus ConcurrentPoster::post_mini, the process waiter thread, and the shell yes builtin) are updated to pass raw pointers. A ~150-line source-lint test pins the four function bodies as inline snapshots.

Security risks

None user-facing. The change is a memory-safety soundness fix (a one-instruction-window UAF plus an aliasing-model protector violation) in the mini event loop's cross-thread posting path. It does not touch auth, crypto, network parsing, or user-input handling.

Level of scrutiny

High. This is core threading/event-loop code with a subtle aliasing-model argument (Tree Borrows / Stacked Borrows protectors on reference arguments, and interior-mutability exemptions defeated by QueuePadded's alignment padding). It changes a crate-visible primitive's signature to unsafe fn(*const Self, ..). The first revision was a statement-reorder that alii's review showed was insufficient under Miri; this is the full rework. The description also explicitly interacts with two other open PRs (#37691, #37701) and scopes out a same-class issue in the frames above these entry points to #37691.

Other factors

  • alii left a detailed changes-requested review; the author reworked and replied point-by-point, and all of alii's threads are marked resolved. But alii has not yet re-reviewed the reworked commits (5a87a34 + 0d08ae5), and the rework departs from one of alii's suggestions ("the queue can stay as it is") with a stated Miri-backed reason (the QueuePadded padding byte). That specific departure deserves a human eye.
  • Comment-cop bot flagged several long comments; the author shortened most and kept three as # Safety sections that alii's review explicitly asked for — reasonable, but a maintainer should confirm those keeps.
  • CI is currently failing on unrelated vendored-dependency downloads (also failing on main per the author's note); the PR's own test lane has not run green on all platforms yet.
  • The change is not simple or mechanical; it is exactly the kind of concurrency/lifetime reasoning that REVIEW.md flags as the most-blocked category. Auto-approval is not appropriate here.

Comment thread src/bundler/bundle_v2.rs Outdated
@robobun

robobun commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator Author

@alii round three is in 06489e3; description updated, threads replied to.

  1. Resolve / Load::run_on_js_thread take this: *mut Self (dispatch.rs passes the pointer it had), read the plugin through bv2 raw, copy the strings into the BunStrings and run the chain holding only the raw request. match_on_load / match_on_resolve take the strings by value: they made those copies internally already, so the change is that the copies are made by a frame that has returned before the chain runs, and the plugin sees the same strings. DeferredBatchTask stays as is: drainDeferred only queues the promise reactions, so the answers run after it returns and it is not a hand-off frame. The bv2_plugin sentence now holds for the thread.
  2. post_result is body-pinned (your injected read fails it), takes the pass as *const BundleV2 and is shared with ServerComponentParseTask, whose callback is split like ParseTask's; its small wrapper is body-pinned too, so the lint reads that file now. Both worker entry points call unget() before the hand-off, per your &Worker point.
  3. Verified on ASAN: plugin, chain, defer and bake dev plugin suites (most plugin tests answer synchronously, i.e. through the converted outermost frames), the failing-build probe with and without the IO pool, worker-terminate, and the bake production tests with "use client" components for the server-component path (they pass with a raised timeout; a React production build does not fit the 5 s default under ASAN here, with or without this change). Cross-target checks, clippy, rustfmt and the other source lints are clean.

One thing you should know for sequencing, since you asked for one PR per class: three other open PRs touch these same functions from adjacent angles. #37746 (field-wise access to an outstanding request while the pass writes its list link) also carries the string copies and the post-last ordering, which this PR now has, so it shrinks to the field-wise part on rebase; #37709 copies the plugin pointer into the requests and reallocates DeferredBatchTask per drain, and its run_on_js_thread half is covered here; #37732 is the dispatch side and does not overlap textually. I have left notes on the first two rather than closing them, since what remains in each is a different problem from the hand-off class this PR closes; say the word if you would rather any of it be folded in here.

While reading ServerComponentParseTask I noticed the task object itself is never freed (the comment at its allocation says on_complete does it, but that frees only the Result); that predates this PR and is reported separately.

@alii alii left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round four does what it says.

  • dispatch.rs hands Resolve / Load on raw; both run_on_js_thread copy the strings and the plugin handle out through (*this).bv2 before the chain runs and hold only the raw request across it; match_on_* take the strings by value. The one &mut left on that path is the JS-side Plugin, which the bundle thread never touches.
  • unget() moved ahead of the hand-off, post_result takes the pass pointer instead of &Worker, and the server-component task goes through the same post_result.
  • Re-ran last round's probe: with let _ = (*bv2).any_loop(); injected after the publish in post_result, mini-loop-post.test.ts now fails that test (12/13); clean it is 13/13. task_callback_wrap and both run_on_js_thread are body-pinned.

CI red is the vendored-tarball download failure on the darwin aarch64 / freebsd build steps and their dependents, present on main's current build as well; every test lane is green.

Thanks for taking the optional point too.

Comment thread test/internal/source-lints/mini-loop-post.test.ts Outdated

@alii alii left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last thing

@robobun

robobun commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator Author

@alii done: 382abdb removes the lint file (the PR now carries no test; the contracts are the doc comments on the functions, and the description points at them instead of at the snapshots). The diff is otherwise the one you read in round four, rebased through your merge of main.

alii
alii previously approved these changes Aug 13, 2026

@alii alii left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lint removed as asked; the eleven src files are byte-for-byte what was approved in round four (the only other delta is main's merge). The Windows aarch64 test/bake/deinitialization.test.ts teardown segfault on this build has hit 26 unrelated PRs in the past week, so it is not this diff.

@alii

alii commented Aug 13, 2026

Copy link
Copy Markdown
Member

@robobun rebase

@robobun
robobun force-pushed the farm/a735700a/mini-loop-publish-last branch from 382abdb to 97642f5 Compare August 13, 2026 21:28
@robobun

robobun commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator Author

@alii rebased onto current main as a single commit (97642f5). The only overlap with main since the last merge was #37625's comment rewording in the on_defer block, resolved by keeping its names (enqueue_on_js_loop_for_plugins target, any_loop()); the code diff is unchanged (11 files, same hunks), and the source lints, including the new pre-port-identifiers one, pass on the rebased tree.

@alii

alii commented Aug 13, 2026

Copy link
Copy Markdown
Member

Stepping back from the implementation, which is fine, to whether this should merge in this shape. I don't think it should, and I'd like to supersede it rather than land it.

Why

  • Nothing observable changes if it closes. The description says it three times: never observed, one instruction wide, no runtime repro. No linked issue, and a search of the tracker for the symptom finds nothing on the mini loop. The one concrete defect on main is the loop_ read after the push in enqueue_task_concurrent; the rest of +446/-388 across 11 files is aliasing-model hygiene with no test and, for these crates, no Miri in CI to hold it (scripts/rust-miri.ts covers neither bun_event_loop nor bun_bundler, and they can't join as they link uws).
  • The cost is not small: 13 new unsafe fn, net +16 unsafe blocks, on run_from_thread_pool (every file of every build) and the plugin thunks. One of those wrong is strictly worse than the status quo, and four review rounds have gone into it already.
  • It's two PRs glued together. The primitive half (MiniEventLoop.rs, unbounded_queue.rs, and the spelling-only callers) is about +80/-30 and stands alone. The bundler half is +364/-355 and rewrites the exact five hand-off functions that bundler: route every cross-thread event to the bundle thread through one post() #37997 replaces with one bundler::post(); the two conflict in all five shared files (21 blocks). bundler: route every cross-thread event to the bundle thread through one post() #37997's post.rs Mini arm is then the single frame that touches the mini loop from another thread, so the raw-pointer / publish-last discipline belongs there, once, not threaded through eleven files first and re-derived after.
  • It's one instance of a shape the tree has several more of, some hotter and self-annotated ("another thread may free self at any time after .push" in RuntimeTranspilerStore, napi async work, Bun.build's own complete_on_bundle_thread). Hand-fixing this one leaves the class open and untooled.

What I'd like instead, as three small PRs:

  1. Cut this PR down to the leaf: raw-receiver MiniEventLoop::enqueue_task_concurrent{,_with_extra_ctx} reading loop_ before the publish, UnboundedQueue::push_raw / raw push_batch, spelling-only updates at every existing caller, no bundler frame rewrites. Add bun_threading to MIRI_CRATES (its deps already build under Miri) and check the two-thread reduction in as a #[test], so CI holds the "nothing after the publishing store" contract instead of a doc comment. That is ~5 files and mergeable on its own; uws: make Loop::wakeup take the raw loop pointer instead of &mut self #37701 rebases onto it.
  2. bundler: route every cross-thread event to the bundle thread through one post() #37997 rebased on (1), with post()'s Mini arm calling the raw primitive and worker.unget() before the post in the two worker callers. It already carries the behaviour test and lint.
  3. A small follow-up hoisting Bun.build's per-pass AnyEventLoop::default() in init_and_run onto the never-exiting bundle thread, so "a mini loop is owned by its thread and outlives its work" (the invariant ConcurrentPoster::Mini already documents, and that every other mini-loop owner satisfies) is true by ownership here too. Restate the SAFETY invariant as "the owning thread must outlive every in-flight post" so an exiting-owner change like bundler: run concurrent Bun.build() calls on overflow threads #35060 trips on it in review.

So: please force-push this branch down to (1) and drop the bundler / JSBundler / dispatch / LinkerContext changes from it. I'm withdrawing the approval until then. If you think the 11-file shape is still the right call, say why here once and I'll take it to the other maintainers rather than going another round.

@alii alii left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseding rather than merging in this shape; see the comment above for the reasoning and the three-PR plan. Withdrawn approval until the branch is cut down to the leaf change.

@robobun

robobun commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator Author

Agreed, no case to take further: the bundler half belongs in #37997's single post(), and a Miri test in bun_threading is a better hold than the doc comments. Cutting this branch down to the leaf now (entry points, push_raw / raw push_batch, one-line spelling updates at the existing callers, bun_threading in the Miri set with the reduction as a #[test]); will force-push and rewrite the description.

…last

Other threads hand work to a MiniEventLoop by pushing onto its queue and
then waking it. Bun.build makes one loop per pass and frees it when the
pass returns, and the task being posted can be the last thing the pass was
waiting for, so the owner may run it and free the loop while the poster is
still inside enqueue_task_concurrent. On main the body then read loop_ out
of the freed struct; and the &mut self receiver (or a &self) asserts the
struct for the whole call, which the aliasing models reject as soon as the
owner drains the queue.

enqueue_task_concurrent and enqueue_task_concurrent_with_extra_ctx take
this: *const Self, read loop_ before the push and wake through the copy.
The push goes through a new UnboundedQueue::push_raw, with push_batch on a
raw pointer; push(&self) is unchanged for its other users (the queue's
alignment padding is not interior mutable, so a &self on the queue would be
rejected too once the owner frees it). Callers change spelling only.

bun_threading joins the Miri set, and unbounded_queue.rs gets the two-thread
reduction as a test: the consumer frees the queue as soon as the node is
visible, so any access to the queue after the publishing store is reported
as a race with the free. WaitGroup's own handoff test is ignored under Miri
for now: Tree Borrows reports it as violated (finish(&self) is still on the
finisher's stack when wait() returns), which is tracked separately.
@robobun
robobun force-pushed the farm/a735700a/mini-loop-publish-last branch from 97642f5 to 4a11cc1 Compare August 14, 2026 00:11
Comment thread src/threading/WaitGroup.rs Outdated
Comment thread src/threading/unbounded_queue.rs
Comment thread src/threading/WaitGroup.rs
@robobun

robobun commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator Author

@alii cut down as described; the branch is now 4a11cc1 (+ a two-line comment trim in 0caaf9b), 12 files, +186/-36, and the title and description are rewritten for this scope.

  • Kept: the two entry points on this: *const Self reading loop_ before the publish, push_raw / raw push_batch, and spelling-only updates at every existing caller (as_const_ptr() at the three per-thread posters, &raw const **mini at the five bundler sites; any_loop_mut and every bundler frame are as on main). Dropped: everything in LinkerContext, dispatch, the JSBundler thunks, post_plugin_answer, run_on_js_thread, the ParseTask / server-component split, post_result; that is bundler: route every cross-thread event to the bundle thread through one post() #37997's post() now.
  • bun_threading is in MIRI_CRATES (and the Miri workflow's path filter), and the reduction is a #[test] in unbounded_queue.rs: the consumer frees the queue the moment the node is visible, for both publishing stores. Putting an access back after the store fails it on the first iteration under Miri (Data race detected between (1) atomic load ... and (2) deallocation). It does not see a regression of the receiver shape itself (Miri's scheduler does not land the free inside the call in practice, even at a 50% preemption rate); that part is the signature, and the description says so.
  • Joining the Miri set turned up a first instance of the class you predicted: WaitGroup's own hand-off test fails under Tree Borrows (wait() returns while finish(&self) / Mutex::unlock(&self) are still on the finisher's stack, so the waiter's free is rejected). It is #[cfg_attr(miri, ignore)]d with the reason so the crate can join now, and reported separately. With that, the crate and the whole default set pass (bun run rust:miri, about 5 s added).
  • Callers re-exercised on a debug build: plugin and defer suites, the failing-build loop, bun exec cp -R / rm -rf, the waiter-thread install tests; cross-target checks, clippy and rustfmt clean.

#37701 can rebase onto this once it lands; I will leave steps 2 and 3 of your plan to #37997 and a follow-up unless you want me to open the third one.

@alii alii left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the shape I asked for, thanks for turning it round quickly.

  • Bundler / JSBundler files are call-site spelling only (&raw const **mini); every frame is as on main.
  • Ran the new test under Miri myself (nightly-2026-07-20, -Zmiri-tree-borrows): passes as shipped in ~4 s; with one (*this).back.0.load(...) inserted after the front store in push_batch it fails on the first iteration with Data race detected between (1) atomic load ... and (2) deallocation, pointing at the inserted line. So the contract is held by CI, not by the doc comment.
  • The WaitGroup ignore is honest about why and is the first catch of the crate joining Miri; fine as a separate report.

Steps 2 and 3 can stay with #37997 and a follow-up. #37701 rebases onto this.

@robobun

robobun commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks for re-running it. Nothing further from my side; CI on 0caaf9b is at 177 lanes green with two still queued, so it is yours to merge when they finish.

Jarred-Sumner pushed a commit that referenced this pull request Aug 14, 2026
…t the release (#38330)

### Problem

- `WaitGroup::wait()` returning is what lets the owner free the group,
so once the last `finish()` has published the count and released the
mutex, the finishing thread must not touch the group at all. #34458
moved the last real access in front of that release. What is left is
that `finish(&self)`, `Mutex::unlock(&self)`, the per-OS unlock impls,
and the contended-path `Futex::wake(&self.state)` all still hold
references into the group through and after the release, and a reference
argument asserts the memory for the whole call.
- Miri rejects the waiter's free for exactly that reason. `bun run
rust:miri -p bun_threading` (the crate is not in the Miri set yet) fails
the crate's own test,
`wait_group::tests::wait_returning_means_finish_is_done_with_self`,
under Tree Borrows with
`Undefined Behavior: deallocation through <tag> at alloc[0xc] is
forbidden ... transitioned due to a protector release -->
src/threading/Mutex.rs:193` (end of `DebugImpl::unlock`; offset `0xc` is
padding inside the mutex, so this hits even though every field is an
atomic), and under the default Stacked Borrows with `... would remove
[SharedReadOnly ...] which is strongly protected`. Every seed I tried
(12) fails within 2..200 iterations.
- It is not only padding: when the waiter re-locked while the finisher
still held the mutex, `FutexImpl::unlock` (src/threading/Mutex.rs:397 on
main) forms `&self.state` for the wake after the releasing swap, i.e.
possibly after the waiter has freed the group. In an instrumented Miri
run that happened in 6 of 1500 iterations. On a real kernel the wake is
harmless (a private `FUTEX_WAKE` only uses the address as a key), but
the reference is still formed on freed memory.
- In-tree callers with that lifetime: `LinkerContext`'s two source-map
groups. `generate_chunks_in_parallel` frees the task slab on the line
after `wait()`
(src/bundler/linker_context/generateChunksInParallel.rs:78), and the
`Bun.build` error path waits on both groups and tears down the whole
`BundleV2` right after
(src/runtime/api/js_bundle_completion_task.rs:1267). That is the path
#34458's ASAN report came from. `ThreadPool`'s group is joined before
the pool drops and the Windows install queue's group is a `static`, so
those two are fine with `&self`.

### Fix

- `WaitGroup::finish_raw(this: *const Self)` does the work through raw
pointers, fast path included (a fast-path CAS can be the second-to-last
decrement, with another finisher letting the waiter free the group while
this frame is still live). Its last access to the group is the store
that releases the mutex.
- `Mutex::unlock_raw(this: *const Self)` and raw-pointer `unlock_raw`
impls for the futex, Darwin and Windows backends, so no frame between
`finish_raw` and the releasing store holds a reference. `unlock(&self)`
delegates to it; `Bun__unlock` calls the impl directly; the
`os_unfair_lock_unlock` extern takes the address (the Windows one
already did).
- `Futex::wake_raw(*const AtomicU32)` for the unlock tail;
`wake(&AtomicU32)` delegates to it, so the other callers are unchanged.
It is a safe fn because every backend's wake side only keys on the
address and never reads the word (Linux `get_futex_key` for private
futexes, `__ulock_wake`, `RtlWakeAddress*`, `_umtx_op` private wake,
`memory.atomic.notify`); the worst a freed or reused address can produce
is a spurious wakeup, which every wait loop already tolerates.
- `finish(&self)` stays for groups kept alive by something other than
`wait()` and delegates to `finish_raw`; its doc says when it is and is
not allowed. The two `LinkerContext` finishers use `finish_raw` through
`ParentRef::as_const_ptr`, so the finish is the last statement in the
task that touches the context.
- Why this is the right shape: the memory-level ordering from #34458 is
unchanged; this only changes which pointers are live across the release,
which is the thing both aliasing models (and LLVM's `dereferenceable` on
reference arguments) reason about. It is the same shape as
`UnboundedQueue::push_raw` in #37883 and the tree's `this: *mut Self`
convention for functions that end in a free
(test/internal/source-lints/self-receiver-reclaim.test.ts).
- Linux `Futex` wake no longer panics on `EFAULT` (the FreeBSD backend
already tolerated it). A real kernel only returns it for an address
outside user space, which no caller can produce (every caller just did
an atomic op on the same word), while Miri returns it for a word that
has since been freed, which is now a documented-legal input; with the
panic kept, the fixed test fails under Miri about once per 250
iterations (the 6/1500 above). The `futex_3arg` SAFETY comment in
`bun_sys` is corrected to match (a WAKE only uses `uaddr` as a key).
- Verified with:
- The crate's own test
(`wait_group::tests::wait_returning_means_finish_raw_is_done_with_the_group`),
which now finishes through `finish_raw`. `bun_threading` is added to
`MIRI_CRATES` and `src/threading/**` to the Miri workflow's paths, so
the existing `cargo miri test` CI job is the automated check: it fails
on main's `src/threading` (the diagnostic above) and passes here. Under
`cfg(miri)` the test runs 500 iterations (about 15s; the unfixed shape
fails within 200 on every seed tried), natively still 10,000. Passes on
12 seeds plus a 10,000-iteration run on the default seed, under both
Tree Borrows and Stacked Borrows; the full `bun run rust:miri` set
passes (`bun_threading` takes 15s, `bun_paths` takes 75s for
comparison).
- `cargo check -p bun_threading --tests` on linux-gnu, linux-musl,
android, both darwin and both windows-msvc triples and freebsd, plus
`--release` (the `ReleaseImpl`-direct path) on linux, darwin and
windows; `cargo check --workspace` on the host; `cargo clippy --no-deps`
on `bun_threading` and `bun_bundler`; the Windows-target clippy finding
set is identical to main's.
- `bun bd test test/bundler/bun-build-api.test.ts` (52 pass) and a
20-round loop of a 200-module `sourcemap: "external"` build plus a
failing sourcemap build under the ASAN debug build, which exercises both
`LinkerContext` call sites and the error-path teardown.
- Overlap with open PRs: #37883 adds `bun_threading` to the Miri set
with this test `#[ignore]`d under Miri; this PR makes it pass, so
whichever lands second drops the ignore (the `MIRI_CRATES` and workflow
lines are identical). #36481 introduces stack-scoped groups whose
`BatchDone` drop calls `(*ptr).finish()` relying on this property; with
this change that should be `WaitGroup::finish_raw(ptr)`, noted there.
- Deliberately not in this PR: the same shape exists with other
primitives in two places outside `bun_threading`,
`SingleHTTPChannel::write_item(&self)` in src/http/AsyncHTTP.rs
(`send_sync` frees the channel right after `read_item` returns) and
`process_http_callback(&mut self)` in
src/runtime/webcore/s3/download_stream.rs (`on_response` frees the task
once that unlock lands). Both need a raw-unlock path for the guard types
rather than `WaitGroup` changes, so they are tracked separately.
`ResetEvent::set(&self)` has the same tail but its only user is a
process-lifetime `BundleThread`, and the `Condvar` notifiers I looked at
(`VmHandle`, `HTTPThread` shutdown) hold an `Arc` or a `static`.

### Background

- Protectors: in Rust's aliasing models (Stacked Borrows, and Tree
Borrows, which `rust:miri` uses) a reference passed as a function
argument is "protected" for the duration of the call: the callee may
assume the memory stays valid and unchanged by others until it returns,
and freeing memory that a protected reference covers is undefined
behavior even if the callee never touches it again. This is what lets
rustc mark reference arguments `dereferenceable` for LLVM. A raw pointer
argument carries no such assertion, which is why a function whose job
ends by letting another thread free the object takes `*const Self`.
- Padding: a `&Mutex` covers the struct's padding bytes too, and padding
is not interior-mutable, so a struct made only of atomics still gets the
strict treatment for those bytes. That is what the `0xc` in the
diagnostic is.
- Futex wake is address-keyed: `wait` sleeps on an address after
comparing the word; `wake` looks the address up in the kernel's (or
runtime's) waiter table without reading user memory. This is the
property every futex-based mutex relies on so that the thread that
acquires the lock next may free it; it is also why a wake on a freed
address is harmless and why `wake_raw` needs no `unsafe`.
- `WaitGroup::finish` publishes the final decrement under the group's
mutex (since #34458), so `wait()`, which checks the count under the same
mutex, cannot return before the finisher's unlock; the releasing store
inside that unlock is therefore the exact point after which the group
may be gone.

<details>
<summary>Miri diagnostic on main, and the iteration counts</summary>

```
$ bun run rust:miri -p bun_threading
test wait_group::tests::wait_returning_means_finish_is_done_with_self ... error: Undefined Behavior: deallocation through <620185> at alloc217837[0xc] is forbidden
    --> library/alloc/src/boxed.rs:2002:17
     = help: the accessed tag <620185> has state Reserved (conflicted) which forbids this deallocation (acting as a child write access)
help: the accessed tag <620185> was created here, in the initial state Reserved
    --> src/threading/WaitGroup.rs:115:17      drop(Box::from_raw(wg));
help: the accessed tag <620185> later transitioned to Reserved (conflicted) due to a protector release (acting as a foreign read access) on every location previously accessed by this tag
    --> src/threading/Mutex.rs:193:6           (end of DebugImpl::unlock)
```

Depending on the interleaving the same test also fails as
`deallocation ... is forbidden ... the accessed tag is foreign to the
protected tag <..> (currently Frozen) ... protected tag was created
here: Mutex.rs:66 pub fn unlock(&self)`,
and under Stacked Borrows (`MIRIFLAGS=""`) as
`not granting access to tag <..> because that would remove
[SharedReadOnly for <..>] which is strongly protected`.

Iteration at which the unfixed shape fails under Miri, by `-Zmiri-seed`:
`0: 2, 1: 199, 2: 75, 3: 16, 4: 75, 5: 10, 6: 10, 7: 113, 8: 113, 9: 71,
10: 36, 11: 51`.

Instrumented run of the fixed code, 1500 iterations, default seed: 2051
contended unlocks, 6 of whose wakes ran after the waiter had already
freed the group (Miri returned `EFAULT`), 0 aliasing reports.

</details>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants