Skip to content

destructOnExit: release nested JSLockHolder VM refs so ~VM runs - #34448

Open
robobun wants to merge 1 commit into
mainfrom
farm/161a219b/destruct-on-exit-vm-refcount
Open

destructOnExit: release nested JSLockHolder VM refs so ~VM runs#34448
robobun wants to merge 1 commit into
mainfrom
farm/161a219b/destruct-on-exit-vm-refcount

Conversation

@robobun

@robobun robobun commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

What

test/js/web/timers/timer-heap-race.test.ts went flakily red on the x64-asan lane (seen in #34422's build 74213 and reported against build 74295):

==6590==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 480 byte(s) in 1 object(s) allocated from:
    ...
    <bun_runtime::timer::timeout_object::TimeoutObject>::init_with src/runtime/timer/mod.rs:142
    <bun_runtime::timer::All>::set_timeout src/runtime/timer/Timer.rs:271
    ...
    WebCore::JSEventListener::handleEvent JSEventListener.cpp:239
    WebCore::EventTarget::dispatchEvent EventTarget.cpp:248

The test itself (added in #33131) is a crash-freedom stress of the timer heap under cross-thread Atomics.waitAsync cancellation; the leak it is tripping over is a pre-existing main-thread VM teardown gap that the fixture happens to reach.

Cause

The fixture calls process.exit(0) from worker.onmessage. That handler runs inside JSEventListener::handleEvent, which takes its own JSLockHolder (holding a RefPtr<VM>). When global_exit reaches Zig__GlobalObject__destructOnExit, the VM refcount is therefore at least 3 (two creation refs plus the handler's lock holder), but destructOnExit only released the two creation refs. ~VM() never ran, so Heap::lastChanceToFinalize() was skipped.

collectNow() still runs first, so most JSTimeout wrappers are swept and their TimeoutObject boxes freed via finalize(). The flaky single leak is whichever wrapper the conservative stack scan happened to keep reachable (a stale encoded pointer from an earlier fire() frame): without lastChanceToFinalize(), that wrapper's destructor never runs and its m_ctx box (one 480-byte TimeoutObject) is reported by LSan.

Fix

Release every remaining VM ref in destructOnExit so ~VM() always runs. The two creation refs are constant; anything beyond that is a RefPtr<VM> in a JSLockHolder still on the stack. global_exit is noreturn so those holders' destructors never run, and terminate_all_workers_and_wait has already joined every worker thread, so no other thread can observe the freed VM.

The same observation and a closely related fix already exist on the unmerged ciro/repl-node-tests branch (commit 19d8c87, for the REPL's mid-eval process.exit()). This PR takes the simpler refCount() loop so the full ~VM() teardown path runs rather than calling lastChanceToFinalize() directly.

Verification

  • New asan-only test process.exit() from worker.onmessage finalizes the JSC heap reproduces the leak deterministically by pinning a Timeout on globalThis before exiting from onmessage: fails on main (480-byte TimeoutObject), passes with the fix.
  • Original flaky fixture: 0/50 failures with the fix (was ~20/30 on main) under the CI LSan settings.
  • bun bd test test/js/web/timers/timer-heap-race.test.ts: 4 pass.
  • bun bd test test/js/web/workers/: 369 pass.

destructOnExit only runs under BUN_DESTRUCT_VM_ON_EXIT=1 (the asan/leak lane), so release builds are unaffected.


[review] gate passed · iteration 0 · 3 files touched

fails on main (without fix)
ASAN without fix: 1 FAILED
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/js/web/timers/timer-heap-race.test.ts
info: syncing channel updates for nightly-2026-05-06-x86_64-unknown-linux-gnu
info: latest update on 2026-05-06 for version 1.97.0-nightly (e95e73209 2026-05-05)
info: component rust-src is up to date
info: checking for self-update (current version: 1.29.0)
bun test v1.4.0 (fafaab4b0)

test/js/web/timers/timer-heap-race.test.ts:
(pass) timer heap survives cross-thread Atomics.waitAsync timeout cancellation [3787.02ms]
(pass) timer heap stays consistent while GC re-arms the RunLoop timer [2407.83ms]
(pass) terminating a worker with pending Atomics.waitAsync tickets does not leak deferred-work tasks [6996.69ms]
72 |     const { stdout, stderr, signal, exitCode } = await runFixture("timer-heap-exit-onmessage-fixture.ts", {
73 |       BUN_DESTRUCT_VM_ON_EXIT: "1",
74 |       ASAN_OPTIONS: "allow_user_segv_handler=1:disable_coredump=0:detect_leaks=1:abort_on_error=1",
75 |       LSAN_OPTIONS: `malloc_context_size=30:print_suppressions=0:suppressions=${path.join(import.meta.dir, "..", "..", "..", "l
... (truncated)

release without fix: 3 skipped
bun test v1.4.0-canary.1 (1498d7b77)

test/js/web/timers/timer-heap-race.test.ts:
(pass) timer heap survives cross-thread Atomics.waitAsync timeout cancellation [3035.50ms]
(skip) timer heap stays consistent while GC re-arms the RunLoop timer
(skip) terminating a worker with pending Atomics.waitAsync tickets does not leak deferred-work tasks
(skip) process.exit() from worker.onmessage finalizes the JSC heap

 1 pass
 3 skip
 0 fail
 1 expect() calls
Ran 4 tests across 1 file. [3.20s]
__F:0:S:3
passes on PR (with fix)
ASAN with fix: all passed
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/js/web/timers/timer-heap-race.test.ts
info: syncing channel updates for nightly-2026-05-06-x86_64-unknown-linux-gnu
info: latest update on 2026-05-06 for version 1.97.0-nightly (e95e73209 2026-05-05)
info: component rust-src is up to date
info: checking for self-update (current version: 1.29.0)
bun test v1.4.0 (fafaab4b0)

test/js/web/timers/timer-heap-race.test.ts:
(pass) timer heap survives cross-thread Atomics.waitAsync timeout cancellation [3804.35ms]
(pass) timer heap stays consistent while GC re-arms the RunLoop timer [2506.79ms]
(pass) terminating a worker with pending Atomics.waitAsync tickets does not leak deferred-work tasks [7028.21ms]
(pass) process.exit() from worker.onmessage finalizes the JSC heap [7245.33ms]

 4 pass
 0 fail
 4 expect() calls
Ran 4 tests across 1 file. [22.62s]
__F:0:S:0

release with fix: 3 skipped
$ bun scripts/build.ts --profile=release
info: syncing channel updates for nightly-2026-05-06-x86_64-unknown-linux-gnu
info: latest update on 2026-05-06 for version 1.97.0-nightly (e95e73209 2026-05-05)
info: component rust-src is up to date
info: checking for self-update (current version: 1.29.0)
[configured] bun-profile → bun (stripped)
  target       linux-x64-gnu
  build type   Release
  build dir    ./build/release
  revision     fafaab4b07
  features     (none)

22 deps, 106 codegen, 1169 objects in 851ms

ninja: Entering directory `/workspace/bun/build/release'
[1/1232] install /workspace/bun
bun install v1.4.0-canary.1 (1498d7b77)

Checked 124 installs across 170 packages (no changes) [15.00ms]
[2/1232] install /workspace/bun/packages/bun-error
bun install v1.4.0-canary.1 (1498d7b77)

Checked 1 install across 2 packages (no changes) [2.00ms]
[3/1232] gen ErrorCode+*.h
[4/1232] gen bindgenv2
[5/1232] install /workspace/bun/src/node-fallbacks
bun install v1.4.0-canary.1 (1498d7b77)

Checked 129 installs across 147 packages (no changes) [9.00ms]
[6/1232] fetch zlib
[zlib] up to date
[7/1232] fetch picohttpparser
[picohttpparser] up to date
[8/1232] gen .bind.
... (truncated)
diff hotspot
src/jsc/bindings/ZigGlobalObject.cpp               | 11 ++++++++--
 .../timers/timer-heap-exit-onmessage-fixture.ts    | 25 ++++++++++++++++++++++
 test/js/web/timers/timer-heap-race.test.ts         | 21 ++++++++++++++++++
 3 files changed, 55 insertions(+), 2 deletions(-)

gate history · 1 passed · 0 rejected · iteration 0

evidence per changed file
file                                                     reads  edits  tests
src/jsc/bindings/ZigGlobalObject.cpp                         4      7      0
test/js/web/timers/timer-heap-exit-onmessage-fixture.ts      0      1      0
test/js/web/timers/timer-heap-race.test.ts                   2      3      0

self-review · no surviving concerns

29 concerns were raised and did not survive verification.

When process.exit() is reached from inside a JSEventListener (e.g.
worker.onmessage), JSEventListener::handleEvent's JSLockHolder holds a
RefPtr<VM> on the stack. destructOnExit only released the two creation
refs, so the VM stayed at refcount>=1, ~VM() never ran, and
Heap::lastChanceToFinalize() was skipped. Any wrapper that survived
collectNow() as conservatively reachable then leaked its native m_ctx.

timer-heap-race.test.ts was flakily hitting this on the asan lane: the
fixture calls process.exit(0) from worker.onmessage while a few
just-fired Timeout wrappers are still conservatively live on the stack.

Release every remaining VM ref. global_exit is noreturn so the stacked
holders' destructors never run, and workers were joined by
terminate_all_workers_and_wait, so no other thread uses the VM.

Co-authored-by: Ciro Spaciari <ciro.spaciari@gmail.com>
@coderabbitai

coderabbitai Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 12 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: cf55c6e5-6c3a-467f-beb3-9d9a2956e4a8

📥 Commits

Reviewing files that changed from the base of the PR and between 6352b79 and fafaab4.

📒 Files selected for processing (3)
  • src/jsc/bindings/ZigGlobalObject.cpp
  • test/js/web/timers/timer-heap-exit-onmessage-fixture.ts
  • test/js/web/timers/timer-heap-race.test.ts

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 1:10 AM PT - Jul 17th, 2026

@robobun, your commit fafaab4 has 2 failures in Build #74417 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 34448

That installs a local version of the PR into your bun-34448 executable, so you can run:

bun-34448 --bun

@robobun

robobun commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator Author

Reproduced the original flake locally (~20/30 runs of the fixture under the asan-lane env). The new deterministic test (timer-heap-exit-onmessage-fixture.ts) fails 3/3 on main, passes 3/3 with this change; the flaky fixture is 0/50 with the change.

Self-review clean (no surviving concerns). CI build 74417: timer-heap-race.test.ts is green; the remaining asan-lane reds are test-http2-reset-flood.js and test-worker-message-port-transfer-terminate.js, both pre-existing on other recently merged PRs (74239, 74213, 74193) and reproducing on main without this change. The two Windows entries are retry-passed flakes.

The RunLoop::dispatchAfter self-cycle this fixture also exercises (every notified Atomics.waitAsync waiter leaks a DispatchTimer+WTFTimer) is a separate runtime leak; it is already suppressed in test/leaksan.supp and is being handled separately.

@Jarred-Sumner Jarred-Sumner left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we instead rely on ContextDestructionObserver to release the lock? Can we do that?

@robobun

robobun commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator Author

Shouldn't we instead rely on ContextDestructionObserver to release the lock? Can we do that?

I looked at that first. The extra ref isn't held by anything that registers with the context: it's JSLockHolder::m_vm (a plain RefPtr<VM> in runtime/JSLock.h), a stack RAII object. Each holder's constructor refs the VM, the destructor derefs it, and nothing else knows the holder exists. ContextDestructionObserver can only notify objects that registered themselves in m_destructionObservers.

To use the observer we'd have to change every call site that can be on the stack at process.exit() to use a wrapper that registers itself and nulls its RefPtr<VM> on contextDestroyed(). That's JSEventListener::handleEvent plus the other bindings holders (JSAbortAlgorithm, JSErrorHandler, JSDOMPromiseDeferred, JSPerformanceObserverCallback, SerializedScriptValue, URLPattern, JSDOMPromise), and it still wouldn't cover a JSC-internal JSLockHolder that ends up on the stack (there are ~259 of them under JavaScriptCore/, e.g. Completion.cpp, SamplingProfiler.cpp).

The refCount() loop accounts for all of them uniformly, including ones we don't own, and the safety argument is that global_exit is noreturn (the stacked holders' destructors never run) and terminate_all_workers_and_wait has already joined every other thread that could hold the VM. Happy to switch to a different shape if you have something specific in mind.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't find any bugs. The refcount-loop change looks correct and is confined to the BUN_DESTRUCT_VM_ON_EXIT=1 asan lane, but it's a VM-lifetime change in JSC bindings (last loop iteration runs ~VM() under a live vm&), so worth a maintainer glance.

What was reviewed:

  • Loop captures refCount() once into a local, so vm isn't touched after the final deref frees it; runLoop is already held via Ref<> (line 4114) so threadWillExit() after ~VM is safe.
  • Confirmed terminate_all_workers_and_wait runs before destructOnExit in VirtualMachine.rs:1564, matching the comment's "workers were already joined" claim.
  • New test mirrors the neighboring asan test's structure/env exactly; fixture pins the Timeout on globalThis so the leak is deterministic rather than stack-layout-dependent.
Extended reasoning...

Overview

Fixes a flaky LSan leak in timer-heap-race.test.ts on the x64-asan lane. The root cause: when process.exit() is called from inside worker.onmessage, JSEventListener::handleEvent has a JSLockHolder (holding a RefPtr<VM>) on the stack, so the VM refcount at destructOnExit is ≥3. The old code released exactly two refs (the creation pair), so ~VM() — and therefore Heap::lastChanceToFinalize() — never ran, and any wrapper that collectNow() left conservatively reachable leaked its native m_ctx.

The fix replaces the two fixed derefSuppressingSaferCPPChecking() calls with for (unsigned refs = vm.refCount(); refs > 0; --refs) vm.deref...(). A new asan-only test with a fixture that pins a Timeout on globalThis before exiting from onmessage makes the leak deterministic.

Security risks

None. This is exit-path teardown code that only runs under BUN_DESTRUCT_VM_ON_EXIT=1 (the asan/leak CI lane). global_exit is noreturn, so nothing after this observes the freed VM. Release builds don't reach this path.

Level of scrutiny

Medium-high. Although the diff is tiny (~2 lines of logic), it's VM lifetime management in the JSC C++ bindings — the repo's most-scrutinized memory-safety category. I checked:

  • The loop reads refCount() once into a local before iterating, so it doesn't dereference vm after the last deref() frees it. This is the same UAF shape the original two-deref code already had (second deref freed the VM when refcount was exactly 2), so no new hazard.
  • runLoop->threadWillExit() after ~VM is safe because runLoop is a Ref<WTF::RunLoop> captured beforehand (line 4113-4114 already documents this).
  • The "workers were already joined" claim in the new comment is accurate — terminate_all_workers_and_wait runs at VirtualMachine.rs:1564, before destructOnExit at line 1613.
  • The stack-resident JSLockHolder RefPtr<VM>s become dangling after ~VM, but global_exit is noreturn so their destructors never run — this is stated in both the PR description and the code comment.

Other factors

The PR description is unusually thorough: it names the exact mechanism, cites prior art on ciro/repl-node-tests (commit 19d8c87), and reports 0/50 vs ~20/30 flake rates plus 369 passing worker tests. The new test follows the neighboring asan test's pattern byte-for-byte (same env, same assertion shape via runFixture). The fixture wires worker.onerror to fail loudly. Blast radius is limited to the asan CI lane.

Deferring rather than approving because VM teardown ref-balancing is exactly the class of change the repo's review rules flag for maintainer eyes, even when it looks correct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants