Wait for off-thread jobs before freeing a terminated worker's VM - #36983
Closed
robobun wants to merge 14 commits into
Closed
Wait for off-thread jobs before freeing a terminated worker's VM#36983robobun wants to merge 14 commits into
robobun wants to merge 14 commits into
Conversation
worker.terminate() (and process.exit() / uncaught throw / unhandled rejection inside a worker) freed the VirtualMachine box, its EventLoop, the uws loop, and the JSC heap while jobs the VM had handed to other threads were still running: WorkPool bodies, HTTP-thread fetch/S3 callbacks, napi execute callbacks, the bundler thread. Each of those holds raw pointers back into that memory and posts its completion with enqueue_task_concurrent, so teardown raced every in-flight job into a use-after-free. Natural exit was safe only because each job's KeepAlive holds the event loop open; the terminate path broke out of the loop without any equivalent wait. WebWorker::shutdown now fences: EventLoop tracks outstanding off-thread jobs (every schedule site that pairs a KeepAlive ref with an off-thread handoff takes a count; the off-thread body releases it after its last VM access), shutdown sets a per-loop cancel flag (cancel-aware pool bodies skip their compute), runs a per-VM cancel-hook fan-out (fetch aborts, S3 shutdown-by-id), and waits for the count to reach zero before WebWorker__teardownJSCVM. Completions posted during the wait are reclaimed unrun by the existing shutdown drain via new per-tag release arms. If the wait exceeds 10s the VM and everything a straggler can still reach are leaked instead of freed. Also: the HTTP thread's last-ref fetch reclaim no longer parks worker tasklets for the process-exit drain (whose deinit would walk the worker's freed JSC handles on the main thread); worker tasklets leak the small box instead.
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughChangesThe PR adds off-thread event-loop fences, worker-termination cancellation hooks, bounded shutdown draining, shutdown-safe cleanup for asynchronous tasks, and regression tests for active worker operations. Worker shutdown fencing
Possibly related issues
Possibly related PRs
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
…ib test assertions The shutdown drain no longer frees the napi_async_work box: the addon owns it (napi_create_async_work hands the handle out and only napi_delete_async_work may free it, possibly from a cleanup hook or an experimental-module finalizer), so freeing it risked a double-free. The box now leaks with the VM, like the addon's data already did. fs.watchFile's InitialStatTask gets the same off-thread bracket as its siblings (begin at schedule, end on every exit of run_owned via a local): the scheduler's shutdown wait only covers its periodic task, and the watcher is not appended to the scheduler until the initial stat completes, so an in-flight initial stat could post to a freed VM. zlib-worker-terminate.test.ts now asserts stderr is empty modulo the one tolerated terminate() abort, matching worker-terminate-offthread.test.ts, and runs its subprocess with leak detection off.
S3 requests carried no abort signal, so AsyncHTTP::init left them with the sentinel async_http_id 0 and never registered their sockets in the abort tracker: the terminate cancel hook's schedule_shutdown_by_id matched nothing. list_objects (a second S3HttpSimpleTask construction site) also had no fence bracket at all, so a completed s3.list() on a worker underflowed the off-thread counter and every later terminate rode the full 10s deadline and leaked the VM. The simple task now carries a signal store like the streaming task and passes it to AsyncHTTP::init (real id, abort-tracker registration), both construction sites bracket the fence and register the cancel hook, and both hooks additionally set the task's abort signal so a request that has not started yet fails fast in the HTTP thread's queued-abort scan instead of connecting to a server the dying worker can no longer accept on. Adds a Bun.S3Client.list lane to the terminate matrix, with the worker template gaining an async wrapper so the lane can complete one list before the door fires, which makes an unbalanced counter a deterministic stall the dt guard catches.
Collaborator
Author
|
CI state as of build 89345 (2cf2b79): 195 of 196 jobs green. The one red is the The branch is ready for review. The one open question from the latency measurements is whether the 10 s |
Contributor
There was a problem hiding this comment.
No new issues found after the earlier fixes (Archive/RuntimeTranspilerStore bracketing, the offthread_drained gate on the S3 streaming drain arm, and the main-thread early-return in the cancel-hook registry). This is a large cross-thread memory-safety change across ~15 off-thread job families plus a new leak-on-timeout policy in WebWorker::shutdown, so it should get a human pass.
What was reviewed:
- begin/end pairing at every
WorkPool::schedule/ HTTP-thread / bundler-thread site named in the diff, and that eachoffthread_job_endgoes through a local after the enqueue - the
drainedgating inweb_worker.rs— every free past step 3 is skipped on timeout, andrelease_queued_tasks_for_shutdownrequeues the multi-post S3 streaming box when!drained terminate_cancel_hookfor S3 touches onlysignal_store+ the captured id, not the HTTP-thread-ownedhttpstorage- the new
release_unrun/release_at_shutdownarms don't run JS and JSC is still live when they run
Extended reasoning...
Overview
This PR adds a barrier (EventLoop::outstanding_offthread) that WebWorker::shutdown waits on before freeing the worker's VM box, EventLoop, uws loop, and JSC heap. Every off-thread schedule site that hands a job to the WorkPool, HTTP thread, or bundler thread now increments the counter at schedule time and decrements it (through a local pointer copy) after the job's last VM access on the off thread. A 10s deadline leaks the VM instead of freeing it if the counter never drains. A per-VM cancel-hook registry lets shutdown abort in-flight fetch/S3 requests to bound the wait. The shutdown drain (__bun_release_task_at_shutdown) gains ~10 new arms to reclaim completions that now reliably reach the queue.
The change spans 27 files: the core mechanism in event_loop.rs / web_worker.rs / VirtualMachine.rs, per-family bracketing in WorkTask, AnyTaskJob, ConcurrentPromiseTask, ConcurrentCppTask, AsyncFSTask, AsyncReaddirRecursiveTask, AsyncCpTask, PasswordJob, ShellTask (+ custom cp/rm schedulers), node_zlib, napi_async_work, fs.watchFile initial stat, FetchTasklet, S3 simple + streaming + list, JSBundleCompletionTask, Archive::AsyncTask, and RuntimeTranspilerStore. Two new ASAN-gated test files exercise a door × family matrix.
Security risks
None in the traditional sense — this is a use-after-free fix. The risk surface is that an unbalanced begin/end pair either stalls every worker terminate to the 10s deadline (then leaks the VM) or, if a family is missed, leaves a UAF window open. The test matrix's dt > 9000 guard catches the former for every covered family; missed families are the residual risk.
Level of scrutiny
High. Per REVIEW.md this is the most-blocked category (native memory safety, cross-thread lifetime, refcount balancing on every terminal path). The change introduces a new invariant that every future off-thread schedule site must uphold, adds a leak-on-timeout policy with a hard-coded 10s constant, and modifies the fetch hot path (cancel-hook register/unregister on every worker fetch). The ShellCpTask / ShellRmTask custom-scheduler bracketing and the rm verbose-post fence in particular are subtle enough to warrant a maintainer's eye on the ordering.
Other factors
Three earlier findings from a prior automated pass were all addressed (Archive + RuntimeTranspilerStore bracketing; offthread_drained gating on the S3 streaming drain arm; main-thread early-return in the hook registry). CodeRabbit's readdir-result-leak and download-stream-race findings were also addressed. CI shows one pre-existing unrelated ASAN leak in AsyncLocalStorage-tracking.test.ts (RSA keygen, reproduces on main). The PR body's known-limits section explicitly names the Windows libuv flows and the bundler-plugin round-trip as out of scope.
Collaborator
Author
|
An ASAN repro on main hit another face of the worker-teardown door this fence covers, and it lands in the WorkTask family, which the test matrix here does not exercise yet: #37139 takes the coder's own lifetime (the in-flight ctx now holds a reference), which also covers the main-thread Suggested scenario for const s = new CompressionStream('brotli');
const w = s.writable.getWriter();
const big = new Uint8Array(6 << 20);
for (let i = 0; i < big.length; i += 3) big[i] = (i * 2654435761) >>> 24;
w.write(big).catch(() => {});
w.close().catch(() => {});
s.readable.getReader().read().catch(() => {});Note the parent has to outlive the abandoned compute for the enqueue face to fire: without the fence the completion posts seconds after the terminate, so a parent that exits right away can miss it. |
Jarred-Sumner
pushed a commit
that referenced
this pull request
Aug 7, 2026
…ad transforms (#37139) ### Repro `CompressionStream('brotli')` with a chunk over 128 KiB runs the codec step on a WorkPool thread. Tearing down the VM while that step is in flight frees the native coder under the pool thread: ```js // ASAN build, BUN_DESTRUCT_VM_ON_EXIT=1 (the CI test runner sets this) const s = new CompressionStream("brotli"); const w = s.writable.getWriter(); const big = new Uint8Array(6 << 20); for (let i = 0; i < big.length; i += 3) big[i] = (i * 2654435761) >>> 24; w.write(big).catch(() => {}); w.close().catch(() => {}); s.readable.getReader().read().catch(() => {}); setTimeout(() => process.exit(0), 15); ``` ``` ==ERROR: AddressSanitizer: heap-use-after-free ... thread (Bun Pool 0) #0 UpdateNodes vendor/brotli/c/enc/backward_references_hq.c:468 ... #4 BrotliEncoderCompressStream vendor/brotli/c/enc/encode.c:1661 #5 CompressionStreamCoder::transform src/runtime/webcore/CompressionStreamCoder.rs:367 freed by: BrotliEncoderDestroyInstance CompressionStreamCoder__destroy JSCompressionStream.cpp:176 (CFinalizer) JSC::Heap::CFinalizerOwner::finalize -> Heap::lastChanceToFinalize ``` The same free-under-the-pool-thread happens on `worker.terminate()` / `process.exit()` inside a worker while a large write is in flight (`WebWorker::shutdown` -> `WebWorker__teardownJSCVM` -> `lastChanceToFinalize`). Other faces of the same report: READ 1 in `BrotliEstimateBitCostsForLiterals` / `UpdateNodes`, WRITE 4 in `StoreAndFindMatchesH10`. `DecompressionStream` has the identical finalizer shape, and zstd/zlib formats share the path. ### Cause The stream cell's CFinalizer (registered in the constructor) destroys `m_coder` unconditionally. During normal operation the in-flight task's `Strong` root keeps the cell from being swept, and the eager ClearAlgorithms release already defers on `m_asyncCodecInFlight`. But `Heap::lastChanceToFinalize` at VM teardown runs every finalizer regardless of roots, so the coder (brotli ring buffer + hasher, zlib window, zstd ctx) is freed while the pool thread is still inside `transform`. ### Fix Reference-count the coder. The JS cell holds one reference, released where it released before (finalizer, or the eager ClearAlgorithms path; both already null the cell's pointer first, so `CompressionStreamCoder__destroy` keeps its signature and call sites). Each in-flight `CompressionAsyncCtx` takes its own reference when the async step is scheduled and drops it with the ctx on the JS thread. The backend is freed when the last reference drops, so teardown releases the cell's hold but can no longer free the state under the pool thread. On the teardown paths where the completion never gets delivered, the coder is abandoned with the dying process instead of freed early, which is the bounded-leak tradeoff the worker teardown path already takes elsewhere. Related: #36983 fences `WebWorker::shutdown` on outstanding off-thread jobs, which closes the worker-terminate door from the other side (and is still needed for it: after this change, the worker repro's surviving report moves to `EventLoop::enqueue_task_concurrent` via `WorkTask::on_finish` on the freed worker loop, which is exactly the bug that PR addresses, now with a `WorkTask` stack). This change covers what the fence cannot: the main-thread `BUN_DESTRUCT_VM_ON_EXIT=1` exit path, and the coder's own lifetime independent of teardown ordering. ### Verification - New test in `test/js/web/streams/compression.test.ts` (ASAN-gated): fails on the unfixed build with the ASan report above, passes with the fix. - Main-thread repro: 5/5 clean runs with the fix (was UAF on every run before). - `test/js/web/streams/compression.test.ts` (37), `test/regression/issue/18413-all-compressions.test.ts`, `test/regression/issue/23314/zstd-large-decompression.test.ts`, and the four node webstreams compression compat tests all pass. <!-- robobun:evidence:begin --> --- **[review]** gate passed · iteration 0 · 4 files touched <details><summary>fails on main (without fix)</summary> ```console ASAN without fix: 1 FAILED $ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/js/web/streams/compression.test.ts bun test v1.4.0 (38b3183) test/js/web/streams/compression.test.ts: (pass) TransformStream.prototype getters reject native transform subclasses (0) [13.39ms] (pass) TransformStream.prototype getters reject native transform subclasses (1) [2.50ms] (pass) TransformStream.prototype getters reject native transform subclasses (2) [2.66ms] (pass) TransformStream.prototype getters reject native transform subclasses (3) [2.27ms] (pass) CompressionStream and DecompressionStream > brotli > compresses data with brotli [15.13ms] (pass) CompressionStream and DecompressionStream > brotli > decompresses brotli data [21.20ms] (pass) CompressionStream and DecompressionStream > brotli > round-trip compression with brotli [51.32ms] (pass) CompressionStream and DecompressionStream > zstd > compresses data with zstd [9.46ms] (pass) CompressionStream and DecompressionStream > zstd > decompresses zstd data [18.82ms] (pass) CompressionStream and DecompressionStream > zstd > round-trip compression with zstd [36.88ms] ( ... (truncated) release without fix: 1 skipped bun test v1.4.0-canary.1 (0ac8ea9) test/js/web/streams/compression.test.ts: (pass) TransformStream.prototype getters reject native transform subclasses (0) [0.42ms] (pass) TransformStream.prototype getters reject native transform subclasses (1) [0.07ms] (pass) TransformStream.prototype getters reject native transform subclasses (2) [0.06ms] (pass) TransformStream.prototype getters reject native transform subclasses (3) [0.03ms] (pass) CompressionStream and DecompressionStream > brotli > compresses data with brotli [0.91ms] (pass) CompressionStream and DecompressionStream > brotli > decompresses brotli data [0.78ms] (pass) CompressionStream and DecompressionStream > brotli > round-trip compression with brotli [1.80ms] (pass) CompressionStream and DecompressionStream > zstd > compresses data with zstd [0.58ms] (pass) CompressionStream and DecompressionStream > zstd > decompresses zstd data [0.43ms] (pass) CompressionStream and DecompressionStream > zstd > round-trip compression with zstd [1.02ms] (pass) CompressionStream and DecompressionStream > zstd > decompresses a multi-frame zstd stream [0.28ms] (pass) CompressionStream and DecompressionStream > zstd > decompr ... (truncated) ``` </details> <details><summary>passes on PR (with fix)</summary> ```console ASAN with fix: all passed $ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/js/web/streams/compression.test.ts bun test v1.4.0 (38b3183) test/js/web/streams/compression.test.ts: (pass) TransformStream.prototype getters reject native transform subclasses (0) [13.88ms] (pass) TransformStream.prototype getters reject native transform subclasses (1) [2.67ms] (pass) TransformStream.prototype getters reject native transform subclasses (2) [2.66ms] (pass) TransformStream.prototype getters reject native transform subclasses (3) [2.15ms] (pass) CompressionStream and DecompressionStream > brotli > compresses data with brotli [15.57ms] (pass) CompressionStream and DecompressionStream > brotli > decompresses brotli data [22.42ms] (pass) CompressionStream and DecompressionStream > brotli > round-trip compression with brotli [51.98ms] (pass) CompressionStream and DecompressionStream > zstd > compresses data with zstd [10.10ms] (pass) CompressionStream and DecompressionStream > zstd > decompresses zstd data [18.91ms] (pass) CompressionStream and DecompressionStream > zstd > round-trip compression with zstd [38.62ms] ... (truncated) release with fix: 1 skipped $ bun scripts/build.ts --profile=release [configured] bun-profile → bun (stripped) in 820ms (unchanged) ninja: Entering directory `/workspace/bun/build/release' [1/12] gen generated_host_exports.rs generated_host_exports.rs: 92 exports (host=3, lazy=10, generic=79, rust=0); 239 extern-C blocks audited [1/12] cargo bun_bin → libbun_rust.a (--target x86_64-unknown-linux-gnu) nightly-2026-07-20-x86_64-unknown-linux-gnu unchanged - rustc 1.99.0-nightly (9f36de775 2026-07-19) �[1m�[92m Compiling�[0m bun_core v0.0.0 (/workspace/bun/src/bun_core) �[1m�[92m Compiling�[0m bun_errno v0.0.0 (/workspace/bun/src/errno) �[1m�[92m Compiling�[0m bun_ptr v0.0.0 (/workspace/bun/src/ptr) �[1m�[92m Compiling�[0m bun_boringssl_sys v0.0.0 (/workspace/bun/src/boringssl_sys) �[1m�[92m Compiling�[0m bun_safety v0.0.0 (/workspace/bun/src/safety) �[1m�[92m Compiling�[0m bun_zlib_sys v0.0.0 (/workspace/bun/src/zlib_sys) �[1m�[92m Compiling�[0m bun_cares_sys v0.0.0 (/workspace/bun/src/cares_sys) �[1m�[92m Compiling�[0m bun_zstd v0.0.0 (/workspace/bun/src/zstd) �[1m�[92m Compiling�[0m bun_picohttp v0.0.0 (/workspace/bun/src/picohttp) �[1m�[92m Compiling�[0m bun_brotli ... (truncated) ``` </details> <details><summary>diff hotspot</summary> ``` .../bindings/webcore/streams/JSCompressionStream.h | 7 ++-- .../webcore/streams/JSCompressionStreamShared.h | 1 + src/runtime/webcore/CompressionStreamCoder.rs | 43 +++++++++++++++---- test/js/web/streams/compression.test.ts | 49 +++++++++++++++++++++- 4 files changed, 87 insertions(+), 13 deletions(-) ``` </details> **gate history** · 2 passed · 0 rejected · iteration 0 <details><summary>evidence per changed file</summary> ``` file reads edits tests src/jsc/bindings/webcore/streams/JSCompressionStream.h 2 2 0 …sc/bindings/webcore/streams/JSCompressionStreamShared.h 2 2 0 src/runtime/webcore/CompressionStreamCoder.rs 3 6 0 test/js/web/streams/compression.test.ts 1 1 0 ``` </details> <!-- robobun:evidence:end -->
springmin
pushed a commit
to springmin/bun
that referenced
this pull request
Aug 8, 2026
…ad transforms (oven-sh#37139) ### Repro `CompressionStream('brotli')` with a chunk over 128 KiB runs the codec step on a WorkPool thread. Tearing down the VM while that step is in flight frees the native coder under the pool thread: ```js // ASAN build, BUN_DESTRUCT_VM_ON_EXIT=1 (the CI test runner sets this) const s = new CompressionStream("brotli"); const w = s.writable.getWriter(); const big = new Uint8Array(6 << 20); for (let i = 0; i < big.length; i += 3) big[i] = (i * 2654435761) >>> 24; w.write(big).catch(() => {}); w.close().catch(() => {}); s.readable.getReader().read().catch(() => {}); setTimeout(() => process.exit(0), 15); ``` ``` ==ERROR: AddressSanitizer: heap-use-after-free ... thread (Bun Pool 0) #0 UpdateNodes vendor/brotli/c/enc/backward_references_hq.c:468 ... #4 BrotliEncoderCompressStream vendor/brotli/c/enc/encode.c:1661 #5 CompressionStreamCoder::transform src/runtime/webcore/CompressionStreamCoder.rs:367 freed by: BrotliEncoderDestroyInstance CompressionStreamCoder__destroy JSCompressionStream.cpp:176 (CFinalizer) JSC::Heap::CFinalizerOwner::finalize -> Heap::lastChanceToFinalize ``` The same free-under-the-pool-thread happens on `worker.terminate()` / `process.exit()` inside a worker while a large write is in flight (`WebWorker::shutdown` -> `WebWorker__teardownJSCVM` -> `lastChanceToFinalize`). Other faces of the same report: READ 1 in `BrotliEstimateBitCostsForLiterals` / `UpdateNodes`, WRITE 4 in `StoreAndFindMatchesH10`. `DecompressionStream` has the identical finalizer shape, and zstd/zlib formats share the path. ### Cause The stream cell's CFinalizer (registered in the constructor) destroys `m_coder` unconditionally. During normal operation the in-flight task's `Strong` root keeps the cell from being swept, and the eager ClearAlgorithms release already defers on `m_asyncCodecInFlight`. But `Heap::lastChanceToFinalize` at VM teardown runs every finalizer regardless of roots, so the coder (brotli ring buffer + hasher, zlib window, zstd ctx) is freed while the pool thread is still inside `transform`. ### Fix Reference-count the coder. The JS cell holds one reference, released where it released before (finalizer, or the eager ClearAlgorithms path; both already null the cell's pointer first, so `CompressionStreamCoder__destroy` keeps its signature and call sites). Each in-flight `CompressionAsyncCtx` takes its own reference when the async step is scheduled and drops it with the ctx on the JS thread. The backend is freed when the last reference drops, so teardown releases the cell's hold but can no longer free the state under the pool thread. On the teardown paths where the completion never gets delivered, the coder is abandoned with the dying process instead of freed early, which is the bounded-leak tradeoff the worker teardown path already takes elsewhere. Related: oven-sh#36983 fences `WebWorker::shutdown` on outstanding off-thread jobs, which closes the worker-terminate door from the other side (and is still needed for it: after this change, the worker repro's surviving report moves to `EventLoop::enqueue_task_concurrent` via `WorkTask::on_finish` on the freed worker loop, which is exactly the bug that PR addresses, now with a `WorkTask` stack). This change covers what the fence cannot: the main-thread `BUN_DESTRUCT_VM_ON_EXIT=1` exit path, and the coder's own lifetime independent of teardown ordering. ### Verification - New test in `test/js/web/streams/compression.test.ts` (ASAN-gated): fails on the unfixed build with the ASan report above, passes with the fix. - Main-thread repro: 5/5 clean runs with the fix (was UAF on every run before). - `test/js/web/streams/compression.test.ts` (37), `test/regression/issue/18413-all-compressions.test.ts`, `test/regression/issue/23314/zstd-large-decompression.test.ts`, and the four node webstreams compression compat tests all pass. <!-- robobun:evidence:begin --> --- **[review]** gate passed · iteration 0 · 4 files touched <details><summary>fails on main (without fix)</summary> ```console ASAN without fix: 1 FAILED $ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/js/web/streams/compression.test.ts bun test v1.4.0 (38b3183) test/js/web/streams/compression.test.ts: (pass) TransformStream.prototype getters reject native transform subclasses (0) [13.39ms] (pass) TransformStream.prototype getters reject native transform subclasses (1) [2.50ms] (pass) TransformStream.prototype getters reject native transform subclasses (2) [2.66ms] (pass) TransformStream.prototype getters reject native transform subclasses (3) [2.27ms] (pass) CompressionStream and DecompressionStream > brotli > compresses data with brotli [15.13ms] (pass) CompressionStream and DecompressionStream > brotli > decompresses brotli data [21.20ms] (pass) CompressionStream and DecompressionStream > brotli > round-trip compression with brotli [51.32ms] (pass) CompressionStream and DecompressionStream > zstd > compresses data with zstd [9.46ms] (pass) CompressionStream and DecompressionStream > zstd > decompresses zstd data [18.82ms] (pass) CompressionStream and DecompressionStream > zstd > round-trip compression with zstd [36.88ms] ( ... (truncated) release without fix: 1 skipped bun test v1.4.0-canary.1 (0ac8ea9) test/js/web/streams/compression.test.ts: (pass) TransformStream.prototype getters reject native transform subclasses (0) [0.42ms] (pass) TransformStream.prototype getters reject native transform subclasses (1) [0.07ms] (pass) TransformStream.prototype getters reject native transform subclasses (2) [0.06ms] (pass) TransformStream.prototype getters reject native transform subclasses (3) [0.03ms] (pass) CompressionStream and DecompressionStream > brotli > compresses data with brotli [0.91ms] (pass) CompressionStream and DecompressionStream > brotli > decompresses brotli data [0.78ms] (pass) CompressionStream and DecompressionStream > brotli > round-trip compression with brotli [1.80ms] (pass) CompressionStream and DecompressionStream > zstd > compresses data with zstd [0.58ms] (pass) CompressionStream and DecompressionStream > zstd > decompresses zstd data [0.43ms] (pass) CompressionStream and DecompressionStream > zstd > round-trip compression with zstd [1.02ms] (pass) CompressionStream and DecompressionStream > zstd > decompresses a multi-frame zstd stream [0.28ms] (pass) CompressionStream and DecompressionStream > zstd > decompr ... (truncated) ``` </details> <details><summary>passes on PR (with fix)</summary> ```console ASAN with fix: all passed $ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/js/web/streams/compression.test.ts bun test v1.4.0 (38b3183) test/js/web/streams/compression.test.ts: (pass) TransformStream.prototype getters reject native transform subclasses (0) [13.88ms] (pass) TransformStream.prototype getters reject native transform subclasses (1) [2.67ms] (pass) TransformStream.prototype getters reject native transform subclasses (2) [2.66ms] (pass) TransformStream.prototype getters reject native transform subclasses (3) [2.15ms] (pass) CompressionStream and DecompressionStream > brotli > compresses data with brotli [15.57ms] (pass) CompressionStream and DecompressionStream > brotli > decompresses brotli data [22.42ms] (pass) CompressionStream and DecompressionStream > brotli > round-trip compression with brotli [51.98ms] (pass) CompressionStream and DecompressionStream > zstd > compresses data with zstd [10.10ms] (pass) CompressionStream and DecompressionStream > zstd > decompresses zstd data [18.91ms] (pass) CompressionStream and DecompressionStream > zstd > round-trip compression with zstd [38.62ms] ... (truncated) release with fix: 1 skipped $ bun scripts/build.ts --profile=release [configured] bun-profile → bun (stripped) in 820ms (unchanged) ninja: Entering directory `/workspace/bun/build/release' [1/12] gen generated_host_exports.rs generated_host_exports.rs: 92 exports (host=3, lazy=10, generic=79, rust=0); 239 extern-C blocks audited [1/12] cargo bun_bin → libbun_rust.a (--target x86_64-unknown-linux-gnu) nightly-2026-07-20-x86_64-unknown-linux-gnu unchanged - rustc 1.99.0-nightly (9f36de775 2026-07-19) �[1m�[92m Compiling�[0m bun_core v0.0.0 (/workspace/bun/src/bun_core) �[1m�[92m Compiling�[0m bun_errno v0.0.0 (/workspace/bun/src/errno) �[1m�[92m Compiling�[0m bun_ptr v0.0.0 (/workspace/bun/src/ptr) �[1m�[92m Compiling�[0m bun_boringssl_sys v0.0.0 (/workspace/bun/src/boringssl_sys) �[1m�[92m Compiling�[0m bun_safety v0.0.0 (/workspace/bun/src/safety) �[1m�[92m Compiling�[0m bun_zlib_sys v0.0.0 (/workspace/bun/src/zlib_sys) �[1m�[92m Compiling�[0m bun_cares_sys v0.0.0 (/workspace/bun/src/cares_sys) �[1m�[92m Compiling�[0m bun_zstd v0.0.0 (/workspace/bun/src/zstd) �[1m�[92m Compiling�[0m bun_picohttp v0.0.0 (/workspace/bun/src/picohttp) �[1m�[92m Compiling�[0m bun_brotli ... (truncated) ``` </details> <details><summary>diff hotspot</summary> ``` .../bindings/webcore/streams/JSCompressionStream.h | 7 ++-- .../webcore/streams/JSCompressionStreamShared.h | 1 + src/runtime/webcore/CompressionStreamCoder.rs | 43 +++++++++++++++---- test/js/web/streams/compression.test.ts | 49 +++++++++++++++++++++- 4 files changed, 87 insertions(+), 13 deletions(-) ``` </details> **gate history** · 2 passed · 0 rejected · iteration 0 <details><summary>evidence per changed file</summary> ``` file reads edits tests src/jsc/bindings/webcore/streams/JSCompressionStream.h 2 2 0 …sc/bindings/webcore/streams/JSCompressionStreamShared.h 2 2 0 src/runtime/webcore/CompressionStreamCoder.rs 3 6 0 test/js/web/streams/compression.test.ts 1 1 0 ``` </details> <!-- robobun:evidence:end -->
This was referenced Aug 8, 2026
Collaborator
Author
|
Closing: this was superseded by #37075 (Worker / worker_threads lifetimes and ordered VM teardown, merged 2026-08-08), which makes a terminating worker wait for or release its in-flight off-thread work (thread pool, HTTP thread, bundle thread, napi) before the VM is freed. Both test files from this branch (test/js/node/zlib/zlib-worker-terminate.test.ts and test/js/web/workers/worker-terminate-offthread.test.ts) pass unmodified against an ASAN debug build of current main (04148c8), 28 pass / 0 fail across three runs, whereas 11 of them failed on main when this PR was opened. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
worker.terminate()(and the other three teardown doors:process.exit()in the worker, an uncaught throw, an unhandled rejection) frees the worker'sVirtualMachinebox, itsEventLoop, the uws loop, and the JSC heap while jobs that VM handed to other threads are still running. Every such job holds raw pointers back into that memory: WorkPool bodies post completions withenqueue_task_concurrent, the HTTP thread's fetch/S3 callbacks readjavascript_vm, napiexecutecallbacks write into ArrayBuffer stores, the bundler thread reads the VM's env loader. Natural exit is safe (each job's KeepAlive holds the event loop open), butWebWorker::spinbreaks out of that loop immediately on terminate andshutdown()never re-checked, so teardown raced every in-flight job.Repro (any of the families below; aborts under ASAN with heap-use-after-free in the pool thread's completion post):
The fix makes teardown wait for those jobs:
EventLoop.outstanding_offthread: a counter of off-thread jobs whose body can still touch VM-owned memory. Every schedule site that pairs aKeepAlive::ref_with aWorkPool/HTTP-thread/bundler-thread handoff takes a count; the off-thread body releases it through a local pointer copy after its last VM access (usually right after the completion enqueue).WebWorker::shutdowngains a fence betweenmarkShuttingDownand the queued-task drain: set a per-loop cancel flag (cancel-aware pool bodies skip their compute), run the new per-VM cancel-hook registry (in-flight fetch aborts; S3 requests getschedule_shutdown_by_id), then wait for the counter to reach zero. Completions posted during the wait are reclaimed unrun by the existing drain, while JSC is still alive.executethat never returns, a bundler plugin round-trip the dead JS thread can never answer), the VM, its loops, the JSC heap, and the cloned env loader are leaked instead of freed: a bounded leak on a pathological terminate instead of a use-after-free.Bracketed families:
WorkTask(blob read/write, web CompressionStream, dns),AnyTaskJob(pbkdf2, scrypt, randomFill, the C++ keygen/sign jobs,Bun.zstd*, secrets),ConcurrentPromiseTask(glob scan, transpiler, image, blob copy),AsyncFSTask(the 42 async fs ops),AsyncReaddirRecursiveTask,AsyncCpTask,ConcurrentCppTask(webcrypto, node:sqlite async),PasswordJob, shell builtins'ShellTask, asyncnode:zlib/brotli/zstd,napi_async_work,fs.watchFile's initial stat,FetchTasklet, S3 simple + streaming,JSBundleCompletionTask.Also fixed on the fetch path: the HTTP thread's last-ref reclaim used to park the tasklet box for the process-exit drain even for worker VMs, whose JSC handles are long gone by then; worker tasklets now leak the box instead (the buffers were already released).
Known limits, called out in comments: the Windows-only libuv flows (
UVFSRequestfor open/read/write/close,WriteFileWindows,CopyFileWindows) are separate machinery, scheduled on the worker's own uv loop rather than the WorkPool, and not bracketed here; a bundler build waiting on a JS plugin response stalls to the deadline and leaks (the plugin protocol has no cancellation);WorkTaskcontext payloads queued at teardown are still requeued rather than freed (status quo).Verification: a new ASAN-gated test file
test/js/web/workers/worker-terminate-offthread.test.tswith a door x family matrix (12 off-thread families; terminate/exit/throw doors) plus the four-door fetch matrix, and the async zlib test intest/js/node/zlib/zlib-worker-terminate.test.ts. On an unfixed build the matrix reports ASAN heap-use-after-free inEventLoop::vm_ref/VirtualMachine::event_loop_sharedacross the families (10 hits in one run); with the fix 20/20 pass. The tests tolerate exactly one known unrelated failure signature: debug builds can still trip JSC'sExceptionScope::assertNoExceptionwhen terminate lands mid-dispatch, which reproduces on unfixed main before teardown even starts and is tracked separately; any sanitizer report still fails the test.Supersedes #35155 and #36575, and generalizes the counter from #36855 (napi addon test coverage stays with that PR; its mechanism is included here). #36818 (fs write buffer pin) and #35161 (dns same-thread teardown) remain separate.
[review] gate passed · iteration 2 · 27 files touched
fails on main (without fix)
passes on PR (with fix)
diff hotspot
gate history · 6 passed · 0 rejected · iteration 2
evidence per changed file