Run pool jobs that own their memory without a VM borrow so worker.terminate() does not wait on a blocked Bun.write(file, file) - #38312
Conversation
…nate() does not wait for them Job::run_on_pool took a VM borrow around every JobContext's run(), so a worker's teardown waited for whatever the body was doing. CopyFile does the whole copy inside run() and opens its source blocking, so a Bun.write(file, file) from a FIFO, tty or idle pipe kept the borrow until the other side acted and worker.terminate() never settled. getaddrinfo (dns.lookup) and Bun.secrets had the same shape. JobContext now declares what run() holds on the VM: `type Vm = Borrow` for bodies that reach VM-owned memory (the carrier borrows as before), `type Vm = Unborrowed` for bodies that own everything they touch (the carrier only skips a job whose VM is already closed; a VM torn down under a running body refuses its completion, which the job releases on the pool thread as it already did for a late post). CopyFile, ReadFile, WriteFile, the libc dns lookup, Bun.secrets, Bun.password, Glob.scan and Bun.Archive run unborrowed; the jobs that read caller buffers or write into JS-allocated ones keep the borrow. Bun.secrets' C++ runTask never used the global it was handed. For the file jobs to own their memory the store has to own its path: Bun.file(bytes) and Bun.write(bytes, ...) kept a pinned PathLike::Buffer into the caller's ArrayBuffer in the Store (so the path followed later writes to the buffer, the protect taken for it was never released, and dropping the store off the JS thread would unpin a dead heap). Store::init_file / init_s3 copy a byte path into owned bytes on the JS thread, which is what the Bun.file(bytes) documentation says happens.
|
Warning Review limit reached
Next review available in: 3 seconds Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (24)
Comment |
|
Updated 1:05 AM PT - Aug 14th, 2026
❌ @robobun, your commit 3a626be has some failures in 🧪 To try this PR locally: bunx bun-pr 38312That installs a local version of the PR into your bun-38312 --bun |
|
Status: reproduced and fixed, waiting on CI for 3a626be. Reproduced on canary |
There was a problem hiding this comment.
I reviewed this PR and the automated bug hunt found no issues. Because it changes the core JobContext contract (the new type Vm: VmHold split) and reclassifies eight jobs as safe to run while their VM tears down — each an individual memory-safety claim — a human look at the Unborrowed classifications and the Store path-ownership change would still be worthwhile.
What was reviewed:
- The
VmHoldsealed trait:JsPtr::under_borrowstill requires&Borrow, so anUnborrowedbody cannot type-check a JS-memory access. own_pathcovers the removedto_thread_safe()infind_or_create_file_from_path; the two directData::File(...)constructors that bypassinit_file(stdio injsc_hooks.rsandBunObject.rs) both useFd, notPathLike::Buffer.Bun__SecretsJobOptions__runTask's droppedglobalparameter — confirmed the C++ body never read it.
Extended reasoning...
Overview
This PR fixes a regression where worker.terminate() hangs indefinitely if the worker has a blocking pool job (e.g. Bun.write(file, Bun.file(fifo)), getaddrinfo, a keychain prompt) in flight. The fix adds an associated type type Vm: VmHold to JobContext (src/jsc/job.rs), sealed to Borrow (VM teardown waits for the body, as before) and the new Unborrowed (teardown does not wait; a completion posted after close is refused and released on the pool thread). All 19 JobContext impls are classified across 15 runtime files. Additionally, Store::init_file/init_s3 now copy a PathLike::Buffer path into owned bytes (own_path) so file stores hold no JS-heap references — a prerequisite for the file jobs' Unborrowed claim, and a fix for a pre-existing doc-vs-behavior mismatch (the buffer was documented as copied but was actually borrowed and pinned, with the pin never released). Three tests cover the terminate hang, the buffer-copy semantics, and the pool-thread release path under ASAN.
Security risks
None identified. The change is about thread-lifetime coordination inside the runtime; no auth, crypto semantics, or user-input parsing is altered. The crypto job impls only add type Vm = Borrow (preserving existing behavior).
Level of scrutiny
High. This touches the core VM-handle / thread-pool lifecycle (src/jsc/VmHandle.rs, src/jsc/job.rs) and each Unborrowed declaration is an assertion that the job's OffThread and its Drop touch nothing of the JS heap — getting one wrong is a use-after-free on a pool thread after VM teardown. The own_path change in Store.rs is load-bearing for three of those assertions. The design (a sealed trait with two impls, dispatched via C::Vm::acquire) is a new abstraction that every future JobContext author must choose correctly, so a maintainer should sign off on the shape.
Other factors
- I spot-checked the two places that construct
Data::Filedirectly without going throughinit_file(__bun_stdio_blob_store_newinjsc_hooks.rsandbuild_storeinBunObject.rs): both usePathOrFileDescriptor::Fd, so they cannot carry aPathLike::Bufferand don't needown_path. - The
to_thread_safe()removed fromfind_or_create_file_from_pathinBlob.rsis subsumed byown_pathinsideinit_file/init_s3, which is called downstream on both the file and S3 branches. - The
Unborrowed::acquiresnapshot ofis_closed()is racy by design (the doc says so): a body that starts against a still-open VM and then loses the race has its completion refused via the existingPostable::release_refusedpath — no new mechanism. - Test coverage is good: the FIFO test would hang on the unfixed build, the byte-path test asserts the documented copy semantics, and the refused-completion row runs
CopyFile's pool-thread release under ASAN. The PR description enumerates which surrounding suites were run. - No prior human or bot review comments to address; CI is still building.
|
Follow-ups since the PR was opened:
|
There was a problem hiding this comment.
I reviewed this PR and didn't find any bugs (the earlier nit on PipelineTask's type Vm doc comment was addressed in 3a626be). Because it changes the VM-borrow contract for thread-pool jobs and reclassifies eight of them as safe to run and release without one — a per-impl memory-ownership judgement whose failure mode is UAF under worker teardown — a human look would still be worthwhile.
What was reviewed:
VmHoldsealing andrun_on_pool'sC::Vm::acquiredispatch —JsPtr::under_borrowstill requires&Borrow, so anUnborrowedbody cannot reach JS memory through the typed path.own_pathinStore::init_file/init_s3: confirmed the removedto_thread_safe()infind_or_create_file_from_pathflows intoinit_fileat Blob.rs:3707, and the fd-backed stdio /output_file_jscconstructors carry noPathLike::Buffer.- The
Borrow/Unborrowedsplit across all 19 impls matches the PR description; eachBorrowimpl that binds_vmnow carries a one-line reason.
Extended reasoning...
Overview
This PR introduces an associated type JobContext::Vm: VmHold (sealed to Borrow | Unborrowed) so that thread-pool jobs whose off-thread body owns everything it touches no longer hold a VM borrow while running. VmHandle::close() waits for outstanding borrows, so a job blocked in open(2) on a FIFO (or a keychain prompt, or getaddrinfo) previously made worker.terminate() hang forever. Eight jobs are reclassified as Unborrowed (CopyFile, ReadFile, WriteFile, libc dns.lookup, Bun.secrets, Bun.password, Glob.scan, Bun.Archive); eleven keep Borrow. To make the file jobs' Unborrowed classification sound, Store::init_file/init_s3 now copy a PathLike::Buffer into owned bytes on the JS thread (fixing a documented-but-unimplemented copy semantic and a leaked protect()). Bun__SecretsJobOptions__runTask drops its unused global parameter. Three test files gain coverage.
Security risks
None identified. The change does not touch auth, crypto correctness, input validation, or trust boundaries. The risk profile is memory safety (UAF / off-thread heap access), not security.
Level of scrutiny
High. This is core VM/worker teardown lifecycle code. The correctness of each Unborrowed classification depends on the impl's off-thread body touching only owned memory and its Drop being sound off the JS thread — a wrong call is a UAF that only manifests under terminate() racing a running job. The own_path change alters what thread may drop a Store. The type-level enforcement (sealed trait, JsPtr::under_borrow requiring &Borrow) is sound, but it cannot catch a job that reads JS memory through a raw pointer rather than a JsPtr.
Other factors
The PR description is unusually thorough (per-impl reasoning, mechanism explanation, verification list). All comment-cop and prior-review threads are resolved. Tests cover the regression (FIFO-blocked terminate), the own_path behavioral fix (buffer mutation after Bun.file()), and the pool-thread release path under ASAN. I spot-checked that find_or_create_file_from_path's removed to_thread_safe() is subsumed by init_file's new own_path call, and that the non-init_file Data::File constructors are fd-only. Nothing looks wrong, but the breadth of per-job ownership reasoning across 24 files in memory-safety-critical native code warrants a maintainer's review rather than bot-only approval.
Problem
worker.terminate()never settles if the worker has aBun.write(file, file)in the thread pool whose source is a FIFO, a tty or a pipe nobody is writing to (this includes the documentedBun.write(Bun.stdout, Bun.stdin)). 1.3.14 settled it in a few ms.Job::run_on_pool(src/jsc/job.rs) takes a VM borrow around everyJobContext::run, andVmHandle::close()in the worker's teardown waits for the borrow count to reach zero.CopyFile::run(src/runtime/webcore/blob/copy_file.rs) does the entire copy insiderun, andOPEN_SOURCE_FLAGShas noO_NONBLOCK, so the pool thread sits inopen(2)(or the read) holding the borrow until the other end of the FIFO shows up.getaddrinfo(dns.lookup,src/runtime/dns_jsc/dns.rs) andBun.secrets(a keychain prompt) hold it the same way.JobContextimpls, 5 actually use the borrow inrun. Most of the others still need it: their off-thread half reads the caller's JS buffers or writes into a JS-allocated one. The file jobs,dns,secrets,password,Glob.scanandArchiveown everything they touch, except that a file store could hold aPathLike::Buffer:Bun.file(bytes)/Bun.write(bytes, ...)kept a pointer into the caller's ArrayBuffer in theStore, pinned, so the path followed later writes to the buffer (the docs say the buffer is copied), theprotect()taken for it was never released, and releasing such a store off the JS thread wouldunpin()a dead heap.Fix
JobContextgetstype Vm, which is whatrunholds on the VM:Borrow(as before: the carrier borrows, teardown waits) or the newUnborrowed(the carrier only checksVmHandle::is_closed()to skip a job whose VM is already gone; a VM torn down under a running body refuses the completion and the job is released on the pool thread, the path that already existed for a post that loses the race withclose()). The type is sealed to those two, andJsPtr::under_borrowstill requires a&Borrow, so a body can only reach JS memory if it declaredBorrow.CopyFile,ReadFile,WriteFile, the libcdns.lookup,Bun.secrets,Bun.password,Glob.scanandBun.ArchivedeclareUnborrowed.Bun.secretsstops carrying the global: its C++runTasknever read it.zstd,pbkdf2,scrypt,randomFill, the extern crypto jobs,CompressionStream,Image,Transpilerand bothnode:fsjobs declareBorrow, each with a one-line reason whererundoes not name the borrow (thenode:fscase is Don't block worker/VM teardown on fs thread-pool ops that never complete #37170's).Store::init_file/init_s3now make the path the store's own: aBufferpath is copied into owned bytes on the JS thread (which drops the pin there), string paths go throughto_thread_safe()as before (moved out offind_or_create_file_from_path). With that, the three file jobs hold only native memory, so running them unborrowed and releasing them on a pool thread is sound.test/js/web/workers/worker-terminate-lifetime.test.ts, new test: a worker'sBun.write(out, Bun.file(fifo))blocked inopen(2), thenterminate(). Times out on the unfixed build, settles in ~200ms on the debug build.test/js/bun/util/bun-file.test.ts, new test:Bun.file(bytes)andBun.write(bytes, file)keep their path when the buffer is changed afterwards (Uint8Array and ArrayBuffer). Fails on the unfixed build.test/js/web/workers/worker-refused-completion.test.ts: newBun.write(file, file)row, so CopyFile's pool-thread release runs under ASAN like the other producers'. All 17 rows pass with CI's LSan settings.worker-terminate-funnels,worker_threads.test.ts(122),bun-write.test.js,archive.test.ts,password.test.ts,glob/scan.test.ts, source lints,cargo clippy -p bun_jsc -p bun_runtimepass on the debug build. The pre-existing failures in this container are unrelated: the bun-write copy_file_range and glob node_modules cases exceed their timeouts under debug (test: stop the Bun.write copy_file_range fallback test from starving its concurrent siblings #37792), the c-ares terminate test's LSan report is node:fs: mark the per-VM Binding box as LSan-ignored (fixes worker-terminate-lifetime.test.ts on main) #35159, and dns/secrets tests need network / libsecret.Background
Jobis the carrier for work a VM sends to the thread pool: an off-thread part the pool body runs, and a JS-side part (promise, callback) the VM itself releases at teardown. When the body finishes, it posts a completion back through the VM'sVmHandle; after teardown has closed the handle the post is refused and the job frees its off-thread part right there on the pool thread.VmHandle::borrow) is how off-thread code says it is reading memory the VM owns, such as the bytes of an ArrayBuffer a caller passed tofs.write. Teardown closes the handle only once no borrow is held, so for a body that blocks on something external, holding one turnsterminate()into a wait on that external party. Before Worker / worker_threads: WebCore-shaped lifetimes, joined threads, one ordered VM teardown #37075 these jobs were not waited for at all; this keeps the wait for the jobs that need it.Storeis the refcounted backing of aBlob: bytes, or forBun.file()a path or fd. File jobs keep a ref to it while they run, so whichever thread drops the last ref frees it; that is why it cannot refer to anything in the JS heap.PathLikeis the parsed form of a path argument; itsBuffervariant borrows the caller's ArrayBuffer and pins it (so it cannot be detached or moved) until thePathLikeis dropped, which is only safe on the JS thread.no test proof · iteration 0 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/web/workers/worker-refused-completion.test.ts test/js/web/workers/worker-terminate-lifetime.test.ts