s3: post the streaming download's final task through a copy of the handle - #37880
s3: post the streaming download's final task through a copy of the handle#37880robobun wants to merge 1 commit into
Conversation
…ndle S3HttpDownloadStreamingTask::http_callback posted through (*this).loop_handle. On the final callback on_response frees *this as soon as the task is queued, while post_task's &self still points into the allocation. Clone the handle out first and post and hand the request back through the copy, as release_at_shutdown and S3HttpSimpleTask::http_callback already do. Adds a source lint for src/runtime/webcore/s3/ banning post_task and embedded_work_finished through the stored handle.
|
Warning Review limit reached
Next review available in: 21 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (2)
Comment |
|
Status: fix pushed, self-review in progress. Reproduced how: the new lint reports |
There was a problem hiding this comment.
LGTM — brings http_callback into the same clone-then-post shape as the three sibling hand-offs in this directory.
What was reviewed:
- Confirmed
simple_request.rsandrelease_at_shutdownalready post through a cloned handle; this change makeshttp_callbackmatch them exactly. - The clone is taken before
process_http_callback(whose unlock is the other point at which the JS thread can free*this), so bothpost_taskandembedded_work_finishedare covered. - Verified no
.loop_handle.post_task/.loop_handle.embedded_work_finishedremain insrc/runtime/webcore/s3/; the new lint's non-vacuity checks (field declaration + post site present) hold. - The lint follows the existing
test/internal/source-lints/conventions (globAllSources, git ls-tree guard, self-tested regex).
Extended reasoning...
Overview
Two files changed. In src/runtime/webcore/s3/download_stream.rs, S3HttpDownloadStreamingTask::http_callback is refactored so the LoopHandle is cloned out of *this unconditionally before process_http_callback, and both post_task and embedded_work_finished go through that local copy instead of (*this).loop_handle. The conditional done_handle Option is removed since one clone now serves both calls. The task construction and control flow are otherwise byte-for-byte the same. A new source-lint test, test/internal/source-lints/s3-post-through-handle-field.test.ts, bans the .loop_handle.post_task( / .loop_handle.embedded_work_finished( spellings (method-call and path-call, rustfmt-wrapped) within src/runtime/webcore/s3/, self-tests its regex against banned/allowed sample lists, and asserts the scanned files still declare a loop_handle: LoopHandle field and still call .post_task( so a rename can't blind it silently.
Security risks
None. This is an internal cross-thread hand-off; no user-facing surface, parsing, or trust boundary is touched.
Level of scrutiny
Memory-safety code in unsafe Rust normally warrants high scrutiny, but the change here is mechanical: it converts one call site to the exact shape already used by release_at_shutdown in the same file and both hand-offs in simple_request.rs (verified by grep). The old code already cloned the handle for embedded_work_finished; the diff extends that same clone to cover post_task. The only behavioral delta is one extra Arc inc/dec on non-final callbacks, which the PR description acknowledges and which is negligible next to the mutex the callback already takes. The clone is placed before process_http_callback, which matters: on the final callback with a task already queued, the JS thread can free *this as soon as process_http_callback unlocks, so cloning after would be too late — the placement is correct.
Other factors
The source-lint test follows the established directory conventions closely: it uses globAllSources, the same git ls-tree tracked-file guard as dead-code-escapes.test.ts, strips full-line comments before matching (with the correct [ \t]* to preserve line numbers), self-tests the regex against explicit banned and allowed lists, and guards against vacuous passes by counting field declarations and post sites. The lint is scoped to src/runtime/webcore/s3/ where the rule holds unconditionally, and the PR description enumerates the out-of-scope sites and why they are handled separately. The PR description includes a Miri reduction demonstrating the aliasing violation under both Tree Borrows and Stacked Borrows, and reports the existing S3 test suite and all 19 source-lint files pass on the debug/ASAN build.
|
Updated 2:48 PM PT - Aug 12th, 2026
❌ @robobun, your commit 5b7b731 has 4 failures in
🧪 To try this PR locally: bunx bun-pr 37880That installs a local version of the PR into your bun-37880 --bun |
Problem
s3file.stream(), or reading anS3Fileas aReadableStream) the HTTP thread posts its hand-back task by callingpost_taskon theloop_handlefield stored inside the request's own allocation.post_taskis still running on the HTTP thread with&selfpointing into it. Freeing memory behind a live reference argument is undefined behaviour whether or not the callee reads through it again.deallocation through <tag> ... is forbidden,protected tags must never be Disabled) and with Stacked Borrows; the same reduction posting through a copy of the handle passes.LoopHandle. The other three hand-offs in the directory already post through a clone.Fix
src/runtime/webcore/s3/that bans posting or handing back through a storedloop_handle. Onmainit reports exactly the one line this PR changes.Background
LoopHandle: a cloneable, Arc backed handle to a JS VM's event loop.post_taskqueues a task for the JS thread and wakes it;embedded_work_finishedtells the VM one piece of off-thread work is done, which the VM waits for before closing.concurrent_task, so posting it hands the JS thread the very allocation the handle field lives in, and the JS side (on_response) frees that allocation on the final delivery.&selfargument must stay valid for the whole call, and codegen marks itdereferenceable. Freeing the memory it points into mid-call is UB even from another thread and even if the callee never touches it again.bun run rust:miri) is the interpreter that checks those models; it reports this class of bug where ASAN and normal tests see nothing.test/internal/source-lints/holds regex tests over the source tree that stop a fixed pattern from coming back. This one is scoped to the s3 directory because the same spelling is sound in places where something else keeps the object alive across the post.Original description
Problem
S3HttpDownloadStreamingTask::http_callback(src/runtime/webcore/s3/download_stream.rs) is how the HTTP thread delivers each chunk of a streaming S3 download (s3file.stream(), anything else that reads anS3Fileas aReadableStream). It posted the task withThe task it posts is the request's own allocation (the inline
concurrent_taskfield), and on the final callback the JS-thread consumer,on_response, frees that allocation (drop(heap::take(this_ptr))) as soon as it observeshas_more == false, which it can do the momentVmHandle::posthas pushed the task and woken the loop.post_task(&self)and theVmHandle::post(&self, ..)it calls are still running on the HTTP thread at that point, and their&selfis&(*this).loop_handle: a reference into the allocation being freed. A reference argument is protected for the duration of its call, and freeing memory a protected reference points into is UB under both aliasing models whether or not the callee reads through it again (codegen relies on the same contract: the argument is annotated dereferenceable for the whole call). Nothing does read through it after the push today, so no crash is known; the contract is what is wrong, and it is wrong on the final callback of every streaming download.The function already clones the handle for the
embedded_work_finished()that follows the post, for exactly this reason ("afterthismay have been freed"), and the directory's other hand-offs (release_at_shutdownin the same file, and bothS3HttpSimpleTask::http_callbackand itsrelease_at_shutdownin simple_request.rs) already post through the clone. Before #37075 this site copied the VM pointer out before enqueueing; the move toLoopHandleturned that into a call through the field.A reduction of this exact shape (the consumer on a second thread frees the allocation while the poster is still inside
post_task(&self), andpost_taskreads nothing from the allocation after the push) fails under Miri with Tree Borrows, the modelbun run rust:miriuses:and under Stacked Borrows with
not granting access to tag <2994> because that would remove [SharedReadOnly for <6802>] which is strongly protected, where<6802>is the same&self. The same reduction posting through a copy of the handle passes under both.Reduction run under Miri
MIRIFLAGS=-Zmiri-tree-borrows cargo miri run -- beforeand the default (Stacked Borrows) run both exit 1 with the errors quoted above;-- afterexits 0 under both.Fix
http_callbackclones the handle out unconditionally beforeprocess_http_callback, posts through the clone, and callsembedded_work_finished()on it whenis_done, the shape of the three sibling hand-offs. The conditionaldone_handleis gone since the one copy now serves both calls; on non-final callbacks this adds one Arc increment and decrement per callback, next to the mutex that callback already takes. Behaviour is unchanged: the same task goes to the same queue at the same point, and the request is handed back at the same point.The same spelling exists outside this directory and is not part of this change. Sites where it is the same bug each have their own fix: the two Windows mkdirp completions in blob/copy_file.rs and blob/write_file.rs (#37705),
napi_async_work::post_to_js_thread(#37750),FetchTasklet::deref_from_thread(posts through a&selfhelper; reported separately) andStatWatcher::post_to_js_thread(reported separately; #37591 reshapes it). The others post while something else keeps the object alive (the bundler's plugin-dispatch thunk in js_bundle_completion_task.rs during the build, the stat watcher scheduler, the threadsafe function under its lock, AsyncModule's leakedWakeContext, theFetchTasklethelper's other two callers, which hold a ref), which is why the spelling cannot simply be banned tree-wide; theFSWatcher::posthelper in node_fs_watcher.rs was not examined here beyond noting that its batch holds an activity ref across the post.Tests
test/internal/source-lints/s3-post-through-handle-field.test.tsbanspost_taskandembedded_work_finishedthrough a storedloop_handle(method-call and path-call spellings, including rustfmt-wrapped chains) in src/runtime/webcore/s3/, where every object storing a handle is freed by the JS thread on delivery, so the rule holds unconditionally and the allowlist is empty.embedded_work_scheduled()through the field,.clone()and posting through a local are allowed, and the file checks its pattern against both lists. It also checks that the scanned files still declare the field and still post, so a rename cannot blind it silently. Againstmainit reports exactlyand passes with this branch.
Verification
On the debug (ASAN) build: a script streaming, 150 rounds of 6 concurrent downloads each, a 512 KiB body, an empty body, a 64-chunk trickled body, the same two with the consumer deliberately late (so that the final callback can arrive with a task already queued, the path where the earlier task is the one that frees), and a 404 whose XML error is reported on the final callback, with GC between rounds, completes without a sanitizer report. test/js/bun/s3/{s3-stream-cancel-leak,s3-stream-error-gc,s3-connection-close,s3-insecure,s3-requester-pays,s3-storage-class,s3-list-objects,s3-list-checksum-algorithm,s3-fd-validation,s3-argument-validation}.test.ts pass (the connection-close and list-objects files only with
--timeoutraised: their concurrent debug-build subprocesses take 7 to 15 s each in this container, with or without this change; the credentialed s3.test.ts streaming coverage runs in CI).bun test test/internal/source-lints/(19 files) passes; rustfmt is clean on the changed file.