http: clear the request stream drain callback under the buffer mutex - #37435
Conversation
ThreadSafeStreamBuffer.callback is read by the HTTP thread in report_drain() while it holds the buffer mutex, but the JS thread cleared it in clear_drain_callback() without taking the lock. FetchTasklet reaches that from start_request_stream when assign_to_stream fails (for example a direct stream's pull() throwing after writing), at which point the request is still in flight and the HTTP thread may be flushing the bytes that were just written and reading the callback, so the two accesses raced. Take the mutex in clear_drain_callback, make the field private so every access goes through the locked accessors, and debug-assert in report_drain that the caller holds the lock. Fix the comments in FetchTasklet that described the callback as main-thread-only. The new test drives the start_request_stream failure path repeatedly while the HTTP thread is flushing, checking the fetch rejects with pull()'s error and that clearing the callback does not deadlock against the HTTP thread.
WalkthroughChangesThe stream buffer now synchronizes drain callback clearing and reporting with its mutex. Fetch tasklet comments document this lifecycle. A regression test covers concurrent request-body stream failures. Fetch drain callback synchronization
Possibly related issues
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
|
Status: ready for review. Reproduced how: a direct stream request body whose Fix: |
|
Updated 11:22 PM PT - Aug 10th, 2026
✅ @robobun, your commit b0c14895bafec4de37da6ab7947c02d9716d91f9 passed in 🧪 To try this PR locally: bunx bun-pr 37435That installs a local version of the PR into your bun-37435 --bun |
There was a problem hiding this comment.
I reviewed this PR and didn't find any bugs. The fix is small and looks correct — the mutex guard in clear_drain_callback properly serialises against report_drain, and none of the JS-thread callers hold the buffer lock so there's no self-deadlock. That said, this is a cross-thread synchronisation change in the fetch request-body path and the added test is (as noted) path coverage rather than a race detector, so a human look would still be worthwhile.
What was reviewed:
Mutex::lock_guardandis_held_by_current_threadexist and match the usage here.- Both
clear_sinkcallers (assign-failure atFetchTasklet.rs:701and the deinit path) run on the JS thread without the buffer mutex held — no recursive lock. on_write_request_data_drainonly enqueues a task and doesn't re-enter the mutex, so calling it under the lock (unchanged behaviour) is safe.- Making
callbackprivate breaks no other accessors — the only reads/writes are insideThreadSafeStreamBuffer.rs.
Extended reasoning...
Overview
The PR closes a data race on ThreadSafeStreamBuffer.callback by taking the buffer mutex in clear_drain_callback(), making the field private, and adding a debug_assert!(mutex.is_held_by_current_thread()) to report_drain(). Two SAFETY comments in FetchTasklet.rs are updated to drop the now-false "main-thread-only" claim, and a new test in fetch-abort-stream-body.test.ts exercises the assign-failure teardown path 50 times.
Security risks
None. This is internal synchronisation of a two-word Option<Callback> field; no user-controlled input reaches the changed code beyond what already flowed through it.
Level of scrutiny
High. Per the repo review guide, native memory safety and thread affinity are the most-blocked category. The functional change is tiny (one lock guard, one debug assert, one visibility tightening) and strictly adds synchronisation rather than removing it, which limits the blast radius. But it sits on a path where both the HTTP thread and the JS thread hold &mut ThreadSafeStreamBuffer simultaneously — the PR description itself flags that a follow-up should move to Guarded<_> to remove that aliasing — and the new test admittedly passes on the unfixed build (no TSan), so a maintainer confirming the lock ordering and the h2/h3 report_drain call sites is warranted.
Other factors
I verified that bun_threading::Mutex provides both lock_guard() and is_held_by_current_thread() (the latter is a no-op returning true in release, so the debug_assert! is sound). All three report_drain callers (h1 write_to_stream, h2 and h3 drain_send_body) call it between acquire() and release(), so the new assertion holds. The drain callback (on_write_request_data_drain) only refs the tasklet and posts a concurrent task — it does not touch the buffer mutex — so invoking it while the HTTP thread holds the lock cannot deadlock against the new clear_drain_callback lock. The comment-cop bot flags on earlier commits were addressed in b0c1489 and are all resolved. The test follows harness conventions (port: 0, Buffer.alloc, test.concurrent, await using, exact-identity .rejects.toBe).
What does this PR do?
Fixes a data race on
ThreadSafeStreamBuffer.callback(the streaming request body's drain callback) between the HTTP thread and the JS thread.Cause. The HTTP thread reads
callbackinreport_drain()while holding the buffer mutex (src/http/lib.rswrite_to_stream, and the h2/h3drain_send_body). The JS thread cleared it inclear_drain_callback()without the mutex; the comments inFetchTaskletclaimed the field was main-thread-only.FetchTasklet::start_request_streamcallsclear_sink()whenassign_to_streamfails, and at that point the request is still in flight: the HTTP thread still owns its ref on the buffer and may be flushing the bytes that were just written, so the unlocked write ofNoneraced the locked read.Option<Callback>is two words (fn pointer + context), so a stale or torn read callson_write_request_data_drain(tasklet)during teardown. Today the tasklet is still referenced on that path (the HTTP side releases its ref only after the abort completes), so the worst case is a spurious resume task, but nothing in the code enforces that. The other caller,deinit->clear_data->clear_sink, is already ordered after the HTTP thread's last access by the tasklet refcount release.A user-reachable way onto that path is a direct stream request body whose
pull()writes and then throws synchronously:Fix.
clear_drain_callbacktakes the buffer mutex, so it is serialised againstreport_drain; once it returns the HTTP thread can no longer call back into the tasklet through the buffer. The field is now private so every access goes through the locked accessors, andreport_draindebug-asserts that the caller holds the lock (it already had to, per its doc comment). The stale comments inFetchTaskletare corrected.How did you verify your code works?
New test in
test/js/web/fetch/fetch-abort-stream-body.test.tsdrives the snippet above 50 times against a server that does not answer until the upload is torn down, and checks that every fetch rejects withpull()'s own error, thatpull()ran every time (it only runs once the HTTP thread has asked for the body, so the teardown really happens in flight), and, with the fix, that clearing the callback does not deadlock against the HTTP thread holding the buffer. There is no thread sanitizer build, so this test is regression coverage of the path rather than a detector for the race itself; it also passes on the unfixed build.To confirm the window is actually hit, I temporarily logged both accesses in a debug build and ran the same scenario 500 times:
Observed interleavings (temporary instrumentation, not part of the PR)
Per request, in logging order (
drain= HTTP threadreport_drain,clear= JS threadclear_drain_callback):500/500 requests had the HTTP thread reading the callback in the same window as the JS-side clear; in 6 the read landed after the clear (with the fix it observes
None; before it was an unsynchronised read of a field being written).Also ran under the debug build:
fetch-abort-stream-body.test.ts,fetch-http2-client.test.ts,fetch-http3-client.test.ts(their streaming request body tests exercise the newreport_drainassertion on h1, h2 and h3),body.test.ts,fetch-cyclic-reference.test.ts,fetch-keepalive.test.ts.fetch-backpressure.test.ts's fourh3 receive backpressurecases time out locally when the whole file runs, identically on unmodified main (they send no request body, so they do not touch this code); they pass in isolation.A larger follow-up could make the mutex own
bufferandcallbackoutright (Guarded<_>), which would also remove the&mut ThreadSafeStreamBufferboth threads currently hold at once; this PR only closes the race.no test proof · iteration 0 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/web/fetch/fetch-abort-stream-body.test.ts