Skip to content

Fix Windows heap corruption when a socket's data callback closes it and re-enters the event loop - #33018

Open
robobun wants to merge 3 commits into
mainfrom
farm/3c97b97f/fix-windows-nested-tick-uaf
Open

Fix Windows heap corruption when a socket's data callback closes it and re-enters the event loop#33018
robobun wants to merge 3 commits into
mainfrom
farm/3c97b97f/fix-windows-nested-tick-uaf

Conversation

@robobun

@robobun robobun commented Jun 28, 2026

Copy link
Copy Markdown
Collaborator

Symptom

On Windows, bun 1.3.14 crashes with STATUS_HEAP_CORRUPTION (exit 0xC0000374) on the first occurrence of this sequence, all synchronous, inside a Bun.listen data callback:

const server = Bun.listen({
  hostname: "127.0.0.1", port: 0,
  socket: {
    data(sock) {
      sock.terminate(); // us_socket_close, synchronously
      // Any synchronous re-entry of the event loop. This one is
      // waitForPromise inside HTMLRewriter.transform:
      new HTMLRewriter()
        .on("p", { element: () => new Promise(r => setImmediate(r)) })
        .transform("<p></p>");
    },
    close() {}, error() {},
  },
});
const c = await Bun.connect({ hostname: "127.0.0.1", port: server.port, socket: { data() {} } });
c.write("x");

Neither half alone crashes (400 rounds each). Together it crashes on round 1. Not reproducible on Linux or macOS. Any synchronous loop re-entry from inside a socket callback works as the second half (expect().toThrow() in bun:test is another; it is the one the us_internal_loop_post comment names).

Cause

us_internal_loop_post defers us_internal_free_closed_sockets to the outermost loop tick:

/* packages/bun-usockets/src/loop.c:357 */
if (loop->data.tick_depth <= 1) {
    us_internal_free_closed_sockets(loop);
}

because a poll callback may re-enter the loop, and the suspended outer dispatch frame still reads s->flags right after on_data returns (loop.c:615).

tick_depth is only maintained by the epoll/kqueue backend: epoll_kqueue.c, the only writer, sits inside #if defined(LIBUS_USE_EPOLL) || defined(LIBUS_USE_KQUEUE). Windows compiles LIBUS_USE_LIBUV, where tick_depth stays 0 forever, so the guard is always satisfied and every nested uv_run runs the closed-socket sweep.

The exact interleaving, verified against the vendored libuv (oven-sh/libuv@4dcfac47 plus bun's patches)

One uv_run iteration on Windows is (win/core.c:719-755): uv__process_reqs (poll callbacks fire here), prepare, poll, uv__process_reqs, check (runs us_internal_loop_post), endgames (runs handle close callbacks), timers. The check phase runs before the endgame phase.

  1. uv__fast_poll_process_poll_req(A.uv_p, req) clears submitted_events_1, re-arms it (bun's rearm patch does this before the callback), then calls poll_cb(A) -> us_internal_dispatch_ready_poll -> on_data -> the JS data handler.
  2. The JS closes A. us_socket_close -> us_poll_stop -> uv_poll_stop; A.uv_p->data = 0; uv_close(A.uv_p, close_cb_free_poll). Because the re-armed AFD request is still outstanding, uv__poll_close takes the cancel path, not the immediate uv__want_endgame. closesocket(fd) posts an AFD_POLL_LOCAL_CLOSE completion to the IOCP. A is pushed onto loop->data.closed_head.
  3. The JS re-enters the loop. The nested uv_run(UV_RUN_NOWAIT):
    • dequeues A's completion -> uv__fast_poll_process_poll_req(A.uv_p, req) -> tail -> uv__want_endgame(A.uv_p)
    • check: us_internal_loop_post -> tick_depth(0) <= 1 -> us_internal_free_closed_sockets -> us_poll_free(A): uv_is_closing(A.uv_p) is true, so it arms A.uv_p->data = A
    • endgames: uv__poll_endgame(A.uv_p) -> close_cb_free_poll(A.uv_p) -> h->data != 0 -> free(A); free(A.uv_p)
  4. The nested run returns, on_data returns, and the outer dispatch reads A->flags.adopted at loop.c:615. Use after free.

It then gets worse. The outer uv__fast_poll_process_poll_req frame's tail reads the freed A.uv_p, and since UV_HANDLE_CLOSING is still set in the stale bytes and UV_HANDLE_ENDGAME_QUEUED was cleared by the nested uv__process_endgames, it re-queues the freed handle for a second endgame. The second close_cb_free_poll reads h->data out of freed memory (by then the allocator's encoded freelist pointer) and passes it to free(), and double-frees A.uv_p. That is what the allocator reports as heap corruption.

Fix

us_poll_stop, us_create_poll, and the common path of us_poll_free are unchanged from main. Three additions, all of which are no-ops for a non-nested close:

  1. us_loop_run and us_loop_pump bracket uv_run with tick_depth++ / --, the direct mirror of us_loop_run and us_loop_run_bun_tick in epoll_kqueue.c. us_internal_loop_post fires from the uv_check_t registered in us_create_loop, so it runs inside uv_run and now sees the nesting depth. These two functions are the only live entry points to uv_run on the uSockets default loop; I audited every uv_run caller in src/ to confirm it.

  2. close_cb_free_poll marks a handle it ran for before us_poll_free armed it. With the sweep deferred, the nested tick's uv__process_endgames can now run close_cb_free_poll while uv_p->data is still 0. That branch used to silently do nothing, which would leak both allocations: the later us_poll_free sees uv_is_closing() still true (it reports CLOSED as well as CLOSING) and arms a callback that already fired. It now sets h->data = h, and us_poll_free frees both when it sees the mark.

  3. patches/libuv/win-poll-no-reendgame-after-close.patch. The post-poll_cb tail of uv__fast_poll_process_poll_req (and its slow-poll sibling) re-queues an endgame for any CLOSING handle with no outstanding AFD requests. When a nested uv_run already ran that handle's endgame, and so cleared UV_HANDLE_ENDGAME_QUEUED and set UV_HANDLE_CLOSED, the re-queue runs uv__poll_endgame a second time, which it asserts against, and invokes close_cb_free_poll twice. The patch guards the check on !(flags & UV_HANDLE_CLOSED).

Why us_poll_stop must keep its uv_close

An earlier revision of this branch moved the uv_close from us_poll_stop to us_poll_free, and CI caught it: test-http-client-set-timeout.js plus five more node http tests hung on every Windows lane, none of them in test/expectations.txt and none present on main's CI.

The mechanism, from an instrumented build: us_poll_stop's uv_close is load-bearing for uv__loop_alive(). uv_close -> uv__handle_closing -> uv__active_handle_add raises loop->active_handles even for an unref'd handle, and all of bun's socket polls are unref'd (us_poll_start calls uv_unref). That is what makes uv_run execute the next iteration, which is the only thing that runs the uv_check_t -> us_internal_loop_post -> closed-socket sweep. Moving the uv_close into the sweep creates a cycle: a socket closed from the timer phase (which runs after the check phase) drops active_handles to zero, the very next uv_run(UV_RUN_ONCE) returns at while (r != 0) without running a single phase, the sweep never runs again, other still-open sockets' IOCP completions are never dequeued, and the process either spins at 100% CPU (bun's own liveness still says alive) or leaks the sockets. test-http-client-set-timeout.js reproduces it deterministically: an Agent({keepAlive: true, timeout: 50}) whose timeout callback destroys the request, which is exactly a close from the timer phase.

#30028 makes the same move, for a different reason, and its Windows CI (build 49641) has the same test-http-client-* failures. I originally thought the two PRs overlapped; they do not. #30028's symptom is real but needs a different fix; the full analysis is on that PR.

Verification (Windows x64)

Fail-before:

# released bun 1.3.14
> bun repro.mjs both 200
EXIT=-1073740940        (0xC0000374, STATUS_HEAP_CORRUPTION, first round)
> bun repro.mjs close-only 200
SURVIVED mode=close-only rounds=200
> bun repro.mjs pump-only 200
SURVIVED mode=pump-only rounds=200

# this branch's test against a build with packages/ reverted to the merge base
> bun bd test test/js/bun/net/socket.test.ts -t "survives closing a socket and re-entering"
  {
-   "exitCode": 0,
-   "stdout": "SURVIVED 20",
+   "exitCode": 3,
+   "stdout": "",
  }
(fail)   0 pass / 1 fail

Pass-after on this branch:

(pass) survives closing a socket and re-entering the event loop from its own data callback [542ms]
SURVIVED mode=both rounds=400 hits=400 reentries=400

The spawned script counts the setImmediate callbacks, which only run inside the nested autoTick that transform()'s waitForPromise performs, and the test asserts that count is exactly 20, so the re-entrant tick is observed rather than inferred.

No-regression sweep on the same Windows debug build:

# the six node http tests the previous revision hung (plus the deterministic one 3x)
test-http-client-set-timeout.js   x3     all EXITED code=0
test-https-timeout.js                    EXITED code=0
test-http-client-timeout.js              EXITED code=0
test-http-client-timeout-event.js        EXITED code=0
test-http-client-timeout-option.js       EXITED code=0
test-stream-readable-async-iterators.js  EXITED code=0

test/js/bun/net/socket.test.ts            37 pass   4 skip   0 fail
test/js/bun/net/tcp-server.test.ts         8 pass            0 fail
test/js/bun/net/socket-retention.test.ts   3 pass   1 skip   0 fail
test/js/bun/net/socket-dns-error.test.ts   3 pass            0 fail
test/js/bun/udp/udp_socket.test.ts       178 pass            0 fail

Both libuv patches were confirmed present in the built tree (poll.c has the baseline UV_HANDLE_CLOSED assert plus the two new guards).

Note for reviewers

  • eventing/libuv.c is entirely inside #ifdef LIBUS_USE_LIBUV, which only Windows defines, so on Linux and macOS this diff produces a byte-identical binary and the new test passes both with and without it (the epoll/kqueue backend already maintains tick_depth). The Windows runs above are the before/after proof.
  • The new patch file is a traditional unified diff (no diff --git header) on purpose, matching every other file in patches/. git apply runs from vendor/<dep>/, a subdirectory of this repository, and only prefixes a patch's paths with that subdirectory for traditional patches; a git-format patch is treated as toplevel-relative, falls outside the prefix, and is silently skipped with exit 0, which the fetch step cannot detect. There's a note at the patches: array in deps/libuv.ts; fetch-cli.ts::applyPatch is worth hardening separately.

Related: #32233 fixes the sibling re-entrancy hazard on the epoll/kqueue backend, where a nested tick clobbers the outer tick's ready-poll batch. Same bug class, opposite backend.


no test proof · iteration 20 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/js/bun/net/socket.test.ts

@robobun

robobun commented Jun 28, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 6:05 PM PT - Aug 12th, 2026

@robobun, your commit c46dde9 has 3 failures in Build #93542 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 33018

That installs a local version of the PR into your bun-33018 executable, so you can run:

bun-33018 --bun

@github-actions

Copy link
Copy Markdown
Contributor

Found 3 issues this PR may fix:

  1. Windows: Bun 1.3.14 segfaults in long-running localhost HTTP proxy #32585 - Windows segfault in long-running localhost HTTP proxy matches the premature socket freeing pattern in the libuv backend
  2. Crash in uv__queue_insert_tail using Claude or Droid #22444 - Crash in uv__queue_insert_tail on Windows is consistent with corrupted poll handles from missing tick_depth guard
  3. Bun crashes in Claude Code cause Windows BSOD (KERNEL_SECURITY_CHECK_FAILURE) — full system reboots on Windows 11 x64 #27692 - Windows BSOD (KERNEL_SECURITY_CHECK_FAILURE) indicates severe heap corruption, which double-free/use-after-free of socket poll handles would cause

If this is helpful, copy the block below into the PR description to auto-close these issues on merge.

Fixes #32585
Fixes #22444
Fixes #27692

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Adjusts libuv poll teardown and nested loop handling, adds a Windows libuv poll patch and wires it into dependency setup, hardens dependency patch application, and adds a subprocess regression test for socket close during event-loop re-entry.

Changes

libuv poll lifecycle and re-entrancy fix

Layer / File(s) Summary
Poll close/free handling
packages/bun-usockets/src/eventing/libuv.c
close_cb_free_poll now frees the stored allocation when h->data is present and otherwise marks the handle via h->data = h; us_poll_free adds an early return for already-closed polls whose data matches the self-marker and frees both allocations immediately.
tick_depth bracketing
packages/bun-usockets/src/eventing/libuv.c, packages/bun-usockets/src/internal/loop_data.h
us_loop_pump and us_loop_run now increment and decrement loop->data.tick_depth around uv_run calls, and loop_data.h updates the comment to cover the libuv entry points.
Windows poll patch wiring
patches/libuv/win-poll-no-reendgame-after-close.patch, scripts/build/deps/libuv.ts
The new Windows poll patch guards fast and slow endgame re-queue paths after UV_HANDLE_CLOSED, and the libuv dependency config adds that patch to its patch list.
Patch application hardening
scripts/build/fetch-cli.ts
applyPatch() now runs git apply with verbose output and a GIT_CEILING_DIRECTORIES limit rooted at the dependency source directory, and it throws a BuildError when git apply reports a skipped patch.
Socket re-entry regression test
test/js/bun/net/socket.test.ts
The new subprocess test closes a socket inside data() then runs HTMLRewriter to force nested re-entry, repeats 20 times, and asserts the subprocess prints SURVIVED 20.

Suggested reviewers

  • Jarred-Sumner
  • cirospaciari
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes the Windows heap-corruption fix caused by closing a socket and re-entering the event loop.
Description check ✅ Passed The description is detailed, on-topic, and covers the change, root cause, fix, and verification results required by the template.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/bun-usockets/src/eventing/libuv.c`:
- Around line 219-226: In us_create_poll, the newly allocated p and p->uv_p are
used without checking for allocation failure, so add OOM handling before
dereferencing either pointer. Use the existing native OOM path in this
function’s flow to validate the malloc/calloc results and return or abort
consistently before assigning p->uv_p->data, keeping the fix localized to
us_create_poll and its allocation setup.

In `@test/js/bun/net/socket.test.ts`:
- Around line 1655-1657: The nested HTMLRewriter test only checks that it
doesn’t crash, so it doesn’t prove the re-entry path actually ran. Update the
HTMLRewriter::transform() repro in socket.test.ts to make the nested tick
observable by flipping a reentered flag inside the setImmediate callback and
asserting it immediately after transform() returns, and remove the setTimeout(r,
5) fallback so the test stays deterministic. Keep the focus on the nested
HTMLRewriter re-entry case rather than relying on sock.terminate() alone.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: c800994f-8229-44c2-8464-7e07a1f458bc

📥 Commits

Reviewing files that changed from the base of the PR and between e2a69b7 and 83e495d.

📒 Files selected for processing (3)
  • packages/bun-usockets/src/eventing/libuv.c
  • packages/bun-usockets/src/internal/loop_data.h
  • test/js/bun/net/socket.test.ts

Comment thread packages/bun-usockets/src/eventing/libuv.c
Comment thread test/js/bun/net/socket.test.ts
@github-actions

Copy link
Copy Markdown
Contributor

This PR may be a duplicate of:

  1. usockets(win): defer READABLE re-arm past on_open so .end() mid-handshake doesn't hang #30028 - Both PRs fix the same defect: uv_close called from us_poll_stop while inside poll_cb on the libuv/Windows backend, by moving uv_close from us_poll_stop to us_poll_free in libuv.c. PR Fix Windows heap corruption when a socket's data callback closes it and re-enters the event loop #33018 is a strict superset, adding tick_depth tracking on top.

🤖 Generated with Claude Code

@robobun

robobun commented Jun 28, 2026

Copy link
Copy Markdown
Collaborator Author

Good flag. #30028 (@Jarred-Sumner) did not surface in my pre-open duplicate search (I had scoped it to robobun-authored PRs). I've now read it and tested the relationship both ways on a Windows debug build. Summary: #30028 is one of this PR's two changes, this PR is a strict superset, and neither makes the other redundant.

The overlap. Both PRs move uv_close from us_poll_stop to us_poll_free. They arrived there from two independent Windows failures: #30028 from a .end()-mid-handshake hang (uv_close racing a freshly-submitted AFD request inside its own poll_cb), and this PR from the heap corruption in the description (libuv's uv__fast_poll_process_poll_req tail re-queuing a second endgame for a handle a nested tick already endgamed). Two independent diagnoses converging on the same change is strong evidence it is the right one.

Not a duplicate: #30028 alone does not fix the heap corruption. Verified, not just traced. I applied only #30028's libuv.c hunks to current main (they apply clean; git apply --check returns 0) and confirmed the resulting file has zero tick_depth writes and uv_close only in us_poll_free, then built it:

bun bd repro.mjs both 200
REPRO_EXIT=3      (same abort as unfixed main; 0xC0000374 on the release build)

Mechanism: the nested tick's us_poll_free now issues the uv_close itself. By then the handle's AFD request count is already 0 (its AFD_POLL_LOCAL_CLOSE completion was processed earlier in the same nested iteration), so uv__poll_close takes the immediate uv__want_endgame path and uv__process_endgames, which runs right after the check phase in the same iteration, frees the socket out from under the suspended outer dispatch. Moving the uv_close only changes where it happens; the tick_depth bracket is what keeps us_poll_free from running inside a nested tick at all.

The converse holds. #30028's regression test, run verbatim against this branch:

(pass) TLS .end() inside open (mid-handshake) fires close and doesn't hang [26.47ms]

so this PR covers #30028's symptom too.

Remaining delta over #30028's us_poll_free, beyond the tick_depth bracket. us_create_poll zeroes the uv_poll_t (us_calloc) so us_poll_free can detect a never-started poll via type != UV_POLL; #30028's version calls uv_is_closing() on that handle, which reads uninitialized flags on the error paths that free a poll before us_poll_start. And the uv_is_closing tolerate-and-return branch in #30028's us_poll_free arms uv_p->data but never frees, and becomes unreachable once us_poll_free is the only uv_close issuer for polls, so this PR drops it.

@Jarred-Sumner this is your call on how the two should land:

I did not want to step on an open PR without flagging the overlap.

@robobun

robobun commented Jun 28, 2026

Copy link
Copy Markdown
Collaborator Author

Evaluated all three. I'm not adding any Fixes lines: none of the three is provable from its report, and one is definitely not this.

#22444 (crash in uv__queue_insert_tail during spawnProcess, Windows): the signature is consistent with this bug's downstream. uv__queue_insert_tail at uv__handle_init faults when loop->handle_queue is corrupted, and the double-endgame described in this PR corrupts exactly that list: the second uv__handle_close runs uv__queue_remove on the freed poll handle's stale handle_queue links, splicing its former neighbors incorrectly, and the next insert into that list is where a corrupted doubly-linked list faults. So this PR plausibly fixes it, but the issue has no reproduction, I cannot confirm its trigger, and an N-API addon calling the exported uv_run on the main loop directly could produce the same corruption through a path this PR does not bracket. I'd rather leave it open for the reporter than auto-close it on merge.

#32585 (Windows segfault in a long-running HTTP proxy, fault address 0xFFFFFFFFFFFFFFFF): plausibly the same class (heavy socket teardown on a loaded server), but the two bun.report links are not decoded in the issue, so I cannot place the top frame inside a suspended us_internal_dispatch_ready_poll. Not confident enough to auto-close.

#27692 (Windows BSOD, KERNEL_SECURITY_CHECK_FAILURE): not this, and not a Bun bug in the way the issue frames it. A user-mode process cannot corrupt kernel data structures or bugcheck the machine; heap corruption inside bun.exe terminates bun.exe, not Windows. Bugcheck 0x139 with parameter 0xA (FAST_FAIL_CORRUPT_LIST_ENTRY) at a kernel-mode address (0xFFFFEF82...) means a kernel-mode driver corrupted a kernel list. That is a driver or hardware problem on that machine, not something any fix in Bun can reach.

If a maintainer wants #22444 linked on the strength of the signature match, no objection from me, but I would rather not auto-close an issue I cannot prove.

Comment thread packages/bun-usockets/src/eventing/libuv.c
@robobun

robobun commented Jun 28, 2026

Copy link
Copy Markdown
Collaborator Author

Converting to draft: CI on this branch has a Windows regression I introduced, and the verification section in the description is wrong until it's fixed.

Six node-compat tests time out on all three Windows lanes (11 aarch64, 2019 x64, 2019 x64-baseline):

test/js/node/test/parallel/test-https-timeout.js
test/js/node/test/parallel/test-http-client-timeout.js
test/js/node/test/parallel/test-http-client-timeout-event.js
test/js/node/test/parallel/test-http-client-set-timeout.js
test/js/node/test/parallel/test-http-client-timeout-option.js
test/js/node/test/parallel/test-stream-readable-async-iterators.js

None of them is in test/expectations.txt, and none appears in the annotations of the last three main builds (including the one at this branch's merge base). So this is not flake. Five of the six exercise socket.setTimeout() followed by a destroy from the timeout handler, which is exactly the path the us_poll_stop change touches. The regression sweep in the description did not cover that class.

I'm reproducing test-https-timeout.js on a Windows debug build now. I will not un-draft this until the root cause is understood and those tests pass.

@Jarred-Sumner heads up since I pinged you above: please hold off on the #30028 / #33018 question until this is resolved.

@robobun
robobun marked this pull request as draft June 28, 2026 20:45
@robobun
robobun force-pushed the farm/3c97b97f/fix-windows-nested-tick-uaf branch 2 times, most recently from 84b2342 to eb014e8 Compare June 28, 2026 21:29
@robobun

robobun commented Jun 28, 2026

Copy link
Copy Markdown
Collaborator Author

Un-drafting. CI was right and this PR's first revision was wrong. The correction changes the diff substantially and also explains #30028's two-month-old CI failures, so here is the complete story.

What CI caught. Six node http tests (test-http-client-set-timeout.js, test-http-client-timeout.js, test-http-client-timeout-event.js, test-http-client-timeout-option.js, test-https-timeout.js, test-stream-readable-async-iterators.js) timed out on all three Windows lanes. None is in test/expectations.txt and none appears in the last several main builds' annotations, so it was a regression I introduced, not flake.

Root cause of the hang. The first revision moved uv_close from us_poll_stop to us_poll_free. I reproduced test-http-client-set-timeout.js hanging deterministically (100% CPU, a spin) on a Windows debug build and instrumented libuv.c: after the two sockets close, uv_run never runs another iteration. us_poll_stop's uv_close is load-bearing for uv__loop_alive(): uv_close -> uv__handle_closing -> uv__active_handle_add raises loop->active_handles even for an unref'd handle, and all of bun's socket polls are unref'd. That is what makes uv_run execute the next iteration, which is the only thing that runs the check phase, which is the only thing that runs us_internal_free_closed_sockets, which after the move was the only thing that called uv_close. A cycle. The test's Agent({keepAlive: true, timeout: 50}) destroys the request from the timeout callback, which fires in uv__run_timers, the phase after check, so active_handles hit zero and the very next uv_run(UV_RUN_ONCE) returned at while (r != 0) without executing a single phase. bun's JS-side liveness still said alive, so it spun on uv_run(UV_RUN_NOWAIT) returning 0 forever.

The fix now. us_poll_stop, us_create_poll, and the common path of us_poll_free are byte-identical to main, which makes that hang structurally unreachable rather than just untested. The heap-corruption fix comes from the tick_depth bracketing (unchanged), a third rendezvous state between close_cb_free_poll and us_poll_free for the deferred-sweep case, and a one-line libuv patch (!(flags & UV_HANDLE_CLOSED) on the post-poll_cb endgame re-queue, which uv__poll_endgame already asserts). The description has been rewritten for this diff; its verification section now includes all six node tests exiting (plus the deterministic one three times), the repro both ways, and a 229-test socket sweep.

A correction to my earlier comments on this PR. I wrote that this PR is a strict superset of #30028 and that #30028's regression test passes on this branch. Both were true of the first revision and are no longer true: the uv_close move is gone because it is the hang. The two PRs now do not overlap at all, and this one does not fix #30028's .end()-mid-handshake hang. #30028's own Windows CI (build 49641) shows the same class of test-http-client-* failures, which is very likely why it has been open since May 1; I've posted the root cause there.

(The earlier review note about us_malloc/us_free pairing is now moot: us_create_poll is no longer part of this diff.)

One build-system finding, for whoever owns scripts/build. I lost an afternoon to this and it will bite the next person. git apply, run from vendor/<dep>/ (a subdirectory of this repository), only prefixes a patch's paths with that subdirectory for traditional unified diffs. A patch with a diff --git a/X b/X header is treated as toplevel-relative, falls outside the prefix, and is silently skipped with exit 0, so the fetch writes the .ref stamp over an unpatched tree with no error. All 18 existing files in patches/ happen to be traditional format, which is why nobody has hit this. Mine is now traditional too and there's a note at the patches: array in deps/libuv.ts, but fetch-cli.ts::applyPatch should probably fail on git's Skipped patch output (or neutralize the prefix detection with GIT_CEILING_DIRECTORIES). Happy to send that as a separate PR if wanted.

@robobun
robobun marked this pull request as ready for review June 28, 2026 21:38

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/build/deps/libuv.ts`:
- Around line 51-64: The patch application flow in applyPatch is allowing a
skipped git apply to look successful, which can leave vendor/libuv unpatched
while still writing the fetch stamp. Update applyPatch in
scripts/build/fetch-cli.ts to detect when git apply skips or applies nothing and
treat that as a hard failure before the .ref file is written. Use the
applyPatch/fetch stamping path to locate the fix and ensure malformed patch
formats are rejected instead of being marked done.

In `@test/js/bun/net/socket.test.ts`:
- Around line 1685-1687: The subprocess regression assertion in socket.test.ts
is dropping native crash diagnostics by discarding stderr; update the combined
expectation around the Promise.all result so stderr remains part of the asserted
object while staying unconstrained (for example using an any-string matcher),
and remove the separate stderr discard so failures still surface diagnostics
from proc.stderr.text().
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 5c5e6eab-0499-46fe-a4b8-99a5021d2b5f

📥 Commits

Reviewing files that changed from the base of the PR and between 38f8e6d and eb014e8.

📒 Files selected for processing (5)
  • packages/bun-usockets/src/eventing/libuv.c
  • packages/bun-usockets/src/internal/loop_data.h
  • patches/libuv/win-poll-no-reendgame-after-close.patch
  • scripts/build/deps/libuv.ts
  • test/js/bun/net/socket.test.ts

Comment thread scripts/build/deps/libuv.ts Outdated
Comment thread test/js/bun/net/socket.test.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/build/fetch-cli.ts`:
- Around line 245-254: Condense the explanatory block comment in fetch-cli.ts to
fit the repository’s 3-line limit while preserving the core rationale. Keep only
the essential points about GIT_CEILING_DIRECTORIES preventing skipped git-format
patches from being treated as successfully applied and the fail-loud backstop
using the skipped-patch check. Use the existing comment block near the
patch-application logic and shorten the wording without changing behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 90936f46-7f94-4661-84c0-3cfa451f43cb

📥 Commits

Reviewing files that changed from the base of the PR and between eb014e8 and 19c2aa4.

📒 Files selected for processing (3)
  • scripts/build/deps/libuv.ts
  • scripts/build/fetch-cli.ts
  • test/js/bun/net/socket.test.ts
💤 Files with no reviewable changes (1)
  • scripts/build/deps/libuv.ts

Comment thread scripts/build/fetch-cli.ts Outdated
@robobun

robobun commented Jun 28, 2026

Copy link
Copy Markdown
Collaborator Author

CI status, for whoever reviews this: the red check is infrastructure, not the diff.

Across the last two builds on this branch (66519 and 66525), the only hard-failed job each time is darwin 26 aarch64 - test-bun, and in both cases it is not a test failure. The runner timed out downloading the build artifact and refused to start:

Error: buildkite-agent artifact download timed out after 120s for step 'darwin-aarch64-build-bun'.
Refusing to continue with a partial download (would silently fall back to the wrong binary).

It never ran a single test either time, and both failures were on the same agent (darwin-aarch64-26-5-1-1) within about 35 minutes, so that host's connection to the artifact store looks unhealthy. My Buildkite token is read-scoped (the single-job retry API returns 403), so I can't re-run just that job. A maintainer with Buildkite access can retry it in one click; I'd rather not push an empty commit to re-roll the whole build onto an agent that has already failed twice.

For the lanes this PR can actually affect, both builds are clean: no failures in, and no annotation mentions of, socket.test.ts, libuv, fetch-cli, or applyPatch. The only other annotations are the usual auto-retried Windows install/init flaky tests. And specifically, there are no test-http-client-*timeout* hangs on any Windows lane, which is the exact CI signature that correctly caught this PR's first, wrong revision. The verification for this revision is in the description.

@robobun

robobun commented Jun 28, 2026

Copy link
Copy Markdown
Collaborator Author

One correction to my comment above. I wrote that no annotation mentions socket.test.ts. That was wrong: build 66525 has one, and since that is the file my regression test lives in, it deserves an explanation rather than a hand-wave.

It is in the flaky (retried and passed) bucket, not a failure, and it is not my test. socket > should not call open if the connection had an error, a pre-existing IPv6 ::1 connect test at socket.test.ts:484, timed out once after 90s on debian 13 x64. The TCPSocket it left behind then tipped the should not leak memory object count (line 789) in the same run from 2 to 3. The whole file passed on retry.

Two reasons it cannot be this PR:

  • It happened on a Linux lane. eventing/libuv.c is entirely inside #ifdef LIBUS_USE_LIBUV, which only Windows defines, so on Linux this diff compiles to exactly what main does.
  • My test runs its sockets in a spawned subprocess, so it contributes zero TCPSocket objects to the runner's heap and cannot affect that count.

The hard-failure picture is unchanged from my comment above: 280 jobs passed, and the only 2 failures are both darwin 26 aarch64 - test-bun shards on the same agent (darwin-aarch64-26.5.1-1), both the artifact-download timeout, zero tests run. That agent is now 4 for 4 across builds 66519 and 66525.

@robobun

robobun commented Jun 28, 2026

Copy link
Copy Markdown
Collaborator Author

The same failure is back on build 66564 (the push for the review nit above), so here is the consolidated CI picture across this branch's last three builds, for whoever can unblock it.

Every lane that can exercise this change is green. On 66525, all 24 Windows test-bun shards passed (8 each of 2019 x64, 2019 x64-baseline, and 11 aarch64), along with every Linux and ASan lane: 280 passed jobs in total. eventing/libuv.c is entirely inside #ifdef LIBUS_USE_LIBUV, which only Windows defines, so a macOS lane could not exercise any of this change even if its agent were healthy.

The only failure is one Buildkite host. Across 66519, 66525, and now 66564, every hard-failed job is a darwin 26 aarch64 - test-bun shard, and all four of them were dispatched to the same agent, darwin-aarch64-26.5.1-1. Every one died with the same error before running a single test:

Error: buildkite-agent artifact download timed out after 120s for step 'darwin-aarch64-build-bun'.
Refusing to continue with a partial download (would silently fall back to the wrong binary).

That host cannot reach the artifact store. Re-pushing just re-dispatches onto it (66564 hit it again within minutes of my last commit), so I am going to stop pushing. My Buildkite token is read-scoped (the single-job retry endpoint returns 403), so I cannot retry it from here either.

The red bun-plugin-svelte GitHub check is not this PR. It belongs to the Packages CI workflow, which is currently failing on main itself (and on every open PR) on a bundler output-format assertion in packages/bun-plugin-svelte/test/index.test.ts. A fix for that is already in flight on another branch.

What this needs: someone with Buildkite access to retry this job, or to read the darwin failure as the infrastructure noise it is. The Windows fail-before and pass-after for the actual change are in the description.

@robobun

robobun commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator Author

Build 66564 finished: 284 passed, 2 failed. Both failures are noise, but one of them disproves a claim in my previous comment, so I am correcting it here and explaining both.

I was wrong that a re-push can only land on the sick agent. There are two darwin 26 aarch64 agents, not one. In 66564 the two darwin 26 aarch64 - test-bun shards were split between them: the shard on darwin-aarch64-26.4-1 ran the full suite against this binary and passed; the shard on darwin-aarch64-26.5.1-1 died on the same artifact-download timeout as every time before, zero tests run. I wrote "I am going to stop pushing" while 66564 still had 230 jobs in flight, generalizing from the first four dispatches all happening to hit the same host. The fifth did not. Bad claim, withdrawn.

The other failure is darwin 14 aarch64 - test-bun: two HTTP file-serving test timeouts on agent darwin-aarch64-14.5-1. test/js/bun/http/fetch-file-upload.test.ts timed out in uploads roundtrip with sendfile(), which allocates 128 MB, SHA-256 hashes it, writes it to disk, then PUTs it through Bun.serve (which hashes it again), all under a 10 second budget, and is already marked test.todoIf(isBroken && isWindows). test/js/bun/http/bun-serve-file.test.ts hit the per-file timeout in the same run. The sibling darwin 14 aarch64 shard on a different agent passed, as did both darwin 14 x64 shards. These are load timeouts.

They cannot be caused by this diff, and that is a preprocessor fact rather than a judgment call. libusockets.h selects the eventing backend:

#if defined(_WIN32)
#define LIBUS_USE_LIBUV
#elif defined(__APPLE__) || defined(__FreeBSD__)
#define LIBUS_USE_KQUEUE

eventing/libuv.c, the only native file this PR changes, is wrapped file-scope in #ifdef LIBUS_USE_LIBUV, so on macOS it is an empty translation unit before and after. Of the rest: the loop_data.h change is a comment, the libuv patch targets src/win/poll.c in a Windows-only dependency, and fetch-cli.ts only runs at build time (the darwin build job passed). The one darwin-visible change, the new test in socket.test.ts, ran 3rd of 2485 on that same darwin 14 agent and passed.

Since a re-push is not futile after all, I have pushed a single empty ci: retrigger: the darwin 14 timeouts are a re-roll away, and darwin 26 has a healthy agent to land on. That is the only retrigger I will push. If it comes back red on the same infrastructure, the analysis here and in the description is what I have.

@robobun

robobun commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator Author

The retrigger (build 66646) went straight back to the sick agent. darwin-aarch64-26.5.1-1 took one of the two darwin 26 aarch64 - test-bun shards again and failed it with the same artifact-download timeout within minutes of dispatch: its 5th identical failure across 4 consecutive builds on this branch (66519, 66525, 66564, 66646), zero tests run each time. The other shard went to a healthy host.

That is structural, not a run of bad luck. As long as that agent is online in the darwin 26 aarch64 pool it takes one of the two shards of every build and fails it before running anything, so the buildkite/bun check on this branch cannot go green no matter how many times I push. I said the last retrigger was the only one I would push, so I am done pushing. The full record is in the comments above and the verification is in the description.

What this needs, in order of preference:

  1. Someone with Buildkite access retries the failed job, or takes darwin-aarch64-26.5.1-1 out of rotation. I can do neither: my token is read-scoped.
  2. Or merge on the evidence that already exists. darwin 26 aarch64 - test-bun did run the full suite against this binary and pass (the shard that landed on a healthy agent in build 66564), and macOS cannot exercise this change in the first place: eventing/libuv.c is an empty translation unit outside #ifdef LIBUS_USE_LIBUV, which only Windows defines.

@robobun

robobun commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator Author

Build 66646 settled with a second failure beyond the darwin agent, and this one is on a Windows lane, so it deserves real scrutiny rather than a hand-wave: this is a Windows PR.

The job is windows 11 aarch64 - test-bun: test/cli/install/bun-install.test.ts, 182 pass / 2 fail. The two failing tests are should handle aliased dependency with existing lockfile (one duplicated tarball request in the asserted registry URL list) and should use updated life-cycle scripts in root during re-installation (EBADF: bad file descriptor reading back bar/bar.txt with Bun.file().text()). The EBADF is the one that would be alarming if it were new, because a double-closed-then-recycled fd is exactly how an unrelated file read gets EBADF. It is not new. Three independent lines of evidence:

  1. It is a known repo-wide flake, not something this branch introduced. The identical failure, the same EBADF on the same expect(await file(join(ctx.package_dir, "bar", "bar.txt")).text()).toBe("bar!") in the same test, is in the flaky (retried and passed) annotation of build 66584 on farm/1740190b/module-load-override, an unrelated branch built from a different base commit. bun-install.test.ts appears in the flaky bucket of 8 of the last ~28 finished builds across 8 different branches, none of them this one.
  2. It flaked and recovered on the other two Windows lanes of this very build. In 66646 itself, bun-install.test.ts also failed on windows 2019 x64 and on windows 2019 x64-baseline, and the automatic retry passed both; only the 11 aarch64 run was not rescued. This same file, on this same 11 aarch64 lane, with this same binary (the retrigger commit is empty), ran clean in builds 66525 and 66564.
  3. The diff contains no file-descriptor operation. close_cb_free_poll stores a marker pointer, us_poll_free calls free() on two heap blocks, and the tick_depth change is an integer increment around uv_run. Nothing in it opens, closes, or dups an fd; the socket fd close lives in bsd_close_socket, untouched. A lifecycle bug in what I changed would surface as the STATUS_HEAP_CORRUPTION this PR's own test detects (and the x64-asan lanes are green), not as a clean EBADF on a regular-file read.

So 66646's two failures are one persistently sick darwin build agent and one known flaky Windows install test that the retry happened not to save on one of three lanes. My one retrigger is spent and I am not pushing again. Everything in my previous comment stands: retry the darwin job, or merge on the evidence.

@robobun
robobun force-pushed the farm/3c97b97f/fix-windows-nested-tick-uaf branch from fb8e250 to c5c4804 Compare July 9, 2026 00:42
@robobun

robobun commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased onto ab6eb2d0d8 (current main) to clear a merge conflict, and dropped the empty retrigger commit while I was at it. The force-push changes nothing about the fix.

The conflict was a pure append-vs-append in test/js/bun/net/socket.test.ts: #31859 added two tests at the end of the file and this PR adds one at the same spot, so git's three-way merge found false common lines in the similar spawn boilerplate. Resolved by taking main's file verbatim and re-appending this PR's test after it. The diff of the other five files is byte-identical to before the rebase (verified via diff-of-diffs against the pre-rebase branch).

Since #31859 rewrote the Rust socket dispatch layer, I re-checked that the test's mechanism is intact: sock.terminate() still reaches us_socket_close synchronously (socket_body.rsclose_and_detachus_socket_t::closec::us_socket_close), and the rebuilt subprocess still prints SURVIVED 20, i.e. all 20 setImmediates fired inside the nested tick. That is the exact precondition the Windows crash needs, so the fail-before proof in the description still holds.

Full-file run on the rebased branch: 34 pass / 2 skip / 9 fail, versus 33 pass / 2 skip / 9 fail for the same command on unmodified main. The 9 failures are identical and are outbound-DNS tests (getaddrinfo ENOTIMP www.example.com) that fail in my local environment on both.

@robobun

robobun commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator Author

Post-rebase CI picture (build 70743, finished: 285 passed, 1 failed; the two darwin 14 aarch64 shards that were pending both passed). This supersedes my earlier CI comments; both things that were blocking have moved:

The darwin agent blocker is gone. Both darwin 26 aarch64 - test-bun shards ran on healthy agents (darwin-aarch64-26.4-1 and 66790-tart-26) and passed. darwin-aarch64-26.5.1-1, the host that failed 5 times in a row with the artifact-download timeout across builds 66519/66525/66564/66646, is not in this build.

The one failure is the current repo-wide Windows flake, not this PR. test/napi/napi.test.ts on windows 2019 x64: napi > napi_wrap > has the right lifetime exhausts its 100-iteration gcUntil loop. Of the last 30 finished builds on other branches, the same test is in the flaky bucket of 21 of them and is a hard error in builds 70668, 70655, 70623, and 70611 (a formdata fix, a net-backlog fix, a signal-name fix, and a transform-stream fix respectively). On the one shard where it failed here, socket.test.ts ran as file 3/596 and passed 41/0.

This PR's own surface is clean. 23 of 24 Windows test-bun shards passed (the 24th is the napi shard above), and no annotation anywhere in the build mentions socket.test.ts, libuv, fetch-cli, or tick_depth. Specifically, no test-http-client-*timeout* hangs on any Windows lane, which is the CI signature that correctly caught this PR's first, wrong revision.

I have already pushed this conversation's one retrigger (on the pre-rebase builds), so I will not push again for the napi flake. Anyone with Buildkite access can retry the one job; I cannot (read-scoped token).

@robobun
robobun force-pushed the farm/3c97b97f/fix-windows-nested-tick-uaf branch from c5c4804 to 9d58b55 Compare July 10, 2026 13:05
@robobun

robobun commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased again for the same append-vs-append conflict in socket.test.ts (resolved the same way as above). Diff is otherwise byte-identical. Of the new commits, #33359 (the us_timer_t → bun timer-heap change) touches loop_data.h and the epoll/kqueue backend; it auto-merged cleanly and does not touch libuv.c, us_internal_loop_post, or the tick_depth guard in loop.c, all of which I re-verified are intact. Full socket.test.ts delta on the rebased build: main 55 pass / 9 fail / 66 tests → this branch 56 pass / 9 fail / 67 tests (the 9 are outbound-DNS environment failures, identical on both).

@robobun

robobun commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

Build 71517 finished: 268 passed, 18 failed. All 18 are accounted for by three tests that are broken on main and present on other branches; none of the three is related to this PR, and none of them mentions socket.test.ts, libuv, or any file this PR touches.

Test Lanes here On other branches (last ~25 finished builds, excluding this one)
test/js/web/fetch/fetch-gzip.test.tsfollowed redirect with a malformed chunked body rejects fetch() 16 (every Linux, ASan, Alpine, and Windows lane + darwin 26/14) Hard error in 18 of them: 71518, 71514, 71513, 71512, 71510, 71508, 71505, 71495, 71493, 71491, 71490, 71489, 71488, 71485, 71478, 71477, 71454, 71453. Almost certainly #33613 (the redirect-on-head change); a fix is already in flight on farm/34736d08/fix-fetch-gzip-redirect-test (71515, where this test passes).
test/js/bun/http/proxy-stress-concurrent.test.tshttps-proxy → https-origin mode=concurrent-32 ×1200 1 (darwin 14 x64) 71518, 71515, 71510, 71495, 71490, 71488
test/js/third_party/grpc-js/test-tonic.test.ts (150s timeout) 1 (darwin 14 aarch64) 71513, 71491, 71453

16 + 1 + 1 = the 18 failed shards. The other-branch hits are a TLS ALPN change, an mTLS close-notify fix, a blob deserialize fix, an ASan overflow sweep, a worker-threads ELU change, an RSA JWK validation fix, and so on: none related to each other, to uSockets, or to this PR.

I am not going to fold an HTTP-client redirect fix or a darwin TLS-proxy stress-test fix into a uSockets libuv.c change; those belong in their own PRs (and one of them is already open). For this PR's own surface, the picture is the same as the previous build: no socket.test.ts failures or flaky mentions on any lane, and no test-http-client-*timeout* hangs on any Windows lane.

@robobun
robobun force-pushed the farm/3c97b97f/fix-windows-nested-tick-uaf branch from 9d58b55 to 2662a98 Compare July 14, 2026 06:29
@robobun

robobun commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased onto main for a conflict in packages/bun-usockets/src/eventing/libuv.c itself this time. #34009 added a Bun__JSC_onBeforeWait park-hook call in us_loop_run, at the same line where this PR inserts the tick_depth bracket. Resolved by placing tick_depth++ before the hook and tick_depth-- after uv_run, so the hook sits inside the bracket; that matches epoll_kqueue.c::us_loop_run_bun_tick, where tick_depth++ is at line 363 and Bun__JSC_onBeforeWait at line 392. close_cb_free_poll, us_poll_free, and us_loop_pump are unaffected by #34009 and are unchanged from before. Rebuilt; the test still prints SURVIVED 20.

This rebase also picks up #33916, the fix for the fetch-gzip.test.ts main regression that accounted for 16 of the 18 failures in 71517.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found, but this touches memory-safety-critical event-loop lifecycle code (uSockets libuv backend + a vendored libuv patch), so I'm deferring to a human reviewer rather than auto-approving.

What was reviewed:

  • The tick_depth bracketing in us_loop_run/us_loop_pump mirrors epoll_kqueue.c and sits correctly around the new Bun__JSC_onBeforeWait hook from #34009.
  • The close_cb_free_pollus_poll_free self-marker handshake (h->data = h) — checked that the sentinel cannot collide with the normal-path values (0 from us_poll_stop, or p from us_poll_free's arm), and that both allocations are freed exactly once on each ordering.
  • The libuv win/poll.c patch guard matches what uv__poll_endgame already asserts, and covers both fast/slow siblings.
  • applyPatch hardening: GIT_CEILING_DIRECTORIES + -v skip detection is additive; existing traditional-format patches are unaffected.
Extended reasoning...

Overview

This PR fixes a Windows-only heap corruption (UAF + double-free) that occurs when a socket's data callback synchronously closes the socket and then re-enters the event loop. It touches six files: the uSockets libuv eventing backend (libuv.c), a comment in loop_data.h, a new libuv patch file (win/poll.c), the libuv dep config, a hardening of fetch-cli.ts::applyPatch, and a new subprocess regression test.

The core fix has three cooperating pieces: (1) tick_depth bracketing around uv_run so the existing us_internal_loop_post guard actually observes nesting on Windows, (2) a three-state rendezvous between close_cb_free_poll and us_poll_free for the case where the deferred sweep means libuv's close callback fires before the sweep runs, and (3) a one-line guard in libuv's uv__fast_poll_process_poll_req tail so a handle whose endgame already ran inside a nested uv_run isn't re-queued.

Security risks

None identified. This is internal event-loop lifecycle management with no user-controlled input reaching the changed paths. The applyPatch change is build-time only and strictly tightens behavior (fails on skip instead of silently succeeding).

Level of scrutiny

High. This is native memory-lifecycle code in the Windows event loop backend, where a mistake manifests as heap corruption or hangs on every socket close. The first revision of this PR was itself wrong (moved uv_close from us_poll_stop to us_poll_free) and hung six Node http tests on all Windows CI lanes; the author caught it, root-caused it, and reverted to a minimal approach — but that history underscores how subtle this area is. The fix spans three layers (uSockets, libuv internals via patch, and Bun's re-entrant loop semantics) whose ordering invariants are not obvious from any single file.

The h->data = h self-marker is a novel sentinel handshake across libuv's async close machinery. I traced the three possible uv_p->data states at each callback ordering and believe it's correct, but this is exactly the kind of invariant a maintainer familiar with uSockets' Windows poll lifecycle (cirospaciari or Jarred-Sumner, per suggested reviewers) should confirm.

Other factors

  • The PR description is exceptionally thorough: fail-before/pass-after on Windows, a 229-test no-regression sweep, and an explicit interleaving trace against the vendored libuv source.
  • The bug-hunting system found no issues; all CodeRabbit comments are resolved; my earlier allocator-idiom nit is moot (that code was reverted).
  • There is an acknowledged relationship with open PR #30028 (same file, different bug, and #30028's approach shares the hang mechanism this PR's first revision hit) — the author flagged this for Jarred-Sumner but there's been no maintainer response yet. That coordination is another reason a human should look.
  • The vendored libuv patch is explicitly noted as not-upstreamable (nested uv_run is outside libuv's contract), which is a maintenance-burden decision worth a maintainer's sign-off.

Given the criticality of the code path, the vendored-dependency patch, and the open coordination question with #30028, this should not be auto-approved.

@robobun

robobun commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator Author

Build 72796 finished: 283 passed, 3 failed; all four darwin aarch64 shards that were pending passed. The two failures beyond binary-size are both main regressions I inherited by rebasing onto the current tip:

  • binary-size flags bun-darwin-aarch64 +933.8 KB and bun-darwin-x64 +882.4 KB against the canary baseline (#72668). The same check is a hard error on main's own build (72807) and on 72822, 72820, 72798, each with byte-identical darwin deltas. This PR contributes 0 bytes to a darwin binary: libuv.c is an empty translation unit inside #ifdef LIBUS_USE_LIBUV, and the rest is build-time or test.
  • test/js/node/test/parallel/test-net-connect-memleak.js (assert.strictEqual(collected, true)false !== true) on alpine 3.23 x64 and x64-baseline: a GC-timing assertion. The same test is a hard error on 72823, 72819, 72816, 72808. Linux musl, where libuv.c is an empty TU.

The fetch-gzip 16-lane failure from the previous build is gone, as predicted (this rebase includes #33916). No annotation mentions socket.test.ts or any file this PR touches. I will stop posting per-build comments here; every red so far is a main regression with linked evidence, and I am out of retriggers. If something actually implicates this PR's code I will say so.

@robobun

robobun commented Aug 9, 2026

Copy link
Copy Markdown
Collaborator Author

Independent confirmation of this diagnosis from a different symptom, plus a rebase onto current main.

test/bake/deinitialization.test.ts went red on the Windows 2019 x64 lane of build 90708 with panic(main thread): Segmentation fault at address 0x0. Symbolizing that build's trace against its PDB lands in us_internal_socket_after_resolve (context.c, group->loop->uv_loop->active_handles-- on a freed us_connecting_socket_t), reached from us_internal_loop_post's DNS drain inside uv__check_invoke. Root-causing it from scratch arrived at exactly the three problems this PR fixes, with the double uv__want_endgame on a CLOSED poll handle observed directly in instrumented logs:

process_req h=E00 flags=1 sub1=5 slot=1      <- cancel completion, nested run
want_endgame h=E00 flags=1                   <- endgame #1
poll_endgame h=E00 flags=1                   -> CLOSED, close_cb(data=0)
cb-end h=E00 flags=3                         <- outer poll_cb frame unwinds
want_endgame h=E00 flags=3                   <- endgame #2 on a CLOSED handle
poll_endgame h=E00 flags=3                   -> assert (debug) / double free (release)

Repro numbers on a Windows Server 2019 x64 box, running that bake test's child fixture in a loop:

  • canary bun-profile.exe from build 90708 (unfixed): 10/10 runs failed, 2 as the CI segfault (identical bun.report frames), 8 as a 100%-CPU event-loop hang
  • unfixed debug build of current main: 15/15 failed
  • with this fix applied: 20/20 passed, and test/js/bun/websocket, test/js/web/websocket, test/js/bun/http/serve.test.ts, test/js/bun/net/socket.test.ts, and test/js/web/fetch/fetch.test.ts pass

This PR no longer applies to main (free -> us_free after the Rust rewrite, and the libuv dep now carries a second patch, win-poll-abort-with-disconnect.patch, so the patch list and hunk contexts moved). Branch farm/9d3febcd/win-uv-poll-double-free (diff) carries the same three changes rebased and verified as above, plus a connect-burst regression test. Note the fail-before for all three changes is only observable on Windows; the code is not compiled on the POSIX backends.

Overlap note: #37105 (opened yesterday) contains the close_cb_free_poll / us_poll_free marker handshake from this PR's third hunk, independently rediscovered, plus a live-poll-count leak assertion. The tick_depth and endgame-guard halves exist only here and in the rebase branch; without them the crash above still reproduces.

@robobun

robobun commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author

A third independent sighting of the same bug, with a trigger shape that differs from the two already described here, for whoever reviews this.

Symptom. test/js/sql/postgres-listen-notify.test.ts on the Windows 2019 x64 lane of #32089: the release build either hangs (the per-test timeout never fires) or segfaults at address 0x0; a Windows Server 2019 debug build crashes deterministically with Segmentation fault at address 0xFFFFFFFFFFFFFFFF (the freed socket is filled with the allocator's poison, so flags.adopted reads as set and prev is followed). Symbolized against the debug PDB:

us_internal_socket_follow_adopted   packages/bun-usockets/src/internal/internal.h:355
us_internal_dispatch_ready_poll     packages/bun-usockets/src/loop.c:721          (the follow_adopted right after on_data returns)
poll_cb                             packages/bun-usockets/src/eventing/libuv.c:165
uv__fast_poll_process_poll_req      vendor/libuv/src/win/poll.c:233
uv__process_reqs / uv_run           vendor/libuv/src/win/core.c
us_loop_run                         packages/bun-usockets/src/eventing/libuv.c:413
WindowsLoop::tick_with_timeout      src/uws_sys/Loop.rs:462

Trigger shape. Unlike the terminate()-in-data repro in the description and the connect burst in the rebased branch, nothing here closes the socket synchronously from its own callback. The postgres connection's on_data drains microtasks, so a test body resumes after an await inside socket A's data dispatch; it then destroys the peer's end of A and calls await expect(p).rejects.toThrow(), where p only settles once A's close has been processed. process_promise in src/runtime/test_runner/expect.rs goes through wait_for_promise into us_loop_run / us_loop_pump, and the nested uv_run is the one that dispatches A's close and, because tick_depth is still 0 on this backend (loop.c:448), frees A in its check_cb. The outer dispatch frame then unwinds into loop.c:721. Rewriting the same test to await p.then(...) instead of expect().rejects makes it pass (that is how #32089 worked around it), which confirms the nested run is the mechanism; any native handler that drains microtasks inside on_data (postgres, mysql) can hit it.

Read against farm/9d3febcd/win-uv-poll-double-free: the tick_depth bracket covers this shape as well. With the close processed in the nested run, the nested loop_post sees depth 2 and skips the sweep, so A is still allocated when the outer frame reads it at loop.c:721 and when uv__fast_poll_process_poll_req's tail reads A.uv_p; the outer tick's loop_post frees both through the close_cb_free_poll marker, and the endgame-requeue guard keeps the outer tail from running the endgame a second time when the nested run already ran it. So this needs nothing beyond what is already on that branch; noting it here so the reviewer has the full set of symptoms this one change accounts for. Not opening a separate PR.

@alii

alii commented Aug 12, 2026

Copy link
Copy Markdown
Member

@robobun rebase conflicts

robobun and others added 3 commits August 12, 2026 19:56
us_internal_loop_post defers us_internal_free_closed_sockets to the
outermost loop tick via loop->data.tick_depth, because a nested tick (a
poll callback re-entering the loop, e.g. waitForPromise) must not free a
socket the suspended outer dispatch frame still reads after its callback
returns. That counter was only maintained by the epoll/kqueue backend; on
libuv (Windows) it stayed 0 forever, so every nested uv_run ran the
sweep. The result is a deterministic heap corruption
(STATUS_HEAP_CORRUPTION, 0xC0000374 on bun 1.3.14) when a socket's data
callback closes it and then synchronously re-enters the event loop.

Three pieces, all of which are no-ops for a non-nested close:

- eventing/libuv.c: us_loop_run and us_loop_pump bracket uv_run with
  tick_depth++ and --, the direct mirror of us_loop_run and
  us_loop_run_bun_tick in epoll_kqueue.c. us_internal_loop_post runs
  from the uv_check_t registered in us_create_loop, so it fires inside
  uv_run and now sees the nesting depth.

- eventing/libuv.c: deferring the sweep means the nested tick's
  uv__process_endgames can now run close_cb_free_poll before us_poll_free
  has re-pointed uv_p->data at the us_poll_t. That branch used to
  silently do nothing, which would leak both allocations: the later
  us_poll_free sees uv_is_closing() still true (uv_is_closing reports
  CLOSED as well as CLOSING) and arms a callback that already fired.
  close_cb_free_poll now marks the handle (h->data = h) and us_poll_free
  frees both when it sees the mark.

- patches/libuv/win-poll-no-reendgame-after-close.patch: the
  post-poll_cb tail of uv__fast_poll_process_poll_req (and its slow-poll
  sibling) re-queues an endgame for any CLOSING handle with no
  outstanding AFD requests. When a nested uv_run already ran that
  handle's endgame, and so cleared UV_HANDLE_ENDGAME_QUEUED and set
  UV_HANDLE_CLOSED, the re-queue runs uv__poll_endgame a second time,
  which it asserts against, and invokes close_cb_free_poll twice. Guard
  the check on !(flags & UV_HANDLE_CLOSED).

us_poll_stop, us_create_poll, and the common path of us_poll_free are
unchanged from main. In particular, us_poll_stop keeps issuing the
uv_close: that uv_close is what keeps uv__loop_alive() true for a closed
socket (uv__handle_closing calls uv__active_handle_add even for an
unref'd handle), which is what makes uv_run execute the next iteration,
which is the only thing that runs the sweep. An earlier revision of this
branch moved the uv_close into us_poll_free; that cycle hung six node
http tests on every Windows CI lane, because a socket closed from the
timer phase (after the check phase) dropped active_handles to zero and
the very next uv_run returned without running a single phase. PR 30028
makes the same move for a different reason and its CI shows the same
failures.

Repro: a Bun.listen data handler that calls sock.terminate() and then
synchronously re-enters the loop (an HTMLRewriter element handler that
returns a pending promise) crashes bun 1.3.14 on Windows with
STATUS_HEAP_CORRUPTION on the first hit. Either half alone does not
crash. Not reproducible on Linux or macOS, where the epoll/kqueue
backend already maintains tick_depth.
git apply, run from a subdirectory of a repository, only prefixes a
patch's paths with that subdirectory for traditional unified diffs. A
patch with a `diff --git a/X b/X` header is treated as toplevel-relative
instead, falls outside the subdirectory prefix, and is silently "Skipped"
with exit 0. applyPatch only checked the exit status, so the fetch went
on to write the .ref stamp over a tree the patch never touched.
vendor/<dep>/ sits inside this repository, so every dep fetch runs git
apply from exactly that kind of subdirectory. All 18 existing files in
patches/ happen to be traditional format, which is the only reason
nothing has hit this before.

Two-part fix in applyPatch:

- GIT_CEILING_DIRECTORIES set to dest's parent stops git from discovering
  the enclosing repository, so the prefix logic never engages and the
  patch's paths resolve against dest regardless of format.
- The invocation adds -v and LC_ALL=C and fails hard on "Skipped patch"
  in stderr, so a skipped file can never stamp the source as patched.

Also from review: the regression test's spawned child now includes
stderr in the combined assertion (as expect.any) so native crash
diagnostics appear in the failure diff instead of being discarded, and
the libuv dep's now-redundant patch-format note is removed because the
applyPatch doc is the right home for it.
@robobun
robobun force-pushed the farm/3c97b97f/fix-windows-nested-tick-uaf branch from 2662a98 to c46dde9 Compare August 12, 2026 20:04
@robobun

robobun commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased onto current main (835 commits, three files conflicted). Since a lot of Windows event-loop work landed in between, I first checked whether main had fixed this on its own: it has not. tick_depth is still never written on the libuv backend, loop.c's tick_depth <= 1 guard is unchanged (its comment even describes this exact bug), and close_cb_free_poll/us_poll_free are logically identical to when this PR was opened. The fix is still needed; in fact #34478 made it more so (below). How each conflict was resolved:

packages/bun-usockets/src/eventing/libuv.c

scripts/build/deps/libuv.ts

  • libuv was bumped three times (now 89ee3439) and main added a second patch, win-poll-abort-with-disconnect.patch. Resolved to a three-entry patches array, this PR's patch last. I verified the whole stack by fetching src/win/poll.c at the pinned commit and applying all three in that order: main's two apply as before, and this PR's applies cleanly at a 25-line offset (the lines main's new patch adds; no hunk overlap, since main's patch touches the AFD subscription/reporting code and this one touches the post-poll_cb endgame tails). I also dropped the NOTE block about patch-file format from the comment: the fetch-cli.ts commit in this PR makes that failure mode fail loudly, so the warning was stale.

test/js/bun/net/socket.test.ts

  • Append-vs-append again. Took main's file verbatim and re-appended this PR's test.

Rebuilt on the rebased tree: the test passes, and the full file goes from 72 pass / 9 fail / 83 tests on unmodified main to 73 / 9 / 84 on this branch (the 9 are outbound-DNS environment failures, identical on both). The diff is otherwise the same as before; the net change from the rebase is the us_free spelling and the us_loop_pump body.

@robobun

robobun commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author

Build 93542 (the rebase push) is red because of a CI-wide outage, not the diff: three build-bun jobs died fetching dependency tarballs from github.com (cares, mimalloc, lolhtml, and the prebuilt WebKit all hit Failed to download after 5 attempts ... cause: fetch failed), and the 42 waiting_failed jobs are just their dependents. Every build on the pipeline in the same window has the same failures, including main's own. The download step runs before the applyPatch code this PR touches, so this PR's build-script change is not involved. I have already used this PR's one retrigger, so this needs a retry from someone with Buildkite access once GitHub is reachable again, or the next rebase will re-run it.

@alii alii left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tick_depth bracket, the closed sentinel and the libuv patch hold up against current main, and the rebase you pushed this afternoon took care of the us_free and active_handles conflicts. Two things left, plus one note.

  • us_loop_pump: nothing exercises the pump bracket; add an unref'd variant of the new test.
  • The fetch-cli.ts hardening is unrelated to this fix and the body still says it was left for a separate PR; split it out or update the body.
  • The c-ares uv_poll_t in src/runtime/dns_jsc/dns.rs has the same shape (on_dns_socket_state uv_closes it from inside its own poll_cb, on_close_uv frees it, and the microtask drain at the end of on_dns_poll_uv can re-enter the loop). A follow-up is fine, but say so in the body.
    Windows CI on the rebase died in the github.com outage, so it still needs a green Windows lane.

* callback), and the forced iteration always reaches the check phase, i.e.
* us_internal_loop_post. The tick_depth bracket is what makes that post
* defer the closed-socket sweep to the outermost tick; see us_loop_run. */
loop->data.tick_depth++;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing covers this path. The new test keeps the listener ref'd, and uv_close bumps active_handles for the closing handle, so both the outer and the nested tick in the test go through us_loop_run and this bracket can be deleted without the test noticing. Since #34478 the pump does run the check phase, so a close from an unref'd socket's callback followed by a nested tick hits the same sweep. Add a variant of the socket.test.ts case with server.unref() and the client unref'd.

input: normalizeLf(patchBody),
stdio: ["pipe", "ignore", "pipe"],
encoding: "utf8",
env: { ...process.env, GIT_CEILING_DIRECTORIES: join(dest, ".."), LC_ALL: "C" },

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is unrelated to the fix, and the body still says applyPatch hardening is for a separate PR. It changes the fetch step for every patched dep on every platform, no patch in the tree is git-format so nothing needs it today, and GIT_CEILING_DIRECTORIES also hides the repo's .gitattributes eol=lf from git apply. It applies to main on its own; split it out, or if you want it here update the body and check the eol point.

robobun added a commit that referenced this pull request Aug 13, 2026
…e stack (Windows)

On Windows each c-ares socket is driven by a uv_poll_t embedded in a heap
UvDnsPoll. c-ares closes a UDP socket as soon as its answer is processed,
which happens inside on_dns_poll_uv, so uv_close() is issued from within
the handle's own poll callback. The callback then drains microtasks; a
promise reaction that spins the event loop (Bun.build() with a plugin
whose setup() returns a pending promise, bun:test's .resolves, ...) runs a
nested uv_run, whose endgame phase invoked on_close_uv and freed the
UvDnsPoll. libuv's uv__fast_poll_process_poll_req frame for that handle
was still suspended underneath and reads the handle again once the
callback returns; with the stale bytes it re-queued the endgame, close_cb
ran a second time and the struct was freed twice. A script that re-enters
the loop from resolve4() reactions segfaults after about ten queries on
the release build.

Count the on_dns_poll_uv frames active for a handle. on_close_uv only
frees the struct when none is active; otherwise it marks the handle and
the outermost frame, once its microtask drain is over, hands the struct to
the event loop's task queue, which is drained only after the libuv
callback has returned. The common path (close callback delivered after
the poll callback returned) is unchanged.

The libuv side of the same scenario is the win-poll-no-reendgame-after-
close patch from #33018 (identical copy): with the struct kept alive,
libuv would otherwise re-queue the endgame of the already CLOSED handle
from the post-callback check and invoke close_cb twice (uv__poll_endgame
asserts on this in debug builds).
@robobun

robobun commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator Author

Two more CI sightings of this bug in test/bake/deinitialization.test.ts, plus a way to reproduce it almost every time with the shipped binary. I root-caused it independently from those sightings and landed on the same three problems this PR describes, so this is corroboration rather than anything new about the cause.

Sightings. Both on 2026-08-13, on unrelated branches that had just merged main:

  • Build 93713, windows 2019 x64: the fixture child dies with panic(main thread): Segmentation fault at address 0x0 during the websocket=8 case. Symbolized against that build's PDB, the top frame is us_poll_start_rc (eventing/libuv.c, the p->uv_p->data = p store right after the memset) under start_connections <- us_internal_socket_after_resolve <- us_internal_loop_post. A freshly malloc'd us_poll_t and its freshly malloc'd uv_poll_t overlapped, i.e. the free list of the uv_poll_t size class had already been corrupted by an earlier uv_poll_t being freed twice, which is the double endgame described above.
  • Build 93669, windows 11 aarch64: the child hangs after the closeActiveConnections websocket=1 case until the outer test kills it (signalCode: "SIGTERM"). Locally, sampling a hung release process shows the main thread spinning at 100% in mimalloc's _mi_page_purge_holes -> mi_page_purge_holes_walk, which walks a page's free list; a double free puts a cycle in that list, so the idle-time purge never terminates. That is what the timeout variant of this bug looks like.

Reproduction. On a Windows Server 2019 box where localhost resolves to both ::1 and 127.0.0.1 (the default Bun.serve hostname ends up listening on ::1 only there, so every connect goes through the happy-eyeballs path and churns sockets), one case of the fixture is enough:

cd test/bake/fixtures/deinitialization
bun test ./test.ts -t "flags: closeActiveConnections sendAnyRequests websocket=1$"

With the bun-profile.exe from build 93713 this failed 12 of 12 runs (10 hangs, 2 segfaults, both segfaults with the same trace as the build 93713 crash); the whole fixture failed 30 of 30 sequential runs. Every other case passes in isolation. What makes this case special, as far as I can tell: main() resumes from the WebSocket open event, i.e. inside the dispatch of the client socket's readable event, and then calls expect(fetch(origin)).rejects.toThrow(...) without awaiting it. .rejects runs wait_for_promise, so the loop is re-entered from inside that socket's dispatch; server.stop(true) closes the HMR connection during the nested ticks, and the client socket is closed, swept and freed before the outer dispatch returns to loop.c's follow_adopted. A debug build of a reduced version (peer closes the socket inside the nested tick, then a few more sockets are opened and closed before it returns) crashes deterministically with Segmentation fault at address 0xFFFFFFFFFFFFFFFF in us_internal_socket_follow_adopted <- us_internal_dispatch_ready_poll (loop.c:723) <- poll_cb, the same stack as the postgres sighting above.

Confirmation that this is the whole story for that test. I tried a variant of the fix that avoids the libuv patch: keep a per-poll dispatch depth in poll_cb and, when us_poll_stop runs while the poll's own callback is on the stack, issue the uv_close only when the outermost callback returns (timer-phase and other closes are unchanged, so it should not hit the liveness problem the first revision here ran into), plus the same tick_depth bracket and close-callback marker as this PR. It is on farm/fae41b8e/libuv-nested-tick-socket-free if useful as a reference. With it, a Windows release build passes the fixture case above 12 of 12, the whole fixture 10 of 10 sequentially and 30 of 30 six-wide in parallel, and the node:net, websocket, serve.test.ts and socket.test.ts suites still pass on a debug build. One data point that supports carrying the libuv patch (or an equivalent): with only tick_depth plus the marker, the reduced repro trips Assertion failed: 0, file src/win/core.c, line 763 in uv__process_endgames on a debug build, i.e. the handle re-queued by uv__fast_poll_process_poll_req after its endgame already ran, now freed by the marker path. So the third change is required, not optional.

Not opening a separate PR; this one covers it.

@robobun

robobun commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator Author

One more trigger shape for this bug, with a different crash site than the three already described here. Posting it because it is the smallest repro so far and because the nested tick enters through us_loop_run rather than us_loop_pump, so it would complement the pump variant requested above.

Symptom. test/js/node/http/node-https-server-context.test.ts segfaulted on the windows 2019 x64 lane of build 94403 (that test has since been changed to avoid .rejects, which only hides this). Symbolized against the debug PDB, the crash is in the TLS layer's own post-dispatch tail, not in us_internal_dispatch_ready_poll:

SSL_get_ex_data
ssl_flush_pending_session        packages/bun-usockets/src/crypto/openssl.c:419
us_internal_ssl_on_data          packages/bun-usockets/src/crypto/openssl.c:2428   (after the JS dispatch returned)
us_internal_dispatch_ready_poll  packages/bun-usockets/src/loop.c:720
poll_cb                          packages/bun-usockets/src/eventing/libuv.c:165
uv_run / us_loop_run             packages/bun-usockets/src/eventing/libuv.c:413
WindowsLoop::tick_with_timeout   src/uws_sys/Loop.rs:462

Trigger. A test body resumes inside TLS socket A's on_data (the secureConnect await), calls socket.destroy() (A goes on closed_head while A's us_internal_ssl_on_data is still on the stack), then synchronously hits expect(...).rejects, whose wait_for_promise drives nested us_loop_run ticks until a second socket's error settles the promise. The nested tick's us_internal_loop_post frees A; when the outer us_internal_ssl_on_data resumes, s and s->ssl are freed. This is the expect().toThrow() case the tick_depth comment in loop.c names.

Numbers (Windows Server 2019 x64, canary 1.4.0-canary.1 (b7a043103), i.e. current main without this PR):

  • repro below: 9 of 20 runs segfault (Segmentation fault at address 0xFFFFFFFFFFFFFFFF x6, 0x200000078 x2, 0x10 x1)
  • same flow with try/catch instead of .rejects: 0 of 20
  • Linux and macOS: passes, as expected (epoll/kqueue already maintain tick_depth)

I have not built this branch on Windows; the nested tick here goes through tick_with_timeout -> us_loop_run, which is the bracket this PR adds, so it should be covered by the same change.

Repro (bun test file; keys from test/js/node/test/fixtures/keys)
import { expect, test } from "bun:test";
import { once } from "node:events";
import { readFileSync } from "node:fs";
import https from "node:https";
import tls from "node:tls";

const keys = "test/js/node/test/fixtures/keys/";

async function peerCN(port: number, extra = {}) {
  const socket = tls.connect({ host: "127.0.0.1", port, rejectUnauthorized: false, ...extra });
  const errored = once(socket, "error");
  await Promise.race([once(socket, "secureConnect"), errored.then(([e]) => Promise.reject(e))]);
  const cn = socket.getPeerCertificate().subject?.CN;
  socket.destroy(); // closes socket A from inside its own on_data
  return cn;
}

test("rejects after a socket event", async () => {
  const server = https.createServer({
    key: readFileSync(keys + "agent1-key.pem", "utf8"),
    cert: readFileSync(keys + "agent1-cert.pem", "utf8"),
    minVersion: "TLSv1.3",
  });
  server.listen(0);
  await once(server, "listening");
  const port = (server.address() as any).port;
  try {
    expect(await peerCN(port)).toBe("agent1"); // resumes inside A's us_internal_ssl_on_data
    await expect(peerCN(port, { maxVersion: "TLSv1.2" })).rejects.toThrow(); // nested us_loop_run ticks
  } finally {
    server.close();
  }
});

@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

One more data point for test/bake/deinitialization.test.ts, since it keeps showing up on unrelated PRs: builds 97333 (windows 2019 x64, Segmentation fault at address 0x0, symbolized against that build's PDB to the same us_poll_start_rc <- start_connections <- us_internal_socket_after_resolve frame as above) and 96648, 96672, 96712, 96745, 96754 (child hangs until the outer test kills it, main thread spinning in _mi_thread_idle_work). Independent root-causing landed on the same three problems this PR fixes.

Verification of this PR's exact diff (the packages/, patches/ and scripts/ changes applied to current main), Windows Server 2019 x64:

  • release (bun-profile.exe): the full deinitialization fixture passes 30 of 30 sequential runs; the unmodified bun-profile.exe from build 97333 hangs 30 of 30 on the same machine.
  • debug: a reduced fixture (client WebSocket whose open handler stops the server and spins expect().resolves until its own socket is closed, 32 times) segfaults at 0xFFFFFFFFFFFFFFFF in us_internal_socket_follow_adopted on every run without the fix and passes 10 of 10 with it.

That reduced fixture is on farm/0b4e0697/libuv-nested-tick-closed-sockets (test/js/web/websocket/websocket-close-during-nested-tick{.test,-fixture}.ts, the rest of that branch is the same fix as this PR) if a second regression test covering the peer-initiated close is wanted. Not opening a separate PR.

@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

This also fixes the intermittent Windows failure of test/bake/deinitialization.test.ts that started showing up on main this week, so adding the evidence here.

CI symptom

The child bun test of that file dies at flags: websocket=8 (the first case that opens 8 HMR websockets), either as panic(main thread): Segmentation fault at address 0x0 (exit 3) or as a hang that the parent kills after 60s. Main builds on the Windows lanes: 94978 (18059646e1, hang), 95095 (18391f652b, segfault), 95216 (b555e06414, hang), 95531, 95569, 96743, 97019, 97596, plus several PR builds. Apart from one differently shaped hang on the aarch64 lane (94142, stuck in the sendAnyRequests case), every finished main build from Aug 6 up to 94812 (b5afcacd71) is clean, so the fixture started hitting this on Aug 13; the hits predate #38299 and the other teardown changes from Aug 14. The defect itself is the one this PR describes: tick_depth from #29354 was never maintained on the libuv backend, so any timing change in the fixture could expose it.

How the fixture gets there

  • The test body resumes inside the client HMR websocket's on_data dispatch (the 101 response fires onopen, which settles the Promise.all, and the continuation runs in the microtask drain before the dispatch returns).
  • server.stop() finds the server drained and drops the DevServer synchronously (deinit_if_we_can), which closes every server-side HMR socket.
  • expect(fetch(...)).rejects.toThrow(...) then runs the event loop from inside that dispatch. The nested ticks deliver the closes to the clients, and the nested check phase plus endgame free the client socket whose dispatch is still on the stack. With 8 sockets the nested window is long enough for that most of the time, which is why websocket=8 is the case that fails.
  • Symbolicated debug stack on Windows x64: us_internal_socket_follow_adopted (internal.h:359) < us_internal_dispatch_ready_poll (loop.c:750, right after us_dispatch_data returns) < poll_cb (libuv.c) < uv__fast_poll_process_poll_req (win/poll.c) < uv_run < us_loop_run < auto_tick < TestCommand::run. A debug build faults at 0xFFFFFFFFFFFFFFFF (mimalloc's freed-memory fill read as prev); release builds fault at 0x0 or spin, matching the re-queued endgame double free described above.

An instrumented build confirmed the free: close_cb_free_poll freeing the poll that is at the bottom of the dispatch stack, immediately before the crash.

Verification of this branch (Windows x64, debug builds)

  • c46dde9f55 (this PR): the deinitialization fixture passes 30/30; a focused repro (a Bun.connect socket that calls terminate() in data() and then blocks in expect().resolves) passes 10/10. The libuv patch was confirmed applied in the built tree (both UV_HANDLE_CLOSED guards present in win/poll.c).
  • Unfixed main (c19cab407b): the same focused repro crashes 5/5 in a debug build and spins at 100% CPU with the release canary (bcab5edce); the deinitialization fixture crashed in 1 of 3 plain debug runs and in 12 of 12 runs once stderr logging was added to libuv.c.

For what it is worth, I independently tried the variant that moves the uv_close into us_poll_free; the note above about why us_poll_stop must keep its uv_close is the reason I am not opening that as a separate PR.

@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

Arrived at the same three changes independently from build 98537 (test/bake/deinitialization.test.ts, us_poll_start_rc frame), so nothing new on the cause; standing down in favor of this PR rather than opening a second one. Two things from that branch that may be useful while the remaining test work here is done:

  • farm/2f31e133/libuv-nested-tick-free has a bun test fixture with two more trigger shapes besides the data callback: closing the accepted socket from its own open handler (outer frame is the accept loop, loop.c:560) and from its own end handler (outer frame is the peer-FIN close at loop.c:924, which then reads the freed socket in us_internal_socket_close_raw). Both crash an unpatched Windows debug build on every run and pass 20/20 with the fix, in case more coverage is wanted.
  • A gotcha for the unref'd/pump variant: the nested run must not wait on an event of the peer socket. On Windows the peer's completion is often dequeued in the same IOCP batch as the event being dispatched, and a nested uv_run never sees the outer batch, so such a wait deadlocks (that is how the open-handler variant first failed). Waiting on a write to a separate, already established connection (or a timer, as the existing test does) works.

@robobun

robobun commented Aug 16, 2026

Copy link
Copy Markdown
Collaborator Author

This is also the cause of the top hard failure on the windows-x64 CI lane: test/bake/deinitialization.test.ts (16 of ~400 builds on 2026-08-15, e.g. builds 98610, 98578, 97327), which dies with panic(main thread): Segmentation fault at address 0x0 in us_poll_start_rc (libuv.c:243, p->uv_p->data = p after the memset) while the "websocket=8" case opens its connections.

How that fixture reaches this bug: the case before it awaits the WebSocket onopen, so the rest of the test body runs as a microtask drained inside the ws client socket's poll_cb. There it calls expect(fetch(...)).rejects.toThrow(...) (twice), which is wait_for_promise, i.e. a nested uv_run inside that socket's own callback. server.stop(true) runs during the spin, the client socket is closed from a nested poll_cb on the same handle, and its close completes inside the spin. When the outer frame returns, uv__fast_poll_process_poll_req re-queues the endgame and close_cb_free_poll runs a second time on the freed uv_poll_t; its data is now mimalloc's free-list link (the previously freed 448-byte block, i.e. another socket's uv_poll_t), so two blocks in that size class are freed twice. The next us_create_poll batch then gets a us_socket_t block handed out as a uv_poll_t, and the memset(p->uv_p, 0, sizeof(uv_poll_t)) in us_poll_start_rc zeroes the us_poll_t being started, hence p->uv_p == NULL and the fault at 0. It is intermittent (~4% of builds) because the fetch rejection that ends the spin races the close completion; this PR's repro makes the same window deterministic.

Independent reproduction on a windows-x64 box with a bun test based fixture (close from the socket's own data handler, then expect(promise).resolves inside the handler; a second case has the peer close the socket during the spin, which exercises the nested same-handle dispatch the CI failure goes through; then churn through more connections): the unmodified release canary (1.4.0-canary.1+bcba4722b) hangs at 100% CPU in 9/9 runs, a control without the spin passes 3/3, and an unmodified debug build segfaults in 6/6 runs with this stack, which is the first half of the mechanism described here:

us_internal_socket_follow_adopted   packages/bun-usockets/src/internal/internal.h:359
us_internal_dispatch_ready_poll     packages/bun-usockets/src/loop.c:750
poll_cb                             packages/bun-usockets/src/eventing/libuv.c:165
uv__fast_poll_process_poll_req      vendor/libuv/src/win/poll.c:233
uv__process_poll_req                vendor/libuv/src/win/poll.c:573
uv__process_reqs                    vendor/libuv/src/win/core.c:682
uv_run                              vendor/libuv/src/win/core.c:806
us_loop_run                         packages/bun-usockets/src/eventing/libuv.c:417

For reference, branch farm/499b3f4c/libuv-poll-close-reentrant-uv-run fixes the same thing without a libuv patch: the uv_poll_t is allocated inside a small state block that counts the poll_cb frames on the stack, us_poll_stop only clears the event mask while that count is nonzero and the outermost poll_cb issues the uv_close after dispatch (so libuv never sees the handle closed during its own callback and the tail of uv__fast_poll_process_poll_req always reads a live handle), us_poll_free/close_cb_free_poll free the two blocks from whichever runs second, plus the same tick_depth bracket as here. That fixture passes 5/5 on a debug build of that branch, and socket, tcp-server, udp, websocket server/client, serve, node-net-server and the deinitialization test all pass on it. The uv_close there is still issued before the same iteration's check phase, so the uv__loop_alive concern described above does not apply to it; closes made outside the handle's own callback are unchanged. Not opening it as a PR since this one covers the bug; it is there in case the libuv patch turns out to be unwanted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants