Skip to content

inspector: distinguish backpressure and drops in the debugger WebSocket writer - #34220

Open
hughescr wants to merge 6 commits into
oven-sh:mainfrom
hughescr:pr/writer-correctness
Open

inspector: distinguish backpressure and drops in the debugger WebSocket writer#34220
hughescr wants to merge 6 commits into
oven-sh:mainfrom
hughescr:pr/writer-correctness

Conversation

@hughescr

Copy link
Copy Markdown
Contributor

Related: #34218 (this PR fixes the "layer (b)" writer-correctness defect described there; the exit-drain half is #34219)

What does this PR do?

Fixes a correctness bug in the inspector debugger's writer layer
(webSocketWriter/bufferedWriter in src/js/internal/debugger.ts), which
backs the ws:/ws+unix: transport used by --inspect/--inspect-wait
over a real Bun.serve WebSocket (Debugger's #websocket handler).
uWS's ws.sendText() returns a three-way contract (see
packages/bun-uws/src/WebSocket.h, WebSocket::send()'s doc comment and
SendStatus enum): -1 (BACKPRESSURE -- the message was accepted into
uWS's outbound buffer, which is just running high), 0 (DROPPED -- the
message was not sent at all), or a positive byte count (SUCCESS).
webSocketWriter previously collapsed this three-way result with !!,
coercing -1 to true and making backpressure indistinguishable from
success. Since bufferedWriter only requeues on a falsy return, it never
reacted to backpressure at all -- and, had it naively resent on any falsy
value instead, it would have duplicated a message that uWS says was already
accepted.

The fix, entirely in the existing writer-layer functions (no new
files):

  • webSocketWriter now surfaces the real tri-state result --
    "success" / "backpressure" / "dropped" -- instead of coercing it.
    result === 0 is ambiguous in the underlying Rust binding itself (it's
    also what a genuine zero-length-string SUCCESS or an already-closed-socket
    short-circuit would return -- see send_status_to_js/send_text in
    src/runtime/server/ServerWebSocket.rs); treating every 0 as
    "dropped" is safe here specifically because CDP's
    sendMessageToFrontend skips empty messages before they ever reach this
    writer (src/jsc/bindings/BunDebugger.cpp:217-218), so that ambiguity
    can't actually arise on this path -- documented in a comment at the call
    site.
  • bufferedWriter now paces (queues, does not resend) further writes on
    backpressure, and requeues on a genuine drop.
  • bufferedWriter.write() also gates on pendingMessages.length > 0, not
    just the paced flag: a "dropped" result queues the message but does
    not set paced, so without this a later write() could reach the wire
    directly and overtake a still-queued retry -- once anything is queued,
    every later write has to queue behind it to preserve order.
  • pendingMessages is now fully cleared once a drain fully succeeds, so a
    later no-op drain() can't resend stale entries.

bun:internal-for-testing gains a debuggerInternals export exposing the
real webSocketWriter/bufferedWriter implementations (via a testHooks
property riding on debugger.ts's single permitted default export --
builtin modules under src/js/internal/** support only export default,
so a second named export isn't an option). This lets the accompanying tests
exercise the shipped code directly instead of a reimplementation in the
test file that could silently drift from it.

Not claimed: this PR does not assert it fixes any specific CI incident
or reported issue. It's a correctness fix found by inspecting the writer
layer's handling of sendText()'s return contract, landed with tests that
pin the corrected behavior going forward.

Relationship to sibling PRs: this is one of three independent PRs
against the same base (fix/inspector-backpressure-retry) that each touch
test/cli/inspect/test-reporter.test.ts's former
"delivers every TestReporter event exactly once, in order, when write() reports WebSocket backpressure" test -- a main-thread-exit-drain fix and a
TestReporter id-collision fix, both found while investigating this writer
bug, but with unrelated root causes and call paths. That test exercised
none of them correctly (it used --inspect-wait=unix:, which never
constructs a bufferedWriter(webSocketWriter(...)) in the first place), so
this PR replaces it with tests that actually cover this fix; the other two
PRs replace the same original test with their own, different coverage.
Whichever of the three lands last will need its trailing test block
reconciled by hand against whatever's already merged, since all three
diffs touch the same lines.

How did you verify your code works?

Added two new describe blocks to test/cli/inspect/test-reporter.test.ts,
replacing the old backpressure test that didn't actually exercise this code
path:

  1. "writer layer: bufferedWriter / webSocketWriter (unit)" -- scripted
    unit tests against the real implementations (via
    bun:internal-for-testing's debuggerInternals), covering:

    • webSocketWriter mapping all three sendText() return values to the
      correct WriteResult.
    • bufferedWriter pacing (not resending) after backpressure.
    • bufferedWriter requeuing a dropped write, and blocking a later write
      from overtaking it on the wire (the ordering-fix regression case).
    • pendingMessages being fully cleared after a successful drain (no
      stale resend on a later no-op drain).
    • Drain preserving order across repeated drops/backpressure partway
      through a multi-message queue.
    • close() clearing the pending queue and delegating to the underlying
      writer's close().
    • webSocketWriter + bufferedWriter composed together, proving a
      dropped CDP message can never be overtaken by the next one through the
      real sendText() mapping.
  2. "writer layer: ws:// integration" -- a real --inspect-wait=ws://
    subprocess with a genuine WebSocket client, proving the actual
    production wiring (bufferedWriter(webSocketWriter(ws)), constructed by
    Debugger's #websocket handler) delivers every TestReporter.found/
    TestReporter.end event for 200 tests exactly once and in non-decreasing
    id order, with no drops or duplicates, end to end over a real socket.

Honest limitation: this integration test does not force a genuine
socket-level BACKPRESSURE or DROPPED condition. The global WebSocket
client used here (matching this suite's other ws:// tests, e.g.
test/regression/issue/21654) has no public API to pause reading the
underlying socket the way a raw Bun.connect() client can, so uWS's
outbound buffer never has a reason to back up. Forcing a genuine DROPPED
additionally needs upward of 16MB of unacknowledged backlog -- the
debugger's #websocket handler sets no backpressureLimit, so uWS falls
back to its 16MB default (WebSocketServerContext.rs's
backpressure_limit default) -- which isn't a clean, fast, reliable hook to
build a test around. Both scenarios are instead covered deterministically
by the scripted-writer unit tests in (1), which drive the real
webSocketWriter/bufferedWriter functions with a scripted underlying
writer that returns "backpressure"/"dropped" on command. This test's
job is only to prove the real production wiring delivers correctly under
normal conditions, over an actual connection.

Ran the gate three times plus a full sweep, per this repo's CLAUDE.md
guidance to check for flakiness:

$ bun bd test test/cli/inspect/test-reporter.test.ts   # x3
9 pass / 0 fail / 440 expect() calls  (all three runs identical)

$ bun bd test test/cli/inspect/
45 pass / 4 todo / 0 fail across 4 files (49 tests)
  -- inspect.test.ts alone: 29 pass / 2 todo / 0 fail (unchanged from
     baseline -- no regressions from this change)

Typechecked src/js (cd src/js && bun x tsc --noEmit) and compared
errors attributed to the two changed files specifically against an
unmodified-base run: internal-for-testing.ts has zero errors on both
sides, and internal/debugger.ts actually has two fewer errors on this
branch than on base -- the widened onData(socket: Socket<{ ...; backend: Backend | Writer }>, ...) signature resolves two pre-existing
Socket<{ backend: Backend }> vs. Socket<{ backend: Writer }> mismatches
at its call sites, rather than papering over them with a cast. (A
whole-tree error-count comparison is not a reliable signal in this repo --
several thousand errors in unrelated node/*/internal/streams/* files
vary between runs regardless of this change -- so the comparison here is
scoped to the files actually touched.)

https://claude.ai/code/session_013DexKQwDASzoRVDFS8Jtyn

hughescr added 2 commits July 13, 2026 16:21
ws.sendText() (the uWS binding backing the inspector debugger socket)
has a documented three-way return contract: -1 (BACKPRESSURE, message
accepted into uWS's outbound buffer but the buffer is running high),
0 (DROPPED, message not sent at all), or a positive byte count
(SUCCESS). webSocketWriter previously collapsed this with `!!`, which
coerces -1 to `true` -- indistinguishable from success -- so
bufferedWriter (which only requeues a message when write() returns
falsy) never reacted to backpressure at all: the coercion silently
lied about what sendText() actually returned.

This distinguishes all three cases and adds real pacing behavior for
backpressure: bufferedWriter now queues (pauses) further writes while
backpressured, flushing them in order once drain() reports the
connection has room again -- so a burst of events under backpressure
is paced rather than blindly hammering an already-strained socket.

This intentionally does NOT re-send the specific message that reported
backpressure. Per uWS's own send() contract, BACKPRESSURE means that
message was already accepted into the outbound buffer and will still
be delivered -- unlike DROPPED (0), which means it was discarded and
must be retried. Re-sending an already-accepted message would
duplicate it on the wire, a worse bug than the one being fixed. Only a
genuine DROPPED result is queued for retry, matching bufferedWriter's
pre-existing behavior for that case.

While extending drain()'s tri-state handling, also fixed an adjacent
latent bug: pendingMessages was never cleared after a fully successful
drain pass, which would have re-sent already-delivered messages on the
next drain -- a duplicate-send bug of its own, and one now far more
likely to trigger given backpressure also routes through this queue.

Adds a regression test to test/cli/inspect/test-reporter.test.ts that
drives the debugger socket into genuine backpressure (pausing reads
before the subprocess starts emitting) and asserts every found/end
event arrives exactly once and in non-decreasing id order.

This is the fuller alternative to the minimal one-line coercion fix
(`sendText(message) !== 0`, staged separately) -- see PR body for
verification status and how to choose between the two.

Claude-Session: https://claude.ai/code/session_01N8rB9z5epigmEMvKbJFuQ1
`webSocketWriter`/`bufferedWriter` (src/js/internal/debugger.ts) back the
`ws:`/`ws+unix:` debugger transport (Bun.serve's `#websocket` handler). This
tightens their contract in three ways:

- `webSocketWriter` now surfaces `ws.sendText()`'s real tri-state result
  ("backpressure" / "dropped" / "success") instead of coercing it with `!!`,
  which collapsed BACKPRESSURE (-1, message already accepted -- do not
  resend) and DROPPED (0, message never sent -- must retry) into the same
  truthy/falsy signal.
- `bufferedWriter` paces (queues) further writes on backpressure without
  resending the message that reported it, and requeues on a genuine drop.
- `bufferedWriter.write()` now also gates on `pendingMessages.length > 0`,
  not just the `paced` flag: a "dropped" result queues a message without
  setting `paced`, so a later write could otherwise reach the wire ahead of
  a still-queued retry. `pendingMessages` is also fully cleared once a
  drain fully succeeds, so a later no-op drain can't resend stale entries.

`ws.sendText()`'s `result === 0` is ambiguous in the underlying Rust binding
itself (DROPPED vs. a genuine zero-byte SUCCESS or a closed-socket
short-circuit); treating every 0 as "dropped" is safe here specifically
because CDP's `sendMessageToFrontend` never hands this writer an empty
string in the first place -- documented in a comment at the call site.

`bun:internal-for-testing` now exposes the real `webSocketWriter`/
`bufferedWriter` implementations (via a `testHooks` property riding on
debugger.ts's single permitted default export) so the accompanying unit
tests exercise the shipped code directly rather than a reimplementation
that could drift from it. test/cli/inspect/test-reporter.test.ts adds
scripted-writer unit tests for the pacing/requeue/ordering/drain-clearing
behavior above, plus a real `ws://` integration test proving the production
`bufferedWriter(webSocketWriter(ws))` wiring delivers every TestReporter
event exactly once and in order over an actual connection. This is not
claimed to fix any specific CI incident -- it's a correctness fix for the
writer layer found via code review, verified by tests that pin the
contract going forward.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@coderabbitai

coderabbitai Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 78d6fc81-b141-44b0-bf67-9733fb8ece32

📥 Commits

Reviewing files that changed from the base of the PR and between 9a98148 and bded2e6.

📒 Files selected for processing (1)
  • test/cli/inspect/test-reporter.test.ts

Walkthrough

Changes

The inspector debugger now exposes writer test hooks, uses explicit WebSocket write outcomes, updates buffering and retry ordering, and adds unit and subprocess WebSocket integration tests.

Inspector writer flow

Layer / File(s) Summary
Writer contract and test hooks
src/js/internal/debugger.ts, src/js/internal-for-testing.ts
The debugger export includes webSocketWriter and bufferedWriter test hooks, with WriteResult replacing boolean writer results and socket framing accepting both backend types.
WebSocket buffering and retry behavior
src/js/internal/debugger.ts
Writer results distinguish success, backpressure, and dropped sends while buffering preserves ordering, retries dropped messages, and clears completed queues.
Writer and inspector integration tests
test/cli/inspect/test-reporter.test.ts
Scripted writer tests cover queue behavior, retries, pacing, and ordering; integration tests verify ordered, exact-once reporter messages over WebSocket.

Possibly related PRs

  • oven-sh/bun#34219: Both PRs add inspector delivery assertions around draining and TestReporter event completion.
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: handling backpressure vs drops in the debugger WebSocket writer.
Description check ✅ Passed The description matches the template and includes both the change summary and verification details.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/cli/inspect/test-reporter.test.ts`:
- Around line 563-564: Remove the custom 30000 timeout arguments from the
spawned child test command and the corresponding outer test timeout near the
related test setup. Rely on the repository runner’s standard timeout policy,
reducing TEST_COUNT only if needed to keep the workload within the normal
debug-build budget.
- Around line 618-637: Update the completion logic around the WebSocket message
listener and donePromise so it does not resolve immediately on the 200th
TestReporter.end event. After receiving all expected end events, send a uniquely
identified protocol command and resolve only when its matching response is
received, preserving the existing foundIds and endedIds collection while
ensuring queued duplicate deliveries are observed before assertions and socket
closure.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: ec739b6e-9d77-41a8-bdfe-57abd91ffbab

📥 Commits

Reviewing files that changed from the base of the PR and between be77b65 and 2e69a35.

📒 Files selected for processing (3)
  • src/js/internal-for-testing.ts
  • src/js/internal/debugger.ts
  • test/cli/inspect/test-reporter.test.ts

Comment thread test/cli/inspect/test-reporter.test.ts Outdated
Comment thread test/cli/inspect/test-reporter.test.ts Outdated
Removed the child's inert --timeout override from the spawned bun test
argv (the outer test()'s 30000ms timeout argument is unchanged).

Resolve via a protocol barrier after the final end event -- or via
socket close once all end events have been observed, since the child
exits immediately and close proves the stream is complete -- rejecting
fast when the socket dies with events still missing.

Surface captured child stderr before the exit-code assertion, so a
nonzero exit fails with the child's own error output instead of a bare
exit-code mismatch.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/cli/inspect/test-reporter.test.ts`:
- Around line 616-640: Update the message handler in the test reporter barrier
flow so the BARRIER_ID branch resolves donePromise only for protocol responses
that have no method. Preserve handling of TestReporter events first, and require
both msg.id === BARRIER_ID and an absent method before calling doneResolve.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 8d64ba4a-05e4-480f-b8db-6466e6a20cc0

📥 Commits

Reviewing files that changed from the base of the PR and between 2e69a35 and 9a98148.

📒 Files selected for processing (1)
  • test/cli/inspect/test-reporter.test.ts

Comment thread test/cli/inspect/test-reporter.test.ts
hughescr added 3 commits July 16, 2026 00:46
Per CDP framing, a response never carries a method, so gate the barrier-resolve branch on msg.method === undefined as well as the id.
* upstream/main: (70 commits)
  node:http/https/http2: raise Node v26.3.0 compat to ~94%, sync the upstream suites, and fix the Windows/macOS transport-layer teardown bugs they exposed (oven-sh#32488)
  expect: fix panic in toBeArrayOfSize/toHaveBeenCalledTimes with length > i32 max (oven-sh#32266)
  lexer: fix TOKEN_TO_STRING[TColon] showing " =" instead of ":" (oven-sh#34253)
  Bun.Terminal: write() returns bytes accepted, fire drain on POSIX (oven-sh#34289)
  test(serve-body-leak): give release-asan the same 60s per-test timeout as debug (oven-sh#34297)
  worker: mark the context terminating before the final concurrent-queue drain (oven-sh#34278)
  buffer: wrap negative ucs2 indexOf offset against raw byte length for Buffer needles (oven-sh#34273)
  fs.promises.watch: yield events with a null prototype (oven-sh#34279)
  child_process: latch stdin write EPIPE as 'error' + destroy, fail later writes with ERR_STREAM_DESTROYED (oven-sh#34268)
  Fix asString assertion when passing String objects as signals (oven-sh#34265)
  Buffer: carry size_t through toString/write so length 2^32 doesn't wrap to 0 (oven-sh#34274)
  test: use tempDir in log-test.test.ts instead of hardcoded /tmp path (oven-sh#34294)
  tty: track raw mode per handle instead of per process (oven-sh#33527)
  test: expect the bumped mimalloc SHA in process.versions
  Return freed memory to the OS on a background thread instead of the JS thread (oven-sh#34181)
  Move WTFTimer out of the shared timer heap to fix a cross-thread race (oven-sh#33131)
  test: update block-scoped enum lowering expectations to let (oven-sh#34287)
  Error.captureStackTrace: install .stack as non-enumerable (oven-sh#34259)
  js_parser: treat "async as T" / "async satisfies T" as a cast, not an arrow (oven-sh#34246)
  js_parser: accept `!`, `#name`, and `export @dec` in standard decorator grammar (oven-sh#34245)
  ...
* upstream/main: (422 commits)
  install: drop packages held only by optional-peer resolution slots from bun.lock (oven-sh#35681)
  Update mimalloc to the upstream dev3 (v3.4.3) sync (oven-sh#36431)
  compile(pe): ftruncate the Windows --compile output after writing (oven-sh#36430)
  Strong: back bun_jsc::Strong with StrongRootBlock; free AbortSignal.timeout at wrapper GC (oven-sh#35849)
  test(harness): replace toRun matcher with async bunRun + toSpawn (oven-sh#36424)
  test: measure memory via harness rss() instead of process.memoryUsage.rss() (oven-sh#36429)
  Deflake a few tests
  no-orphans(windows): allow CREATE_BREAKAWAY_FROM_JOB and set DIE_ON_UNHANDLED_EXCEPTION on the Job (oven-sh#36414)
  GarbageCollectionController: replace per-tick heap sampler with idle timer only (oven-sh#35356)
  exe_format(pe): write a valid OptionalHeader.CheckSum for --compile output (oven-sh#36383)
  FileSink: flush buffered bytes when process.exit() runs in the same tick as write() (oven-sh#36250)
  test(http): speed up and de-flake serve-async-stream-client-abort.test.ts (oven-sh#35919)
  test(20144): stop racing child startup against the 1s SIGKILL guard (oven-sh#34166)
  test(no-orphans): skip fast-exit perl daemon test on macOS (oven-sh#36413)
  fs: return negative BigIntStats *Ns for pre-epoch timestamps (oven-sh#36187)
  event_loop: make DeferredTaskQueue::run tolerate re-entrant map mutation (oven-sh#32703)
  dotenv: stop panicking on nested `${...}` inside `${VAR:-default}` (oven-sh#36199)
  fetch: make the idle timer an absolute deadline for the response header block (oven-sh#36145)
  bundler: don't panic on unterminated naming template placeholders (oven-sh#36325)
  Buffer#indexOf/lastIndexOf: rare-byte SIMD filter with a Two-Way O(n+m) fallback (oven-sh#36420)
  ...

# Conflicts:
#	src/js/internal/debugger.ts
@hughescr

Copy link
Copy Markdown
Contributor Author

Housekeeping for reviewers: I've just merged latest main (bbe3f6a262) into this branch — a normal merge commit (1163a5e50a), no force-push, so existing review comments and their anchors are intact. Two conflicts in src/js/internal/debugger.ts against main's new lazy CDP-adapter / node:inspector code, both resolved conservatively: kept main's lazy CDP block alongside this branch's named function startInspector( (the file tail's Object.assign(startInspector, { testHooks: ... }) export needs the name); the second conflict was comment-only (both sides had the same pendingMessages.length = 0; line). I also re-checked this branch's tri-state Writer.write change against the new CDP code specifically: the only call sites that consume a Writer.write return value are this branch's own bufferedWriter — the new code goes through Backend.write (boolean, untouched here) or ignores returns — so the two changes compose without interaction. bun bd test test/cli/inspect/test-reporter.test.ts is green after the merge.

Since I have four related-but-independent PRs open (#34219, #34220, #34221, #34222), a quick map so nobody has to reconstruct it: no two of them touch the same source file. The only overlap anywhere is test coverage — #34219, this PR, and #34221 each extend test/cli/inspect/test-reporter.test.ts, so whichever lands first leaves a test-file-only conflict in the other(s); I'll rebase and resolve promptly whenever anything merges. This PR and #34221 don't overlap even in tests, and #34222 shares nothing with the other three. Any merge order works.

robobun added a commit that referenced this pull request Jul 31, 2026
The sticky hasQueuedAnyDebuggerMessage gate is set on the first protocol
response and never clears, so every inspected process exit paid a fixed
+150ms with no completion condition. The FIFO sentinel alone (which wakes
as soon as the debugger thread has called write() for every queued message)
is sufficient for the bug in #34218; WebSocket-level backpressure flushing
is a separate concern tracked in #34220. Test passes 5/5 with the sentinel
alone.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant