Skip to content

watcher: coalesce per-save event bursts into a single --hot reload - #30617

Open
robobun wants to merge 10 commits into
mainfrom
farm/1f3aa35a/hot-coalesce-watcher-events
Open

watcher: coalesce per-save event bursts into a single --hot reload#30617
robobun wants to merge 10 commits into
mainfrom
farm/1f3aa35a/hot-coalesce-watcher-events

Conversation

@robobun

@robobun robobun commented May 13, 2026

Copy link
Copy Markdown
Collaborator

Fixes #13511.

Repro

// script.js
console.log("Hello");
bun --hot script.js

Save script.js in an editor (or just touch / rewrite it): Hello is printed two or more times for one save.

Minimal programmatic repro on Linux: two writeFileSyncs ~2 ms apart reliably produce two reloads on 1.3.x.

Cause

A single editor save emits several filesystem events spread over a few milliseconds (open(O_TRUNC) + write(), or write + rename for atomic saves, each mirrored on the parent-directory watch). The watcher tried to coalesce these, but:

  • the inotify/kqueue coalesce window was 0.1 ms and performed at most one extra read, so any events arriving after that spilled into the next watch-loop cycle;
  • the Windows watcher used a 0 ms timeout for subsequent GetQueuedCompletionStatus calls, so it only swept what was already queued;
  • in HotReloader.Task.append, every event for the same file re-appended the same path hash; once the fixed 8-slot buffer filled it called enqueue() mid-on_file_update, letting the JS thread start a reload while the watcher thread was still appending. The loop in Task.run then turned the later pending_count increments into a second reload for the same save.

Each extra on_file_update is a Task.enqueue, a vm.reload(), and a full re-evaluation of the entry point.

Fix

  • BUN_INOTIFY_COALESCE_INTERVAL default raised from 100_000 ns to 10_000_000 ns (10 ms). The env var declares the default, so it is the single source of truth; watcher_impl::coalesce_interval_ns() reads it and watcher_impl::MAX_COALESCE_ITERATIONS (32) lives next to it. Both are shared by all three backends.
  • INotifyWatcher / KEventWatcher / WindowsWatcher (src/watcher/*.rs): after the first read, keep draining until the queue stays quiet for the interval, bounded by buffer capacity and the iteration cap so a continuously written file cannot starve the loop. In the common case the loop exits one interval after the last event in the burst. The interval is stored as u64 nanoseconds and split into a timespec by a shared helper (no overflow path); Windows converts to the milliseconds GetQueuedCompletionStatus takes, rounding up so a sub-millisecond override still waits. The Windows Timeout enum is replaced by the raw DWORD timeout.
  • HotReloader.Task.append (src/jsc/hot_reloader.rs): skip hashes already in the buffer, so a burst of events for one file never triggers the mid-on_file_update flush. (VirtualMachine.reload ignores the hash list; only bake's dev server consumes it, and dedup does not change its semantics.)

The should not remap against a stale sourcemap after a partial-file reload test pins BUN_INOTIFY_COALESCE_INTERVAL=0: it deliberately relies on the self-write event being dispatched before the rejection is reported, which the new window would otherwise absorb.

Verification

New test coalesces a burst of writes into a single reload in test/cli/hot/hot.test.ts: spawns bun --hot, does 10 x writeSync with sleepSync(2) between them, and asserts exactly one reload. The test measures the gaps between its own writes: a burst that a loaded runner stretched past the coalesce window and that then split is retried (those writes really are separate saves), while a split burst whose gaps stayed uFull test/cli/hot/hot.test.ts passes 13/13 on the current head. cargo check -p bun_watcher is clean for the linux, darwin and windows targets. With the retrying form of the burst test, on a box at load average 100 to 170 (gaps between writes routinely 25 to 80 ms): the unfixed build failed 8/8 runs and the fixed build passed 10/10. The test stays skipped on Intel macOS, where sleepSync granularity rather than transient load stretches the burst. CI Linux and Windows lanes have passed it on every build of this PR. The burst test is timing-based by nature: on a heavily oversubscribed box (load average ~200 on 16 cores while verifying the latest rebase) the writer's 2 ms gaps stretch past 10 ms and the test legitimately sees separate saves; widening the window via the env var on that same box coalesced the burst every time, and the test is already skipped on Intel macOS for the same reason. CI Linux and Windows lanes have passed it on every build of this PR.

Rebase notes

Rebased eight times as main moved (the eighth, onto #39324's shared sort helper, conflicted only in an import block). Two were non-trivial:


no test proof · iteration 26 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/cli/hot/hot.test.ts

@robobun

robobun commented May 13, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 2:10 AM PT - Aug 17th, 2026

@robobun, your commit 0a1ef9d has 1 failures in Build #99936 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 30617

That installs a local version of the PR into your bun-30617 executable, so you can run:

bun-30617 --bun

@github-actions

Copy link
Copy Markdown
Contributor

Found 1 issue this PR may fix:

  1. Watch mode log two times errors in the console #10463 - Reports errors appearing twice in --watch mode on save, plausibly caused by multiple filesystem events from a single editor save each triggering a separate re-execution

If this is helpful, copy the block below into the PR description to auto-close this issue on merge.

Fixes #10463

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented May 13, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Linux, macOS, and Windows file watchers now coalesce rapid event bursts using a configurable quiet-period and bounded drain iterations (default 10ms). Hot reloader deduplicates pending file-hash IDs. Tests add a regression asserting a burst produces a single reload.

Changes

File Event Coalescing Across Watchers

Layer / File(s) Summary
Coalescing Configuration Defaults
src/bun_core/env_var.zig, src/watcher/INotifyWatcher.zig
Default coalescing interval changed to 10ms (BUN_INOTIFY_COALESCE_INTERVAL) and default_coalesce_interval_ns exported as the inotify fallback.
Linux inotify Event Coalescing
src/watcher/INotifyWatcher.zig
init reads the env-var; read() replaces threshold-based double-read with a bounded ppoll + non-blocking read drain loop, handling EAGAIN/EINTR and capping iterations.
macOS kqueue Event Coalescing
src/watcher/KEventWatcher.zig
Adds coalesce_interval_ns, reads env-var in init, and replaces the single extra kevent call with a bounded loop that repeatedly calls kevent using a quiet timeout until quiet or iteration cap.
Windows Event Coalescing Loop
src/watcher/WindowsWatcher.zig
Adds coalesce_interval_ms, converts nanoseconds→milliseconds with round-up in init, changes next to accept timeout_ms, and uses an INFINITE first wait then bounded coalesce sweeps capped by iterations.
Hot Reloader Hash Deduplication
src/jsc/hot_reloader.zig
Task.append returns early if the incoming file-hash ID already exists in the pending hashes, preventing duplicate reload scheduling.
Test Coverage for Coalescing Behavior
test/cli/hot/hot.test.ts
Expanded imports and determinism: added a regression that writes a burst to a watched file and asserts a single hot reload; adjusted timing tests to await initial eval and allow overriding coalesce interval via env.
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the primary change: coalescing per-save watcher events into one hot reload.
Description check ✅ Passed The description explains the cause, implementation, testing, platform behavior, and known test limitations in sufficient detail.

Warning

Review ran into problems

🔥 Problems

Stopped waiting for pipeline failures after 30000ms. One of your pipelines takes longer than our 30000ms fetch window to run, so review may not consider pipeline-failure results for inline comments if any failures occurred after the fetch window. Increase the timeout if you want to wait longer or run a @coderabbit review after the pipeline has finished.


Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional findings (outside current diff — PR may have been updated during review):

  • 🟡 test/cli/hot/hot.test.ts:658-662 — The BUN_INOTIFY_COALESCE_INTERVAL override here is only honored on Linux — KEventWatcher.zig hardcodes coalesce_interval_ns = 10_000_000 and WindowsWatcher.zig hardcodes Timeout.coalesce = 10, neither reads the env var. On macOS/Windows the test still runs with the new 10 ms window, so per the comment above, the self-write gets absorbed and the reject→report race this test guards never opens (the test still passes, it just stops exercising the regression). Might be worth having the kqueue/Windows watchers honor the same override, or at least noting that this race is now only meaningfully exercised on Linux.

    Extended reasoning...

    What the issue is

    The should not remap against a stale sourcemap after a partial-file reload test sets env: { ...bunEnv, BUN_INOTIFY_COALESCE_INTERVAL: "100000" } to pin the watcher's coalesce window back to 0.1 ms so the file's self-write is dispatched before the rejection is reported. However, BUN_INOTIFY_COALESCE_INTERVAL is only consumed by INotifyWatcher.init() (src/watcher/INotifyWatcher.zig:114). The macOS watcher hardcodes const coalesce_interval_ns = 10_000_000 (src/watcher/KEventWatcher.zig:12) and the Windows watcher hardcodes Timeout.coalesce = 10 (src/watcher/WindowsWatcher.zig) — neither consults the env var. So on macOS and Windows the env override is a no-op and the test runs under the new 10 ms coalesce window introduced by this PR.

    How it manifests

    The test's own comment explains why this matters: "the default 10 ms coalesce would absorb it into the next writeFull and the race under test never opens." That is exactly what now happens on macOS/Windows. The writeFileSync(__filename, "// stub") self-write triggers the watcher's 10 ms coalesce wait; during that window the rejection is printed (correctly mapped — no stale-sourcemap race), and then the watcher delivers the event(s) after the report. The reject→report window the test was written to exercise is therefore closed on those two platforms.

    Why the test still passes

    I traced driveErrorReloadCycle under the 10 ms window and the test does not fail or flake on macOS/Windows. There are two possible interleavings:

    1. The self-write and the test's subsequent writeFull(N+1) both land inside the same 10 ms coalesce window → one reload with the full N+1 content → error: N+1 prints → verifyLine passes.
    2. The self-write is delivered alone → the watcher reloads the comment-only stub, which throws nothing and emits nothing to stderr (--hot keeps the process alive via the keep-alive timer) → the next writeFull(N+1) triggers another reload → error: N+1 prints.

    In both cases driveErrorReloadCycle observes the expected error: N sequence and the assertions pass. So this is purely a test-coverage degradation, not a functional bug or a flaky test.

    Step-by-step proof

    1. On macOS, KEventWatcher.init() is called → it never reads BUN_INOTIFY_COALESCE_INTERVAL; coalesce_interval_ns stays at the file-level constant 10_000_000.
    2. Test spawns bun --hot with the env var set → the kqueue watcher ignores it.
    3. The hot module evaluates: it does writeFileSync(__filename, "// stub N") then throw new Error('N').
    4. The kqueue thread receives NOTE_WRITE, then enters the while (count < changelist_count …) loop with a 10 ms timeout, blocking dispatch.
    5. Meanwhile the JS thread reports the rejection — correctly mapped, since no second reload has retranspiled the file yet. The race the test is meant to catch (second reload overwriting source_mappings[path] between reject and report) cannot occur because the watcher is still inside its 10 ms drain.
    6. driveErrorReloadCycle sees error: N, calls writeFull(N+1), the watcher (either coalescing it with the stub write or in a fresh cycle) reloads, and the loop progresses.
    7. Test reaches reloadCounter === 20 and passes — but at no point did it actually open the stale-sourcemap window.

    Impact and fix

    The sourcemap-remapping code being guarded is platform-independent VM code, so Linux CI still catches regressions, and the env var name (INOTIFY) signals Linux-only intent — this may even be a conscious tradeoff. But the test silently degrades to a near-tautology on macOS/Windows after this PR, which is worth flagging. Two options:

    • Have KEventWatcher and WindowsWatcher honor the same env var (or a generically-named one) so the override works everywhere.
    • Add a comment on the test noting the race is only exercised on Linux.

    This is a nit — coverage concern only, no production-code impact, and the test should not flake. It should not block the PR.

@robobun

robobun commented May 13, 2026

Copy link
Copy Markdown
Collaborator Author

Addressed the coverage note in 0c76ed9: BUN_INOTIFY_COALESCE_INTERVAL is now honoured by the kqueue and Windows watcher backends too (despite the Linux-centric name), so the stale-sourcemap test's 0.1 ms override applies uniformly across platforms.

The earlier windows-x64-baseline-build-zig failure was a Zig compiler panic (reached unreachable code in Sema.handleExternLibName while analyzing src/runtime/image/backend_wic.zig, which this PR doesn't touch) — the non-baseline windows-x64-build-zig lane passed on the same commit.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/watcher/WindowsWatcher.zig`:
- Line 222: The loop in WindowsWatcher.zig uses an off-by-one condition: change
the while condition in the coalescing loop from "iterations <=
max_coalesce_iterations" to use "<" so it matches KEventWatcher and
INotifyWatcher; locate the while loop in the WindowsWatcher coalescing logic
(the loop that increments "iterations" and references "max_coalesce_iterations")
and replace the comparison operator to ensure the loop runs at most
max_coalesce_iterations times.

In `@test/cli/hot/hot.test.ts`:
- Line 335: Replace usage of tmpdirSync with the harness helper tempDir: import
tempDir from the test harness and swap const dir = tmpdirSync(); to create the
temporary directory via tempDir() (and await it if the helper is async), and
update any other occurrences in this file (e.g., the beforeEach that uses
tmpdirSync) to use tempDir as well; ensure variable names and cleanup logic
still work with the tempDir return shape.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 4249892f-5e6a-4983-bdea-1a1f404e2abc

📥 Commits

Reviewing files that changed from the base of the PR and between 3bf4b33 and 0c76ed9.

📒 Files selected for processing (6)
  • src/bun_core/env_var.zig
  • src/jsc/hot_reloader.zig
  • src/watcher/INotifyWatcher.zig
  • src/watcher/KEventWatcher.zig
  • src/watcher/WindowsWatcher.zig
  • test/cli/hot/hot.test.ts

Comment thread src/watcher/WindowsWatcher.zig Outdated
Comment thread test/cli/hot/hot.test.ts
Comment thread src/watcher/KEventWatcher.zig Outdated
Comment thread test/cli/hot/hot.test.ts Outdated
Comment thread src/watcher/KEventWatcher.zig Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/watcher/KEventWatcher.zig`:
- Around line 76-85: The loop drains events unconditionally which can trap if
the initial kevent syscall returns -1; check the syscall result stored in
`extra` and handle errors before slicing with `@intCast(count)`: if `extra ==
-1` and `count == 0` return or propagate the syscall error from
`std.posix.system.kevent`, otherwise only enter the draining logic when `count >
0` (i.e., guard the slices `changelist[`@intCast`(count)..]` and the secondary
kevent call) so you never slice with a zero/invalid `count` after a failed
syscall in the `KEventWatcher` coalescing loop (variables: `iterations`,
`count`, `changelist_count`, `max_coalesce_iterations`, `extra`).

In `@src/watcher/WindowsWatcher.zig`:
- Around line 144-147: The current conversion truncates sub-millisecond positive
values to 0ms; modify the conversion of
bun.env_var.BUN_INOTIFY_COALESCE_INTERVAL.get() (ns) before assigning
this.coalesce_interval_ms so that an explicit 0 stays 0, but any positive ns is
rounded up to at least 1ms (e.g., compute ms = ns / std.time.ns_per_ms using
integer ceil or set ms = 1 when ns > 0 and computed ms == 0), then cast to
std.math.cast(w.DWORD, ms) and fall back to default_coalesce_interval_ms if the
env var is absent.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: d3b22350-fda9-4dde-9dd1-ec20825a0867

📥 Commits

Reviewing files that changed from the base of the PR and between 0c76ed9 and 8731480.

📒 Files selected for processing (2)
  • src/watcher/KEventWatcher.zig
  • src/watcher/WindowsWatcher.zig

Comment thread src/watcher/KEventWatcher.zig Outdated
Comment thread src/watcher/WindowsWatcher.zig Outdated

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No new issues found, but this changes the coalesce behavior of all three platform watcher backends and bumps the per-reload latency floor from ~0.1 ms to ~10 ms — worth a human sanity-check on the tradeoff and the timing-sensitive test before merge.

Extended reasoning...

Overview

The PR reworks event coalescing in all three filesystem-watcher backends (INotifyWatcher, KEventWatcher, WindowsWatcher): the single 0.1 ms extra read is replaced with a bounded drain loop that waits up to 10 ms (×32 iterations) for the queue to go quiet, the BUN_INOTIFY_COALESCE_INTERVAL default is raised 100×, and HotReloader.Task.append now dedups path hashes. A new burst-write test is added and an existing stale-sourcemap test is pinned to the old 0.1 ms window via env var.

Security risks

None identified. No auth, crypto, network, or untrusted-input parsing is involved; the change is confined to local filesystem-event timing and an internal env var.

Level of scrutiny

Moderate-to-high. The watcher backends are shared by --hot, --watch, bun test --watch, and the bake dev server, so a regression here is broadly user-visible. The change also encodes a UX tradeoff (every hot reload now waits ~10 ms after the last event before firing) and replaces a simple one-shot read with a bounded loop on three different kernel APIs — each with subtly different early-return/timer-resolution semantics that the PR thread already iterated on (kqueue iteration cap, Windows <= off-by-one, timespec ≥1 s EINVAL). That's exactly the kind of cross-platform timing logic a maintainer should eyeball.

Other factors

  • The earlier macOS hot.test.ts failures and the kqueue iteration-cap bug were addressed in 8731480; CI results for that commit aren't reflected in the timeline yet.
  • One non-blocking inline comment (timespec normalization for env-var values ≥1 s) is still open.
  • The new test is intentionally timing-dependent (10×writeSync with sleepSync(2) vs a 10 ms window); the author has acknowledged they'll tighten it if CI proves it flaky, but that's a judgment call better confirmed by a human.
  • The Timeout enum removal in WindowsWatcher.zig changes a public-ish signature (next); a quick check shows no other callers, but it's another small surface-area change.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All my earlier inline concerns are addressed and I didn't find new issues, but this reworks the coalesce loop across all three watcher backends and bumps the default coalesce window 100× — worth a human sanity-check on the latency/flake tradeoff before merge.

Extended reasoning...

Overview

This PR restructures the event-coalescing logic in all three filesystem watcher backends (INotifyWatcher.zig, KEventWatcher.zig, WindowsWatcher.zig), raises the default BUN_INOTIFY_COALESCE_INTERVAL from 0.1 ms to 10 ms, adds hash dedup in HotReloader.Task.append, adds a new timing-sensitive test, and refactors an existing test to fix a pre-existing flake. The diff is ~250 lines of behavior change plus extensive comments.

Security risks

None. This is local filesystem-watcher / dev-loop tooling with no auth, crypto, network, or untrusted-input surface.

Level of scrutiny

Moderate-to-high. The change is well-reasoned and well-documented, but it:

  • Touches three independent OS-specific syscall loops (inotify/ppoll, kqueue/kevent, IOCP/GetQueuedCompletionStatus) with subtly different semantics that were already the subject of several review rounds (iteration-cap parity, timespec normalization, negative-count guard, sub-ms rounding).
  • Changes a user-visible default by 100× — every --hot/--watch reload now waits up to ~10 ms longer for the queue to go quiet. That's almost certainly the right tradeoff vs. duplicate reloads, but it's a product decision.
  • Adds a test whose correctness depends on sleepSync(2) gaps staying under the 10 ms window across CI platforms; the author explicitly chose to keep it at the production default and tighten later if CI proves it flaky.

Other factors

All prior review feedback (mine and CodeRabbit's) has been addressed in 8731480 and e508ede; all inline threads are resolved. The latest CI run on 8731480 shows only unrelated failures (s3-storage-class, http-emit-close), so the new test hasn't been disproven on macOS/Windows yet either. Given the cross-platform surface and the acknowledged residual timing sensitivity, a maintainer should sign off rather than auto-approve.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/cli/hot/hot.test.ts`:
- Around line 440-471: The initial-output wait can hang if the child process
exits before emitting the first "[#!root]" line; replace the plain await initial
with a guarded wait that races the initial promise against the child's early
exit so the test proceeds instead of hanging. Locate the initial/sawInitial pair
and the reader that consumes runner.stdout (symbols: initial, sawInitial,
reader, runner, finished) and change the wait to something like awaiting
Promise.race between initial and a promise that resolves when runner exits (or
when finished becomes true) so the test unblocks on early child termination.
- Around line 360-371: The async IIFE reading from runner.stdout currently
swallows all errors with .catch(() => {}), which hides parser/stream failures
and causes the test to hang; update the catch to propagate the error instead
(e.g., .catch(err => { /* surface */ throw err }) or store the error into a
shared readerError promise/variable that the test checks alongside evals) so any
failure in the stdout reader (the async IIFE that manipulates buffered and
pushes into evals) will fail the test immediately rather than being silently
ignored.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 76414366-f4d7-45cf-883d-3796fcdab39b

📥 Commits

Reviewing files that changed from the base of the PR and between 575f9c3 and 487b821.

📒 Files selected for processing (1)
  • test/cli/hot/hot.test.ts

Comment thread test/cli/hot/hot.test.ts
Comment thread test/cli/hot/hot.test.ts Outdated

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't find any new issues, but this touches all three platform watcher backends plus a user-facing default (0.1 ms → 10 ms coalesce window, worst-case ~320 ms reload latency under continuous writes), so it's worth a human sanity-check on the timing trade-offs and the Intel-macOS test skip.

Extended reasoning...

Overview

This PR reworks event coalescing in all three filesystem-watcher backends (INotifyWatcher, KEventWatcher, WindowsWatcher) so that the burst of kernel events produced by a single editor save collapses into one onFileUpdate call, fixing #13511 where --hot re-evaluated the entry point once per event. It also adds a dedup in HotReloader.Task.append, raises the BUN_INOTIFY_COALESCE_INTERVAL default from 0.1 ms to 10 ms, adds a new timing-sensitive regression test, and adjusts two existing tests to account for the new window.

Security risks

None. The change is confined to local filesystem-watch timing and an internal env var; no auth, crypto, network, or untrusted-input surfaces are touched.

Level of scrutiny

Medium-high. The diff is not large, but it changes platform-specific syscall loops (ppoll/kevent/GetQueuedCompletionStatus) with subtle semantics (early-return on event-ready, timespec normalization, Windows timer quantization), and it shifts a default that affects every --hot/--watch user — each reload now waits up to ~10 ms (or up to ~320 ms if writes never stop) before dispatching. The PR has already been through several correction rounds (kqueue iteration cap 5→32, count > 0 guard, sec/nsec split, divCeil rounding), which suggests the edge cases here are easy to get wrong and merit human eyes.

Other factors

All three of my earlier inline findings have been addressed and resolved. The new test is inherently timing-dependent (10× writeSync with sleepSync(2) vs a 10 ms window) and is already skipIf(isIntelMacOS); the author explained why the window can't be widened without defeating the regression gate. Two unresolved CodeRabbit nits remain on the test's stdout-reader error handling — minor, but still open. Given the cross-platform blast radius and the latency/design trade-offs (10 ms default, 32-iteration cap), this is beyond what I'd auto-approve.

@robobun

robobun commented May 13, 2026

Copy link
Copy Markdown
Collaborator Author

Status: complete; build 99936 on the current head (0a1ef9d) is green on all 10 lanes including macOS, ready for maintainer review

Bug (#13511): a single editor save in --hot mode re-evaluates the entry point 2+ times on Linux/Windows, printing duplicate output.

Root cause: editors emit several filesystem events per logical save, spread over a few milliseconds. The watcher's coalesce window was 0.1 ms (inotify/kqueue) or 0 ms (Windows), so one save split across multiple watch-loop cycles, one reload each. A second path: HotReloader.Task's fixed 8-slot buffer could overflow mid-on_file_update and enqueue an extra reload for the same save.

Fix:

  • BUN_INOTIFY_COALESCE_INTERVAL default 0.1 ms to 10 ms; all three backends read it through watcher_impl::coalesce_interval_ns() and keep draining until the queue stays quiet for that long (capped by the shared MAX_COALESCE_ITERATIONS = 32 so nonstop writers can't starve the loop).
  • Dedup path hashes in HotReloader.Task.append so repeated events for one file can't overflow the buffer into a second reload.

Verification: test/cli/hot/hot.test.ts "coalesces a burst of writes into a single reload" fails without the fix (3+ evals) and passes with it (exactly 2). Full hot.test.ts 13/13 on the current head; cargo check -p bun_watcher clean for linux/darwin/windows targets.

Latest round:

  • Build 99936 (0a1ef9d, final): 178/179 jobs passed. The darwin any aarch64 lane ran this time and passed, so the post-rebase kqueue drain loop is now CI-verified on macOS and the gap noted below is closed. hot.test.ts passed first try on every lane (it appears in neither the failure nor the flaky annotations). The one failed job is test/bake/deinitialization.test.ts on a single asan shard: a heap-stats GC-liveness assertion whose fixture never touches files or the watcher; it passes 5/5 locally on this branch's ASAN build and has been reported for main-break triage. Four other unrelated tests flaked yellow (passed on retry).
  • Rebase 8 onto main at f492595: the only conflict was an import block where share one sort instance across cold rust sort sites #39324 added index_sort next to this PR's widened watcher_impl import; the PR's content is unchanged. Verified after the rebase: cargo check -p bun_watcher clean on linux/darwin/windows targets, hot.test.ts 13/13, burst test 5/5.
  • Build 98259 (sha 917b689) was green except that the new burst test flaked once on two Linux aarch64 lanes (passed on retry) with the stretched-burst signature (evals: [1,2,3] / [1,2,3,4]): the runner stretched the writer's 2 ms gaps past the 10 ms window, so the writes really were separate saves. e4d0890 makes the test measure its own gaps and retry a burst that was both stretched and split; a split burst with gaps under 8 ms still fails and a coalesced one passes at once. On a box at load average 100 to 170 the unfixed build fails 8/8 and the fixed build passes 10/10, and the full file is 13/13. On CI, build 98412 (e4d0890) passed all 6 Linux lanes (120 shards, including the two aarch64 lanes that had flaked) and both Windows lanes with the retrying test needing no retries; its only flaky entry was an unrelated Windows test-cluster-shared-leak.js timeout that passed on retry. The build's one macOS lane (darwin 14 aarch64, 2 shards) was never picked up by an agent in the ~110 minutes before the build was canceled, so it has no macOS result. Consequence worth knowing: the kqueue backend as it stands after rebase 7 (redone over watcher(kqueue): drop per-cycle Vec, two-phase init, and raw libc::kevent #35321's safe kevent wrapper) is cargo checked for aarch64-apple-darwin but has not been exercised by CI yet; the pre-rebase version of the same loop passed the darwin lanes in every earlier build. Please re-run the build when the darwin queue is serving again.
  • Earlier this round: rebase 7 (kqueue loop redone over watcher(kqueue): drop per-cycle Vec, two-phase init, and raw libc::kevent #35321's safe kevent wrapper), defaults hoisted into watcher_impl, comments collapsed to one line each. Details in the PR description.

Review threads: all bot threads are replied to and resolved; none open.

CI history: no watcher/hot-reload test has failed on any platform in any run of this PR. Earlier red was main-branch breakage (test-worker-message-port-transfer-terminate, test-net-connect-memleak, the bunx @angular/cli@latest registry break, all failing identically on unrelated PRs) plus known single-lane flakes and darwin artifact-download timeouts.

Comment thread test/cli/hot/hot.test.ts Outdated
@robobun

robobun commented May 14, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased onto main (post-Rust-rewrite, resolved the hot.test.ts conflict — main already has the random-file deflake) and ported:

  • src/bun_core/env_var.rs: BUN_INOTIFY_COALESCE_INTERVAL default 100 µs → 10 ms
  • src/jsc/hot_reloader.rs: Task::append dedups the hash before pushing
  • src/watcher/INotifyWatcher.rs: DEFAULT_COALESCE_INTERVAL_NS / MAX_COALESCE_ITERATIONS constants; drain-until-quiet loop with sec/nsec split for ppoll
  • src/watcher/KEventWatcher.rs: coalesce_interval_ns field read from env var in init; iteration-capped drain loop with extra <= 0 break
  • src/watcher/WindowsWatcher.rs: coalesce_interval_ms field with ns→ms div_ceil conversion; Timeout enum replaced with raw w::DWORD; iteration-capped outer loop

rust:check-all clean on all targets. Gate verified: without the .rs change a 10-write burst triggers 4 reloads ([1,2,3,4]); with it, one ([1,2]). All 13 hot.test.ts tests pass. .zig kept as reference.

Note: this and #30644 both touch the same watcher files; whichever lands second will need a trivial merge.

@robobun
robobun force-pushed the farm/1f3aa35a/hot-coalesce-watcher-events branch 2 times, most recently from 0de4f5d to cd15cac Compare May 21, 2026 09:54
@robobun
robobun force-pushed the farm/1f3aa35a/hot-coalesce-watcher-events branch from 6c5f2ab to e77c065 Compare May 24, 2026 06:21

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found on this revision, but this touches all three platform watcher backends (inotify/kqueue/ReadDirectoryChangesW, in both Zig and Rust) plus cross-thread hot-reload scheduling, and changes the user-facing --hot reload latency default — worth a human look before merging.

Extended reasoning...

Overview

This PR rewrites the event-coalescing logic in all three filesystem watcher backends (INotifyWatcher, KEventWatcher, WindowsWatcher) — in both the Zig originals and the newly-ported Rust implementations — replacing a single fixed extra read with a bounded drain-until-quiet loop. It also raises the BUN_INOTIFY_COALESCE_INTERVAL default from 0.1 ms to 10 ms, adds hash deduplication in HotReloader.Task.append to prevent mid-onFileUpdate flushes, adds a timing-based regression test, and pins an existing test to the old interval via env var.

Security risks

None. No auth, crypto, network, or untrusted-input parsing surfaces are touched. The only external input is the BUN_INOTIFY_COALESCE_INTERVAL env var, which is bounds-handled (sec/nsec split, divCeil, iteration cap) and only affects local watcher timing.

Level of scrutiny

High. This is core runtime infrastructure for --hot/--watch mode, with platform-specific syscall semantics (ppoll, kevent, GetQueuedCompletionStatus) that differ subtly in early-return and timer-resolution behaviour. The change introduces a deliberate ~10 ms latency floor on every hot reload — a UX tradeoff a maintainer should sign off on. The Task.append dedup interacts with the pending_count atomic and the while-swap loop in Task.run across the watcher and JS threads. The PR has gone through several rounds of bot feedback (iteration-cap parity, count > 0 guard, timespec normalization, ms rounding) and a Rust port mid-review; a human should confirm the Zig and Rust paths remain semantically equivalent and that the new test's timing margin is acceptable for CI.

Other factors

All prior inline review threads (mine and CodeRabbit's) are resolved. The new test is skipIf(isIntelMacOS) and relies on a 2 ms write gap vs a 10 ms coalesce window, which the author has acknowledged as a deliberate margin against the production default. The author also flagged a known merge interaction with #30644. Given the breadth (11 files, 6 platforms × language combinations) and the design decision embedded in the new default, this exceeds the bar for bot-only approval.

@robobun
robobun force-pushed the farm/1f3aa35a/hot-coalesce-watcher-events branch 2 times, most recently from 692e279 to 802d7e8 Compare June 5, 2026 15:35

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found in this pass, but this rewrites the coalesce loop across all three platform watcher backends (inotify/kqueue/IOCP, in both the Zig reference and the shipping Rust) and changes a user-visible timing default — worth a human look before merging.

Extended reasoning...

Overview

This PR changes how the filesystem watcher coalesces rapid event bursts so that a single editor save triggers exactly one --hot reload instead of several. It touches eleven files: the three platform watcher backends (INotifyWatcher, KEventWatcher, WindowsWatcher) in both the Zig reference and the shipping Rust port, hot_reloader.{zig,rs} (hash dedup in Task.append), env_var.{zig,rs} (default BUN_INOTIFY_COALESCE_INTERVAL 0.1 ms → 10 ms), and test/cli/hot/hot.test.ts (new regression test plus pinning an existing test to the old interval).

Security risks

None identified. The change only adjusts how long the watcher thread waits for additional kernel events after the first read and adds a linear-scan dedup over an 8-element array. No new external input is parsed, no auth/crypto/permissions surface is touched, and the env var was already user-controllable. The drain loops are bounded by both byte count and a 32-iteration cap, so a continuously-written file cannot starve the watch loop.

Level of scrutiny

This warrants human review rather than auto-approval. It is a behaviour change in a production-critical, cross-platform code path that every --hot/--watch user exercises, implemented twice (Zig + Rust) across three OS-specific syscall loops with subtly different semantics (e.g. kevent early return vs ppoll timeout, Windows ~15.6 ms timer granularity). It also embeds design decisions a maintainer should sign off on: the 10 ms default window (trades ~10 ms of reload latency for dedup), the 32-iteration cap, and reusing the Linux-named env var for all backends. The PR went through several correctness iterations during review (kqueue iteration cap 5→32, count > 0 guard, timespec sec/nsec split, ns→ms divCeil, an accidental full src/ revert), which is normal for this kind of change but underscores that it is not mechanical.

Other factors

All prior inline review threads (CodeRabbit and earlier claude runs) are resolved, and the current bug-hunting pass found nothing. The new test is well-commented and has a verification table showing it fails without the fix and passes with it, but it is inherently timing-dependent (10× writeSync with sleepSync(2) gaps vs a 10 ms window) and is already skipIf(isIntelMacOS); the author has explicitly said they will tighten the burst rather than the window if a CI lane proves it flaky. The author also flagged a likely merge conflict with #30644. None of these are blockers, but together with the scope they make this a clear case for a human reviewer rather than bot approval.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No new issues found on the Rust port. This still warrants a maintainer look: it rewrites the coalesce/drain loop in all three platform watcher backends (inotify/kqueue/IOCP) and changes the default --hot reload latency from ~0.1 ms to ~10 ms per save — a user-visible tradeoff worth a human sign-off.

What was reviewed:

  • Drain-loop bounds on all three backends — iteration cap + buffer/count guards prevent starvation; count > 0 guards the kqueue negative-cast.
  • timespec sec/nsec split for ≥1 s overrides and Windows ns→ms div_ceil rounding — both correct.
  • Task::append dedup — linear scan over ≤8 slots; only consumer that reads the hash list (bake dev server) is unaffected by dedup.
  • INotifyWatcher control-flow reshape — the removed break 'outer after the inner read is replaced by falling through to the post-loop break 'outer read_len; no behavior lost.
Extended reasoning...

Overview

This PR rewrites the event-coalescing logic in all three platform filesystem-watcher backends (INotifyWatcher.rs / Linux inotify, KEventWatcher.rs / macOS kqueue, WindowsWatcher.rs / ReadDirectoryChangesW+IOCP), adds a hash-dedup guard in hot_reloader.rs's Task::append, raises the BUN_INOTIFY_COALESCE_INTERVAL default from 100 µs to 10 ms in env_var.rs, and adds a regression test plus a BUN_INOTIFY_COALESCE_INTERVAL=0 pin on an existing timing-sensitive test in hot.test.ts. Every prior inline finding (kqueue iteration cap, negative-count guard, timespec ≥1 s normalization, Windows sub-ms truncation, the accidental src/ revert) has been addressed and the threads are resolved.

Security risks

None identified. No untrusted-input parsing, auth, crypto, or network surface is touched. The changed code reads kernel-produced event buffers on a dedicated watcher thread; buffer offsets are advanced by kernel-supplied lengths as before. The env var is an internal tuning knob and its value is bounded by isize::try_from / try_into fallbacks.

Level of scrutiny

Medium-high. The mechanism is conceptually simple (poll-drain until quiet, capped), but it is implemented three times against three different OS APIs with unsafe FFI, and it changes a user-visible default: every --hot/--watch reload now waits ~10 ms after the last event before dispatching (worst case ~320 ms if writes arrive continuously and hit the 32-iteration cap). That latency-vs-duplicate-reload tradeoff, and whether 10 ms / 32 iterations are the right constants, is a product decision a maintainer should ratify. The Windows change also removes the Timeout enum in favor of a raw DWORD.

Other factors

  • The new test is inherently timing-based (2 ms sleepSync gaps vs a 10 ms window) and is already skipIf(isIntelMacOS); the author explicitly deferred further tightening to CI evidence. This is a reasonable call but flags residual flake risk on loaded runners.
  • The Task::append dedup is O(n) over ≤8 entries and the PR states VirtualMachine.reload ignores the hash list; only bake's dev server consumes it, and receiving one hash instead of N duplicates is semantically equivalent there.
  • Six rebases including a full Zig→Rust port; the author's status comment reports rust:check-all clean and the full hot.test.ts suite passing locally, with no watcher/hot-reload CI failures across builds. CI on the current head (ea1fb40) is still building per the robobun status comment.
  • No human reviewer has weighed in yet; given the cross-platform syscall surface and the default-behavior change, deferring rather than approving.

Comment thread src/watcher/KEventWatcher.rs Outdated
Comment thread src/watcher/KEventWatcher.rs Outdated
Comment thread src/watcher/WindowsWatcher.rs Outdated
Comment thread src/watcher/WindowsWatcher.rs Outdated
Comment thread src/watcher/WindowsWatcher.rs Outdated
Comment thread src/watcher/WindowsWatcher.rs Outdated
Comment thread src/watcher/WindowsWatcher.rs Outdated
Comment thread src/watcher/KEventWatcher.rs Outdated
Comment thread src/watcher/INotifyWatcher.rs Outdated
Comment thread src/bun_core/env_var.rs Outdated
Comment thread src/jsc/hot_reloader.rs Outdated
Comment thread src/watcher/INotifyWatcher.rs Outdated
Comment thread src/watcher/Watcher.rs Outdated
Comment thread src/watcher/Watcher.rs Outdated
Comment thread src/watcher/WindowsWatcher.rs Outdated
Comment thread src/watcher/WindowsWatcher.rs Outdated

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both earlier notes are addressed — the coalesce interval and iteration cap now live once in watcher_impl (single source of truth via the env-var default), and the rationale comments are collapsed to one line per site. I didn't find anything further; the drain loops in all three backends look correct and bounded, the Task::append dedup is sound, and BUN_INOTIFY_COALESCE_INTERVAL=0 restores the old behaviour for the sourcemap-race test. Leaving the final call to a maintainer since this changes a user-facing default (0.1 ms → 10 ms reload latency) across three platform watcher backends and adds a timing-sensitive test.

The unresolved comment-cop flags from 15:23 were on ad93e40; 917b689 collapses those blocks.

Extended reasoning...

Overview

The PR fixes #13511 (one editor save triggering multiple --hot reloads) by (1) raising the BUN_INOTIFY_COALESCE_INTERVAL default from 100 µs to 10 ms, (2) replacing the single-shot extra read in each watcher backend with a bounded drain-until-quiet loop, and (3) deduping repeated path hashes in HotReloader.Task::append so a burst for one file cannot overflow the 8-slot buffer into a mid-on_file_update enqueue. Shared constants (coalesce_interval_ns(), MAX_COALESCE_ITERATIONS, coalesce_timespec()) now live in watcher_impl so the three #[cfg]-gated backends read one source of truth.

What changed since my last review

ad93e40 hoisted the per-backend DEFAULT_COALESCE_INTERVAL_* and MAX_COALESCE_ITERATIONS constants into Watcher.rs and dropped the dead .unwrap_or(...) fallbacks (the env var declares a default, so .get() is always Some; .expect() documents that). 917b689 collapsed the multi-paragraph coalesce rationale to one line per site with a #13511 link. Both address exactly what I raised at 14:56. The 15:23 comment-cop round targets ad93e40's intermediate state and is stale on the current head.

Security risks

None. No untrusted-input parsing, no auth/crypto, no new syscall surface. The env var is already user-controllable and was already read; only its default changed. Overflow is not possible: the interval is stored as u64 ns and split via /+% into a timespec (so tv_nsec < 1e9 always holds), and Windows uses div_ceil + try_from().unwrap_or(INFINITE) so absurd values clamp rather than wrap.

Level of scrutiny

Medium-high. This is a behavioural change to concurrent watcher-thread code across all three platform backends (inotify, kqueue, ReadDirectoryChangesW), and it changes a user-visible default — every --hot/--watch reload now waits up to ~10 ms longer. That is almost certainly the right trade (it fixes a real, long-standing double-reload complaint), but it's a product decision a maintainer should sign off on. The new burst test is timing-based by construction and the PR description already documents it flaking at load-average ~200; it's skipped on Intel macOS for the same reason.

Correctness spot-checks

  • INotifyWatcher: rest is re-borrowed at the top of each outer iteration with the updated read_len; the rest.len() < Event::LARGEST_SIZE guard prevents a short read that could split an event; ppoll returning ≤0 (timeout or EINTR) breaks cleanly with the events already accumulated.
  • KEventWatcher: Ok(0) | Err(_) => break means a failed drain poll never discards the events already in changelist[..count]; the loop is bounded by CHANGELIST_COUNT and the iteration cap.
  • WindowsWatcher: iterations <= MAX_COALESCE_ITERATIONS gives 1 blocking + 32 coalesce sweeps; div_ceil keeps a sub-ms override (e.g. the test's 0) from silently becoming a busy 0 ms while still yielding 0 for exactly-0. The removed Timeout enum was a two-value #[repr(u32)] that only ever cast to DWORD, so the raw DWORD is equivalent.
  • Task::append dedup: linear scan over ≤8 entries; VirtualMachine::reload ignores the hash list and the only other consumer (bake dev server) treats it as a set, so dedup does not change semantics.
  • BUN_INOTIFY_COALESCE_INTERVAL=0: yields {tv_sec:0, tv_nsec:0} / 0 ms, restoring pre-PR immediacy for the stale-sourcemap test that depends on the self-write racing the rejection report.

Other factors

The PR has been rebased seven times, has extensive prior CodeRabbit review, and CI has passed hot.test.ts on every lane per the status comments. No CODEOWNERS cover these paths. Given the cross-platform surface and the default-latency change, I'm deferring rather than approving.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I re-reviewed after ad93e40/917b689/e4d08901 and didn't find any bugs — the constant-hoisting and comment-collapse addressed everything I'd flagged, and the comment-cop threads are now clear. Because this raises a user-visible default 100× (0.1 ms → 10 ms coalesce, adding that latency to every --hot reload) and rewrites the drain loop on all three platform backends, I'd still like a human to sign off on the tradeoff.

Checked: coalesce_interval_ns() .expect is unreachable (env var declares a default so .get() is always Some); coalesce_timespec splits u64 ns so tv_nsec < 1e9 always holds; Windows <= bound gives 1 blocking + 32 coalesce sweeps, matching the < bound in the inotify/kqueue loops; Task::append dedup only touches hashes (VM reload ignores it, DevServer sees a deduped set); the kqueue drain's Err(_) => break preserves already-read events instead of discarding them.

Extended reasoning...

Overview

The PR touches the three platform watcher backends (INotifyWatcher.rs, KEventWatcher.rs, WindowsWatcher.rs), the shared Watcher.rs (new coalesce_interval_ns() / coalesce_timespec() / MAX_COALESCE_ITERATIONS), hot_reloader.rs (Task::append hash dedup), the BUN_INOTIFY_COALESCE_INTERVAL default in env_var.rs (100 µs → 10 ms), and hot.test.ts (new burst-coalesce test plus BUN_INOTIFY_COALESCE_INTERVAL=0 pinned on the stale-sourcemap test). The Windows Timeout enum is removed in favour of a raw DWORD.

Security risks

None. No untrusted input parsing; the only external input is the env var, already routed through the existing env_var unsigned parser with DebugWarn on error. No new syscalls beyond the existing ppoll/kevent/GetQueuedCompletionStatus in a bounded loop.

Level of scrutiny

High. This is production hot-reload/watch-mode code that every --hot/--watch user runs, on three OS-specific backends that CI cannot fully cross-verify from a single lane. The 100× default bump is a behavioural tradeoff (fewer duplicate reloads vs +~10 ms latency per reload) — reasonable, but a maintainer should ratify it. The new test is timing-based by construction (skipped on Intel macOS, retries up to 8× when sleepSync(2) is stretched past 8 ms, uses a fixed 200 ms settle window), which is exactly the shape REVIEW.md asks reviewers to scrutinise.

Other factors

All my prior inline feedback (per-backend constant duplication; multi-paragraph change-narration comments) was addressed in ad93e40 and 917b689, and every comment-cop / CodeRabbit thread is resolved. The bug-hunting pass on the current head found nothing. rust:check-all is reported clean across targets and hot.test.ts passes on CI Linux/Windows per the PR body. Given the scope (3-platform loop rewrite + default change + timing-sensitive test) this doesn't meet the "simple, mechanical, obvious" bar for auto-approval, so I'm deferring rather than approving.

robobun and others added 10 commits August 17, 2026 08:45
A single editor save typically emits several filesystem events a few
milliseconds apart (truncate+write, plus matching events on the
parent-directory watch). The watcher's coalesce window was 0.1 ms and
performed at most one extra read, so most of those events landed in
separate watch-loop cycles and --hot re-evaluated the entry point once
per cycle — the user saw their script's output repeated for one save.

- INotifyWatcher/KEventWatcher/WindowsWatcher: after the first read,
  keep draining until the queue stays quiet for ~10 ms (bounded by
  byte/iteration caps so a continuously-written file cannot starve the
  loop). Raise the default BUN_INOTIFY_COALESCE_INTERVAL to 10 ms.
- hot_reloader.Task.append: dedup by hash so the many directory-watch
  events that all name the same file don't overflow the 8-slot buffer
  and flush mid-onFileUpdate, which let the JS thread start a reload
  while the watcher thread was still appending and produced a second
  reload for the same save.

Pin the stale-sourcemap regression test to the old 0.1 ms interval: it
depends on the self-write event being dispatched before the rejection
is reported, which the new window absorbs.

Fixes #13511
The stale-sourcemap test pins the coalesce window to 0.1 ms so the
self-write event reaches the JS thread inside the reject→report
window it was written to exercise. That override was only read by the
inotify backend, so on macOS/FreeBSD/Windows the test ran with the new
10 ms window and no longer opened the race it guards.

Have the kqueue and Windows watchers read the same env var (the Windows
backend rounds to milliseconds) so the override works uniformly.
With a non-zero coalesce interval the drain loop can race the
self-rewriting hot file in a way that loses the pending rejection
(the pre-existing edge case noted in #29740), which on release
builds manifests as a 30 s timeout. A zero interval makes the
drain loop non-blocking — it polls once and processes whatever
is already buffered — which is the closest analogue to the
pre-loop behaviour this test was tuned against, and keeps the
self-write event landing in the reject→report window on all
build profiles.
Intel macOS CI runners stretch sleepSync(2) well past the 10 ms
coalesce window (timer coalescing + scheduler load), so the 10-write
burst splits into several watch-loop cycles there and the test sees
evals=[1..9] instead of [1,2]. arm64 macOS (14 and 26) and all Linux
lanes pass; the reported bug (#13511) was Windows + Linux only, and
the issue's own comments note it does not reproduce on macOS.
The env var declares the default, so the per-backend DEFAULT_* constants
and their unwrap_or fallbacks were dead; read it once via
coalesce_interval_ns() and store the interval as u64 so the timespec split
cannot overflow. MAX_COALESCE_ITERATIONS and the timespec helper now live
once in watcher_impl. Trim the duplicated rationale comments down to a
single doc comment on coalesce_interval_ns.
The burst test measures the gaps between its writes. On a loaded runner
sleepSync(2) can stretch past the 10 ms coalesce window, and writes that
far apart are separate saves, so a burst that was both stretched and split
is retried instead of counted against the watcher. A split burst whose
gaps stayed inside the window still fails, and a coalesced burst passes
immediately, so the unfixed build still fails every run.
@robobun
robobun force-pushed the farm/1f3aa35a/hot-coalesce-watcher-events branch from e4d0890 to 0a1ef9d Compare August 17, 2026 08:52

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the earlier feedback — the constants are now single-sourced in watcher_impl and the comments are down to one line each. This pass found no bugs. Given it changes a user-visible default (adds ~10 ms to every --hot/--watch reload), rewrites the drain loop in all three platform watcher backends, and lands an inherently timing-based test with a retry loop, a human sign-off would still be worthwhile.

What was reviewed:

  • coalesce_interval_ns()'s .expect — the env var declares a default, so .get() is always Some.
  • coalesce_timespec sec/nsec split and the Windows div_ceilunwrap_or(INFINITE) conversion for overflow.
  • Windows <= bound is intentional (INFINITE wait is iteration 0) and matches the other backends' 32 drain sweeps.
  • KEvent's Err(_) => break on the drain poll keeps already-read events; the initial blocking call still propagates via ?.
Extended reasoning...

Overview

Fixes #13511 by widening the filesystem-watcher coalesce window from 0.1 ms to 10 ms and turning the single extra read into a bounded drain-until-quiet loop across all three backends (INotifyWatcher, KEventWatcher, WindowsWatcher). Also dedups repeated hashes in HotReloader.Task.append so a per-save burst can't overflow the 8-slot buffer into a mid-on_file_update enqueue(). Shared helpers (coalesce_interval_ns, coalesce_timespec, MAX_COALESCE_ITERATIONS) now live once in watcher_impl. One existing test pins BUN_INOTIFY_COALESCE_INTERVAL=0 to preserve the race it depends on; a new test asserts a 10-write burst coalesces to one reload.

Security risks

None identified. No untrusted-input parsing, auth, or crypto is touched. The env var is already user-controllable; the change only raises its default.

Level of scrutiny

Medium-high. The mechanism is straightforward, but it (a) changes a user-facing default that adds ~10 ms latency to every --hot/--watch/bake-dev-server reload, (b) touches three #[cfg]-gated platform backends where only one is exercised on any given CI lane, and (c) ships a test that is timing-based by construction — it retries up to 8 times when sleepSync(2) gaps stretch past 8 ms, uses a 200 ms settle sleep, and is skipped on Intel macOS. REVIEW.md is explicit about timing-based tests, and the PR description itself notes the test can legitimately observe split bursts on heavily loaded boxes. That combination is not something I'm comfortable auto-approving without a maintainer weighing in on the default and the test's flake budget.

Other factors

My two prior 🟡 nits (2026-08-15: quadruplicated default constant + dead unwrap_or fallbacks; multi-paragraph change-narration comments across 12 sites) were fully addressed in ad93e40/917b689 — the diff now matches what I suggested. All comment-cop threads and CodeRabbit threads are resolved. The Windows Timeout enum removal is a pure simplification (both variants were literal DWORD values). The INotify inner-loop reshaping (break 'inner instead of break 'outer) correctly re-borrows rest at the new read_len on each outer iteration. No human maintainer has reviewed this PR yet — every timeline entry is from a bot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Duplicate Output Issue After Repeated Saves with bun --hot

1 participant