install: retry cache publish renames on Windows while a scanner holds a file open - #35568
install: retry cache publish renames on Windows while a scanner holds a file open#35568robobun wants to merge 8 commits into
Conversation
|
Warning Review limit reached
Next review available in: 9 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (8)
Comment |
|
Updated 11:43 PM PT - Aug 12th, 2026
❌ @robobun, your commit 2172ab4 has some failures in 🧪 To try this PR locally: bunx bun-pr 35568That installs a local version of the PR into your bun-35568 --bun |
There was a problem hiding this comment.
Thanks for addressing the earlier nits (Bun.$ for tar, 15s comment). The Rust change itself is minimal — two constants in the existing Windows retry loop, adopting SQLite's winIoerrRetry schedule — and looks correct. Deferring to a human mainly for the new Windows-only test: it coordinates a stalling registry, an FFI-based handle-holder subprocess, and stdout signaling, which is intricate enough that a maintainer should confirm it lands green (and non-flaky) on Windows CI before merge.
The open comment-cop flag on extract_tarball.rs:557 looks like a false positive — the 4-line comment is the same length as the one it replaces and carries the mechanism + schedule citation, which is the durable content REVIEW.md asks for.
What was reviewed
- Retry-loop change: same errno set (
PERM/BUSY/EXIST/NOTEMPTY), onlyMAX_RETRIES4→10 and backoff10<<(n-1)→25*n; no overflow, error path unchanged. - Test env knobs: confirmed
BUN_INSTALL_STREAMING_MIN_SIZEandBUN_TMPDIRare read bysrc/; test assertsHELDso a pass can't be vacuous if the blocker misses the window. - Cleanup:
using/await usingon tempdir, server, both subprocesses;held/readyare resolved on reader EOF so the server pull can't hang the teardown.
Extended reasoning...
Overview
The PR extends the Windows-only retry budget in src/install/extract_tarball.rs when renaming a freshly extracted temp directory into the package cache. It changes MAX_RETRIES from 4 to 10 and the per-retry sleep from exponential 10 << (n-1) ms (~150ms total) to linear 25 * n ms (~1375ms total), matching SQLite's winIoerrRetry. The retried errno set, the destination-swap fallback, and the final error message are all unchanged. It adds a Windows-only regression test (test/regression/issue/11250.test.ts) plus an FFI fixture (test/cli/install/bun-install-windows-locked-temp-fixture.ts) that opens the extracted file via CreateFileW without FILE_SHARE_DELETE to deterministically reproduce the AV-scanner race.
Security risks
None identified. No new inputs are parsed; the change only lengthens an existing bounded retry-and-sleep loop inside #[cfg(windows)]. The test uses a local Bun.serve({ port: 0 }) registry and a temp cache/tmp dir — no external network, no privilege changes.
Level of scrutiny
The production diff is ~10 lines of constant/comment changes in an already-retrying loop and is low-risk in isolation. The test, however, is ~275 lines across two new files and coordinates three moving parts (a mid-response-stalling HTTP server, a busy-polling FFI subprocess that grabs a Win32 handle, and bun install itself) via stdout token parsing. That's the kind of Windows-CI-only harness where flakiness only shows up on the actual runners, so it warrants a maintainer eye on the first CI run rather than a bot approval.
Other factors
- My two earlier nits (use
Bun.$instead ofspawnSyncfor tar; fix the 30s→15s doc mismatch in the fixture header) were both addressed in commit 0d7eee8 and the threads are resolved. - I verified the env vars the test relies on are real:
BUN_INSTALL_STREAMING_MIN_SIZEis defined insrc/bun_core/env_var.rs:97and consumed insrc/install/TarballStream.rs:179;BUN_TMPDIRis read inPackageManagerDirectories.rs. The test assertsblockerOutcontainsHELD, so if the fixture fails to catch the extraction window the test fails rather than vacuously passing. - Cleanup paths look sound:
using tempDir,await using server,await usingon both subprocesses, and the background stdout reader resolves bothreadyandheldon EOF so the server's stalledpull()can't wedge teardown. - The unresolved comment-cop inline on
extract_tarball.rs:557is a mechanical multi-line-comment lint on claude PRs; the new comment is no longer than what it replaces and documents the NTFS invariant plus the SQLite schedule source, which is exactly the "durable non-obvious content" REVIEW.md calls for. A human can dismiss it, but I didn't want to approve over an open bot flag.
There was a problem hiding this comment.
No issues found in this pass. The retry-loop change itself is small and the earlier nits are all addressed, but I'd like a human to sign off on two things: (1) the new user-facing env var BUN_INSTALL_WIN32_AV_RETRY_MS and its 5-second default — a genuine EPERM now surfaces after ~5s instead of ~150ms, which is a UX/design call; and (2) the Windows-only test, which is elaborate (FFI to CreateFileW, a stalling registry, streaming coordination) and only exercised on Windows CI — build #80760 was still running at review time.
What was reviewed:
.get().unwrap()on the new env var is safe — it hasdefault: 5_000, soget()is alwaysSome.- Retry errno set (NOTEMPTY/PERM/BUSY/EXIST) and the error path are unchanged; only the loop bound moved from a 4-count to
start.elapsed() < budget. - On the AV path the per-iteration
renameat(cache_dir, folder_name, …)targets a non-existent destination, so ~50 retries don't repeatedlydelete_tree. - Test requires
HELDin the blocker output, so a pass where the fixture missed the extraction window fails rather than silently passing.
Extended reasoning...
Overview
The PR extends the Windows-only retry loop in src/install/extract_tarball.rs that renames a freshly extracted temp directory into the package cache. It swaps a fixed 4-attempt exponential backoff (~150ms total) for a deadline-based loop: linear backoff of 10ms increments capped at 100ms, retried until BUN_INSTALL_WIN32_AV_RETRY_MS (new env var, default 5000ms) elapses. A one-line env-var declaration is added to src/bun_core/env_var.rs. Two new test files add a Windows-only regression test that simulates an AV scanner holding an extracted file open without FILE_SHARE_DELETE via bun:ffi → kernel32!CreateFileW, with a local registry that stalls mid-tarball until the handle is held.
Security risks
None identified. The change is confined to #[cfg(windows)] install-cache rename retry logic; no new inputs are parsed, no auth/crypto/path-validation is touched. The env var goes through the existing new!(unsigned) machinery which parses via bun_core::fmt::parse_int and falls back to the default on parse failure.
Level of scrutiny
Medium. The production-code delta is small (~15 lines) and mechanical — the errno match arms, error message, and cleanup path are unchanged; only the loop bound changed from a counter to Instant::elapsed() < budget. However, it introduces a new user-facing env var (API surface) and picks a default that trades diagnostic latency (genuine EPERM now takes ~5s to surface) for AV robustness. That default is a design call the PR justifies well (between SQLite's 1.375s and graceful-fs's 60s), but a maintainer should confirm it. The test is intricate — FFI, a stalling ReadableStream registry, subprocess stdout coordination — and only runs on Windows CI, which hadn't reported at review time.
Other factors
All three of my prior nits (use Bun.$ for tar, fixture header 30s→15s, PR description out of sync) are resolved and marked as such; the description now matches the shipped code. The bug-hunting system found nothing this run. The comment-cop bot flagged the long inline comment twice; it's now collapsed to a single long line, which satisfies the rule if not readability. The test asserts blocker: expect.stringContaining("HELD") in the combined-object assertion, so a run where the fixture failed to grab the handle in time would fail the test rather than vacuously pass — good. BUN_INSTALL_STREAMING_MIN_SIZE: "1" forces the streaming extractor so bin.exe is on disk before the registry stall; the fixture's tryOpenNoShareDelete failing on directories (it will hit package.json or bin.exe) is handled by looping over inner until a handle succeeds. Given the new env var and the pending Windows CI run, I'm deferring rather than approving.
|
CI on build #80760: the diff is green. All Windows 11 aarch64 test shards passed (including the new The previous build #80391 hit the same "step failed outside runner" build coordination issue across most lanes, which Ready for a maintainer to look at; the Windows x64 lane may need a manual retry. |
alii
left a comment
There was a problem hiding this comment.
Requesting changes.
- Merged as-is the new test file fails on every platform: it imports stderrForInstall, which main removed in #37000.
- The retry covers the tarball cache move only. patchedDependencies renames into the cache through the same call with no retry, and the env var does not reach it.
- The test needs a case that shows the held handle blocks the rename at all and exercises BUN_INSTALL_WIN32_AV_RETRY_MS.
- Smaller: nothing says where 5s comes from, and the error a user sees after the budget runs out does not name the knob.
…to cache When antivirus / Search Indexer / MDM agents open a freshly extracted file for scanning without FILE_SHARE_DELETE, NTFS fails the rename of the containing directory with STATUS_ACCESS_DENIED. The existing retry handled this case but with only 150ms of total backoff, which is shorter than a typical scanner hold. Extend the retry to 10 attempts with linear backoff (25ms increments, ~1.4s total), matching SQLite's winIoerrRetry which is tuned for exactly this class of interference. Fixes #11250
…ve test to cli/install
…shes Extracted tarballs, patched packages and global virtual store entries are all published into the cache with a directory rename, and on Windows all three fail with STATUS_ACCESS_DENIED while a scanner holds a file inside the directory open. Move the retry into cache_rename::RenameRetry and use it at every site; the budget is BUN_INSTALL_WINDOWS_RENAME_RETRY_MS (default 5s) and the error reported once it runs out names the variable. The global store publish used to treat EPERM as a collision with a concurrent install, deleting the held staging dir and reporting success; it now only does so when the destination actually exists.
2bf096c to
9ee07e0
Compare
|
@alii reworked per your review, at 2172ab4:
Description rewritten to match. |
alii
left a comment
There was a problem hiding this comment.
Looks ready to merge from this side. Checked 2172ab4 against main: the stale harness import is gone, RenameRetry wraps the tarball move, patch_install's renameat_concurrently and both renames plus the --force swap-out in commit_global_store_entry, is_rename_collision no longer turns a held staging dir into a silent success, the budget default and the exhausted error name the variable, and the rewritten test has a budget=0 case per publish that fails on main. The three publishes are the only fatal cache renames (the manifest cache rename in npm.rs is best-effort). Nothing on main touched these files since the merge base and it merges clean. The Windows runs and the windows-msvc cargo check in the description are not reproduced here.
Fixes #11250
Problem
bun installon Windows fails while publishing a package into the cache:STATUS_ACCESS_DENIEDwhile any process holds a handle withoutFILE_SHARE_DELETEon a file inside it, and that is how scanners open freshly written files. bun's own handles requestFILE_SHARE_DELETE; the blocking handle is the scanner's.extract_tarball.rs) retried, but for 4 attempts totalling 150ms of sleep, shorter than a scan of a multi-MB binary. The patched-package one (patch_install.rs, reported asrenaming changes to cache dir: EPERM) did not retry. The global virtual store one (isolated_install/Installer.rs::commit_global_store_entry) treated anyEPERMon Windows as "a concurrent install published first", deleted its own staging dir and returned success, so the install exited 0 withnode_modules/.bun/<pkg>pointing at a directory that does not exist.Fix
bun_install::cache_rename::RenameRetry: a deadline (BUN_INSTALL_WINDOWS_RENAME_RETRY_MS, default 5000) with graceful-fs's backoff (10ms more per attempt, capped at 100ms).is_transientisEPERM/EACCES/EBUSYon Windows and never true on POSIX, where open handles do not block renames and those errnos are real failures.--forceswap-out rename in the global store path) loop on it. The tarball path keeps retryingEXIST/NOTEMPTYas before, since that arm also covers concurrent installs.is_rename_collisionnow only countsEPERMas a collision when the destination actually exists, which is what makes the global store path retry instead of silently discarding the package.test/cli/install/bun-install-windows-rename-retry.test.ts(Windows only; the changed behaviour is unreachable elsewhere and the file skips). It covers each of the three publishes twice against a fixture that opens an extracted file throughCreateFileWwithoutFILE_SHARE_DELETE: with the default budget the install must outlast a 2s hold, and withBUN_INSTALL_WINDOWS_RENAME_RETRY_MS=0it must fail immediately and name the variable, which also shows the held handle is what blocks the rename. For the tarball case the registry withholds the second half of the tarball until the fixture reports it has the handle.cargo clippy -p bun_install -p bun_coreandcargo check -p bun_install --target x86_64-pc-windows-msvcpass.Background
linker = "isolated"andBUN_INSTALL_GLOBAL_STORE=1, each package'snode_modulestree is built in<cache>/links/<entry>.tmp-<pid>and renamed to<cache>/links/<entry>; a project'snode_modules/.bun/<pkg>is a symlink to that entry. Two installs racing on the same entry is expected and the loser keeps the winner's copy, which is the collision logic this change narrows.FILE_SHARE_DELETE: the Windows share mode a process must opt into when opening a file for the file, or any directory above it, to be renamed or deleted while the handle is open. Scanners typically do not set it.Test run on Windows Server 2019 against main (system bun) and against this branch (
bun bd)On main the tarball and patch default-budget cases fail with the errors quoted above; the global store case exits 0 and the
existsSyncon the entry'sbin.exeis false; the three budget=0 cases fail because nothing names the variable.Related: #33884 reworks the concurrent-install arm of the tarball retry loop; #33029 handles non-NTFS volumes, which fail with a different status.
no test proof · iteration 3 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/cli/install/bun-install-windows-rename-retry.test.ts