Skip to content

pack/publish: stop panicking on package.json bin and files entries longer than the path buffer - #38784

Open
robobun wants to merge 4 commits into
mainfrom
farm/c5842f2b/pack-publish-long-bin
Open

pack/publish: stop panicking on package.json bin and files entries longer than the path buffer#38784
robobun wants to merge 4 commits into
mainfrom
farm/c5842f2b/pack-publish-long-bin

Conversation

@robobun

@robobun robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator

Problem

  • bun pm pack and bun publish abort with panic: range end index 5000 out of range for slice of length 4096 (exit 134 plus a crash report) when package.json has a bin string, a bin object value, or a directories.bin longer than the path buffer (4096 bytes on Linux, 1024 on macOS, 98302 on Windows). bun pm pack does the same for a files entry of that length, and bun publish <tarball> also for a bin object key.
  • Cause: those strings are normalized with resolve_path::normalize_buf / normalize_buf_z into a stack PathBuffer, and the normalizer indexes past the end of whatever buffer it is given (src/paths/resolve_path.rs, buf[buf_i..buf_i + count].copy_from_slice(..) in normalize_string_generic_tz). Sites: get_package_bins (three) and the files loop in pack() in src/runtime/cli/pack_command.rs; normalize_bin (four) in src/runtime/cli/publish_command.rs. normalize_bin runs for bun publish <tarball> too, which never goes through pack, so both files need the change.

Fix

  • Add resolve_path::normalize_buf_spill and normalize_buf_z_spill: normalize into the caller's buffer when the result is known to fit, otherwise into a caller-owned Vec grown to size. Same shape as the existing join_z_buf_spill, and the same approach resolver: don't abort on a package.json browser map key longer than 1024 bytes #37526 took for the browser map, which is this bug in another package.json field. The size bound is input + 2: normalizing only removes bytes, except that "" becomes "." and two Windows spellings grow by one byte, and the _z variant appends a NUL. Unit tests in bun_paths pin the fits / spills / exact-length / empty cases.
  • The eight sites use them, with one spill Vec per function hoisted out of the loops, so inputs that fit are handled exactly as before and longer ones just land in the Vec.
  • No length cutoff, because the downstream behavior is already right once the string survives normalization. A bin that long cannot exist on disk, and both commands already have a path for a bin that does not exist: pack queues file bins as optional and ignores a bin directory it cannot open (openat / open_dir / fstatat return ENAMETOOLONG for these, I checked the POSIX and Windows wrappers), publish prints its existing bin '...' does not exist warning and keeps the entry in the registry metadata like npm does. A files entry, on the other hand, is a glob, and a long one (a brace group with many names) is legitimate and has to keep matching; cutting it off would be wrong, so it is not a "bad input" case at all.
  • normalize_bin's directories.bin branch only downgraded ENOENT to the warning; ENAMETOOLONG now takes the same branch, since like ENOENT it means nothing exists at that path (other errors, where something exists but cannot be opened, still fail the publish). Without this line the directories.bin tarball test below exits 1 with failed to open bin directory.
  • Verified with:
    • test/cli/install/bun-pack.test.ts: bins > longer than the path buffer (string, object value, directories.bin are skipped; a normal bin listed next to a long one is still packed executable) and files > an entry longer than the path buffer is still matched (a 100 KB brace glob still selects dist/). The values are 100,000 bytes so the tests also exceed the Windows buffer and are not platform gated. These 5 panic on the current release and pass with this change; the whole file passes (81).
    • test/cli/install/bun-publish.test.ts: bin longer than the path buffer, publishing a tarball (built with Bun.Archive so the test does not depend on the pack fix) with each of the four shapes, asserting the exact set of warnings, plus a directory publish. --dry-run stops before the registry is contacted; the userinfo in the registry URL only satisfies the auth check, the same trick an existing dry-run test uses. These 5 panic on the current release and pass with this change; the whole file passes (44, run with a raised timeout because the debug build is too slow for the file's lifecycle-script tests at the default 5s).
    • cargo test -p bun_paths (22 pass) and bun run rust:miri -p bun_paths (22 pass).
  • Overlap: pack: report an error instead of panicking when --destination does not fit the path buffer #38749 (--destination) and publish: report ENAMETOOLONG for a tarball path that does not fit the path buffer #38753 (the bun publish <tarball> argument) fix the other two length panics in these commands; they are different buffers and different inputs. publish: report ENAMETOOLONG for a tarball path that does not fit the path buffer #38753 also edits the use bun_paths::resolve_path::{..} line in publish_command.rs, so one of the two will need the one-line import merge (union of both lists). Against the current heads of pack: pack a file listed under several bin names once, strip the trailing slash from directories.bin #38720 and pack: skip bins reached through symlinks; publish: do not read the readme through a symlink #38707, which edit other lines of get_package_bins, src/ merges cleanly and only the test additions at the end of describe("bins") conflict.
  • Not covered here: bun install of a dependency whose bin value is this long still aborts, in the bin linker's join_abs_string_z (src/install/bin.rs, resolve_bin_target), a different buffer reached from the lockfile rather than from a package.json being packed. Reproduced separately and filed for its own fix.

Background

  • PathBuffer is a [u8; MAX_PATH_BYTES] stack scratch buffer used for paths about to be handed to the OS. MAX_PATH_BYTES is the platform PATH_MAX, which is why the threshold differs per OS. Strings read out of package.json have no such bound.
  • resolve_path::normalize_buf is path.normalize into a caller-provided buffer: collapses ./, // and .., and here also converts \ to / (the Posix platform parameter), which is why pack and publish run bin paths through it before comparing them against tarball entry names. It assumes the buffer is big enough; the *_spill functions in resolve_path.rs are the family of wrappers that remove that assumption by falling back to a heap Vec, which stays empty unless it is needed.
  • pack handles bins separately from the file walk: get_package_bins reads bin / directories.bin, file bins are pushed onto the pack queue as optional entries (dropped if they cannot be opened) and a bin directory is walked if it can be opened. publish additionally rewrites bin into the object form npm registries expect (normalize_bin), which is where it warns about bins that do not exist; bun publish <tarball> only runs this second part, on the package.json found inside the tarball.
Repro on the current release (1.4.0-canary.1)
mkdir binrepro && cd binrepro
bun -e 'require("fs").writeFileSync("package.json", JSON.stringify({name:"bin-repro",version:"1.0.0",bin:Buffer.alloc(5000,"b").toString()}))'
bun pm pack --dry-run
# panic: range end index 5000 out of range for slice of length 4096
bun publish --dry-run
# same panic (directory publish goes through pack)
mkdir -p pkg/package && cp package.json pkg/package && tar -czf long-bin.tgz -C pkg package
bun publish ./long-bin.tgz --dry-run
# same panic, this time from normalize_bin in publish_command.rs

Same result for bin: {x: <long>}, directories: {bin: <long>}, files: [<long>] (pack) and bin: {<long>: "x.js"} (tarball publish). With this change all of them exit 0; the publish variants print warn: bin '<long>' does not exist / warn: bin directory '<long>' does not exist.


no test proof · iteration 0 · Platform-specific test(s) that do not run on this machine. Deferring to CI, which covers all platforms: test/cli/install/bun-publish.test.ts

@coderabbitai

coderabbitai Bot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 28 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 9a012cb7-92b2-4565-a368-23ae3fce3b6d

📥 Commits

Reviewing files that changed from the base of the PR and between 4bf3f36 and 5e5823b.

📒 Files selected for processing (5)
  • src/paths/resolve_path.rs
  • src/runtime/cli/pack_command.rs
  • src/runtime/cli/publish_command.rs
  • test/cli/install/bun-pack.test.ts
  • test/cli/install/bun-publish.test.ts

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author

Status: ready for review. On both builds of this branch (the original commit and the current head) all 177 lanes that ran are green; the only jobs without a result are the two darwin 14 aarch64 - test-bun jobs, which expired unrun on the first build and have been queued for hours on the second because that agent queue is backed up for every build on the pipeline. Nothing in this diff is platform specific (the tests use 100 KB values, so Linux and Windows exercise the same overflow macOS would), and the annotated failures on both builds are flaky retries of unrelated tests. If the darwin jobs expire again, retrying just those two is all CI needs; I am not going to keep re-pushing for it. Review threads are all resolved. bun install of a dependency with a bin value this long still aborts in the bin linker; that is a different buffer and has been filed separately (noted in the description).

Reproduced on 1.4.0-canary.1 (b7a0431) with a 5000 byte bin value: bun pm pack --dry-run, bun publish --dry-run and bun publish <tarball> --dry-run all abort with panic: range end index 5000 out of range for slice of length 4096; files entries (pack) and bin object keys (tarball publish) of that length do the same.

With this branch the 10 new tests in test/cli/install/bun-pack.test.ts and test/cli/install/bun-publish.test.ts pass (all 10 crash on the release build), both files pass as a whole, and cargo test -p bun_paths / the Miri lane pass with the new unit tests.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — mechanical crash fix following the existing *_spill pattern in resolve_path.rs.

What was reviewed:

  • Verified the str.len() + 2 size bound in normalize_buf_or_spill covers every growth case (empty→., C:C:., bare UNC volume +sep, plus the _z NUL); traced Posix normalization to confirm output never exceeds input+1.
  • Checked all seven call sites clone the borrowed result (ZBox::from_bytes, Box::from, cli_dupe) before the next loop iteration reborrows path_buf/path_spill.
  • Confirmed path::platform::Posix resolves via the crate-root re-export in bun_paths/lib.rs, and the two new highway test stubs match the real kernels' not-found conventions (haystack_len / usize::MAX).
  • The ENAMETOOLONG addition to the directories.bin open-error downgrade is scoped to the one branch that needs it; other errnos still fail loud.
Extended reasoning...

Overview

This PR fixes a slice-index panic in bun pm pack and bun publish when package.json bin/directories.bin/files entries exceed MAX_PATH_BYTES. It touches src/paths/resolve_path.rs (adds normalize_buf_spill / normalize_buf_z_spill plus unit tests and two highway link stubs), src/runtime/cli/pack_command.rs (three sites in get_package_bins, one in the files loop), src/runtime/cli/publish_command.rs (four sites in normalize_bin plus one ENAMETOOLONG errno match), and adds integration tests to both bun-pack.test.ts and bun-publish.test.ts.

Security risks

None. The change replaces a fixed-size buffer write that panicked on overflow with a heap fallback. No new trust boundaries, no path-traversal surface (the normalized result feeds into the same bin_path_escapes_root / Pattern::from_utf8 / exists_at checks as before). The files case is a glob, not a filesystem path, so accepting arbitrary length is correct rather than a new risk.

Level of scrutiny

Medium. It's a crash fix in CLI-only code (not the runtime hot path), and the shape is a direct copy of the established join_z_buf_spill / normalize_string_spill pattern already in the same file. The main thing to get right is the size bound, which I traced through normalize_buf_tnormalize_string_generic_tz for both Posix (all callers) and Windows: normalization is byte-removing except for the three documented +1 cases, and the shared helper reserves +2 to cover the _z NUL on top. The unit tests pin fits/spills/exact-length/empty, and the integration tests use 100 KB inputs so they exceed the buffer on every platform.

Other factors

  • All call sites immediately clone the returned slice (ZBox::from_bytes, Box::<[u8]>::from, cli_dupe) before the next iteration mutably reborrows path_buf/path_spill, so there's no aliasing concern; the borrow checker would have caught it anyway since the code compiles.
  • The two new #[unsafe(no_mangle)] test stubs (highway_last_index_of_char, highway_memrmem16) are needed because normalize_buf_t reaches last_index_of_separator_* which links against those kernels; they match the not-found conventions used by the real wrappers (found_at treats haystack_len as miss, match_at treats usize::MAX as miss) and mirror the existing precedent in src/parsers/native_test_shims.rs.
  • The path::platform::Posix type-path change in pack_command.rs resolves via bun_paths/lib.rs:335 (pub use resolve_path::{.., platform}) and the existing use bun_paths::{self as path, ..} alias.
  • The PR description documents that the whole test files pass (81 / 44), plus cargo test -p bun_paths and Miri. No CODEOWNERS entries cover these paths, and there are no outstanding human review comments.

@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 8:05 PM PT - Aug 14th, 2026

@robobun, your commit 27c7326 is building: #97017

@robobun

robobun commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 11:48 PM PT - Aug 14th, 2026

@robobun, your commit 5e5823b is building: #97532

Comment thread src/paths/resolve_path.rs Outdated
Comment thread src/paths/resolve_path.rs Outdated
Comment thread src/paths/resolve_path.rs Outdated
Comment thread src/paths/resolve_path.rs Outdated
Comment thread src/paths/resolve_path.rs Outdated

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — well-scoped panic fix following the established *_spill pattern.

What was reviewed:

  • The +2 size bound in normalize_buf_or_spill against normalize_buf_t's growth cases (empty→., C:C:., UNC trailing sep, plus NUL) — correct, and all eight call sites use platform::Posix where output ≤ input anyway.
  • Import/alias resolution: path::platform::Posix resolves via bun_paths::platform re-export at src/paths/lib.rs:335; the dropped normalize_buf/normalize_buf_z imports have no remaining callers in either file.
  • The ENAMETOOLONG → warning downgrade in normalize_bin's directories.bin branch is scoped to that one errno and matches the existing ENOENT treatment; other errors still crash.
Extended reasoning...

Overview

Fixes a panic in bun pm pack / bun publish when package.json bin, directories.bin, or files entries exceed the platform PathBuffer length. Adds normalize_buf_spill / normalize_buf_z_spill to src/paths/resolve_path.rs (thin wrappers that pick the caller's stack buffer or grow a caller-owned Vec), swaps the 8 affected call sites in pack_command.rs (3 in get_package_bins, 1 in the files loop) and publish_command.rs (4 in normalize_bin), adds ENAMETOOLONG alongside ENOENT in the directories.bin open-error branch, and adds unit tests plus two highway kernel test stubs. Integration tests cover all five input shapes across both commands.

Security risks

None. The values were already being read from package.json and passed downstream; they just panicked on length. After the fix, over-long bins hit the existing "bin does not exist" path (they cannot exist on disk), and over-long files globs are matched as before. No new data flows to the registry or filesystem beyond what shorter inputs already produced. The ENAMETOOLONG downgrade only widens which errno triggers the existing "does not exist" warning; other open errors still fail the publish.

Level of scrutiny

Medium. bun publish writes to registries, but this change only affects how oversized package.json strings are normalized before the existing existence checks / glob matching, and the downstream behavior for a nonexistent bin was already correct. The one non-mechanical piece is the str.len() + 2 size bound; I traced normalize_buf_tnormalize_string_buf_tnormalize_string_generic_tz and confirmed it never grows output beyond input+1 (empty→., drive-relative C:C:., bare UNC volume gains a trailing separator; ADD_NT_PREFIX is never set from this path), and the _z variant writes one NUL. The bound matches the neighboring normalize_string_spill's +1 (which has no NUL variant). Unit tests pin the exact-length and empty→. boundaries.

Other factors

  • Follows the established join_z_buf_spill / normalize_string_spill pattern already in resolve_path.rs; the PR description notes #37526 took the same approach for the browser-map field.
  • All eight sibling sites are covered in one PR ("fix the whole class"); the spill Vec is hoisted per-function so the fits-in-buffer path is unchanged.
  • Tests use 100 KB inputs to exceed the Windows buffer (98302 bytes) so no platform gating is needed; the publish tests build the tarball with Bun.Archive so they don't depend on the pack fix; the --dry-run + userinfo-in-URL trick matches an existing test in the file.
  • CI reported green on 177 lanes; the only red was a darwin queue expiration, since re-run. The comment-cop threads (long comments) were addressed in e6d9e19 / 5e5823b and are all resolved.
  • The two new #[unsafe(no_mangle)] highway stubs follow the exact shape of the existing ones in the same test module and are only linked into cargo test -p bun_paths.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant