Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions src/install/PackageManager/runTasks.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1132,6 +1132,29 @@ pub fn run_tasks<C: RunTasksCallbacks>(
manager.extracted_count += 1;
bun_core::analytics::Features::extracted_packages_inc();

// Back-fill a registry package whose manifest or lockfile entry
// carried no usable integrity: the extractor hashed the bytes
// as they were downloaded, so pin that value now and force a
// lockfile save so subsequent installs verify against it. Runs
// here (rather than in a single callback) so the resolve phase
// and both hoisted/isolated install phases share one write-back.
if resolution.tag == bun_install::ResolutionTag::Npm
&& package_id != INVALID_PACKAGE_ID
{
let computed = task.data_extract().integrity;
if computed.tag.is_supported() {
let meta =
&mut manager.lockfile.packages.items_meta_mut()[package_id as usize];
if !meta.integrity.tag.is_supported() {
meta.integrity = computed;
manager
.options
.enable
.set(Enable::FORCE_SAVE_LOCKFILE, true);
}
}
}

if C::HAS_ON_EXTRACT {
if C::IS_PACKAGE_INSTALLER {
C::as_package_installer(extract_ctx).fix_cached_lockfile_package_slices();
Expand Down
10 changes: 7 additions & 3 deletions src/install/TarballStream.rs
Original file line number Diff line number Diff line change
Expand Up @@ -211,11 +211,14 @@ impl TarballStream {

// For GitHub/URL/local tarballs we need a SHA-512 to record in the
// lockfile even when there is no expected value to verify against,
// matching `ExtractTarball.run`.
// matching `ExtractTarball.run`. npm packages get the same fallback so
// a registry manifest with no usable integrity still ends up pinned,
// unless the user opted out of integrity work with `--no-verify`.
let compute_if_missing = matches!(
tarball.resolution.tag,
ResolutionTag::Github | ResolutionTag::RemoteTarball | ResolutionTag::LocalTarball
);
) || (tarball.resolution.tag == ResolutionTag::Npm
&& !tarball.skip_verify);

let npm_mode = tarball.resolution.tag != ResolutionTag::Github;
let want_first_dirname = tarball.resolution.tag == ResolutionTag::Github;
Expand Down Expand Up @@ -1171,7 +1174,8 @@ impl TarballStream {
};

match tarball.resolution.tag {
ResolutionTag::Github
ResolutionTag::Npm
| ResolutionTag::Github
| ResolutionTag::RemoteTarball
| ResolutionTag::LocalTarball => {
if tarball.integrity.tag.is_supported() {
Expand Down
10 changes: 10 additions & 0 deletions src/install/extract_tarball.rs
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,16 @@ impl ExtractTarball {
result.integrity = Integrity::for_bytes(bytes);
}
}
// Same fallback for npm packages whose manifest carried no usable
// integrity (missing, unsupported algorithm, or malformed), so they
// don't stay permanently unverified. `--no-verify` opts out.
ResolutionTag::Npm => {
if self.integrity.tag.is_supported() {
result.integrity = self.integrity;
} else if !self.skip_verify {
result.integrity = Integrity::for_bytes(bytes);
}
}
_ => {}
}

Expand Down
7 changes: 7 additions & 0 deletions src/install/integrity.rs
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,13 @@
pub const SHA384: Tag = Tag(3);
/// The value is a [Subresource Integrity](https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity) value
pub const SHA512: Tag = Tag(4);
/// The registry manifest carried a non-empty `dist.integrity` that did not
/// parse as a supported SRI value (unknown algorithm, bad base64, or wrong
/// digest length). Distinguished from `UNKNOWN` so the resolve step can
/// refuse the version instead of treating the hash as merely absent.
/// Never written to disk: not `is_supported()`, so `Display` emits nothing
/// and `verify()` rejects it.
pub const INVALID: Tag = Tag(5);

Check warning on line 296 in src/install/integrity.rs

View check run for this annotation

Claude / Claude Code Review

Tag::INVALID doc comment claims "Never written to disk" but it is serialized to the manifest cache

The "Never written to disk" claim here (and in the PR description's "runtime sentinel only … so it never reaches disk") is not quite true: `npm.rs:2589-2594` sets `Tag::INVALID` on `PackageVersion.integrity`, and `PackageVersion` is byte-serialized to the on-disk manifest cache via `Serializer::write_array(writer, &this.package_versions, …)` (`npm.rs:1038`). It correctly never reaches a *lockfile* (`from_npm` replaces it before append; `Display` emits nothing), but discriminant 5 does land in `~
Comment on lines +290 to +296

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 The "Never written to disk" claim here (and in the PR description's "runtime sentinel only … so it never reaches disk") is not quite true: npm.rs:2589-2594 sets Tag::INVALID on PackageVersion.integrity, and PackageVersion is byte-serialized to the on-disk manifest cache via Serializer::write_array(writer, &this.package_versions, …) (npm.rs:1038). It correctly never reaches a lockfile (from_npm replaces it before append; Display emits nothing), but discriminant 5 does land in ~/.bun/install/cache/*.npm. Worth rewording so a future maintainer doesn't treat Tag(5) as freely reusable — e.g. "Never reaches a lockfile; may be persisted in the manifest cache."

Extended reasoning...

What the comment claims vs. what happens

The new doc comment on Tag::INVALID (integrity.rs:294-296) states:

Never written to disk: not is_supported(), so Display emits nothing and verify() rejects it.

The PR description repeats this: "Tag::INVALID is a runtime sentinel only … so it never reaches disk."

The justification given — Display emits nothing, verify() rejects — only covers the text-lockfile serialization path (bun.lock writes format!("{}", integrity)) and the verify path. It does not cover the raw-byte manifest-cache serialization.

Step-by-step trace to disk

  1. PackageManifest::parse hits a version whose dist.integrity is non-empty but unparseable and whose shasum is absent/invalid. At npm.rs:2589-2594 it sets:
    package_version.integrity = Integrity { tag: Tag::INVALID, ..Default::default() };
  2. PackageVersion is #[repr(C)] with an Integrity field that is bytemuck::NoUninit (integrity.rs:23), and the file explicitly documents at npm.rs:685-686 that "Serializer::write_array reinterprets the whole slice as &[u8]".
  3. After parse, save_asyncSerializer::saveSelf::write_array(writer, &this.package_versions, &mut pos) at npm.rs:1038 writes the PackageVersion array — including the Tag(5) byte — to ~/.bun/install/cache/<pkg>.npm.

So Tag(5) reaches disk in the manifest cache.

What is true

The tag correctly never reaches a lockfile: Package::from_npm (Package.rs:847-861) checks == Tag::INVALID, logs the error, and stores Integrity::default() on the lockfile package instead. And on the text-lockfile side, Display for Integrity emits "" for any unsupported tag. So the security/verification story is intact.

Why it's worth fixing (and why it's only a nit)

There is no functional bug today: a Tag(5) in a cached .npm file round-trips fine — new bun re-reads it as INVALID and errors in from_npm; older bun reads Tag(5) as an unrecognized tag (the newtype accepts any u8), is_supported() is false, and it's treated as absent.

But per REVIEW.md, "Comments carry only durable non-obvious content: invariants" — and this comment states an invariant that is false. The concrete risk: a future maintainer reading "Never written to disk" might treat discriminant 5 as freely reusable (e.g. for a new supported algorithm like SHA3). Cached manifests written by this version would then be misread as that new algorithm with an all-zero digest, and verify() would reject every tarball for those versions until the manifest cache expires. A false invariant comment is worse than no comment.

Suggested fix

Reword to scope the claim to what the justification actually proves, e.g.:

/// Never reaches a lockfile: `from_npm` replaces it with `default()` before
/// append, and `Display` emits nothing. May be persisted in the raw-byte
/// manifest cache; do not reuse discriminant 5.

(and drop the "so it never reaches disk" from the PR description if convenient). Nit only — not blocking.


#[inline]
pub fn is_supported(self) -> bool {
Expand Down
16 changes: 15 additions & 1 deletion src/install/lockfile/Package.rs
Original file line number Diff line number Diff line change
Expand Up @@ -844,7 +844,21 @@ impl Package<u64> {

package.meta.arch = package_version.cpu;
package.meta.os = package_version.os;
package.meta.integrity = package_version.integrity;
package.meta.integrity =
if package_version.integrity.tag == crate::integrity::Tag::INVALID {
log.add_error_fmt(
None,
bun_ast::Loc::EMPTY,
format_args!(
"Registry provided an invalid integrity hash for {}@{}",
bstr::BStr::new(manifest.name()),
version.fmt(&manifest.string_buf),
),
);
crate::integrity::Integrity::default()
} else {
package_version.integrity
};
package
.meta
.set_has_install_script(package_version.has_install_script);
Expand Down
54 changes: 49 additions & 5 deletions src/install/lockfile/bun.lock.rs
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,9 @@
/// lockfile keeps loading:
/// - an npm package resolved to a tarball URL outside the configured
/// registry must carry a supported integrity hash
/// - an npm package's integrity string, when non-empty, must parse as a
/// supported SRI hash (unparseable/unknown-algorithm entries are
/// rejected rather than warned-and-ignored)
/// - a git `.bun-tag` must be a safe path/checkout component (the same
/// check on a `github` tag is enforced at every version, since its
/// download path has no checkout-time re-validation)
Expand Down Expand Up @@ -2622,11 +2625,19 @@

pkg.meta.integrity = Integrity::parse(integrity_str);
if !integrity_str.is_empty() && !pkg.meta.integrity.tag.is_supported() {
// Surface — don't fail — for npm parity (`npm install`
// proceeds on a malformed lockfile integrity, treating
// it as absent). The download path still applies any
// registry-supplied integrity, so this only loses the
// *lockfile* pin.
// A non-empty value that doesn't parse as a supported
// SRI hash is refused so a tampered or unknown-algorithm
// entry cannot silently disable verification. Gated to
// v2+ so lockfiles written before this check keep
// loading with the prior warn-and-ignore behaviour.
if lockfile_version.at_least(Version::V2) {
log.add_error(
Some(source),
item_loc(source, key_loc, i),
b"Unsupported or malformed integrity hash for npm package",
);
return Err(ParseError::InvalidPackageInfo);
}
log.add_warning(
Some(source),
item_loc(source, key_loc, i),
Expand All @@ -2653,6 +2664,39 @@
);
return Err(ParseError::InvalidPackageInfo);
}

// An empty integrity string means this entry vouches for
// nothing. The download path computes and back-fills a
// SHA-512 once the tarball is fetched, so a non-frozen
// install self-heals; under --frozen-lockfile the lockfile
// cannot be updated and the unverified entry is refused.
// Gated to v2+ so lockfiles written before the back-fill
// existed (which always persisted `""` for a registry with
// no integrity) keep loading silently.
if lockfile_version.at_least(Version::V2) && integrity_str.is_empty() {
let frozen = manager
.as_deref()
.is_some_and(|m| m.options.enable.frozen_lockfile());
if frozen {
log.add_error_fmt(
Some(source),
item_loc(source, key_loc, i),
format_args!(
"Package {} has no integrity pin in the frozen lockfile",
bstr::BStr::new(name_str),
),
);
return Err(ParseError::InvalidPackageInfo);
}
log.add_warning_fmt(
Some(source),
item_loc(source, key_loc, i),
format_args!(
"Package {} has no integrity pin in the lockfile; it will be computed after download",
bstr::BStr::new(name_str),
),
);
}

Check failure on line 2699 in src/install/lockfile/bun.lock.rs

View check run for this annotation

Claude / Claude Code Review

Empty-integrity frozen-lockfile check retroactively breaks existing v2 lockfiles; gating comment's premise is false

The comment says this check is "Gated to v2+ so lockfiles written before the back-fill existed… keep loading silently", but `Version::CURRENT` was already `V2` on the base commit (ae4b17de introduced V2 on 2026-07-24; this PR's base is 59242d6c) — so pre-PR Bun already writes v2 lockfiles, and per this PR's own Face B those v2 lockfiles persist `""` for npm packages whose registry omits integrity. After upgrading, `bun install --frozen-lockfile` / `--production` on such a lockfile fails with "Pa
Comment on lines +2670 to +2699

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The comment says this check is "Gated to v2+ so lockfiles written before the back-fill existed… keep loading silently", but Version::CURRENT was already V2 on the base commit (ae4b17d introduced V2 on 2026-07-24; this PR's base is 59242d6) — so pre-PR Bun already writes v2 lockfiles, and per this PR's own Face B those v2 lockfiles persist "" for npm packages whose registry omits integrity. After upgrading, bun install --frozen-lockfile / --production on such a lockfile fails with "Package X has no integrity pin in the frozen lockfile", breaking CI with no code change. Either bump to a new Version::V3 and gate on it (matching how the pre-existing V2 invariant at line 2656 was introduced alongside the V2 bump), or rewrite the comment and PR description to state the retroactive enforcement is intentional.

Extended reasoning...

What the bug is

The new empty-integrity check at bun.lock.rs:2676 refuses a v2 lockfile npm entry with "" integrity under --frozen-lockfile. The gating comment (lines 2673-2675) justifies at_least(Version::V2) with:

Gated to v2+ so lockfiles written before the back-fill existed (which always persisted "" for a registry with no integrity) keep loading silently.

That premise is false. Version::CURRENT was already Version::V2 on the base commit — git show 59242d6c:src/install/lockfile/bun.lock.rs line 120 reads pub const CURRENT: Version = Version::V2;, and git log -S 'Version::V2' shows V2 was introduced in ae4b17d (2026-07-24), 44 commits before this PR's base. So pre-PR Bun already writes v2 lockfiles, and — per this PR's own problem statement, Face B: "bun.lock records """ — those v2 lockfiles persist "" for npm packages whose registry omitted dist.integrity/dist.shasum. The v0/v1 gate protects nothing the comment claims: v0/v1 lockfiles were written before the back-fill existed, but so were v2 lockfiles.

Step-by-step proof

  1. A user runs a Bun built from main between ae4b17d and this PR (e.g. e532ad91f, the "release-asan main" the PR description itself was verified against) against a private registry (Artifactory / Nexus / self-hosted) that omits both dist.integrity and dist.shasum.
  2. That Bun writes bun.lock with "lockfileVersion": 2 (since CURRENT = V2 at bun.lock.rs:120 on that build) and, per Face B, "" for the affected npm package's integrity slot.
  3. The user upgrades to a Bun containing this PR.
  4. CI runs bun install --frozen-lockfile (or bun install --production, which sets FROZEN_LOCKFILE at PackageManagerOptions.rs:812).
  5. parse_into_binary_lockfile reaches line 2676: lockfile_version.at_least(Version::V2) is true (lockfile is v2), integrity_str.is_empty() is true, and frozen is trueadd_error_fmt("Package {name} has no integrity pin in the frozen lockfile") and return Err(ParseError::InvalidPackageInfo).
  6. Install fails with a non-zero exit; CI breaks on upgrade with no code change.

The PR description repeats the same claim — "v0/v1 lockfiles, written before the back-fill existed, keep loading silently" — so this is the author's stated design intent, and the code fails to deliver it.

Why the existing pattern doesn't cover this

Contrast with the pre-existing off-registry-URL check at line 2656 (npm_url_needs_integrity): that check was introduced in the same commit (ae4b17d) that bumped CURRENT from V1 to V2, so every v2 lockfile ever written satisfies it by construction. This PR adds a new invariant after V2 became CURRENT, so existing v2 lockfiles do not satisfy it. REVIEW.md is explicit here: "Any change to cached/serialized output bumps the format version constant."

The same reasoning applies more weakly to the non-empty-unparseable check at 2633-2640, though that is far less likely to appear in a real pre-PR v2 lockfile since Bun's own writer never emitted an unparseable hash — only hand-editing or third-party tooling would produce one.

Impact

Users with a private registry that omits integrity metadata — exactly the population #19519 targets — have their --frozen-lockfile / --production CI break on Bun upgrade with no repo change. The non-frozen path self-heals (warns, back-fills, re-saves), so the blast radius is limited to frozen/production installs, but that is precisely the CI path.

Fix

Two acceptable resolutions:

  • If backward compat is intended (which the comment and PR description both say it is): add Version::V3, bump CURRENT to V3, gate both new checks on at_least(Version::V3), and update from_int / the V2 doc-comment accordingly. This matches exactly how the pre-existing V2 invariants were introduced in ae4b17d.
  • If retroactive enforcement is intentional (defensible for a fail-closed security fix, since the non-frozen path self-heals and the affected population is narrow): rewrite the comment at 2673-2675 and the PR-description bullet for Face C to say so explicitly, and drop the misleading "lockfiles written before the back-fill existed keep loading silently" claim. Optionally include the remediation ("run bun install once without --frozen-lockfile to back-fill") in the error message.

Either way, the current state — code whose behavior contradicts its own documented rationale — should be resolved before merge.

}
ResolutionTag::LocalTarball | ResolutionTag::RemoteTarball => {
// integrity is optional for tarball deps (backward compat)
Expand Down
17 changes: 15 additions & 2 deletions src/install/npm.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2568,17 +2568,30 @@
package_version.unpacked_size = n.value() as u32;
}

if let Some(shasum_str) = dist.get(b"integrity").and_then(|v| v.as_str()) {
package_version.integrity = Integrity::parse(shasum_str);
let mut had_invalid_integrity = false;
if let Some(sri_str) = dist.get(b"integrity").and_then(|v| v.as_str()) {
package_version.integrity = Integrity::parse(sri_str);
if package_version.integrity.tag.is_supported() {
break 'integrity;
}
had_invalid_integrity = !sri_str.is_empty();
}

if let Some(shasum_str) = dist.get(b"shasum").and_then(|v| v.as_str()) {
package_version.integrity =
Integrity::parse_sha_sum(shasum_str).unwrap_or_default();
if package_version.integrity.tag.is_supported() {
break 'integrity;
}
had_invalid_integrity |= !shasum_str.is_empty();
}

if had_invalid_integrity {
package_version.integrity = Integrity {
tag: crate::integrity::Tag::INVALID,
..Default::default()
};
}

Check failure on line 2594 in src/install/npm.rs

View check run for this annotation

Claude / Claude Code Review

Tag::INVALID rejects placeholder integrity strings, breaking minimum-release-age.test.ts on all CI platforms

The new `Tag::INVALID` path breaks `test/cli/install/minimum-release-age.test.ts` on all 9 CI platforms (see robobun's comment): its mock registry serves ~48 placeholder `dist.integrity` values like `"sha512-fake1=="` which now trip `had_invalid_integrity` and cause `Package::from_npm` to fail every install with `error: Registry provided an invalid integrity hash for <pkg>@<ver>`. Per REVIEW.md ("When changing output/defaults/messages, grep the suite for assertions on the old behavior and update
Comment on lines +2571 to 2594

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The new Tag::INVALID path breaks test/cli/install/minimum-release-age.test.ts on all 9 CI platforms (see robobun's comment): its mock registry serves ~48 placeholder dist.integrity values like "sha512-fake1==" which now trip had_invalid_integrity and cause Package::from_npm to fail every install with error: Registry provided an invalid integrity hash for <pkg>@<ver>. Per REVIEW.md ("When changing output/defaults/messages, grep the suite for assertions on the old behavior and update them in the same PR"), the fix is to update that test's mock manifests to omit dist.integrity (so the version falls into the compute-and-pin path) or supply real hashes.

Extended reasoning...

What breaks

test/cli/install/minimum-release-age.test.ts fails with exit code 1 on every CI platform. The robobun comment on this PR's HEAD (d6c4ed9) confirms it as the sole non-flaky failure across 🐧 13 x64/aarch64/asan, 🐧 25.04 x64/aarch64, 🐧 3.23 x64/aarch64, and 🪟 x64/aarch64 — nine platforms, same file, same code 1.

Mechanism

The mock registry in that test file serves manifests whose dist.integrity is a placeholder string, e.g. "sha512-fake1==" (line 106), "sha512-fake2==" (114), "sha512-bugfix1==" (167), "sha512-limit1==" (219) — roughly 48 such entries across the file. There is no dist.shasum on any of them.

Before this PR, Integrity::parse returned Tag::UNKNOWN for these strings and the manifest reader silently treated the version as having no integrity — fine for tests that only exercise version-selection logic and never fetch the tarball. After this PR, the new logic in src/install/npm.rs sets had_invalid_integrity = !sri_str.is_empty() when the parsed tag is unsupported, finds no shasum to fall back to, and stamps the version with Tag::INVALID. Package::from_npm (src/install/lockfile/Package.rs:847-858) then emits error: Registry provided an invalid integrity hash for <pkg>@<ver>, which fails resolve → the install exits non-zero → every test in the file fails.

Step-by-step proof

Take "sha512-fake1==" from line 106:

  1. Integrity::parse_entry calls Tag::parse, which returns (Tag::SHA512, 7)"sha512-" is a recognized prefix.
  2. The remaining input after offset 7 is "fake1=="; trailing = padding is stripped, leaving "fake1" (5 chars).
  3. base64.decoder.calc_size_for_slice("fake1") fails: 5 unpadded base64 chars is length ≡ 1 mod 4, which is invalid. parse_entry returns Tag::UNKNOWN.
  4. Back in npm.rs: is_supported() is false, sri_str is non-empty → had_invalid_integrity = true.
  5. There is no "shasum" field in the mock manifest, so the fallback branch is skipped.
  6. had_invalid_integrity is true → package_version.integrity.tag = Tag::INVALID.
  7. Package::from_npm sees Tag::INVALID and calls log.add_error_fmt("Registry provided an invalid integrity hash for test-pkg@1.0.0").
  8. The install fails; the test's expect(exitCode).toBe(0) (or equivalent) fails.

Even placeholders whose length happens to be valid mod-4 (e.g. "bugfix1", 7 chars → decodes to 5 bytes) still fail: 5 bytes ≠ the expected 64-byte SHA-512 digest length, so decoded_size > expected_len is false but the decoded size doesn't match — actually, re-reading parse_entry, it only checks decoded_size > expected_len, so a 5-byte decode into a 64-byte buffer would succeed with Tag::SHA512. However, the majority of the placeholders (fake1fake4, limit1limit9, beta1, beta2, etc.) have length ≡ 1 mod 4 after stripping ==, which is definitively rejected by calc_size_for_slice. Since each mock package advertises multiple such versions and the test resolves against them, at least one INVALID version per package is enough to fail the install — and the CI result confirms it does.

Why nothing else catches this

The PR's own test file (bun-install-tarball-integrity.test.ts) uses a separate in-process registry with real or intentionally-invalid integrity values, so it never touches minimum-release-age.test.ts's mock. The PR description's verification section says "Neighboring lockfile-version-2.test.ts, bun-install.test.ts, and bun-install-retry.test.ts are unchanged" — but doesn't mention grepping for other tests that serve fake dist.integrity. REVIEW.md is explicit here: "When changing output/defaults/messages, grep the suite for assertions on the old behavior and update them in the same PR."

Fix

Update test/cli/install/minimum-release-age.test.ts's mock registry to either:

  • Omit dist.integrity entirely from each version's dist object. The version then falls into the new compute-and-pin path (Tag::UNKNOWN, not Tag::INVALID), and since these tests never actually download the tarball, nothing else changes. This is the minimal diff.
  • Or supply a real SHA-512 of the served tarball bytes for each version.

The first option is a mechanical ~48-line deletion and keeps the test focused on what it actually exercises (version selection by release age), so it's the recommended fix.

}
}

Expand Down
Loading
Loading