Skip to content

Replace bun_core::Error intern machinery with per-crate thiserror enums - #33909

Merged
Jarred-Sumner merged 38 commits into
mainfrom
claude/err-thiserror-refactor
Jul 11, 2026
Merged

Replace bun_core::Error intern machinery with per-crate thiserror enums#33909
Jarred-Sumner merged 38 commits into
mainfrom
claude/err-thiserror-refactor

Conversation

@robobun

@robobun robobun commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Replaces the runtime-interned global error set bun_core::Error (#[repr(transparent)] NonZeroU16) and the err!() macro with idiomatic per-crate thiserror enums.

Why

err!(Name) cached a global RwLock<Vec<&'static str>> intern lookup in a per-site AtomicU16. Not a const, so it could never be a pattern; every dispatch was an if-else guard chain with no exhaustiveness check (worst case ~40 arms in FetchTasklet::to_body_value_error). This was Zig's anyerror, ported.

Design

One #[derive(thiserror::Error)] enum per crate at <crate_dir>/error.rs, re-exported as crate::Error + crate::Result<T, E = Error>:

#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)]
pub enum Error {
    #[error("TooManyRedirects")]
    TooManyRedirects,
    #[error(transparent)] Sys(#[from] bun_errno::SystemErrno),
    #[error(transparent)] Alloc(#[from] bun_alloc::AllocError),
}
  • err!("OutOfMemory") (92 sites, 17 crates) routes to bun_alloc::AllocError
  • errno/fs names (ENOENT, FileNotFound, AccessDenied, NoSpaceLeft, NameTooLong, NotDir, EISDIR, BadPathName, all E*) route to bun_errno::SystemErrno (2 bytes). NOT bun_sys::Error (40 bytes; the context tier).
  • Cross-crate duplicate names are intentional: css::Error::InvalidCharacter != sql::Error::InvalidCharacter.
  • #[error(transparent)] X(#[from] ...) for pass-through; ? composes via From.
  • Every enum also impls bun_core::output::ErrName so Output::err(e, ...) keeps working.
  • Every #[error("...")] string preserves the original err!("...") literal verbatim.
  • No new deps (thiserror = "2" was already a workspace dep). No dyn Error / Box<dyn Error> / anyhow.

Deleted

  • macro_rules! err + .bun_err link-section (src/bun_core/lib.rs)
  • SEED, EXTRA, intern_slow, intern_cached, Error::intern, Error::from_errno, Error::from_raw, Error::name, Error::as_u16, the RwLock<Vec<&'static str>> intern table (src/bun_core/result.rs)
  • named_error_set! macro (all 31 call sites deleted; enums now derive thiserror::Error directly)
  • OrWriteFailed trait
  • ErrorCode::from(bun_core::Error) / ErrorCode::to_error() (src/jsc/ErrorCode.rs): these aliased the intern u16 onto the JS error-code table index. Callers of Errorable::err() now pass ErrorCode(ErrorCode::JS_ERROR_OBJECT) directly.
  • impl From<bun_sys::Error> for bun_core::Error (src/sys/lib.rs); replaced with From<bun_sys::Error> for bun_errno::SystemErrno.

bun_core::Error still exists but is now bun_core's own local 13-variant thiserror enum (for the bun_core::io::Write trait's return type).

Foundation

  • bun_errno::SystemErrno now impls Display + core::error::Error (via strum::IntoStaticStr).
  • bun_alloc::AllocError now impls Display + core::error::Error.

Scope

  • 1,641 err!() call sites across 40 crates, 449 distinct names
  • 523 files changed, +7435 -4331

Verification

  • cargo check --workspace --keep-going: green
  • cargo build --workspace --keep-going: green (post-mono const asserts)
  • bun run rust:check-all: green across all 10 targets (linux/macos/windows x x64/aarch64 + android)

Deferred

  • bun_runtime (192 local variants) and bun_install (117) kept as flat per-crate enums rather than module-level split. Splitting would change == semantics for the 9 names that span modules inside bun_runtime; both enums are still <=4 bytes with Sys nesting.
  • Shrinking bun_sys::Error (40 bytes), variant payloads, $ERR_* JS machinery: out of scope.

@robobun

robobun commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author
Updated 9:05 PM PT - Jul 10th, 2026

@autofix-ci[bot], your commit be52741 is building: #71777

@robobun

robobun commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

This is a behavior-preserving type-level refactor: the verification is the three-step compiler ladder (cargo check --workspace, cargo build --workspace, bun run rust:check-all across 10 targets), all green. Added test/js/bun/util/error-name-preservation.test.ts as a smoke test pinning the user-visible error codes/messages that flow through the refactored paths (errno routing, the fetch error match arm). By design those assertions hold on both main and this branch; the test exists to catch any future drift in #[error("...")] strings or .name() arms, not to demonstrate a pre-existing bug.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additional findings (outside current diff — PR may have been updated during review):

  • 🔴 src/install/PackageInstall.rs:2150-2160 — The realpath_err closure now maps E::ENOENT to crate::Error::Sys(SystemErrno::ENOENT), whose .name() is "ENOENT" — but the comment directly above says test/cli/install/bun-link.test.ts asserts on FileNotFound:``, and that test (line 462, not touched by this PR) checks expect(err4).toContain("FileNotFound: failed linking dependency/workspace to node_modules..."). Either restore the Zig-style names in this closure (e.g. via local variants like `crate::Error::FileNotFound`) or update bun-link.test.ts in this PR; the same user-visible rename applies to `AccessDenied`→`EACCES`, `NotDir`→`ENOTDIR`, and `NameTooLong`→`ENAMETOOLONG` in this closure.

    Extended reasoning...

    What changed

    Before this PR, realpath_err in install_with_link_run mapped errno values to Zig-style named errors:

    E::ENOENT => bun_core::err!("FileNotFound"),
    E::EACCES => bun_core::err!("AccessDenied"),
    E::ENOTDIR => bun_core::err!("NotDir"),
    E::ENAMETOOLONG => bun_core::err!("NameTooLong"),

    The PR replaces these with:

    E::ENOENT => crate::Error::Sys(bun_errno::SystemErrno::ENOENT),
    E::EACCES => crate::Error::Sys(bun_errno::SystemErrno::EACCES),
    E::ENOTDIR => crate::Error::Sys(bun_errno::SystemErrno::ENOTDIR),
    E::ENAMETOOLONG => crate::Error::Sys(bun_errno::SystemErrno::ENAMETOOLONG),

    The comment immediately above the closure was left in place and still reads: "so map the openat errno to the named error tag to preserve the user-visible error tag (test/cli/install/bun-link.test.ts asserts on FileNotFound:)" — i.e. the whole point of this closure was to preserve the Zig-style tag name, and the PR undid exactly that while keeping the comment.

    Why the name changes

    install::Error::name() at src/install/error.rs:390 implements Self::Sys(e) => <&'static str>::from(e). SystemErrno derives strum::IntoStaticStr, so <&'static str>::from(SystemErrno::ENOENT) yields the variant name "ENOENT", not "FileNotFound". Likewise EACCES, ENOTDIR, ENAMETOOLONG.

    How this reaches user output

    When the realpath openat fails, realpath_err(err) is wrapped as InstallResult::fail(…, Step::LinkingDependency, None) (PackageInstall.rs:2177). In PackageInstaller.rs:2079-2088, the else-branch of the failure handler calls:

    Output::err(cause.err, "failed {} for package <b>{}<r>", (cause.step.name(),))

    Output::err uses the ErrName trait for the prefix, and Step::LinkingDependency.name() is "linking dependency/workspace to node_modules". So the printed line becomes:

    error: ENOENT: failed linking dependency/workspace to node_modules for package <name>
    

    The DanglingSymlink and EACCES special-case branches at PackageInstaller.rs:1978/1994 don't intercept ENOENT, so it reaches the else-branch.

    Step-by-step proof of the test failure

    1. test/cli/install/bun-link.test.ts:462 runs bun install against a linked package whose target directory does not exist, and asserts:
      expect(err4).toContain(`FileNotFound: failed linking dependency/workspace to node_modules for package ${link_name}`);
    2. The realpath openat fails with ENOENT; realpath_err returns crate::Error::Sys(SystemErrno::ENOENT).
    3. cause.err.name()"ENOENT".
    4. Printed: ENOENT: failed linking dependency/workspace to node_modules for package ….
    5. toContain("FileNotFound: …") fails.
    6. bun-link.test.ts is not in this PR's changed-files list (grep of pr_diff.txt shows only the source comment referencing it), so nothing updates the assertion.

    The PR description also claims "Every #[error("...")] string preserves the original err!("...") literal verbatim" — this site is a counterexample.

    Fix

    Either:

    • Add FileNotFound / AccessDenied / NotDir / NameTooLong variants to install::Error (or reuse existing local variants) and return those from realpath_err so .name() matches the old strings; or
    • Update test/cli/install/bun-link.test.ts:462 (and any siblings asserting on AccessDenied:/NotDir:/NameTooLong: from this path) to expect the errno-style names in the same PR, and update the stale comment above realpath_err.

    Per CLAUDE.md's Landing PRs guidance ("When changing output/defaults/messages, grep the suite for assertions on the old behavior and update them in the same PR"), one of these needs to land with this change.

Comment thread src/bun_core/util.rs Outdated
Comment thread src/install/PackageInstall.rs Outdated
@robobun

robobun commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

Pushed fixes for the review findings and the CI crashes (263e54c):

Review findings:

  • errno_to_zig_err: deleted the stub; added bun_errno::from_errno(i32) -> SystemErrno and migrated all ~30 callers so SendFile/Windows-install paths keep their real errno names.
  • realpath_err closure: added FileNotFound/AccessDenied/NotDir/NameTooLong as local install::Error variants and restored them in the closure, so bun-link.test.ts assertion on FileNotFound: holds.
  • is_package_missing_from_cache: dropped the duplicate || ENOENT operand (second arm now checks Error::FileNotFound).

CI crashes (the 19 segfaults + ASAN stack-buffer-overflow):
Root cause was an ABI mismatch across #[no_mangle] extern "Rust" link-time boundaries. Two functions had the declaration and definition in different crates, and after the refactor each side named its own crate's Error type. Result<T, E> layouts diverged, so the callee wrote past the caller's stack slot.

  • __bun_resolver_init_package_manager (resolver↔install): both sides now use bun_core::Error with an ABI-match comment.
  • __bun_macro_context_call (js_parser↔js_parser_jsc): definition now returns bun_js_parser::Error and maps everything to MacroFailed.

Also fixed ResolveMessage::fmt to dispatch on err.name() rather than structural == (resolver errors arrive nested via CrateError::Resolver(...)), and replaced a now-reachable unreachable!() in ffi_body.rs with a proper throw.

Verified locally: rust:check-all 10/10 green, scope-mismatch-panic.test.ts 18/18 pass, resolve-error.test.ts 15/15 pass.

Comment thread src/jsc/AsyncModule.rs
Comment thread src/crash_handler/error.rs Outdated
Comment thread src/install/PackageInstall.rs
Comment thread src/http/error.rs
Comment thread src/bun_core/util.rs
Comment thread src/js_parser_jsc/Macro.rs Outdated
Comment thread src/crash_handler/lib.rs Outdated
Comment thread src/install/PackageInstaller.rs Outdated
@robobun

robobun commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

Pushed 21d0fe7 addressing the round-5 review findings and the CI test failures at 6090ffe:

  • Decoder error codes (ZlibError/BrotliDecompressionError/ZstdDecompressionError) preserved via transparent nesting.
  • http::Error::Cert is now a 66-variant fieldless CertError enum (was &'static str). Error enum sizes: http 24→4, install/jsc 24→6, crash 24→4, runtime 32→8.
  • Blob::on_structured_clone_deserialize catch-all no longer panics on #[from]-wrapped errors.
  • PackageInstaller EACCES check, handle_root_error FileNotFound disjunct, and dead macro-error else branch addressed.
  • bun_core::spawn_sync_inherit/getcwd errno-name loss: deferred to follow-up (tier-0 layering; review agreed).

Gate: the smoke test in this PR is intentionally behavior-pinning, not fail-before. This is a type-level refactor (1,641 err! sites → per-crate thiserror enums) whose correctness criterion is cargo check --workspace + cargo build --workspace + bun run rust:check-all (10/10 targets) all green, plus the existing test suite staying green. No JS-observable delta exists by design, so there is no test that fails on main and passes here. Merge decision is on the review findings + CI, not the gate's stash check.

@robobun

robobun commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

Status at ff387ce:

Green: compiler ladder (check/build/rust:check-all 10/10), no crashes in CI (ABI + Blob fixes landed), enum sizes down to 4-8 bytes, all intern machinery deleted.

Remaining (known, mechanical):

  • ~14 clippy::needless_pass_by_value lints on fns taking the now-larger crate::Error by value (opposite lint to the pass-by-ref one just fixed; fix is &Error at ~14 sites).
  • ~7 test failures in build 71498, all error-message-preservation: napi-loader query-string, stack-overflow-at-entry, issue/03830 macro, fetch redirect vtab, bun-build-compile. Same fix pattern as the ~15 sites already addressed in the last 6 commits (dispatch on .name() or add a missing variant).

Stopping per the brief's "if the error count isn't strictly decreasing, stop and report" instruction rather than continue the fix→new-lint cascade. The tail is tractable; every remaining failure is the same class.

Comment thread src/bundler/lib.rs
Comment thread src/crash_handler/lib.rs Outdated
Comment thread src/install/PackageInstaller.rs
Comment thread src/jsc/error.rs Outdated
Comment thread src/jsc/JSGlobalObject.rs Outdated
Comment thread src/install/auto_installer.rs
@robobun

robobun commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

Updated remaining-work list with the two new review findings (same bug class):

Lossy From conversions with _ => Unexpected catch-alls (fix: accept impl ErrName or &'static str directly, like handle_root_error in 7b0d093):

  • JSGlobalObject::throw_error — affects ~25 Bun.* API error messages (FormData, braces, gunzipSync, shell, Transpiler, DevServer, fetch)
  • AutoInstaller::{path_for_resolution, enqueue_package_for_download} trait return types — diagnostic-only
  • From<install::Error> for bun_core::Error catch-all (once the above two are widened, this impl can be deleted)

These plus the ~7 CI test failures are the complete known tail. All are the same mechanical fix pattern demonstrated in commits 6090ffe and 7b0d093.

Comment thread src/bun_core/lib.rs
Comment thread src/jsc/PluginRunner.rs
Comment thread src/http/error.rs
Comment thread src/crash_handler/error.rs Outdated
@robobun

robobun commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

Merge-conflict notification received (8 commits behind main; #33613 and #33925 likely touch files this PR modifies in src/http/ and re-export shims).

Holding the rebase until the documented remaining-work tail is addressed. Rebasing a known-red 540-file PR before that would restart the gate/CI loop without changing the outcome, and conflict resolution in the ~40 error.rs files is safer done once after the dispatch-site fixes land than twice. If you'd prefer the rebase done now for review purposes, say so and I'll do it in a follow-up session.

@Jarred-Sumner

Copy link
Copy Markdown
Collaborator

@robobun rebase and resume

@robobun
robobun force-pushed the claude/err-thiserror-refactor branch from 7b0d093 to a7c9b08 Compare July 10, 2026 22:55
@robobun

robobun commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased onto main (clean, no conflicts) and resumed per request. Pushed a7c9b08:

  • jsc_hooks.rs: JSError dispatch also matches Js(_) and Bundler(Js(_)) so napi-loader and plugin exceptions reach take_error (import-query.test.ts, plugins.test.ts).
  • http::Error::Picohttp(ParseResponseError) preserves Malformed_HTTP_Response code (fetch-redirect.test.ts, client-fetch.test.ts).
  • JSGlobalObject::throw_error takes impl ErrName (same pattern as handle_root_error), so FormData/braces/shell/Transpiler/etc. throw their real error name.
  • bundler::Error::Js preserves JsError identity through PluginRunner.

cargo check, cargo clippy, rust:check-all (10/10) green. CI will tell us what's left of the ~7-test tail.

Comment thread src/js_parser_jsc/Macro.rs Outdated
Comment thread src/jsc/JSGlobalObject.rs
@robobun

robobun commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator Author

CI at 6395780: the two remaining refactor-caused test failures (03830.test.ts macro message, resolve.test.ts auto-install errno) are both green now after 4b8e111.

Only failure is proxy-stress-concurrent.test.ts on macOS-13-x64: 1 of 1200 https-over-https-proxy requests failed on two of the ×1200 memory-probe iterations. That test asserts on request success count, not error names/codes, so it's unrelated to this refactor. Re-rolled once (4829df6).

Comment thread src/install/auto_installer.rs
@robobun

robobun commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator Author

CI 71695 at cacb44a: all refactor-caused test failures are green. Two remaining failures are single-platform flake unrelated to this PR:

  • proxy-stress-concurrent.test.ts (macOS-13-x64 only): 1 of 1200 https-over-https-proxy requests failed. Same flake as build 71681; a networking stress test that counts request success, not error codes.
  • no-orphans.test.ts (linux-aarch64 only): perl fast-exit orphan-reaping test. Process lifecycle test, no error-name path.

Neither touches anything this refactor changes (error enum types / .name() strings / dispatch). The diff is green on cargo check, cargo build, cargo clippy, bun run rust:check-all (10/10 targets), and the entire test suite on all other lanes. Re-roll already spent (4829df6). Needs a maintainer to merge past the gate (structurally unprovable for a behavior-preserving refactor, per earlier comments) and these two single-lane flakes.

@robobun

robobun commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks for the numbers. Honest assessment of what this refactor could plausibly explain:

Install +12.5% — most plausibly mine. install::Error went from 2 bytes (NonZeroU16) to 6 bytes, so every Result<T, install::Error> return (hundreds of call sites in the install hot path) has a wider discriminant + payload. On a ~11ms operation that's almost all Rust, ~1.3ms of extra stack-copy/register-spill across the ? chain is believable. The fix would be dropping the cross-crate #[from] nesting in install::Error (Http/Spawn/Resolver/etc.) so it goes back to ≤2 bytes; that was the trade-off we discussed in the thread.

Express/Fastify −2-3% rps, +4% sys CPU — less clear. http::Error is 4 bytes (was 2), and runtime::Error is 8, but the serve hot path shouldn't be constructing errors. If it's the Result<T, E> return width on non-error paths in request handling, that's the same size fix.

Fastify +53 MB RSS / +27% minor faults — I can't attribute this to the refactor. The error enums are stack/register values; nothing here changes heap allocation patterns, and Express doesn't show the same RSS delta. This looks like a separate issue (or a different build — the table says bun-32488, is that this PR's build?).

If you want me to pursue the size shrink (drop cross-crate nesting → install::Error back to ≤2 bytes), I can, but it's the "selective flattening" we deferred earlier and will re-open a handful of the .name()-dispatch questions. Happy to take direction.

@Jarred-Sumner

Copy link
Copy Markdown
Collaborator

@robobun sorry this is a different PR please ignore.

@robobun

robobun commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator Author

Understood, no worries. PR state unchanged: cacb44aa, all refactor-caused test failures green, only the two unrelated single-lane flakes (proxy-stress on macOS-13-x64, no-orphans on linux-aarch64) remaining. Ready for review.

Comment thread src/jsc/VirtualMachine.rs Outdated
Comment thread src/install/error.rs Outdated
@Jarred-Sumner

Copy link
Copy Markdown
Collaborator

@robobun fix comments + conflicts

@robobun
robobun force-pushed the claude/err-thiserror-refactor branch from cacb44a to c2408ed Compare July 11, 2026 01:39
@robobun

robobun commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator Author

Rebased onto main (1 conflict in jsc_hooks.rs, 2 new err! sites from main in Blob.rs) and applied the dead-code cleanups per review: 16+10+3 dead variants deleted from jsc/crash_handler/install error.rs, WorkspaceMap.rs always-false disjuncts dropped, VirtualMachine.rs dead store removed. All checks green (check/clippy/rust:check-all 10/10). Pushed 75631c0.

@robobun

robobun commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator Author

CI 71735 at 75631c0 (rebased + dead-code cleanups): all refactor-caused test failures green. Single remaining failure is test-worker-message-port-transfer-terminate.js SIGABRT on x64-asan only — a worker-termination race test under ASAN, not error-code related. Diff doesn't touch Worker/MessagePort/terminate.

Re-roll already spent. Diff is green on cargo check/build/clippy, rust:check-all 10/10, and all other CI lanes. Ready for merge past the gate + this one unrelated lane.

Comment thread src/jsc/AsyncModule.rs Outdated
robobun added a commit that referenced this pull request Jul 12, 2026
Resolved src/bun_core/string/mod.rs: #33909 switched return types to
crate::CrateResult<()> in write_pre_quoted_string and quote_for_json,
which this branch restructures into the const-generic
write_pre_quoted_string_inner; carried CrateResult<()> into both the
adapter and _inner signatures.

Resolved src/router/lib.rs and src/sourcemap/Chunk.rs: #33909 touched
return types inside blocks this branch deletes (orphaned router test
scaffolding, uncalled print_source_map_contents); kept the deletions
after re-verifying zero callers on current main.

Restored Node::find_last in src/collections/pool.rs: gained a test
caller via #33311.
robobun added a commit that referenced this pull request Jul 12, 2026
Resolved src/bun_core/string/mod.rs: #33909 switched return types to
crate::CrateResult<()> in write_pre_quoted_string and quote_for_json,
which this branch restructures into the const-generic
write_pre_quoted_string_inner; carried CrateResult<()> into both the
adapter and _inner signatures.

Resolved src/router/lib.rs and src/sourcemap/Chunk.rs: #33909 touched
return types inside blocks this branch deletes (orphaned router test
scaffolding, uncalled print_source_map_contents); kept the deletions
after re-verifying zero callers on current main.

Restored Node::find_last in src/collections/pool.rs: gained a test
caller via #33311.
dylan-conway added a commit that referenced this pull request Jul 16, 2026
## What this adds

**A spec-exact TOML v1.1.0 parser** (`src/parsers/toml.rs`), replacing
the previous JS-lexer-derived implementation, plus the **official
[toml-lang/toml-test](https://github.com/toml-lang/toml-test)
conformance suite** as a generated `bun:test` file, following the same
pattern as the YAML and JSON5 conformance suites.

### Conformance: 100% of the official suite, before → after

Measured against the toml-test v1.1.0 manifest (commit `4d77658d`): 217
valid + 481 invalid + 1 out-of-range-integer + 9 invalid-encoding cases
— every case in the manifest, zero exclusions.

| | before | after |
|---|---|---|
| total | 160/699 (23%)¹ | **708/708 (100%)** |
| valid documents parsed correctly | 148/217 | 217/217 |
| invalid documents rejected with the asserted SyntaxError | 12/481 |
481/481 |
| invalid documents wrongly **accepted** | 144 | 0 |
| invalid-encoding documents (raw byte input) | untestable¹ | 9/9 |
| date/time literals | all rejected as syntax errors | parse as strings
of their source text |

¹ The 9 invalid-encoding cases contain ill-formed UTF-8 that a JS string
cannot carry; they became testable when `TOML.parse` gained binary input
in this PR.

### Parser

- All four TOML date/time types (offset/local date-time, local date,
local time), returned as strings of the source text
- Correct `inf`/`nan`, underscore and leading-zero validation,
`0x`/`0o`/`0b` integers
- Integers outside `Number.MAX_SAFE_INTEGER` throw instead of silently
losing precision (TOML requires lossless handling or an error)
- Table / array-of-tables / dotted-key / inline-table definition-state
rules enforced; duplicate keys rejected, including non-ASCII keys (the
old byte-view comparison of UTF-16 keys missed duplicates and falsely
rejected distinct keys)
- Control characters, bare carriage returns, and ill-formed UTF-8
rejected (simdutf-validated); leading BOM handled
- Multi-line string delimiter/trimming rules, CRLF→LF normalization,
exact escape validation including TOML 1.1's `\xHH` and `\e`
- TOML 1.1 additions: optional seconds in times, multi-line inline
tables with trailing commas
- Dotted keys are parsed iteratively (the old parser capped them at 512
segments)

### `Bun.TOML.parse`

- Throws `SyntaxError` with precise messages (previously a
`BuildMessage`), matching `JSON.parse`, `Bun.YAML.parse`, and
`Bun.JSON5.parse`
- Converts the AST directly to JS values (previously printed to JSON and
re-parsed)
- Accepts `Blob`/`TypedArray`/`DataView`/`ArrayBuffer` input like the
YAML and JSON5 siblings
- Parse errors carry the redaction flag so secrets in malformed config
files stay out of logs

### `Bun.TOML.stringify` (new)

Serializes a JavaScript object to a TOML document — this API did not
exist before. Same surface as the YAML/JSON5 siblings: `(value,
replacer, space)`, where `replacer` throws and `space` is accepted but
ignored (TOML output is line-oriented).

- Idiomatic layout: scalar keyvals first, then `[table]` and
`[[array-of-tables]]` sections; mixed arrays use inline tables; keys are
bare when possible, quoted otherwise
- `Date` values become TOML offset date-times
- `null`, `BigInt`, circular structures, and non-object top-level values
throw (TOML cannot represent them); `undefined`/function/symbol
properties are skipped
- Integral doubles beyond ±(2^53 − 1) are emitted as floats so documents
round-trip through any TOML reader; unpaired surrogates get the same
USVString replacement as `TOML.parse` string input

### Tests

- `test/js/bun/toml/toml-test-suite.test.ts` — generated official suite
(708 tests); every rejection test asserts the exact full error message
- `test/js/bun/toml/generate_toml_test_suite.ts` — pins the upstream
commit, decodes the suite's tagged-JSON expectations, inlines every
case; `--check` mode fails if the committed suite is stale
- `test/js/bun/toml/toml.test.ts` — hand-written coverage beyond the
official suite: input types (Buffer/TypedArray
subviews/DataView/ArrayBuffer/SharedArrayBuffer/Blob), JS value mapping
(`__proto__` safety, key ordering, no Unicode normalization),
safe-integer boundaries, source-text date/time preservation, multi-line
string edge cases, recursion-limit and GC robustness, the SyntaxError
message contract, and `TOML.stringify` (exact layout output, escaping,
round-trips, every error contract, and a GC stress test)

### Performance

1.15×–1.49× faster than the previous parser at every document size,
growing with size. Release builds from this pipeline (darwin-aarch64),
interleaved A/B, best of 3 rounds (each sample = median of 15 timed
batches), real-world inputs both parsers accept:

| document | size | old | new | change |
|---|---|---|---|---|
| `uv.lock` (apache/airflow) | 2.9 MB | 246 MB/s (12.0 ms) | **367 MB/s
(8.0 ms)** | 1.49× |
| `uv.lock` (langgenius/dify) | 662 KB | 237 MB/s | **338 MB/s** | 1.43×
|
| `poetry.lock` (python-poetry/poetry) | 201 KB | 249 MB/s | **341
MB/s** | 1.37× |
| `Cargo.lock` (zed-industries/zed) | 495 KB | 194 MB/s | **262 MB/s** |
1.35× |
| `Cargo.lock` (this repo) | 71 KB | 199 MB/s | **259 MB/s** | 1.30× |
| `pyproject.toml` (python-poetry/poetry) | 5.7 KB | 168 MB/s | **221
MB/s** | 1.31× |
| `poetry.lock` (truncated at a package boundary) | 2.4 KB | 16.2 µs/op
| 13.5 µs/op | 1.20× |
| `bunfig.toml` | 126 B | 3.29 µs/op | 2.85 µs/op | 1.16× |

Basic strings borrow the source bytes when no escape requires decoding
(the same approach the YAML parser uses for plain scalars), so
string-dominated documents — lockfiles in particular — see the largest
gains.

### Behavioral note: stricter parsing of invalid config files

The old parser silently accepted some invalid TOML, so a small set of
existing `bunfig.toml` files (about 1% in a sample of 99 real-world
bunfigs from GitHub) will go from silently tolerated to a startup
`SyntaxError` that states the fix. The one pattern observed in the wild
is unquoted string values — easy to carry over from `.npmrc` syntax —
which now produce a purpose-built message:

```
TOML Parse error: Strings must be quoted: "isolated"
```

The other formerly-tolerated patterns (backslash escapes in basic
strings being silently dropped, missing newlines between key/value
pairs, bare keys containing `:`) were silent data corruption or
non-portable syntax that every other TOML implementation already
rejects; none appeared in the sample.

### Deletions

The old TOML lexer (`src/parsers/toml/lexer.rs`), and the
now-caller-less `E::Object::set_rope`, `get_or_put_array`, and `set`.
The bake error reporter's JS-escape decoder (which borrowed the old TOML
lexer) is extracted standalone with its exact semantics.

Closes #22426
Closes #28680
Closes #28681
Closes #28687

Also addresses the TOML half of #32025 (`\u{…}` is now correctly
rejected; the JS-lexer half remains).





---

### Merge with main (38cfba6)

One conflict, in `src/ast/e.rs`. Main's hardening round (#33072) added
`flags: own_key_property_flags(&key)` to the property-construction
sites, which marks a `__proto__` key as computed so it becomes an own
property rather than setting the prototype. This branch deletes `set`,
`set_rope`, and `get_or_put_array`, replacing `set` with
`append_property`.

Git merged the new flags into `append_property` on its own; the conflict
was only the bodies of the two functions this branch removes. Kept them
removed, after checking that neither has a caller anywhere in the tree
and that every live construction site (`put`, `get_or_put_object`,
`append_property`) carries the new flags, so the hardening is preserved
on every path that still exists.

Verified on the merge result: the TOML suites pass (810 tests, including
the full conformance suite), and the INI and `bun init` tests pass (73
tests) since they are the remaining `get_or_put_object` callers.
`test/bake/dev/production.test.ts` also passes again now that #33204
pinned the React build the bake harness installs.


### Merge with main (dbac342)

Conflicts with #33722, which fixed `\u{...}` overflow and
unterminated-brace handling in the old TOML lexer's copy of the JS
escape loop.

- `src/parsers/toml/lexer.rs` (modify/delete): kept deleted. The new
parser rejects `\u{...}` at the opening brace (`\u{…}` is JavaScript
syntax, not TOML), so neither the overflow nor the unterminated-brace
path is reachable.
- `test/js/bun/resolve/toml/toml-parse.test.ts`: kept this branch's
content and folded the #30825 crash-regression inputs into the existing
`rejects JS-style \u{XX} escapes` test. Main's `still accepts in-range
\u{...}` test asserted `\u{41}` decodes to `"A"`, which contradicts
spec-compliant TOML and is already asserted to throw by this file.
- `src/runtime/bake/DevServer/js_escape.rs`: ported the one-line
unterminated-brace fix, since that decoder was extracted from the code
#33722 patched and is documented as preserving its exact semantics.
Overflow was already prevented by the existing `0x10FFFF` clamp.


### Merge with main (aba8757)

Conflicts with #33925 (repo-wide snake_case directory cleanup), which
renamed `src/runtime/bake/DevServer/` to `dev_server/` and replaced the
`#[path = "../DevServer/..."]` module-path attributes with plain `mod`
declarations.

- `src/runtime/bake/dev_server/mod.rs`: adopted main's simplified
submodule block and added the `js_escape` declaration.
- `src/runtime/bake/DevServer/js_escape.rs`: moved to
`dev_server/js_escape.rs` following the rename. Git tracked the rename
of this branch's `ErrorReportRequest.rs` changes into
`error_report_request.rs`, and its
`super::js_escape::decode_js_escape_sequences` call is intact.



### Merge with main (ace42f9)

Conflicts with #33909, which replaced `bun_core::Error` with per-crate
`thiserror` enums across the codebase.

- `src/parsers/toml.rs`: main changed the old file's error handling;
this branch replaces the whole file. Kept the new parser and adapted
`TOML::parse` to return `crate::Result<Expr>`, mapping the internal
`PErr` to `parsers::Error::{SyntaxError, Alloc, StackOverflow}`.
- `src/parsers/toml/lexer.rs` (modify/delete): kept deleted.
- `src/parsers/error.rs`, `src/runtime/error.rs`: removed the
`From<toml::lexer::Error>` impls that referenced the deleted file.
- `src/runtime/bake/dev_server/js_escape.rs`: adapted to return
`Result<(), crate::Error>` with `crate::Error::SyntaxError`, since
`bun_core::err!` is gone.
- `src/runtime/api/TOMLObject.rs`: matched on
`bun_parsers::Error::Alloc(_)` in place of the removed
`bun_core::err!("OutOfMemory")`.
- `src/ast/lexer_log.rs`: doc comment combining both sides.

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: robobun <117481402+robobun@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants