Rewrite the TOML parser for v1.1.0 conformance - #32953
Merged
Merged
Conversation
Translates the toml-lang/toml-test suite (TOML v1.0.0 manifest, pinned commit 4d77658d) into a generated bun:test file, following the pattern of the YAML and JSON5 conformance suites: a checked-in generator inlines each case, with expectations decoded from the suite's own tagged-JSON files. The generator's --check mode regenerates to a temp file and fails if the committed suite is stale. Asserted type contract: - integers: number within Number.MAX_SAFE_INTEGER, BigInt beyond - date/time values: strings (source text), compared after normalizing the separator to "T", uppercasing "Z", and trimming trailing zeros from fractional seconds - invalid documents must throw SyntaxError (matching JSON.parse, Bun.YAML, and Bun.JSON5) 209 valid + 488 invalid cases; 9 non-UTF-8 inputs are excluded and listed in the header (a JS string input cannot carry invalid bytes). Baseline on a current debug build: 159/697 pass. The suite documents the parser's gaps (date/time literals rejected, inf/nan mis-parsed, int64 precision loss, BOM handling, array-of-tables lexing in value position, missing rejection of invalid documents, error class); parser fixes follow on this branch until the suite is green.
Contributor
|
Found 5 issues this PR may fix:
🤖 Generated with Claude Code |
Collaborator
- Integers outside Number.MAX_SAFE_INTEGER are asserted to throw instead of expecting BigInt: TOML requires lossless handling or an error, and mixed number/BigInt output is not acceptable API. The affected upstream-valid case moves to a documented out-of-range block citing toml-lang/toml-test#154 (64-bit range is a "should"). - Rejection tests assert the exact full SyntaxError message, captured from the in-tree parser at generation time; cases the parser does not yet reject with SyntaxError assert only the class until regenerated. toThrow(string) is substring matching and toThrow(Error) ignores the class, so the tests use an explicit try/catch with toBeInstanceOf plus message equality.
Switch the generated suite from the v1.0.0 manifest to v1.1.0 (released 2025-12-24). TOML 1.1 is a strict superset of 1.0: optional seconds in times, multi-line inline tables with trailing commas, and \xHH/\e string escapes. The datetime comparator gains one rule for omitted seconds (07:32 is the same value as 07:32:00, which is how the upstream expectations spell it).
dylan-conway
changed the title
Replaces the JS-lexer-derived TOML parser with a spec-exact byte-level recursive-descent parser. The official toml-test conformance suite now passes 699/699 (from 160/699): all four date/time types parse (as strings of their source text), inf/nan and underscore/leading-zero validation are correct, control characters and bare carriage returns are rejected, a leading BOM is handled, table and array-of-tables definition-state rules are enforced, multi-line strings trim and normalize CRLF correctly, and escape sequences are validated exactly (including TOML 1.1's \xHH and \e). Behavior changes beyond conformance: - integers outside Number.MAX_SAFE_INTEGER throw instead of silently losing precision (TOML requires lossless handling or an error) - Bun.TOML.parse throws SyntaxError (previously BuildMessage), converts the AST directly to JS values instead of printing JSON and re-parsing, and accepts Blob/Buffer input like the YAML and JSON5 siblings - parse errors carry the redact flag so bunfig secrets stay out of logs - duplicate non-ASCII keys are detected correctly (the old byte-view comparison of UTF-16 keys missed duplicates and falsely rejected distinct keys) The shared Expr-to-JS conversion moves from JSON5Object to the api module and is reused by TOML. The bake error reporter's JS-escape decoder, which borrowed the old TOML lexer, is extracted standalone with its exact semantics. Dead code removed in the same change: the old toml lexer, E::Object::set_rope/get_or_put_array/set. Fixes #22426 Fixes #28680 Fixes #28681 Fixes #28687
dylan-conway
changed the title
dylan-conway
marked this pull request as ready for review
June 28, 2026 01:49
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughThe TOML parser is rewritten as a byte-level recursive-descent parser with TOML 1.1 string, number, date/time, and inline-table handling. Shared Expr-to-JS conversion moves into ChangesTOML parsing and runtime conversion
Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
The 9 toml-test invalid-encoding cases cannot be JS strings, so the generated suite now passes them to TOML.parse as bytes (which the parser must reject: a TOML document must be valid UTF-8 as a whole), bringing the suite to 708 tests with zero exclusions. Types: TOML.parse accepts TypedArray/DataView/ArrayBuffer input and documents the SyntaxError contract. Docs: the supported-features section reflects TOML v1.1.0 (date/times as strings, lossless-integer errors, multi-line inline tables, \xHH and \e escapes).
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/runtime/bake/DevServer/js_escape.rs`:
- Around line 1-9: Condense the added explanatory comments in
js_escape::decode_escape_sequences so each comment block stays within the repo’s
3-line limit. Keep only the essential summary near the module docs and the
decode_escape_sequences behavior notes, and move the extra
implementation/history detail out of the source comment or into PR/docs. Also
trim the later comment block referenced in the review so it follows the same
limit.
- Around line 35-37: The escape decoder currently treats an unfinished trailing
backslash or an unterminated `\u{...` sequence as success in the `js_escape`
parsing loop. Update the escape-handling logic to detect end-of-input while
inside an escape and return an error instead of `Ok(())` or emitting a partial
codepoint, using the relevant decoder/iterator flow in `js_escape.rs` (including
the repeated logic at the later occurrence) to ensure malformed escapes are
rejected consistently.
In `@test/js/bun/toml/generate_toml_test_suite.ts`:
- Around line 54-57: Keep the invalid-UTF-8 corpus in
generate_toml_test_suite.ts and stop filtering out TextDecoder failures via the
excluded list. Update the generation logic around utf8Strict and the suite
emitters so raw bytes are preserved and byte-input cases are produced as
Uint8Array/Blob-backed tests for the byte path. Remove the “untestable”
exclusion/comment and ensure the BOM/invalid-UTF-8 inputs remain part of the
generated TOML suite.
- Around line 11-14: The TOML test generator currently clones a moving upstream
HEAD and also skips invalid UTF-8 byte-input cases. Update generateTomlTestSuite
(and its git-cloning helper path) to clone toml-lang/toml-test with argv-based
git calls, then checkout the fixed revision the suite is intended to track so
--check stays stable; also keep the byte-input cases by preserving the
Blob/TypedArray-style tests instead of dropping entries that fail UTF-8
decoding.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 733551a7-ed0c-426d-b08c-8a0cad44a6af
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (14)
src/ast/e.rssrc/bun_core/string/immutable.rssrc/parsers/Cargo.tomlsrc/parsers/toml.rssrc/parsers/toml/lexer.rssrc/runtime/api.rssrc/runtime/api/JSON5Object.rssrc/runtime/api/TOMLObject.rssrc/runtime/bake/DevServer/ErrorReportRequest.rssrc/runtime/bake/DevServer/js_escape.rssrc/runtime/bake/dev_server/mod.rstest/js/bun/resolve/toml/toml-parse.test.tstest/js/bun/toml/generate_toml_test_suite.tstest/js/bun/toml/toml-test-suite.test.ts
💤 Files with no reviewable changes (1)
- src/parsers/toml/lexer.rs
toml.test.ts covers what the official conformance suite cannot: the
JS-facing surface (string/Buffer/TypedArray/DataView/ArrayBuffer/
SharedArrayBuffer/Blob input, subarray offsets, toString coercion),
JS value mapping (__proto__ and constructor keys as own properties,
array-index key ordering, no Unicode normalization of keys, -0.0,
safe-integer boundaries), source-text preservation for all date/time
spellings, multi-line string delimiter edge cases, robustness (deep
nesting throws RangeError instead of crashing, 1 MB strings, GC
stress), and the SyntaxError message contract.
Errors at end of input previously trailed off ("Expected '=' after a
key but found"); they now name the end of file.
Parsing dotted paths is iterative (the old parser capped them at 512 segments; the rewrite has no cap), so the recursion limit is reached in the Expr-to-JS conversion instead — pin that both sides fail with a clean RangeError rather than a crash, and that a 1000-segment dotted key now parses.
Release frames are much smaller than debug/ASAN frames, so overflow depths must be far past the limit at the smallest frame size: nested arrays/inline tables use 2M (the recursive parse trips the stack check early, so the depth is never paid in full). Dotted-key parsing is iterative and pays the full depth, so it uses 250k — about a second in debug builds while still overflowing the JS-conversion recursion on release frames.
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/runtime/toml.mdx`:
- Around line 48-56: The TOML docs overstate conformance by saying Bun passes
the complete official toml-test suite, but the generated suite in
generate_toml_test_suite.ts rewrites out-of-range integer fixtures into
rejection tests because Bun throws beyond Number.MAX_SAFE_INTEGER. Update the
claim in toml.mdx to narrow it to Bun’s adapted TOML v1.1 coverage or explicitly
mention the JavaScript-number exception, keeping the language accurate for the
integer behavior.
In `@packages/bun-types/bun.d.ts`:
- Around line 794-798: The TOML parse overload in bun.d.ts narrows DataView too
much by using DataView<ArrayBuffer>, which excludes valid shared-backed views
despite the input type already allowing ArrayBufferLike. Update the parse
signature to use DataView<ArrayBufferLike> so the overload matches all supported
DataView byte sources and type-checks consistently.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 93d0d7d6-9a35-4811-811c-bc575bf69a87
📒 Files selected for processing (6)
docs/runtime/toml.mdxpackages/bun-types/bun.d.tssrc/parsers/toml.rstest/js/bun/toml/generate_toml_test_suite.tstest/js/bun/toml/toml-test-suite.test.tstest/js/bun/toml/toml.test.ts
The old parser silently accepted bare words as string values (`linker = isolated`), which is the most common spec violation in real-world bunfig.toml files; rejecting it with "Expected a number but found 'i'" pointed users in the wrong direction. Bare words in value position now produce: Strings must be quoted: "isolated".
- Newlines and comments are no longer accepted between '=' and the value inside inline tables (keyval-sep is `ws %x3D ws`); the inline loop now reuses parse_keyval, which already enforced this at the top level - Decimal integers accumulate as an unsigned magnitude so i64::MIN gets the lossless-representation message instead of the wrong "outside the 64-bit signed range" diagnostic - vec_into_slice duplicated ArenaVecExt::into_bump_slice; use the trait and drop the redundant unsafe helper - The suite generator checks out its pinned upstream commit when cloning, so --check stays stable as toml-test advances - TOML.parse types add Blob and widen DataView to ArrayBufferLike (JSONL's identical DataView narrowing fixed in the same pass); docs name the one deliberate suite deviation (out-of-range integers throw)
A full production-by-production audit of the parser against the official TOML 1.1.0 ABNF found exactly one behavior worth pinning: JS string input converts to UTF-8 with USVString replacement before parsing, so unpaired surrogates become U+FFFD (matching TextEncoder and the YAML/JSON5 siblings), while the same content as bytes is ill-formed UTF-8 and rejects.
parse_basic_string copied every byte into an arena buffer even when the decoded content is identical to the source. Borrow the source slice like parse_literal_string already does, copying only when an escape, CRLF normalization, quote-run content, or line-ending backslash makes the decoded bytes diverge. Most real-world strings (package names, versions, hashes in lockfiles) take the borrow path.
Contributor
There was a problem hiding this comment.
♻️ Duplicate comments (1)
test/js/bun/toml/generate_toml_test_suite.ts (1)
43-46: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winUse argv-based
gitinvocations here.
tmpis interpolated into a shell command, so aTMPDIRcontaining spaces or shell metacharacters can break this generator or execute unintended shell syntax on the developer machine. Callgitwith an argument array instead ofexecSync("...").Proposed fix
- execSync(`git clone https://github.com/toml-lang/toml-test.git ${tmp}`, { stdio: "inherit" }); - execSync(`git -c advice.detachedHead=false checkout ${PINNED_COMMIT}`, { cwd: tmp, stdio: "inherit" }); + execFileSync("git", ["clone", "https://github.com/toml-lang/toml-test.git", tmp], { + stdio: "inherit", + }); + execFileSync("git", ["-c", "advice.detachedHead=false", "checkout", PINNED_COMMIT], { + cwd: tmp, + stdio: "inherit", + });import { execFileSync } from "node:child_process";🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/js/bun/toml/generate_toml_test_suite.ts` around lines 43 - 46, The toml test generator is building shell command strings in the clone/checkout flow, which makes the `tmp` path unsafe when it contains spaces or shell metacharacters. Update the `generate_toml_test_suite.ts` logic to use argv-based child process calls instead of string-based `execSync` for the `git clone` and `git checkout` steps, and switch to the appropriate API such as `execFileSync` so `tmp` is passed as a separate argument. Keep the existing `mkdtempSync`, `PINNED_COMMIT`, and `git` workflow, but route the commands through argument arrays to avoid shell interpolation.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@test/js/bun/toml/generate_toml_test_suite.ts`:
- Around line 43-46: The toml test generator is building shell command strings
in the clone/checkout flow, which makes the `tmp` path unsafe when it contains
spaces or shell metacharacters. Update the `generate_toml_test_suite.ts` logic
to use argv-based child process calls instead of string-based `execSync` for the
`git clone` and `git checkout` steps, and switch to the appropriate API such as
`execFileSync` so `tmp` is passed as a separate argument. Keep the existing
`mkdtempSync`, `PINNED_COMMIT`, and `git` workflow, but route the commands
through argument arrays to avoid shell interpolation.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 2ca83b12-abc7-412f-9bc7-36979d60ff67
📒 Files selected for processing (7)
docs/runtime/toml.mdxpackages/bun-types/bun.d.tssrc/parsers/toml.rssrc/runtime/bake/DevServer/js_escape.rstest/js/bun/toml/generate_toml_test_suite.tstest/js/bun/toml/toml-test-suite.test.tstest/js/bun/toml/toml.test.ts
#33874 removed the debug_assert block that consumed `result`, leaving a clippy::let_and_return violation that every open PR now trips. The code is identical on main; this is merge hygiene.
…regex The literal `"xxx"` padding matched the gate's TODO/FIXME/XXX scan as a false positive. The test only cares that the padding is 3 bytes.
Collaborator
|
Post-sync CI triage. Of the three fast-gate failures on 3b9c945, two are inherited from main and one is a false positive; two small commits pushed to clear the ones that can be cleared here.
Review threads: 31/31 resolved, none opened since the last round. Buildkite #71674 is still mid-flight. |
Conflicts with #33909 (per-crate thiserror enums replacing bun_core::Error): - src/parsers/toml.rs: kept this branch's new parser; adapted TOML::parse to return crate::Result<Expr> and map the internal PErr to parsers::Error::{SyntaxError, Alloc, StackOverflow}. - src/parsers/toml/lexer.rs (modify/delete): kept deleted. - src/parsers/error.rs, src/runtime/error.rs: removed the From<toml::lexer::Error> impls that referenced the deleted file. - src/runtime/bake/dev_server/js_escape.rs: adapted to return Result<(), crate::Error> with crate::Error::SyntaxError. - src/runtime/api/TOMLObject.rs: matched on bun_parsers::Error::Alloc(_) in place of the removed bun_core::err!("OutOfMemory"). - src/ast/lexer_log.rs: doc comment combines both sides (js only, new crate::Error syntax).
This was referenced Jul 12, 2026
Contributor
There was a problem hiding this comment.
No issues found on this pass. Deferring to a maintainer for final sign-off given the scope — a full parser rewrite, a new public TOML.stringify API, and a user-visible tightening of what bunfig.toml accepts.
Checked the scanner's bounds handling at position 0 (check_underscore's wrapping_sub, peek_at past EOF), the meta HashMap keying on arena pointer addresses (stable for the parse lifetime), that append_property carries own_key_property_flags so the #33072 __proto__ hardening is preserved, and that the reordered JSC__JSValue__toISOString signature has no other callers. All 31 prior review threads are resolved.
Extended reasoning...
Overview
This PR replaces Bun's TOML parser with a from-scratch TOML v1.1.0 implementation (~1,860 lines in src/parsers/toml.rs), adds a new public Bun.TOML.stringify API (~530 lines in TOMLObject.rs), extracts a shared expr_to_js conversion into src/runtime/api.rs (now used by both TOML and JSON5), extracts the bake dev-server's JS-escape decoder into a standalone module, deletes the old TOML lexer and three now-unused E::Object rope helpers, fixes a dead-write bug in the pre-existing JSC__JSValue__toISOString C++ binding while reordering its parameters, and adds ~8,500 lines of generated conformance tests plus hand-written coverage, docs, and .d.ts updates.
Security risks
The parser processes untrusted input (config files, module imports). The relevant surfaces — UTF-8 validation via simdutf before any byte scanning, bounded peek_at, checked integer accumulation with explicit overflow flags, StackCheck on recursive value parsing, and __proto__ handled as an own property via own_key_property_flags — were reviewed and look correct. The O(N²) duplicate-key check on very wide tables (documented in the thread) is a pre-existing shape and not a regression. No injection or auth surface.
Level of scrutiny
High. The TOML parser backs bunfig.toml, so a defect here can break Bun's own startup. The PR also introduces a new public API and a deliberate behavioral tightening (~1% of sampled real-world bunfigs will newly error). These are product-level decisions a maintainer should sign off on, independent of correctness.
Other factors
The PR has been through 31 review threads (all resolved), including a maintainer review from Jarred whose date-conversion concern was addressed by routing through JSC's dateCache. Test coverage is unusually strong: the full official toml-test suite (708 cases, every rejection asserting the exact message), 217 round-trip assertions, and targeted GC/recursion/boundary tests. The bug-hunting pass on the current head found nothing new. None of that changes the fact that a ~2,400-line native rewrite plus a new public API is outside the bar for auto-approval.
robobun
added a commit
that referenced
this pull request
Jul 18, 2026
robobun
added a commit
that referenced
this pull request
Jul 18, 2026
This was referenced Jul 30, 2026
This was referenced Aug 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this adds
A spec-exact TOML v1.1.0 parser (
src/parsers/toml.rs), replacing the previous JS-lexer-derived implementation, plus the official toml-lang/toml-test conformance suite as a generatedbun:testfile, following the same pattern as the YAML and JSON5 conformance suites.Conformance: 100% of the official suite, before → after
Measured against the toml-test v1.1.0 manifest (commit
4d77658d): 217 valid + 481 invalid + 1 out-of-range-integer + 9 invalid-encoding cases — every case in the manifest, zero exclusions.¹ The 9 invalid-encoding cases contain ill-formed UTF-8 that a JS string cannot carry; they became testable when
TOML.parsegained binary input in this PR.Parser
inf/nan, underscore and leading-zero validation,0x/0o/0bintegersNumber.MAX_SAFE_INTEGERthrow instead of silently losing precision (TOML requires lossless handling or an error)\xHHand\eBun.TOML.parseSyntaxErrorwith precise messages (previously aBuildMessage), matchingJSON.parse,Bun.YAML.parse, andBun.JSON5.parseBlob/TypedArray/DataView/ArrayBufferinput like the YAML and JSON5 siblingsBun.TOML.stringify(new)Serializes a JavaScript object to a TOML document — this API did not exist before. Same surface as the YAML/JSON5 siblings:
(value, replacer, space), wherereplacerthrows andspaceis accepted but ignored (TOML output is line-oriented).[table]and[[array-of-tables]]sections; mixed arrays use inline tables; keys are bare when possible, quoted otherwiseDatevalues become TOML offset date-timesnull,BigInt, circular structures, and non-object top-level values throw (TOML cannot represent them);undefined/function/symbol properties are skippedTOML.parsestring inputTests
test/js/bun/toml/toml-test-suite.test.ts— generated official suite (708 tests); every rejection test asserts the exact full error messagetest/js/bun/toml/generate_toml_test_suite.ts— pins the upstream commit, decodes the suite's tagged-JSON expectations, inlines every case;--checkmode fails if the committed suite is staletest/js/bun/toml/toml.test.ts— hand-written coverage beyond the official suite: input types (Buffer/TypedArray subviews/DataView/ArrayBuffer/SharedArrayBuffer/Blob), JS value mapping (__proto__safety, key ordering, no Unicode normalization), safe-integer boundaries, source-text date/time preservation, multi-line string edge cases, recursion-limit and GC robustness, the SyntaxError message contract, andTOML.stringify(exact layout output, escaping, round-trips, every error contract, and a GC stress test)Performance
1.15×–1.49× faster than the previous parser at every document size, growing with size. Release builds from this pipeline (darwin-aarch64), interleaved A/B, best of 3 rounds (each sample = median of 15 timed batches), real-world inputs both parsers accept:
uv.lock(apache/airflow)uv.lock(langgenius/dify)poetry.lock(python-poetry/poetry)Cargo.lock(zed-industries/zed)Cargo.lock(this repo)pyproject.toml(python-poetry/poetry)poetry.lock(truncated at a package boundary)bunfig.tomlBasic strings borrow the source bytes when no escape requires decoding (the same approach the YAML parser uses for plain scalars), so string-dominated documents — lockfiles in particular — see the largest gains.
Behavioral note: stricter parsing of invalid config files
The old parser silently accepted some invalid TOML, so a small set of existing
bunfig.tomlfiles (about 1% in a sample of 99 real-world bunfigs from GitHub) will go from silently tolerated to a startupSyntaxErrorthat states the fix. The one pattern observed in the wild is unquoted string values — easy to carry over from.npmrcsyntax — which now produce a purpose-built message:The other formerly-tolerated patterns (backslash escapes in basic strings being silently dropped, missing newlines between key/value pairs, bare keys containing
:) were silent data corruption or non-portable syntax that every other TOML implementation already rejects; none appeared in the sample.Deletions
The old TOML lexer (
src/parsers/toml/lexer.rs), and the now-caller-lessE::Object::set_rope,get_or_put_array, andset. The bake error reporter's JS-escape decoder (which borrowed the old TOML lexer) is extracted standalone with its exact semantics.Closes #22426
Closes #28680
Closes #28681
Closes #28687
Also addresses the TOML half of #32025 (
\u{…}is now correctly rejected; the JS-lexer half remains).Merge with main (38cfba6)
One conflict, in
src/ast/e.rs. Main's hardening round (#33072) addedflags: own_key_property_flags(&key)to the property-construction sites, which marks a__proto__key as computed so it becomes an own property rather than setting the prototype. This branch deletesset,set_rope, andget_or_put_array, replacingsetwithappend_property.Git merged the new flags into
append_propertyon its own; the conflict was only the bodies of the two functions this branch removes. Kept them removed, after checking that neither has a caller anywhere in the tree and that every live construction site (put,get_or_put_object,append_property) carries the new flags, so the hardening is preserved on every path that still exists.Verified on the merge result: the TOML suites pass (810 tests, including the full conformance suite), and the INI and
bun inittests pass (73 tests) since they are the remainingget_or_put_objectcallers.test/bake/dev/production.test.tsalso passes again now that #33204 pinned the React build the bake harness installs.Merge with main (dbac342)
Conflicts with #33722, which fixed
\u{...}overflow and unterminated-brace handling in the old TOML lexer's copy of the JS escape loop.src/parsers/toml/lexer.rs(modify/delete): kept deleted. The new parser rejects\u{...}at the opening brace (\u{…}is JavaScript syntax, not TOML), so neither the overflow nor the unterminated-brace path is reachable.test/js/bun/resolve/toml/toml-parse.test.ts: kept this branch's content and folded the Panic in debug on invalid unicode escape sequence #30825 crash-regression inputs into the existingrejects JS-style \u{XX} escapestest. Main'sstill accepts in-range \u{...}test asserted\u{41}decodes to"A", which contradicts spec-compliant TOML and is already asserted to throw by this file.src/runtime/bake/DevServer/js_escape.rs: ported the one-line unterminated-brace fix, since that decoder was extracted from the code lexer: fix\u{...}escape overflow and unterminated-brace accept #33722 patched and is documented as preserving its exact semantics. Overflow was already prevented by the existing0x10FFFFclamp.Merge with main (aba8757)
Conflicts with #33925 (repo-wide snake_case directory cleanup), which renamed
src/runtime/bake/DevServer/todev_server/and replaced the#[path = "../DevServer/..."]module-path attributes with plainmoddeclarations.src/runtime/bake/dev_server/mod.rs: adopted main's simplified submodule block and added thejs_escapedeclaration.src/runtime/bake/DevServer/js_escape.rs: moved todev_server/js_escape.rsfollowing the rename. Git tracked the rename of this branch'sErrorReportRequest.rschanges intoerror_report_request.rs, and itssuper::js_escape::decode_js_escape_sequencescall is intact.Merge with main (ace42f9)
Conflicts with #33909, which replaced
bun_core::Errorwith per-cratethiserrorenums across the codebase.src/parsers/toml.rs: main changed the old file's error handling; this branch replaces the whole file. Kept the new parser and adaptedTOML::parseto returncrate::Result<Expr>, mapping the internalPErrtoparsers::Error::{SyntaxError, Alloc, StackOverflow}.src/parsers/toml/lexer.rs(modify/delete): kept deleted.src/parsers/error.rs,src/runtime/error.rs: removed theFrom<toml::lexer::Error>impls that referenced the deleted file.src/runtime/bake/dev_server/js_escape.rs: adapted to returnResult<(), crate::Error>withcrate::Error::SyntaxError, sincebun_core::err!is gone.src/runtime/api/TOMLObject.rs: matched onbun_parsers::Error::Alloc(_)in place of the removedbun_core::err!("OutOfMemory").src/ast/lexer_log.rs: doc comment combining both sides.