toml: reject bare identifiers at value position - #31253
Conversation
|
Warning Review limit reached
Your plan currently allows 1 review/hour. Refill in 49 minutes and 42 seconds. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more review capacity refills, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than trial, open-source, and free plans. In all cases, review capacity refills continuously over time. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (4)
Comment |
|
Updated 4:15 PM PT - May 23rd, 2026
✅ Your commit a4e823d8 has passed in 🧪 To try this PR locally: bunx bun-pr 31253That installs a local version of the PR into your bun-31253 --bun |
There was a problem hiding this comment.
Thanks for the inf/nan fix — that resolves my earlier concern. The follow-up also swaps Bun.TOML.parse's output pipeline from print_json → JSONParse to ExprJsc::to_js, which changes how every parsed value is materialized; it mirrors JSONCObject exactly and looks right, but it's enough of a scope expansion beyond the original fix that a maintainer should give it a quick glance.
Extended reasoning...
Overview
Four files: src/parsers/toml.rs drops the t_identifier arm from parse_value_inner (the original #31251 fix); src/parsers/toml/lexer.rs adds a len-3 keyword arm promoting inf/nan to t_numeric_literal (addressing my prior review); src/runtime/api/TOMLObject.rs replaces the print_json → to_js_by_parse_json round-trip with ExprJsc::to_js so Infinity/NaN survive; and ~70 lines of new tests in toml-parse.test.ts.
Security risks
None. TOML parsing is a config-file format with no auth/crypto/permission surface; the change tightens validation (rejecting previously-accepted invalid input) and swaps one internal AST→JSValue path for another already used by JSONC and the .toml module loader.
Level of scrutiny
Medium. The parser/lexer edits are mechanical and follow existing patterns exactly (the new len-3 arm is structurally identical to the len-4/len-5 true/false arms; parse_key_segment's t_numeric_literal arm already uses raw() so bare keys named inf/nan keep working, and there's a test for it). The TOMLObject.rs change is a byte-for-byte copy of JSONCObject.rs:31-38 including the ToJSError mapping and the unreachable!() for non-literal variants — and the TOML parser only emits E::Object/E::Array/E::String/E::Number/E::Boolean, so that's sound.
Other factors
My earlier inf/nan concern is fully resolved per option (a). The reason I'm not auto-approving is the TOMLObject.rs pipeline swap: it's correct and well-motivated (strict JSON can't round-trip Infinity/NaN), but it changes the materialization path for all Bun.TOML.parse output, not just inf/nan, and it grew out of a follow-up rather than being part of the original PR scope/description. Existing toml.test.js fixture tests do exercise Bun.TOML.parse through the new path, and the new tests cover all six spec float forms plus negative cases (infinity, nan1, inf-foo), so coverage is good — this just feels like the one piece a human should explicitly sign off on.
There was a problem hiding this comment.
Thanks for the inf/nan follow-up — the lexer promotion and tests look right. I'm deferring rather than auto-approving because the TOMLObject::parse change swaps the whole output pipeline (print_json→JSONParse → ExprJsc::to_js); it mirrors JSONCObject so it's almost certainly fine, but it's a runtime-API behavior change worth a quick human glance, and robobun still shows a ❌ on fbd5dd8.
Extended reasoning...
Overview
Four files: src/parsers/toml.rs drops the t_identifier arm from parse_value_inner so bare identifiers on the RHS of = now error per the TOML spec; src/parsers/toml/lexer.rs adds a length-3 keyword arm promoting inf/nan to t_numeric_literal (mirroring the existing true/false arms); src/runtime/api/TOMLObject.rs replaces the print_json → JSONParse round-trip with ExprJsc::to_js; and test/js/bun/resolve/toml/toml-parse.test.ts adds seven targeted tests.
Security risks
None. This is parser correctness for a data-format API; no auth, crypto, permissions, filesystem, or network surface is touched. Input remains bounded by the existing arena/stack-check machinery.
Level of scrutiny
Medium. The lexer and parser changes are mechanical and pattern-matched against existing code (true/false handling, parse_key_segment's t_numeric_literal arm using raw()). The TOMLObject.rs change, however, swaps the entire Expr→JSValue materialization path for Bun.TOML.parse. It is a byte-for-byte copy of JSONCObject.rs:31-38 and the TOML parser only emits E::{Object,Array,String,Number,Boolean} (all handled by data_to_js), so the unreachable!() is sound — but it's still a runtime-API pipeline change rather than a localized bug fix, and the kind of thing a maintainer should sign off on.
Other factors
- My earlier inline concern (inf/nan regressing to a hard error) was fully addressed in fbd5dd8 via option (a), with tests covering all six spec forms plus bare-key and near-miss cases.
- No CODEOWNERS entries cover these paths.
- robobun's status comment still shows a ❌ on Build #57129 for fbd5dd8; 1cac7bb is a CI retrigger but I don't see a green status yet, so I'd rather not auto-approve over a red build.
- Test coverage for the new behavior is thorough; existing
toml-parse.test.tscases also exercise the newExprJsc::to_jspath indirectly (string escapes, objects), which gives some confidence the pipeline swap doesn't regress encoding.
The TOML lexer classifies `@`, `a-z`, `A-Z`, `$`, `_` followed by identifier-part characters as `t_identifier` so bare keys like `foo-bar` work. The value-position parser was also accepting `t_identifier` and wrapping it in an `E.String`, which silently turned invalid input like `a = @`, `a = foo`, or `a = $` into the strings `"@"`, `"foo"`, `"$"` instead of a syntax error. Per the TOML spec the right-hand side of `key = value` must be a quoted string, number, boolean, datetime, array, or inline table. Dropping the `t_identifier` arm falls through to the catch-all `lexer.unexpected()`, which reports e.g. `Unexpected @` at the right location. The identifier arm in `parse_key_segment` is untouched, so bare keys still work. Fixes #31251
…ML.parse Follow-up to removing the t_identifier arm at value position. The TOML 1.0.0 spec lists `inf`, `+inf`, `-inf`, `nan`, `+nan`, `-nan` as valid floats. The bare forms were silently parsing as the strings "inf" / "nan" (the bug this PR removed); the signed forms were producing `0` because the `t_plus` / `t_minus` arms in parse_value_inner read `self.lexer.number` before calling `expect(t_numeric_literal)`. Removing the t_identifier arm made the bare forms throw instead of silently-wrong, which is a visible regression on spec-valid input. Promote `inf` / `nan` to `t_numeric_literal` in the lexer with `f64::INFINITY` / `f64::NAN` in `self.number`. That covers all six spec forms: bare `inf` / `nan` hit the `t_numeric_literal` value arm, and `+inf` / `-inf` / `+nan` / `-nan` now pass through the `t_plus` / `t_minus` arms cleanly. Bare keys spelled `inf` / `nan` keep working because `parse_key_segment`'s `t_numeric_literal` arm uses `self.lexer.raw()` (source text) for the key name, not `self.number`. The TOMLObject pipeline was going Expr → `print_json` → JSC `JSONParse`, and strict JSON has no `Infinity` / `NaN` — `a = inf` would lex correctly but fail at `JSONParse` with 'Failed to parse JSON'. Switch to `ExprJsc::to_js` (`bun_js_parser_jsc::expr_to_js`), the same direct AST → JSValue walker that JSONCObject and the `import foo.toml` module loader already use. Drops the round-trip and gets the IEEE 754 values through correctly.
macOS lanes expired on buildkite #57129; every other lane (Linux, Windows, musl, baseline, ASAN, build-bun/cpp/rust, aarch64, clippy) passed.
1cac7bb to
a4e823d
Compare
There was a problem hiding this comment.
LGTM — the inf/nan lexer promotion and ExprJsc::to_js swap fully address the earlier feedback, and the TOMLObject error-mapping is a verbatim copy of the proven JSONCObject path.
Extended reasoning...
Overview
Four files: drops the t_identifier arm from parse_value_inner (src/parsers/toml.rs, 6 lines removed), adds a length-3 keyword arm promoting inf/nan to t_numeric_literal in the lexer (src/parsers/toml/lexer.rs), swaps Bun.TOML.parse's output pipeline from print_json → JSONParse to ExprJsc::to_js (src/runtime/api/TOMLObject.rs), and adds ~70 lines of tests.
Security risks
None. This is TOML parsing of caller-provided strings; worst case is a thrown SyntaxError or a wrong value. No auth, crypto, filesystem, or network surface. The new unreachable!() in TOMLObject.rs covers only CannotConvertArgumentTypeToJS / CannotConvertIdentifierToJS / MacroError, which the TOML parser cannot produce — it only constructs E::Object, E::Array, E::String, E::Number, E::Boolean (and this PR removes the only arm that ever did anything else).
Level of scrutiny
Low-to-moderate. The parser and lexer changes are mechanical: the removed arm falls through to an existing catch-all, and the new 3 => lexer arm follows the exact pattern of the adjacent 4 => (true) / 5 => (false) arms. The TOMLObject.rs change is the largest, but the match parse_result.to_js(global) block is byte-identical to JSONCObject.rs:31-38, so it's a copy of an already-shipping pattern rather than a new design. The log.has_errors() guard from #31255 is correctly preserved before the new call, and the #31252 regression test still passes on this path.
Other factors
- My prior review flagged that removing the
t_identifierarm would regress spec-validinf/nan. The author implemented exactly the suggested fix (option a: lexer promotion) and went further to fix the JSValue pipeline soInfinity/NaNactually surface — strict JSON can't round-trip them. That comment is resolved. - Test coverage is thorough: bare identifier rejection (top-level, in arrays, in inline tables), all six
inf/nanspec forms,inf/nanas bare keys, near-miss identifiers (infinity,nan1,inf-foo), booleans, and bare-key alphabet preservation. - No CODEOWNERS entries match these paths.
- The bug-hunting system found no issues on this revision.
|
The TOML parser was rewritten for v1.1.0 conformance in #32953, which landed on main (aca54d5). It deletes The behavior this PR adds is covered on current main. Running the tests added here against main: every rejection case passes ( The one test here that does not pass on main is the one asserting that |
Fixes #31251.
Repro
Per the TOML spec the right-hand side of
key = valuemust be a quoted string, number, boolean, datetime, array, or inline table.@is none of those.Cause
The TOML lexer classifies
@,a-z,A-Z,$,_followed by identifier-part characters ast_identifier— this is deliberately broad so bare keys likefoo-barwork in key position. Butparse_value_innerwas also acceptingt_identifierand wrapping it in anE.String, so any bare identifier-ish token on the right of=silently became a string. Same arm existed in the legacy Zig (src/parsers/toml.zigL262-267).Additionally, the TOML 1.0.0 spec (§Float) lists
inf,+inf,-inf,nan,+nan,-nanas valid floats, but the lexer classified bareinf/nanast_identifier(onlytrue/falsegot length-based promotion). Pre-fix they silently became the strings"inf"/"nan"; the signed forms produced0becauset_plus/t_minusreadself.lexer.numberbeforeexpect(t_numeric_literal)failed on the identifier.Fix
t_identifierarm inparse_value_inner. Falls through to the existing catch-allself.lexer.unexpected(), which produces e.g.Unexpected @at the right location. Thet_identifierarm inparse_key_segmentis untouched, so bare keys still work.inf/nantot_numeric_literalin the lexer withf64::INFINITY/f64::NANinself.number. Covers all six spec forms: bareinf/nanhit the numeric-literal value arm, and+inf/-inf/+nan/-nanflow throught_plus/t_minus. Bare keys spelledinf/nankeep working becauseparse_key_segment's numeric-literal arm usesself.lexer.raw()(source text) for the key name, notself.number.TOMLObject::parsefromprint_json→ JSCJSONParsetoExprJsc::to_js(the same direct Expr → JSValue walker JSONCObject and theimport foo.tomlmodule loader already use). Strict JSON has noInfinity/NaN, so the round-trip was both wasteful and unable to carry those values —Bun.TOML.parse("a = inf")would otherwise fail atJSONParsewithFailed to parse JSON.Rebase conflict resolution
Rebased onto main to pick up #31255 (the fix for #31252, which added a
log.has_errors()recovery check toTOMLObject::parse). Both PRs touch the same function. The resolution keeps both: my newExprJsc::to_jscall happens after thelog.has_errors()surface-errors check, so lexer error-recovery diagnostics (a = [1 2]) still throw correctly.Verification
test/js/bun/resolve/toml/toml-parse.test.tscovers:@,foo,@foo,$,_baron the RHS all throw (bare identifier rejection)a = { x = foo }) throwsa = [foo]) throwstrue/falsestill parse as booleans@foo,$bar,foo-barstill parse as keysinf,+inf,-inf,nan,+nan,-nanreturnInfinity/NaN/-Infinitycorrectlyinf/nanstill workinfinity,nan1,inf-fooat value position still throw (they only match the identifier arm, not the promoted keywords)a = [1 2](from TOML arrays accept missing commas in values like a = [1 2] #31252) still throws on the newExprJsc::to_jspath