node:test: republish run() child events at the child's nesting - #39284
node:test: republish run() child events at the child's nesting#39284robobun wants to merge 2 commits into
Conversation
|
Updated 2:00 AM PT - Aug 16th, 2026
✅ @robobun, your commit cfbdcc79567c464150ebada0b3705fa24c2668f6 passed in 🧪 To try this PR locally: bunx bun-pr 39284That installs a local version of the PR into your bun-39284 --bun |
|
Reproduced on main with a two-file repro (a file whose top-level test has one subtest, driven by Scope note: the first revision also re-counted |
|
Warning Review limit reached
Next review available in: 19 minutes Limit details: You’ve used all 5 included reviews currently available under your plan. Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (2)
Comment |
There was a problem hiding this comment.
LGTM — focused node:test run() compat fix that stops adding +1 to child event nesting and threads correct testNumber/topLevel values through, matching Node v26.3.0.
What was reviewed:
- Traced the new
testNumber = runCounts.topLevel + counts.topLevelarithmetic through the three-file fixture (nested/empty/second) — matches the expected 1/2/3, 2, 5 sequence and the run-level plan count of 5. - Confirmed
addRunCountsfoldsfileCounts.topLevelinto the run accumulator between files, sorunCounts.topLevelis the completed-files total when read inrepublishChildEvent. - Checked that removing the unconditional
fileCounts.topLevel++and moving it underreportFileNodedoesn't break the per-filetest:summary(it's still counted before emission). - Verified the dropped
?? 0fallback ondata.nestingis safe: the child'snestingOf()always emits a number.
Extended reasoning...
Overview
This PR fixes node:test's run({ files }) to republish child-process events with the child's own nesting value instead of nesting + 1, matching Node's FileTest#handleReportItem. It also threads a per-file testNumber (the file's 1-based position in the run) into runOneFile/reportCancelledFile, counts each nesting-0 verdict toward topLevel inside republishChildEvent, and only counts the synthetic file node in topLevel when it is actually reported (#skipReporting() semantics). A new ~100-line test in node-test.test.ts drives three files through run() and asserts nesting, testNumber, the run-level test:plan count, and per-file/run test:summary counts against Node v26.3.0's output.
Security risks
None. This is purely event-metadata bookkeeping in the node:test compat layer — no auth, crypto, filesystem, or network surface changes. The child process spawn path is unchanged.
Level of scrutiny
Medium-low. The change is confined to three functions in src/js/node/test.ts that already existed; the diff is mechanical (remove a +1, thread an index parameter, move one increment inside a conditional, add a nesting-0 branch that assigns testNumber). Each behavioral claim is cited against a specific Node source location (FileTest#handleReportItem, countCompletedTest, #skipReporting, report()) and verified against a comparison table of Node v26.3.0 output. Files run sequentially, so reading runCounts.topLevel mid-file while addRunCounts runs at file end has no ordering hazard.
Other factors
The test is comprehensive for a compat fix: it exercises a subtest (nesting 1), top-level pass/fail/skip, an empty file (file node reported), and a follow-on file (cross-file numbering), and asserts exact values via toEqual rather than shape checks. The PR states the rest of the file's 46 tests and the vendored run() tests still pass. The author also confirmed the new test fails on main (2/1/1/1 vs 1/0/0/0) and passes with the fix. The one acknowledged remaining gap — the subtest's own testNumber — is a separate child-side numbering issue that Node also forwards from the child, so it is correctly out of scope here.
run() added 1 to the nesting of every event it republished from a file's child process, so a file's top-level tests were reported at nesting 1 and their subtests at nesting 2. Node forwards the child's nesting unchanged: top-level tests are nesting 0 under run(), as when the file runs on its own. The file node that run() reports for a file with nothing else to report (or one that failed or was cancelled) always carried testId 1 and testNumber 1; node numbers its FileTests by their position in the run.
b58f9ee to
8d514e3
Compare
There was a problem hiding this comment.
Additional findings (outside current diff — PR may have been updated during review):
-
🟡
src/js/node/test.ts:634-639— The PR description's Fix bullets 2–3 and the "bun with this change" comparison column claim that nesting-0 verdicts gettestNumber= run-position and are counted intopLevel, and that the file node only adds totopLevelwhen reported — but neither is in the diff:republishChildEvent()never assignsdata.testNumberor touchescounts.topLevel, andfileCounts.topLevel++at line 566 is still unconditional. Republished top-level tests keeptestNumber: 0and run-leveltopLevel/test:plancount is still the number of files, not the table's 1/2/3/5 and 5. Either add the missing logic (and assert it — the new test collectstestNumberfor every verdict but only checks it on the file node), or trim the description/table to what actually shipped: nesting pass-through + file-node ordinal.Extended reasoning...
What the description claims vs. what the diff does
The PR description's Fix section states three things:
republishChildEvent()forwards the child'snestingas-is — implemented (the+ 1on line 630 is removed).- "Every nesting-0 verdict is counted in
topLeveland giventestNumber= its position among the run's top-level tests" — not implemented. - "The file node only adds to
topLevelwhen it is actually reported" — not implemented.
And the comparison table's "bun with this change" column shows
top → testNumber 1,failing → 2,skipped → 3,second-top → 5,nested.js topLevel 3,plan count 5,run topLevel 5.Code trace
republishChildEvent()(src/js/node/test.ts:628-669) after this diff:- Sets
data.file, buildsdata.details, deleteserror/duration_ms/type. - Increments
counts.suitesorcounts.tests+ one ofskipped/todo/passed/failed. - Never assigns
data.testNumber. The child emitstestNumber: 0(lines 717, 740, 2475), and that value passes through unchanged. - Never touches
counts.topLevel.
fileCounts.topLevel++(src/js/node/test.ts:566) is unconditional — it runs beforereportFileNodeis even computed on line 567. So every file contributes exactly 1 totopLevelregardless of whether its file node is reported.Concrete example (the description's own three-file scenario)
Files:
nested.js(top/sub/failing/skipped),empty.js,second.js(second-top). What the code actually produces:event name nesting testNumber table claims test:pass sub 1 0 0 ✓ test:pass top 0 0 1 ✗ test:fail failing 0 0 2 ✗ test:pass skipped 0 0 3 ✗ test:pass empty.js 0 2 2 ✓ (this is the ordinalfix)test:pass second-top 0 0 5 ✗ Per-file
nested.jstopLevel: 1 (unconditional++at line 566), table claims 3. Run-leveltest:plancount /counts.topLevel: 3 (one per file viaaddRunCounts), table claims 5.Why the new test doesn't catch this
The test at node-test.test.ts:395-403 collects
{ type, name, nesting, testNumber, testId }for every verdict, but the first assertion maps to only[type, name, nesting]— droppingtestNumber— and the second assertion checkstestNumber/testIdonly onverdicts.at(-1)(the empty.js file node, which is fixed viaordinal). Neithertest:plancount norsummary.counts.topLevelis asserted. So the test exercises exactly the two things the diff does change and none of the unimplemented claims. Per REVIEW.md "assert the strongest invariant", the test should either asserttestNumberon the republished verdicts or not collect it.Impact and severity
The code that is in the diff is correct and nothing regresses: nesting is now forwarded unchanged (the headline fix), and file nodes get their run-position ordinal instead of a hardcoded
1. The unimplementedtestNumber/topLevelbehavior is unchanged from main — it's not a regression, just scope the description claims but the diff doesn't deliver. Per the PR-description-mismatch default this is a nit: merging as-is ships a correct partial fix with an overstated description (CLAUDE.md rule 11: "NEVER overstate what you got done"), not incorrect behavior.How to fix
Either path resolves it:
(a) Implement what the description says. In
republishChildEvent(), for non-suite verdicts withdata.nesting === 0, dodata.testNumber = ++counts.topLevel(withcountsbeing the run-level accumulator so numbering spans files, matching node'scountCompletedTest), and move line 566'sfileCounts.topLevel++insideif (reportFileNode). Then extend the test to asserttestNumberontop/failing/skippedand asserttest:plancount /summary.counts.topLevel.(b) Trim the description. Drop Fix bullets 2–3 and correct the "bun with this change" column to show
testNumber0/0/0/0 for the republished verdicts andtopLevel/plan = number of files. The diff then honestly describes nesting pass-through + file-node ordinal, which is what shipped and what the test covers.On the refutation
One verifier refuted bug_002 as a duplicate of bug_001 — correct, and both were merged into this single finding. The refutation does not dispute the substance.
|
The description that review read was the first revision's; the diff had just been reduced (8d514e3) and the body was updated shortly after. The current description matches the diff: the child's |
There was a problem hiding this comment.
LGTM — focused node:test compat fix with tests pinned to node v26.3.0's output.
What was reviewed:
republishChildEvent()no longer bumpsnesting; confirmed the child side always setsnesting: nestingOf(node)so the removed?? 0fallback was dead.ordinalthreading throughrunOneFile/reportCancelledFile— all six formertestId: 1/testNumber: 1sites updated consistently.- Checked no other consumer in
test.tsreads the republishednestingvalue (counting still keys offtopLevel, unchanged). - Tests: both new cases drain pipes concurrently, use
tempDir/bunEnv, assert exact event sequences; the abort case exercisesreportCancelledFilewithout spawning children.
Extended reasoning...
Overview
Two-file change to node:test's run() implementation. In src/js/node/test.ts, the parent-side republishChildEvent() stops adding +1 to the child's reported nesting (one line removed), and runFiles() now threads each file's 1-based position as ordinal into runOneFile() / reportCancelledFile(), which use it for the file node's testId and testNumber in place of six hardcoded 1s. A one-line comment on republishChildEvent() records why nesting is passed through unchanged. Two new subprocess tests in test/js/node/test_runner/node-test.test.ts pin the exact verdict/nesting sequence for a two-file run and the ordinals for an aborted-before-start run.
Security risks
None. This is event-metadata reshaping in the node:test compat layer — no parsing of untrusted input beyond what already existed, no auth/crypto/fs/network surface touched. The child JSON events are produced by bun's own spawned bun test process.
Level of scrutiny
Low-to-moderate. The change is mechanical (delete a +1, thread a loop index through as a parameter) and sits in the Node.js compat layer where the reference implementation is the spec. The PR description diffs the output against node v26.3.0 line-for-line and explains why the earlier, broader revision was scaled back (top-level describe() verdicts aren't emitted yet). I verified the removed ?? 0 fallback was not load-bearing: every child-side emit path (emitRunChildEvent callers at lines 712/735/2470) sets nesting: nestingOf(node), so data.nesting is always present on republished events.
Other factors
The comment-cop bot flagged paragraph-long comments on an earlier revision; the author trimmed them in cfbdcc7 and both threads are resolved. Tests follow repo conventions (test.concurrent, tempDir, bunEnv, concurrent pipe draining, exact-value assertions, 30s headroom for the debug+ASAN child matching neighbouring tests). The PR notes the existing 45 cases in the file plus five vendored run() tests still pass. No CODEOWNERS entry covers src/js/node/. Counting logic (topLevel, test:plan) is deliberately left unchanged with a clear rationale tied to #39331.
Problem
run({ files })fromnode:test, every republishedtest:pass/test:failevent carries anestingone higher than node's: a file's top-level tests come out at nesting 1 and their subtests at nesting 2. Node v26.3.0 reports 0 and 1, exactly what the file reports when run on its own.republishChildEvent()insrc/js/node/test.tsdiddata.nesting = (data.nesting ?? 0) + 1on every event it forwarded from the child process. Node'sFileTest#handleReportItem(lib/internal/test_runner/runner.js) forwards the child's nesting unchanged.test:enqueue/test:dequeue, and its verdict when the file had nothing else to report, failed, or was cancelled) always carriedtestId: 1andtestNumber: 1, whatever its position in the run. Node gives the second file 2, and so on.Fix
republishChildEvent()forwards the child'snestingas-is.runFiles()passes each file's 1-based position torunOneFile()/reportCancelledFile(), which use it as the file node'stestIdandtestNumber.empty.jsreporting astestNumber: 2, testId: 2, are node's output line for line (details below).topLeveland the run-leveltest:planstill count one item per file. Node counts the file's nesting-0 verdicts instead, but bun's child does not yet report top-leveldescribe()suites (node:test: report a verdict for each suite registered with bun:test under run() #39331 adds that), so counting nesting-0 verdicts here would report 0 top-level items for a describe-only file; that recount belongs with node:test: report a verdict for each suite registered with bun:test under run() #39331. SubtesttestNumber(node forwards the child's value; bun's child sends 0) is also unchanged.test/js/node/test_runner/node-test.test.ts, newnode:test run()block (two cases: a two-file run pinning the nesting column and the file node's number, and a run aborted before it starts, pinning both cancelled file nodes' numbers). Both fail on main (nesting 2/1/1/1, file node number 1) and pass with the fix. The other 45 cases in the file and the vendoredrun()tests (test-runner-todo-skip-tests.js,test-runner-expect-error.js,test-runner-expect-error-but-pass.js,test-runner-filetest-location.js,test-runner-tags-experimental-warning.mjs) still pass.--testCLI mode, node:test/reporters, standalone execution, and reporter-output parity (+18 tests, test_runner 32%→55%) #34515 rewrites this code path as part of a much larger change and also drops the+1; this is the standalone fix for the bug on main today.Background
run()is node:test's programmatic runner. With the defaultisolation: 'process'it runs each file in a child process; the child reports its tests as events and the parent republishes them on theTestsStreamthatrun()returned, then appends the run-leveltest:plan, diagnostics andtest:summary. Bun'srun()does the same with abun testchild that prints one JSON event per line.nestingis a test's depth in the event stream: a file's top-level tests are 0,t.test()subtests inside them are 1, and so on. Reporters indent by it. Under process isolation node keeps the child's values; the file node that the parent synthesizes for each file also sits at nesting 0.FileTest#skipReporting); a file whose tests merely failed is represented by those tests. ItstestIdandtestNumberare its position among the run's files.node v26.3.0 vs bun for the files used in the test
nested.js(topwith subtestsub, a failing test, a skipped test) andempty.js(no tests), run in that order. Columns: event, name, nesting.Earlier revision of this PR
The first push also moved
topLevel/testNumberonto node's model on the parent side: each nesting-0 verdict counted as a top-level item and was numbered across files, and the file node only counted when reported. That is right once the child reports every top-level item, but today a top-leveldescribe()emits no verdict (#39331 adds it), so a describe-only file would have reportedtest:plancount 0 andtopLevel: 0where main and node both report 1. That part was dropped from this PR; the counting on main is unchanged.