Skip to content

bun test: nest results under describe scopes instead of repeating the scope path - #35793

Open
jakeboone02 wants to merge 2 commits into
oven-sh:mainfrom
jakeboone02:nest-describe-blocks-streaming
Open

bun test: nest results under describe scopes instead of repeating the scope path#35793
jakeboone02 wants to merge 2 commits into
oven-sh:mainfrom
jakeboone02:nest-describe-blocks-streaming

Conversation

@jakeboone02

@jakeboone02 jakeboone02 commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Branch: nest-describe-blocks-streaming · Commit: 70323bea
Closes #3342

This is one of two alternative implementations of #3342. The sibling PR, #35794,
solves the same problem by buffering each file's results. Pick one.
See Streaming vs. buffered below.

What does this PR do?

bun test repeats the full describe path on every single result line. In a suite with
any real nesting that means the same 40 characters of scope are re-printed hundreds of
times, and the actual test names are pushed off to the right.

This PR prints each describe scope once, as its own dim header line, and indents
its tests beneath it — two spaces per level.

Before

demo.test.ts:
(pass) math > arithmetic > add
(fail) math > arithmetic > subtract [1.84ms]
(pass) math > is a module
(skip) all skipped > x
(todo) all todo > y
(pass) top-level test

After

demo.test.ts:
math
  arithmetic
    (pass) add
    (fail) subtract [0.09ms]
  (pass) is a module
all skipped
  (skip) x
all todo
  (todo) y
(pass) top-level test

Headers are emitted lazily: a scope line is written the first time a result inside it is
about to print, and re-announced if results from a different scope interleave in between.
Nothing is buffered — every line still goes out the instant the test finishes.

Design notes & edge cases
  • Scope path is tracked, not the tree. The reporter keeps a single
    printed_scope_path: Vec<u8> (\x1f-separated). Before printing a result it diffs the
    test's scope path against the last-printed one, emits headers for the segments that
    differ, and stores the new path. This is O(depth) per test with one allocation reused
    for the whole run.
  • Unnamed describes are transparent. describe(() => {...}) contributes no header
    and no indent level, so the anonymous scope doesn't shift its children right.
  • Depth is capped at 64 scopes (BoundedArray), matching the existing limit in
    collect_scopes. Beyond that, indentation stops growing rather than running away.
  • Recap buffers keep the flat format. The skip/todo/fail lists printed after the run
    still read math > arithmetic > subtract — they're a flat index, and nesting them out of
    context would be worse. This means each line is formatted twice, so the side effects of
    formatting (assertion-count errors, GitHub Actions timeout annotations) were extracted
    into print_status_side_effects so they fire exactly once.
  • --reporter=dots is untouched. It uses Layout::Flat and never nests.
  • --only-failures nests the same way; scopes with no failures never print a header.
  • JUnit / LCOV output is unchanged — this is purely the terminal reporter.

How did you verify your code works?

Test runs
  • bun bd test test/cli/test/ — 220 pass, 6 todo, 0 fail
  • bun bd test test/js/bun/test/ — no regressions vs. the parent commit
    (df6c7eed6b has 6 pre-existing failures, 4 of them missing-node_modules load
    errors; this branch has the same set)
  • All test/regression/issue/*.test.ts files that spawn bun test — green
    (5 inline snapshots regenerated for the new shape)

New tests in test/cli/test/bun-test.test.ts cover indentation depth, unnamed-describe
transparency, re-announcing a scope after interleaving, and --only-failures. They fail
under USE_SYSTEM_BUN=1 and pass under bun bd test.

Manually verified: default reporter, FORCE_COLOR=1, NO_COLOR=1, --parallel=2,
--reporter=dots, --only-failures, --rerun-each=2, test.concurrent,
--bail, process.exit() mid-file, GitHub Actions ::group:: output, and JUnit output.

Performance

Release builds, macOS arm64, best-of-5, stdout discarded, stderr piped.
Workloads: wide = 200 files x 200 tests x 3 levels (40k tests); deep = 20 files x
5,000 tests x 8 levels (100k tests).

workload mode build wall (ms) peak RSS (MB) first output (ms)
wide default baseline 89 26.2 6
wide default streaming 92 26.5 6
deep default baseline 143 59.4 6
deep default streaming 159 60.4 7
wide parallel baseline 152 53.5 11
wide parallel streaming 154 53.3 11
deep dots baseline 136 57.7 13
deep dots streaming 131 59.1 13
Full benchmark matrix (all 3 builds x 2 workloads x 4 modes)
workload mode build wall (ms) peak RSS (MB) first output (ms)
wide default baseline 89 26.2 6
wide default streaming 92 26.5 6
wide default buffered 98 26.4 7
wide parallel baseline 152 53.5 11
wide parallel streaming 154 53.3 11
wide parallel buffered 181 53.2 12
wide dots baseline 86 25.8 7
wide dots streaming 87 26.3 8
wide dots buffered 86 26.0 8
wide only-failures baseline 71 26.0 70
wide only-failures streaming 70 25.9 69
wide only-failures buffered 70 26.1 69
deep default baseline 143 59.4 6
deep default streaming 159 60.4 7
deep default buffered 163 58.2 18
deep parallel baseline 95 35.3 20
deep parallel streaming 129 35.8 20
deep parallel buffered 113 35.4 30
deep dots baseline 136 57.7 13
deep dots streaming 131 59.1 13
deep dots buffered 134 59.5 14
deep only-failures baseline 92 52.4 91
deep only-failures streaming 92 51.8 91
deep only-failures buffered 91 51.8 90

baseline = df6c7eed6b (main), streaming = this branch, buffered = the sibling PR.

Summary: memory is unchanged (single reused Vec<u8> for the scope path, ~1 KB).
Wall time is +3% on wide and +11% on deep for the default reporter; --dots and
--only-failures are within noise. Time-to-first-output is identical to today.

Streaming vs. buffered

this PR (streaming) sibling PR (buffered)
Groups split when results interleave yes, scope is re-announced never
test.concurrent / --parallel grouping best-effort guaranteed contiguous
Aggregate status on file/describe lines (#12378) no yes
Time to first output unchanged +12 ms on a 5,000-test file
Peak RSS unchanged unchanged (bounded ~190 KB/file)
Wall time vs. main (default reporter) +3% / +11% +10% / +14%
Failure diagnostics position inline, under their test above the file's block
Lines changed in src/ ~370 ~770

Streaming is the conservative option: it changes only how the scope prefix is rendered and
keeps every other timing property of the reporter identical. The cost is that under
test.concurrent or --parallel a describe can appear more than once, because the
reporter can't know what's still coming.

Known limitations

  • A scope header is re-printed whenever results from another scope interleave. With
    test.concurrent this is common; with sequential tests it never happens.
  • The end-of-run recap lists keep the flat a > b > name form (intentional, see design
    notes).

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@coderabbitai

coderabbitai Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 915ea137-2649-43b2-9173-aca65905bcbd

📥 Commits

Reviewing files that changed from the base of the PR and between 70323be and b839efa.

📒 Files selected for processing (1)
  • test/cli/test/bun-test.test.ts

Walkthrough

Bun’s test console reporter now renders nested describe scopes as hierarchical headers with indented test results. Scope paths are serialized through parallel worker IPC, while recap and flat modes retain path-oriented formatting. Documentation and snapshots reflect the new output.

Changes

Nested test reporting

Layer / File(s) Summary
Scope-aware reporter formatting
src/runtime/cli/test_command.rs, src/runtime/test_runner/bun_test.rs
Describe scopes are serialized and emitted once per scope, with nested and flat layouts selected by reporter mode.
Parallel scope-path transport
src/runtime/cli/test/parallel/*
Workers send scope paths with test results, and the coordinator resets and emits scope headers per file.
Reporter behavior validation
test/cli/test/*, test/js/bun/test/*, test/regression/issue/*
Snapshots and CLI tests cover nested indentation, filtering, recap output, failures, and parallel execution.
Reporter output documentation
docs/guides/test/coverage*.mdx, docs/test/reporters.mdx
Examples show grouped coverage and hierarchical console output.

Possibly related PRs

  • oven-sh/bun#35794: Both PRs implement nested describe scope output and related scope-path transport and rendering changes.

Suggested reviewers: cirospaciari, jarred-sumner

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the main change: nesting bun test results under describe scopes instead of repeating scope paths.
Description check ✅ Passed The description matches the required template and includes both what changed and how it was verified.
Linked Issues check ✅ Passed The changes satisfy #3342 by rendering describe labels once with nested test output and preserving the requested terminal-report behavior.
Out of Scope Changes check ✅ Passed The diff stays focused on reporter behavior, matching docs, and test snapshots, with no obvious unrelated code changes.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/cli/test/bun-test.test.ts`:
- Around line 1526-1537: Update the run helper to drain proc.stdout alongside
proc.stderr, such as by reading both streams in the existing Promise.all, while
preserving the current stderr normalization and exitCode return behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 51767a38-8693-4d0e-b345-c55fd0232c3f

📥 Commits

Reviewing files that changed from the base of the PR and between 04bb5c4 and 70323be.

📒 Files selected for processing (17)
  • docs/guides/test/coverage-threshold.mdx
  • docs/guides/test/coverage.mdx
  • docs/test/reporters.mdx
  • src/runtime/cli/test/parallel/Coordinator.rs
  • src/runtime/cli/test/parallel/Frame.rs
  • src/runtime/cli/test/parallel/runner.rs
  • src/runtime/cli/test_command.rs
  • src/runtime/test_runner/bun_test.rs
  • test/cli/test/bun-test.test.ts
  • test/cli/test/test-filter-lifecycle-snapshot.test.ts
  • test/js/bun/test/bun_test.test.ts
  • test/js/bun/test/concurrent.test.ts
  • test/js/bun/test/describe.test.ts
  • test/js/bun/test/nested-describes.test.ts
  • test/regression/issue/12782.test.ts
  • test/regression/issue/19875.test.ts
  • test/regression/issue/20092.test.ts

Comment thread test/cli/test/bun-test.test.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Nest test/it calls under a single describe label instead of repeating the describe label for each test

1 participant