bake: speed up the css dev server tests by sharing servers and asserting served stylesheets - #37868
bake: speed up the css dev server tests by sharing servers and asserting served stylesheets#37868robobun wants to merge 4 commits into
Conversation
…ssert the served stylesheets test/bake/dev/css.test.ts booted one dev server per case (15) and a happy-dom client for nearly every assertion (20 clients, 2 hard reloads) to check what is mostly server state, and every write made with a client attached paid the harness's one second error overlay poll. Group the cases onto 6 servers, one HTML route per case, keep one client per case that exercises the client side of a hot update (16), and read the rest over HTTP: the exact stylesheet chunk served for each route, response status and content type, and the Build Failed page for routes with errors. Writes whose following assertion can only pass once the rebuild reached the client pass errors: null; writes that recover from an error keep the overlay check, since that is what they assert. Cases that depend on the exact contents of the server (the asset table layout, the bunfig plugin) keep their own server. Routes with a failing stylesheet are fetched only after their client is gone, since re-bundling such a route ships the HTML module to connected clients (issue 31908); the existing resolve-error case only passed because the client was disposed before it applied that update.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (1)
WalkthroughThe CSS development-server tests were reorganized into shared multi-route suites. Coverage now includes dependency graphs, assets, dynamic imports, stylesheet failures, recovery, HTML link changes, late-created files, and project-relative paths. Tests also verify served CSS during partial failures and recovery. ChangesCSS development-server test coverage
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
|
Status: test-only change, ready for a maintainer (revision 4f96628; 2613c6b on top is an empty commit that re-ran CI after a pipeline-wide outage). Still applies cleanly to current main. CI: build 93647 finished with 179 of 181 jobs passed and no failures; the other 2 are the darwin 26 aarch64 test jobs, which never got an agent and were expired by Buildkite after three hours (the darwin 14 aarch64 lanes ran and passed), which is what the red aggregate status is. The only annotations are tests unrelated to this file that passed on retry ( Verified locally against both builds, interleaved with the old file on the same machine: release 54.2s -> 31.5s, debug/ASAN 92.1s -> 53.9s; every local run of the new file passed. The description has the per-case assertion table and the review follow-ups: the Windows route-key problem with a nested HTML file on a shared server is addressed in 3d6385d (that case has its own single-route server again, as on main); 4f96628 makes the initial-error case end recovered like the other shared error cases, documents once on Related PRs adding cases to this file: #37844, #37845, #37859, #37867, #37051, #33405 (the file still ends with the untouched |
|
Updated 6:46 PM PT - Aug 12th, 2026
❌ @robobun, your commit 2613c6b has some failures in 🧪 To try this PR locally: bunx bun-pr 37868That installs a local version of the PR into your bun-37868 --bun |
…e server Its HTML file has to live in a subdirectory, and the harness builds multi-route keys from path.relative output without normalizing separators, so on Windows a nested file registers as /html\index and the shared-server version of the case would 404 there. As its own server it takes the single-file catch-all route, as it did before.
There was a problem hiding this comment.
I re-reviewed after 3d6385d and didn't find any further issues — the project-relative case is back on its own single-route server, so the Windows route-key concern from the earlier pass is resolved. Given the scale of the restructuring (15 cases folded into 7 shared servers, ~two dozen writes switched to errors: null, several client/hardReload steps replaced by served-chunk fetches), a human who owns the bake dev server tests should still confirm the coverage mapping and the errors: null reasoning.
What was reviewed:
- Verified the split-out "css import before create project relative" case now takes the harness's single-HTML catch-all branch, avoiding the Windows
path.seproute-key bug. - Checked each shared-server group ends with its error-producing route recovered (or is the last sub-case) so no build failure leaks into the next sub-case's client.
- Confirmed
Devis an exported class anddev.read()exists, so the inlinedwriteNoChangesreplacement type-checks and matches the original behavior pluserrors: null. - Traced each
errors: nullwrite to a following assertion that reads the new value from the client or anexpectReload; none looked vacuous.
Extended reasoning...
Overview
This PR restructures test/bake/dev/css.test.ts to roughly halve its wall-clock time (release ~61s → ~37s, debug/ASAN ~118s → ~66s). It merges 15 independent devTest cases into 7 by sharing dev servers across cases whose fixtures don't interact, replaces several happy-dom client connects and hardReloads with plain HTTP fetches of the served stylesheet chunk (asserted as inline snapshots via new servedCss/stylesheetUrls/fetchCss/expectBuildFailed helpers), and passes errors: null on writes whose next assertion mechanically proves the rebuild reached the client. No production code is touched.
My earlier pass flagged that combining html/index.html with two top-level HTML files would 404 on Windows because the harness's multi-route branch doesn't normalize path.sep in route keys. Commit 3d6385d addresses this by splitting "css import before create project relative" back into its own devTest with the original single-file layout (which takes the catch-all route branch), and documents the reason in a comment. Every other HTML file in the merged groups is top-level, so no remaining case hits the harness bug.
Security risks
None. Test-only change with no changes to runtime, bundler, or dev server code; no new external network calls (all fetches go to the local dev server the harness spawns).
Level of scrutiny
Medium-high despite being test-only. REVIEW.md is explicit that weakening or restructuring existing tests is one of the most-scrutinized categories: "Never silently weaken, skip, or delete an existing test or safety net… When de-flaking, keep asserting the property the original assertion protected." This PR does exactly that at scale — 15 cases' assertions are relocated, several overlay-poll checks are elided via errors: null, and previously isolated cases now share server state. The description's per-case table is thorough and I spot-checked several entries against the diff, but the soundness of each errors: null (that the following .toBe(newValue) / .notFound() / expectReload genuinely cannot pass if the rebuild silently failed) rests on dev-server invariants ("a stylesheet that fails to rebuild keeps its old rules", "reload lists are only sent while bundling_failures is empty") that someone who owns src/bake/DevServer.rs is better placed to confirm.
Other factors
- The added snapshot assertions are strictly stronger than what they replace (exact served CSS bytes vs. only computed-style checks), and
expectBuildFailedasserting the 500 body's<title>is stronger than the oldnot.toContain("HELLO"). - I walked each shared-server group for cross-case bleed:
/crashis repaired before/resolveconnects;/resolveis repaired before/keep;/keepends emptied (valid) before/initial;/initialends broken but is last in its group./beforeis fully recovered before/relinkconnects. The non-error groups never introduce build failures. - The two writes inside
expectNoWebSocketActivityin the toggle case newly passerrors: null; the comment explains the client fixture exits on any socket message inside that block so the overlay check couldn't observe anything — this matches what the project-relative case already did on main. - CI build #93359 was in progress at review time, so cross-platform (Windows) results aren't confirmed here; that plus the sheer size of the coverage rearrangement is why I'm deferring rather than approving.
… failing route may be fetched The error group's header promised every case leaves the shared server without failures, but the initial-error case ended broken and only worked because it was last: the dev server sends no route reloads to anyone while any failure exists. Recover it once more and assert the stylesheet is served again. Fetching a route whose stylesheet is failing re-bundles it and pushes the still-compiling HTML module to clients as a JS hot update (issue 31908), so the hazard is the fetch, not the recovery; verified that a recovery with the page's client attached is fine, that fetching kills a client with that page loaded, and that clients on other routes or on the error page are unaffected. State that once on expectBuildFailed instead of three differing comments. Drop the redundant second fetch in the before-create case, whose ack would otherwise be counted by the recovery write that follows, and in the link-tag case assert the loaded page keeps its stylesheet while the HTML is broken, which also drains that ack.
There was a problem hiding this comment.
Thanks for splitting the project-relative case back onto its own server — that resolves the Windows route-key issue from the earlier review, and this pass found nothing new. Given the scale of the restructuring (15 cases collapsed to 7 shared servers, errors: null skips gated on rebuild-must-have-landed reasoning, and the #31908 ordering constraints on expectBuildFailed), a maintainer familiar with the bake harness should still sign off on the shared-server isolation and the per-case coverage mapping.
Checked this run: the two remaining expectBuildFailed calls made with a client connected (/before, /relink, and both / calls in the project-relative case) all fall under the documented "HTML file itself fails" or "client showing the error page" exemptions; the /initial route's broken-from-the-start stylesheet is not bundled until that sub-case requests it, so it does not leak into the earlier /resolve and /keep overlay checks on the shared error server; and the script.ts find/replace pair is byte-identical to the original single-server case.
Extended reasoning...
Overview
Test-only change to test/bake/dev/css.test.ts: 15 devTest cases are consolidated into 7 by sharing dev servers across cases with non-interacting fixtures, and server-side state is asserted via new servedCss/stylesheetUrls/expectBuildFailed helpers as inline snapshots instead of only through happy-dom clients. The Windows route-key issue I flagged on the previous revision is fixed in 3d6385d by moving html/index.html back to its own single-route server; 4f96628 adds a second-recovery step to the initial-error case and documents when expectBuildFailed may be called with a client attached. No production code, harness code, or other test files are touched.
Security risks
None. This is a test file for the dev server's CSS hot-reloading; it spawns local dev servers and happy-dom clients in the test process, with no auth, crypto, or external I/O.
Level of scrutiny
Medium-high for a test-only change. The risk isn't runtime breakage but silently weakened coverage or new flake surface: cases now share a server, so an error left behind by one sub-case would show up in the next case's overlay check, and errors: null skips the harness's default overlay poll. The PR description's per-case table maps every original assertion to its new location and the file-header comment states the errors: null convention (only where the following assertion mechanically requires the rebuild to have landed). I traced each shared group for cross-contamination: the error group runs its four sub-cases sequentially with at most one client at a time, each ending recovered; the /initial route is only bundled when its sub-case requests it, so its initial syntax error can't appear in earlier sub-cases' overlay checks; and every expectBuildFailed call with a live client falls under one of the two exemptions its docstring names.
Other factors
- The author reports 3 release + 3 debug/ASAN passing runs of the pushed revision (plus 10 more of the prior revision), and CI build #93557 is running on 4f96628.
- Assertion count went from 76 to 260 with 25 inline snapshots of served stylesheet chunks — this is a strengthening, not a weakening, but the snapshots are new surface that a maintainer should eyeball once.
- The shared-server design and the #31908 / #37844 workarounds encoded in the helpers' docstrings are the kind of harness-behavior reasoning a bake maintainer is best placed to confirm; that, plus the ~700-line net diff, keeps this outside the auto-approve bar for me even though I found no defects.
There was a problem hiding this comment.
Thanks for addressing the Windows route-key issue — 3d6385d puts the project-relative case back on its own catch-all server, and 4f96628 makes /initial end recovered so the shared error server finishes clean. I re-reviewed the current revision and found no further issues.
Given the size of the restructure and the ordering constraints between cases sharing a server (lazy per-route bundling so /initial's fixture error stays dormant until requested; every errors: null write followed by a client assertion that only passes on a successful rebuild; expectBuildFailed only called where the page's client is gone or on the error overlay, or the HTML itself is the failing file), a maintainer familiar with the bake harness should still take a look before merge.
Checked: each errors: null site has a following c.style(...).toBe(new) / notFound() / expectReload that fails on a bad rebuild; the /before and project-relative fetches of a broken route happen while the client is on the error overlay (matching expectBuildFailed's stated rule); the two-link /relink step asserts the chunk set only, deferring order to #37845; no case deleted or weakened per the description's mapping table.
Extended reasoning...
Overview
Test-only change to test/bake/dev/css.test.ts (~700 lines). Fifteen devTest cases are consolidated into seven, with unrelated cases sharing a dev server via one HTML route each. Three helpers (stylesheetUrls, servedCss, expectBuildFailed) fetch the route's HTML/chunks over plain HTTP so served stylesheets are pinned as inline snapshots, and clients/hardReloads that only loaded a page once become fetches. Writes pass errors: null where the next assertion mechanically proves the rebuild landed. No production code is touched.
Prior review and follow-ups
My earlier inline comment flagged that combining html/index.html with other HTML files sends the harness through its multi-route branch, which does not normalize path.sep, so the route would register as /html\\index on Windows. 3d6385d resolves this by keeping the project-relative case on its own single-route server (the nested location is what the case tests, so flattening it was not an option). 4f96628 additionally makes the /initial case recover a second time at the end (so every case on the shared error server ends clean, matching the header comment's convention), consolidates the fetch-while-broken rule into expectBuildFailed's doc comment, and drops the second broken-route fetch in /before whose ack the following recovery write would have miscounted.
Sequencing on shared servers
I traced the four grouped tests for cross-case interference. In the "bundling errors" group, initial.css starts broken in the fixture but the dev server bundles lazily, so /crash, /resolve and /keep never see that error (each recovers before the next case's client connects, and /initial runs last). Every expectBuildFailed call site satisfies the rule in its doc comment: /resolve and /initial fetch after the block-scoped await using c disposes; /keep deliberately does not fetch while broken because its client must survive; /before and the project-relative case fetch while the client is still on the error overlay; /relink fetches while the HTML file itself is what fails. Each errors: null write is immediately followed by a client-side style assertion on the new value (or notFound()/expectReload), which cannot pass if the rebuild failed since the "does not kill old styles" case in this same file pins the invariant that a failed rebuild keeps the old rules.
Security risks
None. Test-only; no auth, crypto, network egress, or user-facing surface touched. The tests run against a local dev server on port: 0 via the existing harness.
Level of scrutiny
Medium. The change is confined to one test file and strictly increases assertion count (76 → 264 per the description, 25 of them inline snapshots of served CSS), with a per-case table mapping every original assertion to its new location. The risk is not correctness of shipped code but test coupling/flakiness: cases now share server state, and errors: null skips the harness's overlay poll. The author's justification for each is mechanical rather than empirical, CI on the previous revision (which differs from this one only by the two documented follow-up commits) was green on every lane including both Windows lanes, and multiple interleaved local runs on release and debug+ASAN passed.
Other factors
The diff is large and the shared-server ordering is subtle enough that a maintainer who owns bake-harness.ts should confirm the conventions (particularly the errors: null policy and the #31908 fetch-while-broken rule) match their intent before this becomes the pattern other bake test files follow (#37827 is the same pass over bundle.test.ts, and #37866 changes the harness poll this file still relies on 19 times). Deferring rather than approving on that basis.
Problem
test/bake/dev/css.test.tsis one of the ten slowest non-integration test files: 52-61s per lane intest/expected-durations.json, about the same on release and ASAN lanes.Fix
hardReloads that only loaded a page once become fetches (clients 20 to 16, overlay polls 41 to 19).errors: nullto skip the overlay poll only where the next assertion can only pass if the rebuild succeeded: a stylesheet that fails to rebuild keeps its old rules in the page, and reloads are only sent while nothing is failing. Writes that recover from an error keep the poll.Background
devTest(intest/bake/bake-harness.ts) boots one dev server per case from a map of fixture files. A fixture with several HTML files gets one route per file, which is what lets unrelated cases share a server.dev.client(route)attaches a happy-dom browser in a node process that receives hot updates over a websocket.dev.fetch(route)is a plain HTTP request, so served HTML and chunks can be asserted without a client.errors: [...]asserts specific messages;errors: nullskips the check.expectBuildFailedand every fetch of a broken route in the file satisfies it./_bun/asset/<hash>.css. The helpers only look at those links because a route bundled while its stylesheet was failing keeps its source<link>after recovery (bake: keep the import record source index of a CSS file that failed to bundle #37844).Original description
What
test/bake/dev/css.test.tsis one of the ten slowest non-integration test files (61s on the Windows 11 aarch64 lane, 52-56s on the others pertest/expected-durations.json). The numbers barely differ between release and ASAN lanes because the time is mostly fixed waits in the harness, not bundling. Measured on a local release run of the old file (56s for 15 cases):dev.client()and everydev.write()/patch()made while a client is attached ends inClient.expectErrorOverlay, which polls 5 x 200ms when no overlay is visible. The old file made 41 such calls (17 connects, 22 writes, 2hardReloads), about 41s of the 56s. Each connect costs ~1.45s (node + happy-dom + the poll), each write with a client attached ~1.07s; the bundling work itself is a few ms per step.This PR only changes the test file. The poll in
bake-harness.tsis the remaining shared lever for this file,bundle.test.ts(#37827 is the same pass over that file) andhot.test.ts, and is being looked at separately; after this change the file still pays it 19 times (13 connects without expected errors, 6 writes that recover from an error), which is about 19s of the remaining ~37s.Changes
@importgraph cases, the shared-dependency/asset/script-import cases, the four error cases, and the "stylesheet appears after the server started" and "link tag changes" cases. 15 servers become 7. Three cases keep their own server: the asset-table swap-remove case (needsfirst.cssandsecond.cssto be the only two entries, in that order) and the bunfig-plugin case, which both keep their operation sequence, and the project-relative case, whose HTML file has to stay in a subdirectory (that is what makes its/style/styles.csslink project-relative rather than HTML-relative) while the harness's multi-route branch only registers nested HTML files correctly on POSIX (it builds the route key frompath.relativeoutput without normalizing separators, so on Windows it would be/html\index; a separate harness fix). On its own server it takes the single-file catch-all route exactly as on main.hardReloads are replaced by fetching the route and asserting what a fresh load gets.servedCssfetches a route's HTML, requires exactly one injected stylesheet link, and returns the chunk (asserting 200 andtext/css), so each case pins the exact served stylesheet as an inline snapshot (chunk order of@imported files, how a circular import is printed, what an emptied stylesheet serves, which root gets rebuilt);stylesheetUrlsasserts how many stylesheets a route links;expectBuildFailedasserts a failing route serves the 500 "Build Failed" page rather than just a status, replacing thenot.toContain("HELLO")checks. These add ~20ms to the whole file.errors: nullwhen the assertion right after them can only pass if the rebuild succeeded and reached the client. This is sound for mechanical reasons, not just faster: a stylesheet that fails to rebuild keeps its old rules in the page (the "does not kill old styles" case asserts exactly that, so atoBe(newValue)ornotFound()after the write proves the rebuild landed), and the dev server only emits the route reload list whilebundling_failuresis empty (DevServer.rs, the List 1 block infinalize_bundle), so a passingexpectReloadproves nothing is failing. Writes that recover from an error keep the default check, because the overlay disappearing is what those steps assert. The header comment in the file states the convention. The two writes insideexpectNoWebSocketActivityin the remove/re-add case also passerrors: null: the client fixture exits on any socket message inside that block, so the overlay check there could not observe anything (the project-relative case already did this).expectBuildFailed). The old "css url resolve error" case fetched its broken route with the page's client attached and only passed because the client was disposed before it applied the pushed update (with a debug build the client fixture printsASSERTION FAILED ... at replaceModulesand exits, which the harness's dispose does not notice; a harness fix for that is being handled separately). The restructured case asserts the same three things (overlay text, 500, recovery) with the fetch after the dispose; the "does not kill old styles" case does not fetch its route while broken, since its client has to survive until the stylesheet is hot-swapped back in; the link-tag case does fetch while broken, because there the HTML file itself is what fails.<link>s in (currently reversed; bake: serve an HTML route's stylesheets in source order #37845 fixes it and adds a dedicated case, so the two-link step here asserts the set of chunks), and whether a root that fails to resolve aurl()through a plugin keeps its old rules on the client (it currently does not, while the builtin resolver does; the plugin case asserts what the original asserted). The helpers only look at injected/_bun/asset/links because a route bundled while its stylesheet was failing currently keeps its source<link>after recovery (bake: keep the import record source index of a CSS file that failed to bundle #37844). On the shared servers that leftover tag is a 404, so the recovery reload of the initial-error case currently sits through the client fixture's 1s stylesheet-load check before acknowledging (visible as a "Reached maximum CSS load check attempts" line in the output); the old single-route servers answered that URL with the HTML page instead. bake: keep the import record source index of a CSS file that failed to bundle #37844 removes that second.Where each original case's assertions live
/keepkeep.css:4:1; still red while broken; fix applies (overlay check kept); emptied stylesheet removes the rule/* keep.css */header only). Route is not fetched while broken, see above/initialerrors: null/scripterrors: null/import#00fand body red; after editing the imported file h1 green, body still redhardReload("still works after a reload") is now the exact chunk a fresh load gets, asserted before and after the edit. Write passeserrors: null/asseturl()resolves tobun.png, then to the replacement image after rewriting iturl()resolves to the same bytes at both points. Write passeserrors: null/crashbackground-image: url;); the 500 is the Build Failed page; fixing the file serves the new chunk (also leaves the shared server clean)/resolveresolve.css:2:21; route 500; recovery with no client attached/circular.ared /.b#00f; after the edit.agreen /.b#00ferrors: null/second;first.cssbroken (errors: nullas before); thesecond.cssedit reaches the client; fix write keeps its overlay check; still green afterwards/first(only there to bundle it first) and the final fresh client on/firstare fetches asserting the exact chunk; exact/secondchunk initially, right after the edit (reads the moved asset slot directly) and unchanged after the fix.secondgreen on c2; no.secondon c1; recovery with no clients attached/secondchunk right after the batch;/firstis the Build Failed page before the fix/shared-first+/shared-second, both clients#ff0after editingshared.csserrors: null/toggle.coloredpresent; gone after removing the import; no websocket activity while the orphaned file is edited twice; still gone; back (plus.main) after re-addingerrors: null/relink.otherred / no.test; switching back reloads into.test/ no.other; both links give both.otherchunk, and with two links the served set is exactly those two chunks; route is the Build Failed page while the link is unresolvable and the loaded page keeps its old stylesheet meanwhile.hardReloadis covered by the chunk assertion after the rewrite reload; the three non-recovery rewrites passerrors: null(writeNoChangesis inlined to pass it); link order left to #37845/beforebefore.css:2:21after creating the stylesheet; reload when the image appears (check kept); client'surl()resolves to the image;HELLOservedhtml/index.htmlat the catch-all route,style//assets/layout unchanged)mkdir, both overlay texts, the no-activity create/delete block (errors: nullas before), reload, image bytes,HELLONo case is deleted, skipped or made todo.
Timing
Same machine, same binaries, old and new file run back to back in alternation. Host load varied a lot during the session (load average 50 to 210 on a 12 CPU quota), so absolute numbers move between rounds, but every interleaved pair shows the same ratio. The debug numbers run the binary that
bun bd testbuilds, invoked directly under a separate uid, because this sandbox's shared inotify instance limit stops dev servers from starting as root (the old file fails identically when that happens; #37827 hit the same thing).Dev server boots 15 -> 7, happy-dom clients 20 -> 16,
hardReloads 2 -> 0, overlay polls 41 -> 19 (13 connects without expected errors and the 6 writes that recover from an error),expect()calls 76 -> 264 (25 of them inline snapshots of served stylesheets). Every local run of the new file passed (3 release and 2 debug/ASAN runs of the pushed revision, 16 across the two earlier revisions), with no client assertion traces or "not accepted" hot updates in any log. The largest group takes ~7-10s in release against the harness's 30s per-case floor; everything in it is event driven (acks and reload events), with no timing-based waits added. The remaining 19 polls are what #37866 (harness) removes.Review follow-ups
html/index.htmlwould be registered as/html\indexon Windows (the harness's multi-route branch does not normalizepath.sep; separate harness fix), and moving the HTML to the top level would have removed the thing the case tests.expectBuildFailedafter probing what actually triggers it (see Background), replacing three comments that each described it differently. The before-create case's second fetch of its broken route is dropped: the rebuild it triggered made the client send an ack that the recovery write right after it would have counted. The link-tag case keeps its fetch (its HTML file is the failing file) and asserts the loaded page still has its stylesheet afterwards, which also consumes that ack before the next write.test/expected-durations.jsonis left alone; the scheduled job regenerates it from CI. #37844, #37845, #37859, #37867, #37051 and #33405 add cases to the end of this file; the file still ends with the untouchedextractCssUrl, so those apply on top of this apart from the import line.[stamp-90s] gate passed · iteration 1 · 1 files touched
passes on PR (with fix)
diff hotspot
gate history · 1 passed · 0 rejected · iteration 1
evidence per changed file