test(e2e): isolate preference-mutating specs from the shared admin user - #1961
Conversation
Every spec authenticates as one shared admin, and user_preferences rows
are keyed by user id, so under fullyParallel a concurrent test's write
lands inside another test's assertions. LocaleContext.syncWithServer
treats the server as authoritative, so the victim's UI flips language
mid-test and its localStorage override is deleted by the same sync.
Adds e2e/fixtures/isolatedUser.ts, which overrides Playwright's
storageState option so the plain `page` fixture is already a dedicated
user — keeping trace/video/screenshot instrumentation that a hand-rolled
browser.newContext() loses, and requiring no changes to test bodies. An
auto guard fixture asserts /api/auth/me reports the dedicated user, so a
silent regression to shared-admin writes fails loudly.
Converted dashboard.spec.ts and i18n.spec.ts (per-worker), plus the
colliding describes in diary-uat-fixes.spec.ts and invoices.spec.ts
(per-test). Scenario 6/7's inline helper is now redundant and removed.
i18n.spec.ts keeps serial mode as defence in depth, no longer as its
isolation mechanism.
No spec needed admin-role treatment: requireRole('admin') covers only
/api/users mutations and /api/backups/*, none of which the converted
tests touch.
Fixes #1957
Co-Authored-By: Claude e2e-test-engineer <noreply@anthropic.com>
|
[product-architect] Architecture review of #1961 (issue #1957). VERDICT: CHANGES_REQUIREDThe design is right and I could not break the mechanism. The single blocker is AC6: What I verified independentlyThe
This is also the documented Playwright "authenticate in each parallel worker" pattern, so it is the right mechanism, and it is strictly better than the three hand-rolled The AC1 is exact. And the grep-defined audit has no blind spot, which I checked because a grep for API paths cannot see UI-driven preference writes. The only two in the suite are AC3 holds — no member/admin risk. Swept all four files route-by-route and assertion-by-assertion. No converted test reaches AC4 is satisfied, not relabelled. Per-worker is the right scope, and the justification is even stronger than stated: each shard runs its own container and DB ( Both premise corrections check out. The guard fixture does close the failure mode it claims to.
BlockingB1. AC6 unmet — shard 14 is red, and this PR is the proximate cause via shard redistribution
Plus 2 flaky ( Root cause — this is not a fixture defect, it is a shard-boundary shift. Confirmed empirically by running
Why this blocks rather than becoming a follow-up: Required: get 16/16 green on this head. The right fix is in Also required (cheap, same PR): document the redistribution side effect. Opting a file into a worker-scoped option reshuffles shard membership across the whole suite — that is a non-obvious consequence of an apparently file-local change, and the next person converting a file needs to expect an unrelated shard to go red. One line in the Non-blockingListing these as follow-ups; none needs to hold the merge once B1 is green.
On scope and the audit itselfNo production code changed — AC7 satisfied, and AC6 is the only outstanding criterion, and B1 is what it takes to close it. |
Scenario 4 asserted a suite-global precondition — that no unassigned household item exists anywhere — but ?areaId=__none__ lists every area-less item in the shared DB, and 48 of the suite's 62 createHouseholdItemViaApi() calls across 12 files pass no areaId. It only ever passed while no such spec shared its shard. #1957's test.use() opt-ins changed worker hashes, which moves shard boundaries suite-wide (createTestGroups buckets by _workerHash first, filterForShard slices by cumulative count), relocating this file from shard 15 into shard 14 alongside area-filter.spec.ts, which holds area-less items through a long scenario. Now ANDs the sentinel filter with a q= search for the scenario's own name token, plus a positive control asserting the same search without the sentinel does list the seeded item — otherwise an empty result would be indistinguishable from a search that matched nothing. Back-ports the pattern already green at work-items/no-area-filter.spec.ts:223. Adds no test.use(), so shard assignment is unchanged from the run CI already exercised. Co-Authored-By: Claude e2e-test-engineer <noreply@anthropic.com>
|
🎉 This PR is included in version 2.13.0-beta.51 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
|
🎉 This PR is included in version 2.13.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
Fixes #1957. Entirely within
e2e/— no production code changed (AC7).Audit (AC1)
Posted as an issue comment and as the header block of
e2e/fixtures/isolatedUser.ts. File list is exactly reproducible withgrep -rl "users/me/preferences" e2e/tests/— 5 files, nothing added, nothing dropped:dashboard.spec.tsdashboard.hiddenCards,localei18n.spec.tslocalei18n-categories.spec.tslocale(already isolated)diary-uat-fixes.spec.tsdashboard.hiddenCardsinvoices.spec.tstable.invoices.columnse2e/pages/DashboardPage.tsande2e/pages/InvoicesPage.tsalso match the string, but only insidewaitForResponse()predicates — they observe the app's request and never issue one. Called out as non-entries.Mechanism
e2e/fixtures/isolatedUser.tspackages the pattern the three reference sites hand-rolled, as two opt-in options that override Playwright'sstorageStateoption so the plainpage/contextfixtures are already the dedicated user:isolatedUserPerWorker— one dedicated user per worker (file scope)isolatedUserPerTest— fresh user per test (file or describe scope)Overriding the option rather than hand-building
browser.newContext()keeps Playwright's automatic trace/video/screenshot instrumentation, which the existing hand-rolled helpers silently lose, and means no test-body changes. Verified againstnode_modules/playwright/lib/index.jsthat_combinedContextOptionsdepends on thestorageStatefixture, so the override genuinely re-pointspagerather than being a silent no-op.An
autoguard fixture asserts once per test that/api/auth/mereports the dedicated user — so a future silent regression to shared-admin writes fails loudly instead of quietly reverting.Conversions (AC2, AC4, AC5)
dashboard.spec.tsisolatedUserPerWorker, file scope; Scenario 6/7's inlineloginAsIsolatedDashboardUserdeleted as redundanti18n.spec.tsisolatedUserPerWorker, file scope;serialkept as CPU-contention defence in depth, documented as no longer the isolation mechanism (AC4)diary-uat-fixes.spec.tsisolatedUserPerTeston the Scenario 3 + 7 describes (AC5)invoices.spec.tsisolatedUserPerTeston the"Effective Amount" columndescribeAC3: no spec needed it. Checked route-by-route rather than assumed — server-side
requireRole('admin')exists only on/api/usersmutations and/api/backups/*; client-side gating is only the Settings sub-nav tabs and work-item note edit/delete for others' notes. Nothing the converted tests touch is gated, so the fixture's defaultrole: 'member'is correct throughout.role: 'admin'is supported for a future case that genuinely needs it.Two corrections to the issue's premises
ON DELETE CASCADEdoes not clean up the preference row.DELETE /api/users/:idis a soft delete (deactivateUser+destroyUserSessions), so the user and itsuser_preferencesrows survive. Isolation still holds — the account can never be logged into again and e-mails are never reused — but users accumulate, which drove the per-worker/per-test scope decision below.POST /api/auth/loginis rate-limited to 20 requests / 15 min per IP, and all workers in a shard share the bucket.Why per-worker rather than per-test
Per-test provisioning for all 69 dashboard tests would add ~85 users per run against a
/settings/userspage that renders 100 rows and is scanned row-by-row byedit-user.spec.ts/deactivate-user.spec.ts— arming a future failure — and would push logins toward the 20/15-min cap. Per-worker costs ~15 users and ~1 login per worker; worst realistic per-shard login count is ~10–12 including retries.Verification available in this sandbox
npx playwright test --config e2e/playwright.config.ts --list→ 2673 tests in 108 files, no load or fixture-pool errors. This validates the whole fixture graph (scope violations, cycles) without containers. Independently re-run.npx tsc --noEmit -p e2e/tsconfig.json: 123 errors before, 123 after — all pre-existing (e2eis not covered bynpm run typecheck); zero inisolatedUser.ts.npx eslintandnpx prettier --checkclean on every touched file.Playwright cannot run in this sandbox (browser binaries are network-policy-blocked), so CI is the confirmation — AC6 requires all 16 shards green at whatever shard assignment is in effect.
Residual risks, in order
APIRequestContexttoBrowserContext— a documented Playwright pattern, but first use in this repo. Would fail loudly for all isolated tests at once. (SECURE_COOKIESisfalsein the E2E container, so the cookie transfer should work.)memberrather thanadmin. No role-dependent rendering was found on any page they touch, but this is the first thing to re-check if dashboard tests go red.429fixture error.Kept deliberately: the
beforeEach/afterEachpreference resets, which remain load-bearing against sequential carry-over within a worker; and the explicit preference DELETEs/PATCHes inside converted tests, now belt-and-braces preconditions on the dedicated user.