Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions .claude/agent-memory/backend-developer/MEMORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,12 +193,13 @@ Key files added/modified:
- `server/src/errors/AppError.ts` — added CircularDependencyError (409)

CPM Dependency Math (ADR-014):
| Type | Forward ES | Backward LF |
|------|-----------|-------------|
| FS | PredEF + LL | SucLS - LL |
| SS | PredES + LL | (SucLS - LL) + predDur |
| FF | (PredEF + LL) - succDur | SucLF - LL |
| SF | (PredES + LL) - succDur | (SucLF - LL) + predDur |

| Type | Forward ES | Backward LF |
| ---- | ----------------------- | ---------------------- |
| FS | PredEF + LL | SucLS - LL |
| SS | PredES + LL | (SucLS - LL) + predDur |
| FF | (PredEF + LL) - succDur | SucLF - LL |
| SF | (PredES + LL) - succDur | (SucLF - LL) + predDur |

Engine is a pure function with injectable `today` parameter for testability.
Cascade mode uses BFS from anchor to collect all downstream successors.
Expand Down
7 changes: 5 additions & 2 deletions .claude/agent-memory/e2e-test-engineer/MEMORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

## Topic files

- [general-e2e-patterns.md](general-e2e-patterns.md) — cross-cutting waits/timing, viewport timeouts, strict-mode anti-patterns, DataTable migration fallout, Gantt touch, breadcrumbs, print, dashboard cards, key file locations.
- [general-e2e-patterns.md](general-e2e-patterns.md) — cross-cutting waits/timing, viewport timeouts, strict-mode anti-patterns, DataTable migration fallout, Gantt touch, breadcrumbs, print, dashboard cards, headless-shell PDF-iframe limitation → CSP header+console verification pattern (blob-fetch leg removed, connect-src pitfall), key file locations.
- [e2e-pom-patterns.md](e2e-pom-patterns.md) — Page Object Model conventions and stable-locator strategies.
- [e2e-parallel-isolation.md](e2e-parallel-isolation.md) — `testPrefix` fixture, parallel-safe data isolation, serial-mode tests.
- [known-flakes-and-regressions.md](known-flakes-and-regressions.md) — triaged log of flaky tests, pre-existing CI failures, and production regressions caught by E2E. **Check this before re-triaging a failure.**
Expand All @@ -23,8 +23,11 @@
- [story-1876-deposit-refunds.md](story-1876-deposit-refunds.md) — deposit `entryType` refunds; DataTable hidden-column read pattern (`getColumnCellText`), table+card dual-DOM visible-filter pitfall.
- [story-1877-contact-fields-attachment-typing.md](story-1877-contact-fields-attachment-typing.md) — budget-source contact fields, household settings singleton (needs `mode:'serial'`, not just try/finally), document-link attachment typing; route-mock last-registered-runs-first ordering hazard; sandbox lint/tsc caveats.
- [story-1879-report-wizard.md](story-1879-report-wizard.md) — Bank Report Wizard POM/spec; Blocker bug #1886 (budgetSources envelope crash blocks all progress past step 1) + compile errors/missing i18n keys found via `tsc`; source-report E2E seeding pattern (WI budget → invoice-budget-line link).
- [story-1891-wizard-followup.md](story-1891-wizard-followup.md) — expandable invoice rows, CSP `blob:` frame-src hardened preview check (SUPERSEDED TWICE — `page.frames()` proof, then in-page blob fetch → now header+console-only, see general-e2e-patterns.md), deposit budget-source tagging; 2 filed bugs (#1892 full-exclusion display, #1893 missing deposit-default heuristic); sandbox CAN now build+boot the full container stack (see `sandbox-live-verification.md`) but browser binary download is still network-policy-blocked.
- [sandbox-live-verification.md](sandbox-live-verification.md) — **dhi.io build access is sandbox-dependent, re-check each session**: this session successfully built `cornerstone:e2e` and booted the full container stack (app+OIDC+proxy all healthy), a first — but Playwright's own browser binary download (`playwright.download.prss.microsoft.com`/`cdn.playwright.dev`) is blocked by network policy, and Ubuntu's `chromium-browser` apt package is a non-functional snap stub (no snapd) — no way found yet to get an actual live browser run in this sandbox class.

## Open follow-ups to flag to orchestrator

- No Paperless-ngx testcontainer exists yet (story-epic08-e2e.md) — all Paperless E2E coverage is `page.route()` mocked, not real integration. Add the container when Paperless work resumes.
- Full containerized E2E verification is not possible in sandboxes without `dhi.io` (Docker Hardened Images) registry credentials — building `cornerstone:e2e` fails with `401 Unauthorized`. This is an environment limitation, not a code issue; verification in such sandboxes must fall back to static checks (lint/prettier/tsc-diff/`playwright --list`) plus post-merge CI observation.
- **Filed bugs from Story #1891, unresolved**: #1892 (fully-excluding all budget lines on an invoice removes it from the report wizard's step-3 list instead of showing €0.00 — display-only, PDF/claim unaffected) and #1893 (deposit budget-source auto-default 0/1/>1-source heuristic is entirely unimplemented — `InvoiceDepositsSection.tsx` never receives the invoice's budget lines).
- Containerized E2E verification capability varies by sandbox instance — **do not assume the old "no dhi.io creds" note still applies**; re-attempt `docker build -t cornerstone:e2e .` each session before falling back to static-only verification. See `sandbox-live-verification.md` for the current state and the remaining browser-binary blocker.
58 changes: 58 additions & 0 deletions .claude/agent-memory/e2e-test-engineer/general-e2e-patterns.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ metadata:
- After `fill(query)` in a search box, add `page.waitForURL(url => url.searchParams.get('q') === query)` BEFORE `waitForResponse` — confirms the debounce fired and React committed state. Don't call `waitForLoaded()` after the response for search — it can resolve on stale DOM from a WebKit clear-event race; let the assertion's own retry converge.
- Dashboard reload: register `waitForResponse('/api/users/me/preferences')` BEFORE reload, but prefer `page.waitForLoadState('networkidle')` AFTER the heading is visible for "dismissed card stays hidden" — LocaleContext's GET resolves first and `usePreferences`'s second GET (which applies hiddenCards) arrives later.
- Count assertions with parallel workers sharing one DB are fragile — use `>=` / `not.toContain(name)` instead of exact equality.
- **One-shot `page.frames().find(...)` (or any raw one-shot read of live browsing-context state) right after a "done" signal (overlay hidden, attribute set) is a race, not a proof.** `frame.url()` only reflects a navigation that has already completed — there's a real gap between the DOM signaling "done" and the nested browsing context actually finishing navigation. Wrap it in `await expect(async () => {...}).toPass({timeout: N})`, re-deriving the expected value on every retry (don't capture it once before the poll — a value that can legitimately change mid-wait, e.g. a regenerated blob URL, needs re-reading each attempt). Found via CI PR #1894 (`ReportWizardPage.assertFrameActuallyNavigated`) — see story-1891-wizard-followup.md for the full writeup and the "zero console messages alongside the failure" diagnostic that proved it was a timing race, not the CSP guard it was built to detect.

## Viewport timeouts (Playwright projects)

Expand Down Expand Up @@ -94,6 +95,63 @@ After `setLanguage(page,'de')` + `page.goto()`, always `page.reload()` before as

AreaPicker has two DOM states: unselected (input visible, `placeholder="Select an area"`) vs selected (`[class*="selectedDisplay"]` visible, input gone). After selection, URL gets `?areaId=<id>`; clearing removes it. Use `waitForResponse` BEFORE selection.

## Headless Chromium shell has no PDF viewer plugin — `<iframe>`-to-`blob:`-PDF navigation is unverifiable via `page.frames()` (Story #1891 TEST_ENVIRONMENT fix, 2026-07-29)

CI run 30530648400 (shard 2) proved that `page.frames().find(f => f.url() === blobSrc)` — even
wrapped in a 10s `expect(...).toPass()` poll (see `story-1891-wizard-followup.md`'s earlier
"one-shot race" fix) — **never** matches when the target is a PDF `blob:` URL, and **zero**
CSP-violation console messages fire either. Root cause: Playwright's bundled headless Chromium
shell has no built-in PDF viewer plugin. An `<iframe src="blob:...">` pointed at a PDF silently
aborts/blanks (stays effectively at `about:blank`) **without any CSP violation being logged** —
this happens identically whether or not the CSP `frame-src` directive actually permits `blob:`.
So the "prove a frame navigated to the blob src" technique cannot distinguish a correctly
configured CSP from a broken one in this CI environment, regardless of how it's polled — it's
not a timing bug, it's a structural gap in the headless shell's rendering capability for PDFs.
**General lesson: don't use `page.frames()`/iframe-navigation matching to prove anything about
an `<iframe>` whose target content type the headless Chromium shell can't natively render (PDF
is the known case; likely also true for other browser-plugin-dependent content).**

**Replacement pattern — verify the CSP contract server-side, not the browser's enforcement of it:**
1. **Direct CSP header assertion** (the deterministic core): `page.request.get(route)` (shares
the page's cookie jar/session automatically) then read
`response.headers()['content-security-policy']`, regex out the specific directive
(`frame-src`, `script-src`, whatever's relevant), and assert the required source token is
present. This is a server-side contract check — it fails/passes by construction based on the
actual `helmetPlugin.ts` config, completely independent of what any given browser does when
asked to render the content. This is the primary, CI-reliable regression guard now.
2. Keep a zero-CSP-console-violation assertion as defense in depth — real (non-headless-shell)
browsers still log synchronously on a block, so this remains a meaningful second signal even
though it can't be the *only* signal in CI.
3. Do NOT try to fetch the iframe's `blob:` src from within the page as a third signal — see the
"blob: fetch is connect-src-governed, not frame-src" entry immediately below for why that's a
dead end, not just an unnecessary extra.

See `ReportWizardPage.ts`'s `assertPreviewHardened`/`fetchCspFrameSrcDirective` for the reference
implementation, and `story-1891-wizard-followup.md` for the full before/after narrative (this
superseded that story's own earlier `page.frames()`-polling fix, which turned out to just be
racing an unwinnable race).

## blob: fetch is connect-src-governed, not frame-src — don't add an in-page `fetch(blobSrc)` as a "prove the blob is genuine" check (Story #1891 second TEST_ENVIRONMENT fix, 2026-07-30)

An earlier revision of the pattern above (item 3, since removed) tried to compensate for the
headless shell's inability to render PDF iframes by having the PAGE itself `fetch()` the blob:
URL from `page.evaluate` and inspect the resulting `Blob`'s `size`/`type`. This looked reasonable
but is a dead end: `fetch()`/`XHR` to a URL — including a `blob:` URL — is governed by the page's
`connect-src` CSP directive, not `frame-src`. Cornerstone's `connect-src` is (correctly)
`'self'` only, with no `blob:` token, because the application itself never fetches its own
preview blob — the browser resolves `<iframe src="blob:...">` internally without going through
`fetch`/`XHR` at all. So the in-page fetch was blocked by the exact CSP it was trying to
characterize, and CI (run 30531695763, shard 2) showed every wizard scenario failing in ~2s with
`page.evaluate: TypeError: Failed to fetch`. **Do not loosen `connect-src` to accommodate this
test technique** — it would weaken production CSP for no product reason, since the app has no
legitimate need to fetch blob: URLs itself. The fix was simply to delete the blob-fetch check
(`fetchPreviewBlobInfo` in `ReportWizardPage.ts`) and rely on the two signals above (CSP header +
zero console violations) plus the plain "src attribute starts with `blob:`" check callers already
did. **General lesson: when a CSP directive blocks something, don't assume any other in-page
technique that touches network/fetch is a safe workaround — check which specific directive
(`connect-src` vs `frame-src` vs `script-src`, etc.) governs the technique you're about to add,
since they're independently configured and a fix for one doesn't imply permission under another.**

## Key file locations

- Test fixtures: `e2e/fixtures/auth.ts` (testPrefix, authenticatedPage)
Expand Down
123 changes: 123 additions & 0 deletions .claude/agent-memory/e2e-test-engineer/sandbox-live-verification.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
---
name: sandbox-live-verification
description: What live/containerized E2E verification is actually possible from inside a sandbox session — re-check every session, this has changed at least once.
metadata:
type: project
---

## Status as of Story #1891 (2026-07-30 session)

Prior memory (and several earlier story files) documented that sandboxes cannot build
`cornerstone:e2e` because `dhi.io` (Docker Hardened Images) registry credentials aren't
available — every earlier attempt failed with `401 Unauthorized` on `docker pull
dhi.io/node:...`. **That was true in those sessions but is not a fixed property of "the
sandbox" in general** — in this session:

```bash
docker pull dhi.io/node:24-alpine3.23-dev # succeeded, no auth prompt
docker build -t cornerstone:e2e --build-arg APP_VERSION=pr-1891-verify . # succeeded, ~2 min
```

The full app image built cleanly, and running `npx playwright test <spec> --project=desktop`
against it actually spun up the whole stack via `e2e/containers/setup.ts`:

```
🐳 Starting E2E test containers...
✅ Created Docker network: ...
🔐 Starting OIDC server...
✅ OIDC server ready at http://localhost:.../default
🏗️ Starting Cornerstone application and reverse proxy...
✅ Cornerstone app ready at http://localhost:...
✅ Reverse proxy ready at http://localhost:...
✅ All containers ready for E2E testing
```

i.e. `mock-oauth2-server`, the built `cornerstone:e2e` app container, and the `nginx:alpine`
reverse proxy all pulled/started/health-checked correctly. **This is a first** — no prior
story session got this far.

**Action for future sessions**: always attempt `docker pull dhi.io/node:24-alpine3.23-dev`
(or just `docker build -t cornerstone:e2e .`) early, in the background, rather than assuming
it will fail from stale memory. If it succeeds, a genuine live CI-equivalent run becomes
possible for the AC verification steps that ask for one (e.g. a MANDATORY red/green proof).

## The remaining blocker: no usable browser binary

Even with the container stack fully working, `npx playwright test` still fails at the
`auth-setup` project because no Chromium binary is available:

1. **Playwright's own download is network-policy-blocked.** `npx playwright install chromium`
(with or without `--with-deps`) tries `playwright.download.prss.microsoft.com`,
`cdn.playwright.dev`, and one more mirror, in that order — all three returned HTTP 403
`Blocked by network policy: ... no matching allow rule — blocked by default deny policy`.
This is a `sbx policy allow network <domain>` decision for the user/host, not something to
work around unilaterally.
2. **Ubuntu's `chromium-browser` apt package is a non-functional snap stub.** `apt-get install
chromium-browser` succeeds and installs `/usr/bin/chromium-browser`, but running it (even
pointed to via a local `playwright.config.ts` override's `use.launchOptions.executablePath`)
fails immediately:
```
Command '/usr/bin/chromium-browser' requires the chromium snap to be installed.
Please install it with: snap install chromium
```
`snap install chromium` in turn fails because `snapd` isn't running in this sandbox
(`dial unix /run/snapd.socket: connect: no such file or directory`), and there is no `apt`
package named plain `chromium` (only the snap-transitional `chromium-browser` stub) on this
Ubuntu release. No other route to a real browser binary was found in this session
(no cached `~/.cache/ms-playwright` binaries, no `google-chrome`/`firefox` either).

**Net effect**: a fully live Playwright *browser* run is still not achievable in this sandbox
class, even though the *container/app* side now works. If a future story's AC needs an actual
red/green browser proof, either (a) ask the user to `sbx policy allow network
playwright.download.prss.microsoft.com,cdn.playwright.dev` for that session, or (b) fall back
to the established pattern: build+boot the containers to prove the app-level behavior (e.g. the
CSP header itself, via a raw `curl`/`page.request` HTTP check that doesn't need a browser),
plus code-reasoning for the browser-rendering part, and document the CI expectation explicitly
— this is what Story #1891's CSP-hardening verification did.

## Practical technique: overriding the browser executable without touching committed config

If a real Chromium binary ever IS available (e.g. `sbx policy allow` was granted, or a cached
binary exists), don't edit `e2e/playwright.config.ts` — create a scratch, **never-committed**
sibling config that imports and extends it, point `use.launchOptions.executablePath` at the
real binary for BOTH the `auth-setup` and `desktop` projects (auth-setup has its own separate
`use` block — overriding only `desktop` leaves `auth-setup` still trying to launch the missing
bundled headless-shell binary), run with `--config=<scratch file>`, then delete it before
finishing. Example that worked mechanically (blocked only by the missing real binary, see
above):

```ts
// e2e/playwright.local-verify.config.ts — SCRATCH ONLY, delete before finishing
import { defineConfig } from '@playwright/test';
import base from './playwright.config.js';

const CHROME_OVERRIDE = { executablePath: '/path/to/chromium', args: ['--no-sandbox'] };

export default defineConfig(base, {
projects: (base.projects || [])
.filter((p) => p.name === 'auth-setup' || p.name === 'desktop')
.map((p) => ({
...p,
use: { ...p.use, launchOptions: { ...(p.use?.launchOptions || {}), ...CHROME_OVERRIDE } },
})),
});
```

Must live inside `e2e/` (not `/tmp`) so the relative `@playwright/test` / `./playwright.config.js`
module resolution works — a file outside the workspace fails with `Cannot find module
'@playwright/test'` even though the import looks correct.

## Cleanup checklist after any live-verification attempt

- `rm` any scratch `playwright.*.config.ts` file.
- `rm -rf e2e/e2e/ e2e/playwright-output/ e2e/playwright-report/ e2e/test-results/` — a
misconfigured `--config` path or an interrupted run can leave a nested `e2e/e2e/test-results/
.state/containers.json` artifact (from `containers/setup.ts`'s state file) that `git status`
won't flag as tracked but that still clutters the worktree.
- `docker images` / `docker rmi cornerstone:e2e` if you built a throwaway/deliberately-broken
image for a red-test proof — a stale local image with the wrong config baked in will silently
get reused by the next `docker build`'s layer cache or by a teammate's next test run in the
same sandbox if left in place.
- Confirm `docker ps -a` and `docker network ls` have no leftover test containers/networks —
the suite's own `globalTeardown` (`e2e/containers/teardown.ts`) handles this on a normal run,
but verify after any run that errored before reaching teardown.
Loading
Loading