From fed84750c3b89eb13cce09dfef38c3a6fb1b8346 Mon Sep 17 00:00:00 2001 From: Frank Steiler Date: Thu, 30 Jul 2026 11:08:51 +0200 Subject: [PATCH 1/7] feat(reports): allow blob preview under CSP, expandable invoice items and deposit source allocation - Widen Fastify helmet frameSrc CSP to allow blob: so the report wizard's pdfmake-generated preview renders in its iframe instead of being blocked (#1891, #1892) - Add expandable per-invoice budget-line/deposit rows to the report wizard's step-3 invoice list (ReportInvoiceList), sourced from a richer SourceReportInvoice payload (budgetLines/deposits) - Add invoice_deposits.budget_source_id (migration 0044) so deposits can be tagged directly to a budget source, with a source-selection heuristic in InvoiceDepositsSection (#1893) Fixes #1891 Fixes #1892 Fixes #1893 Co-Authored-By: Claude dev-team-lead (Sonnet 4.6) Co-Authored-By: Claude backend-developer (Haiku 4.5) Co-Authored-By: Claude frontend-developer (Haiku 4.5) Co-Authored-By: Claude translator (Sonnet 4.5) Co-Authored-By: Claude qa-integration-tester (Sonnet 4.5) Co-Authored-By: Claude e2e-test-engineer (Sonnet 4.5) --- .../agent-memory/e2e-test-engineer/MEMORY.md | 5 +- .../sandbox-live-verification.md | 123 +++ .../story-1891-wizard-followup.md | 177 ++++ .claude/agent-memory/product-owner/MEMORY.md | 3 +- .../product-owner/bank-report-wizard.md | 29 +- .../qa-integration-tester/MEMORY.md | 3 +- .../story-1891-report-wizard-followup.md | 143 ++++ .claude/agent-memory/translator/MEMORY.md | 14 + .../ux-designer/feature-spec-history.md | 10 + .../SelectionActionBar.test.tsx | 28 + .../SelectionActionBar/SelectionActionBar.tsx | 4 +- .../reports/ReportInvoiceList.module.css | 131 ++- .../reports/ReportInvoiceList.test.tsx | 688 ++++++++++++++++ .../components/reports/ReportInvoiceList.tsx | 455 +++++++++-- client/src/i18n/de/budget.json | 35 +- client/src/i18n/en/budget.json | 35 +- client/src/lib/invoiceDepositsApi.test.ts | 1 + client/src/lib/reportExclusions.test.ts | 303 +++++++ client/src/lib/reportExclusions.ts | 38 + client/src/lib/reportPdf/merge.test.ts | 2 + client/src/lib/reportPdf/overviewPdf.test.ts | 2 + client/src/lib/reportPdf/realRender.test.ts | 2 + .../InvoiceDepositsSection.test.tsx | 539 ++++++++++++- .../InvoiceDepositsSection.tsx | 127 ++- .../ReportWizardPage.module.css | 31 + .../ReportWizardPage.test.tsx | 266 ++++++ .../ReportWizardPage/ReportWizardPage.tsx | 179 +++-- e2e/pages/InvoiceDetailPage.ts | 27 + e2e/pages/ReportWizardPage.ts | 226 +++++- .../budget/reportWizardExpansion.spec.ts | 759 ++++++++++++++++++ .../0044_invoice_deposit_budget_source.sql | 7 + ...0044_invoice_deposit_budget_source.test.ts | 272 +++++++ server/src/db/schema.ts | 4 + server/src/plugins/helmetPlugin.test.ts | 55 ++ server/src/plugins/helmetPlugin.ts | 2 +- server/src/routes/invoiceDeposits.test.ts | 216 +++++ server/src/routes/invoiceDeposits.ts | 2 + server/src/routes/sourceReports.test.ts | 28 + .../src/services/budgetSourceService.test.ts | 210 +++++ server/src/services/budgetSourceService.ts | 150 +++- .../services/invoiceDepositService.test.ts | 157 ++++ server/src/services/invoiceDepositService.ts | 7 + server/src/services/invoiceService.test.ts | 6 +- server/src/services/invoiceService.ts | 1 + .../shared/depositAggregateUtils.test.ts | 337 ++++++++ .../services/shared/depositAggregateUtils.ts | 255 ++++++ .../src/services/sourceReportService.test.ts | 325 ++++++++ server/src/services/sourceReportService.ts | 260 ++++-- shared/src/index.ts | 3 + shared/src/types/invoice.ts | 4 + shared/src/types/sourceReport.ts | 33 +- wiki | 2 +- 52 files changed, 6480 insertions(+), 241 deletions(-) create mode 100644 .claude/agent-memory/e2e-test-engineer/sandbox-live-verification.md create mode 100644 .claude/agent-memory/e2e-test-engineer/story-1891-wizard-followup.md create mode 100644 .claude/agent-memory/qa-integration-tester/story-1891-report-wizard-followup.md create mode 100644 client/src/lib/reportExclusions.test.ts create mode 100644 client/src/lib/reportExclusions.ts create mode 100644 e2e/tests/budget/reportWizardExpansion.spec.ts create mode 100644 server/src/db/migrations/0044_invoice_deposit_budget_source.sql create mode 100644 server/src/db/migrations/0044_invoice_deposit_budget_source.test.ts diff --git a/.claude/agent-memory/e2e-test-engineer/MEMORY.md b/.claude/agent-memory/e2e-test-engineer/MEMORY.md index fa5b9e16c..1f42a9c78 100644 --- a/.claude/agent-memory/e2e-test-engineer/MEMORY.md +++ b/.claude/agent-memory/e2e-test-engineer/MEMORY.md @@ -23,8 +23,11 @@ - [story-1876-deposit-refunds.md](story-1876-deposit-refunds.md) — deposit `entryType` refunds; DataTable hidden-column read pattern (`getColumnCellText`), table+card dual-DOM visible-filter pitfall. - [story-1877-contact-fields-attachment-typing.md](story-1877-contact-fields-attachment-typing.md) — budget-source contact fields, household settings singleton (needs `mode:'serial'`, not just try/finally), document-link attachment typing; route-mock last-registered-runs-first ordering hazard; sandbox lint/tsc caveats. - [story-1879-report-wizard.md](story-1879-report-wizard.md) — Bank Report Wizard POM/spec; Blocker bug #1886 (budgetSources envelope crash blocks all progress past step 1) + compile errors/missing i18n keys found via `tsc`; source-report E2E seeding pattern (WI budget → invoice-budget-line link). +- [story-1891-wizard-followup.md](story-1891-wizard-followup.md) — expandable invoice rows, CSP `blob:` frame-src hardened preview check, deposit budget-source tagging; 2 filed bugs (#1892 full-exclusion display, #1893 missing deposit-default heuristic); sandbox CAN now build+boot the full container stack (see `sandbox-live-verification.md`) but browser binary download is still network-policy-blocked. +- [sandbox-live-verification.md](sandbox-live-verification.md) — **dhi.io build access is sandbox-dependent, re-check each session**: this session successfully built `cornerstone:e2e` and booted the full container stack (app+OIDC+proxy all healthy), a first — but Playwright's own browser binary download (`playwright.download.prss.microsoft.com`/`cdn.playwright.dev`) is blocked by network policy, and Ubuntu's `chromium-browser` apt package is a non-functional snap stub (no snapd) — no way found yet to get an actual live browser run in this sandbox class. ## Open follow-ups to flag to orchestrator - No Paperless-ngx testcontainer exists yet (story-epic08-e2e.md) — all Paperless E2E coverage is `page.route()` mocked, not real integration. Add the container when Paperless work resumes. -- Full containerized E2E verification is not possible in sandboxes without `dhi.io` (Docker Hardened Images) registry credentials — building `cornerstone:e2e` fails with `401 Unauthorized`. This is an environment limitation, not a code issue; verification in such sandboxes must fall back to static checks (lint/prettier/tsc-diff/`playwright --list`) plus post-merge CI observation. +- **Filed bugs from Story #1891, unresolved**: #1892 (fully-excluding all budget lines on an invoice removes it from the report wizard's step-3 list instead of showing €0.00 — display-only, PDF/claim unaffected) and #1893 (deposit budget-source auto-default 0/1/>1-source heuristic is entirely unimplemented — `InvoiceDepositsSection.tsx` never receives the invoice's budget lines). +- Containerized E2E verification capability varies by sandbox instance — **do not assume the old "no dhi.io creds" note still applies**; re-attempt `docker build -t cornerstone:e2e .` each session before falling back to static-only verification. See `sandbox-live-verification.md` for the current state and the remaining browser-binary blocker. diff --git a/.claude/agent-memory/e2e-test-engineer/sandbox-live-verification.md b/.claude/agent-memory/e2e-test-engineer/sandbox-live-verification.md new file mode 100644 index 000000000..8beaabc4c --- /dev/null +++ b/.claude/agent-memory/e2e-test-engineer/sandbox-live-verification.md @@ -0,0 +1,123 @@ +--- +name: sandbox-live-verification +description: What live/containerized E2E verification is actually possible from inside a sandbox session — re-check every session, this has changed at least once. +metadata: + type: project +--- + +## Status as of Story #1891 (2026-07-30 session) + +Prior memory (and several earlier story files) documented that sandboxes cannot build +`cornerstone:e2e` because `dhi.io` (Docker Hardened Images) registry credentials aren't +available — every earlier attempt failed with `401 Unauthorized` on `docker pull +dhi.io/node:...`. **That was true in those sessions but is not a fixed property of "the +sandbox" in general** — in this session: + +```bash +docker pull dhi.io/node:24-alpine3.23-dev # succeeded, no auth prompt +docker build -t cornerstone:e2e --build-arg APP_VERSION=pr-1891-verify . # succeeded, ~2 min +``` + +The full app image built cleanly, and running `npx playwright test --project=desktop` +against it actually spun up the whole stack via `e2e/containers/setup.ts`: + +``` +🐳 Starting E2E test containers... +✅ Created Docker network: ... +🔐 Starting OIDC server... +✅ OIDC server ready at http://localhost:.../default +🏗️ Starting Cornerstone application and reverse proxy... +✅ Cornerstone app ready at http://localhost:... +✅ Reverse proxy ready at http://localhost:... +✅ All containers ready for E2E testing +``` + +i.e. `mock-oauth2-server`, the built `cornerstone:e2e` app container, and the `nginx:alpine` +reverse proxy all pulled/started/health-checked correctly. **This is a first** — no prior +story session got this far. + +**Action for future sessions**: always attempt `docker pull dhi.io/node:24-alpine3.23-dev` +(or just `docker build -t cornerstone:e2e .`) early, in the background, rather than assuming +it will fail from stale memory. If it succeeds, a genuine live CI-equivalent run becomes +possible for the AC verification steps that ask for one (e.g. a MANDATORY red/green proof). + +## The remaining blocker: no usable browser binary + +Even with the container stack fully working, `npx playwright test` still fails at the +`auth-setup` project because no Chromium binary is available: + +1. **Playwright's own download is network-policy-blocked.** `npx playwright install chromium` + (with or without `--with-deps`) tries `playwright.download.prss.microsoft.com`, + `cdn.playwright.dev`, and one more mirror, in that order — all three returned HTTP 403 + `Blocked by network policy: ... no matching allow rule — blocked by default deny policy`. + This is a `sbx policy allow network ` decision for the user/host, not something to + work around unilaterally. +2. **Ubuntu's `chromium-browser` apt package is a non-functional snap stub.** `apt-get install + chromium-browser` succeeds and installs `/usr/bin/chromium-browser`, but running it (even + pointed to via a local `playwright.config.ts` override's `use.launchOptions.executablePath`) + fails immediately: + ``` + Command '/usr/bin/chromium-browser' requires the chromium snap to be installed. + Please install it with: snap install chromium + ``` + `snap install chromium` in turn fails because `snapd` isn't running in this sandbox + (`dial unix /run/snapd.socket: connect: no such file or directory`), and there is no `apt` + package named plain `chromium` (only the snap-transitional `chromium-browser` stub) on this + Ubuntu release. No other route to a real browser binary was found in this session + (no cached `~/.cache/ms-playwright` binaries, no `google-chrome`/`firefox` either). + +**Net effect**: a fully live Playwright *browser* run is still not achievable in this sandbox +class, even though the *container/app* side now works. If a future story's AC needs an actual +red/green browser proof, either (a) ask the user to `sbx policy allow network +playwright.download.prss.microsoft.com,cdn.playwright.dev` for that session, or (b) fall back +to the established pattern: build+boot the containers to prove the app-level behavior (e.g. the +CSP header itself, via a raw `curl`/`page.request` HTTP check that doesn't need a browser), +plus code-reasoning for the browser-rendering part, and document the CI expectation explicitly +— this is what Story #1891's CSP-hardening verification did. + +## Practical technique: overriding the browser executable without touching committed config + +If a real Chromium binary ever IS available (e.g. `sbx policy allow` was granted, or a cached +binary exists), don't edit `e2e/playwright.config.ts` — create a scratch, **never-committed** +sibling config that imports and extends it, point `use.launchOptions.executablePath` at the +real binary for BOTH the `auth-setup` and `desktop` projects (auth-setup has its own separate +`use` block — overriding only `desktop` leaves `auth-setup` still trying to launch the missing +bundled headless-shell binary), run with `--config=`, then delete it before +finishing. Example that worked mechanically (blocked only by the missing real binary, see +above): + +```ts +// e2e/playwright.local-verify.config.ts — SCRATCH ONLY, delete before finishing +import { defineConfig } from '@playwright/test'; +import base from './playwright.config.js'; + +const CHROME_OVERRIDE = { executablePath: '/path/to/chromium', args: ['--no-sandbox'] }; + +export default defineConfig(base, { + projects: (base.projects || []) + .filter((p) => p.name === 'auth-setup' || p.name === 'desktop') + .map((p) => ({ + ...p, + use: { ...p.use, launchOptions: { ...(p.use?.launchOptions || {}), ...CHROME_OVERRIDE } }, + })), +}); +``` + +Must live inside `e2e/` (not `/tmp`) so the relative `@playwright/test` / `./playwright.config.js` +module resolution works — a file outside the workspace fails with `Cannot find module +'@playwright/test'` even though the import looks correct. + +## Cleanup checklist after any live-verification attempt + +- `rm` any scratch `playwright.*.config.ts` file. +- `rm -rf e2e/e2e/ e2e/playwright-output/ e2e/playwright-report/ e2e/test-results/` — a + misconfigured `--config` path or an interrupted run can leave a nested `e2e/e2e/test-results/ + .state/containers.json` artifact (from `containers/setup.ts`'s state file) that `git status` + won't flag as tracked but that still clutters the worktree. +- `docker images` / `docker rmi cornerstone:e2e` if you built a throwaway/deliberately-broken + image for a red-test proof — a stale local image with the wrong config baked in will silently + get reused by the next `docker build`'s layer cache or by a teammate's next test run in the + same sandbox if left in place. +- Confirm `docker ps -a` and `docker network ls` have no leftover test containers/networks — + the suite's own `globalTeardown` (`e2e/containers/teardown.ts`) handles this on a normal run, + but verify after any run that errored before reaching teardown. diff --git a/.claude/agent-memory/e2e-test-engineer/story-1891-wizard-followup.md b/.claude/agent-memory/e2e-test-engineer/story-1891-wizard-followup.md new file mode 100644 index 000000000..6d5e5a551 --- /dev/null +++ b/.claude/agent-memory/e2e-test-engineer/story-1891-wizard-followup.md @@ -0,0 +1,177 @@ +--- +name: story-1891-wizard-followup +description: Bank Report Wizard follow-up (Story #1891) — expandable invoice rows, CSP blob: frame-src hardened preview check, deposit budget-source tagging; 2 filed production bugs. +metadata: + type: project +--- + +## Files + +- `e2e/tests/budget/reportWizardExpansion.spec.ts` — new, 7 scenarios (CSP headline, chip + width, line-exclusion → row/total/PDF, full-exclusion, deposit tagged to zero-line source, + claim warning count, regression sweep of `reportWizard.spec.ts` selectors against the new + grid). +- `e2e/pages/ReportWizardPage.ts` — extended: `invoiceExpandToggle`, `expansionPanel`, + `itemsSubTable`/`depositsSubTable` (structural `.nth(0)`/`.nth(1)`, NOT text-based — both + sub-tables always render in this DOM order whether populated or `EmptyState`), + `itemRow`/`itemExclusionCheckbox`/`depositRow`, `invoiceRowAmount` (class filter trick below), + `markClaimedWarningBlock`. Hardened `waitForPreviewReady`/`waitForPreviewRegenerated` per the + story AC — see "CSP hardened-check design" below. +- `e2e/pages/InvoiceDetailPage.ts` — added `depositBudgetSourceSelect` (`#deposit-budgetSource`) + and `depositBudgetSourceHint`. + +## `invoiceRowAmount` locator trick + +`ReportInvoiceList.tsx`'s amount cell has THREE classes containing the substring "amount": +`.amountColumn` (outer wrapper), `.amount` (the value itself), `.amountNegative` (refund +modifier, combined with `.amount` on the same element). To land on just the value element: + +```ts +invoiceRow(...).locator('[class*="amount"]:not([class*="Column"])').first() +``` + +`:not([class*="Column"])` excludes only the wrapper (its class contains "Column", capital C); +both `.amount` and `.amount.amountNegative` still match `[class*="amount"]` and pass the +`:not()`. This general pattern (grep for which OTHER classes in the same component share a +target substring before trusting a bare `[class*="X"]` locator) is worth checking any time a +new locator is added against `ReportInvoiceList.module.css` or similarly-named sibling classes +elsewhere. + +## Expansion panel is a DOM SIBLING, not a descendant + +`ReportInvoiceList.tsx` renders `
...
+{isExpanded &&
...
}
` — the +panel is a sibling of the row, both children of the same per-invoice wrapper. Reach it with a +relative xpath from the row locator: + +```ts +invoiceRow(...).locator('xpath=following-sibling::*[contains(@class,"expansionPanel")]') +``` + +## CSP hardened-check design (the story's headline AC) + +The bug this hardens against: React sets `