feat(reports): AI-generated usage descriptions and cover letter for bank report wizard - #1916
Conversation
Backend layer of #1901 (WIP — frontend/tests follow in a later session): - POST /api/source-reports/generate-content: one batched LLM call producing per-invoice usage descriptions + cover letter subject/body in the requested report language; server re-assembles all prompt context from the DB (client text is never trusted); nothing persisted - New provider method generateReportContent with strict response schema and validator (length caps, all-requested-ids-present enforcement) - Fixes pre-existing buildRequestBody bug: the anthropic profile hardcoded EXTRACTED_LINES_SCHEMA for every call type; responseSchema is now required per call site (summarizeMerge sends its own schema) - llmEnabled config flag (alias of autoItemizeEnabled) exposed via GET /api/config; prompt includes explicit untrusted-data injection guard; LLM errors keep suppressDetails Part of #1901 Co-Authored-By: Claude backend-developer (Haiku 4.5) <noreply@anthropic.com>
…ank report wizard - Step 4 gains an "Enable AI assistance" toggle (absent entirely when no LLM is configured server-side, per GET /api/config's new llmEnabled flag); Step 5 gains a "Generate with AI" button that issues one batched call to the new generate-content endpoint and applies the result as a new editable baseline (applyAiContent), never as a silent override — manual edits survive on top, regenerating over unsaved edits warns before overwriting, and a Step 1-4 change clears AI content the same way it clears manual edits - Progress/error UX mirrors the existing auto-itemize flow: spinner + elapsed-seconds caption, inline translated error banner, retry without losing existing content - Fixes #1915: reportContentGenerationService.ts imported non-existent schema exports (work_items/household_items instead of workItems/householdItems), breaking buildApp() everywhere; linked-item descriptions for the LLM prompt now correctly read from the work/household item's own description column rather than the (already-surfaced-elsewhere) budget-record description (reportContentGenerationService.test.ts scenario 6c is the regression guard) - New English + German keys for the toggle, button, progress, and error copy - Full unit/integration/E2E coverage: applyAiContent, ReportWizardPage AI-generation flow, Step4Settings gating, sourceReportsApi wrapper, and an E2E spec covering the toggle visibility, batched generation, overwrite-confirm, and error paths Fixes #1901 Fixes #1915 Co-Authored-By: Claude dev-team-lead (Sonnet 4.6) <noreply@anthropic.com> Co-Authored-By: Claude backend-developer (Haiku 4.5) <noreply@anthropic.com> Co-Authored-By: Claude e2e-test-engineer (Sonnet 4.5) <noreply@anthropic.com> Co-Authored-By: Claude frontend-developer (Haiku 4.5) <noreply@anthropic.com> Co-Authored-By: Claude qa-integration-tester (Sonnet 4.5) <noreply@anthropic.com> Co-Authored-By: Claude translator (Sonnet 4.5) <noreply@anthropic.com>
|
[ux-designer] Design review of the client/src changes against the visual spec posted on #1901 (comment history also captured in Spec placement decisions — all verified correct:
Token adherence: All three new CSS rules ( Dark mode: No issues found — everything routes through semantic tokens already covered by the dark theme block. Accessibility: Checkbox/label association in Responsive: Component reuse: No new shared components introduced; correctly reuses One informational, non-blocking nit: Verdict: APPROVE No token, dark-mode, accessibility, or responsive issues found. Implementation is a faithful, precise match to every placement/gating/baseline decision from the #1901 spec. |
|
[security-engineer] Security review of PR #1916 (issue #1901: AI report content generation). Verdict: APPROVENo critical or high findings. The implementation follows the established LLM-integration security pattern from Reviewed
1. AuthN/AuthZ — OK
2. Input validation — OKFastify JSON-schema is tight: IDOR-relevant defense-in-depth (good catch by the implementer): the client-supplied 3. Prompt injection — adequate, residual risk accepted (informational)
No finding raised here — this is the correct architecture for the residual risk (untrusted-LLM-output-as-plain-text), not a gap. 4. Information disclosure — OK, verified by dedicated test
5. SSRF / outbound — OK
6. No-persistence claim — verified
7. Rate limiting / concurrency (non-blocking, informational)The rate-limit plugin is Informational: stale test-file comment
Summary: No Critical/High findings. Auth parity confirmed, input validation tight, IDOR-safe (server re-derives invoice set from DB), prompt-injection mitigations reasonable given output is always rendered as plain text/PDF-text (never HTML), no secret/prompt/provider-payload leakage (test-verified), SSRF not applicable (env-only base URL), and the "no persistence" claim is test-verified. One informational note on rate limiting (matches existing sibling-endpoint posture, not a regression) and one informational note on a stale test comment. |
…agent memory Security review of story #1901 (server-side AI report content generation) approved with no High/Critical findings; log the IDOR-safe prompt-building pattern and plain-text-only LLM output rendering as a reusable template for future LLM-writes-user-facing-text features. Co-Authored-By: Claude security-engineer (Sonnet 4.5) <noreply@anthropic.com>
|
[product-owner] Requirements review of PR #1916 against story #1901 (Bank report wizard — AI-generated usage descriptions and cover letter). I verified all six AC sections against the implementation and tests on However, I am returning this with CHANGES_REQUIRED. There is a functional defect in the numbers fed to the model that would put materially wrong currency figures into a document a homeowner sends to their bank. Per-section coverage1. Availability and opt-in — MET
2. Batched generation — NOT MET (see Blocking 1 and 3)
3. Generated text becomes an editable baseline — MET
4. Progress and error handling — MET, with one copy defect (see Must Fix 4)
5. Server endpoint — MET, except the wiki (see Must Fix 5)
6. Translations — METEvery new string resolves in both locales: Blocking — must be fixed before merge1. Every currency amount sent to the model is 1/100th of its real value.
const amountFormatted = (input.totalAmount / 100).toFixed(2);
...
const invAmount = (inv.amount / 100).toFixed(2);But The system prompt then instructs the model "Do NOT invent or alter amounts or dates" — so it will faithfully copy the wrong figure into the cover letter body. The PDF table renders the correct number alongside it, so the exported bank document contradicts itself. This fails AC 2.4 (the cover letter must draw on the totals) and weakens AC 2.3. Fix: drop both 2.
// Round to nearest cent
includedTotal = Math.round(includedTotal);On major units this rounds to whole euros — the comment and the code disagree. 3. Per-invoice The Fix: send the exclusion-adjusted contribution — the per-invoice value the Must fix before merge (non-blocking, no re-review needed)4. Shared LLM error copy names the wrong feature. All four codes reachable from this flow are worded for auto-itemization: 5. The API Contract wiki edit is uncommitted. The Non-blocking observations
Adjudicated deviation — confirmed satisfiedAC 2.3 asks the prompt to draw on "the names and descriptions of the linked work items and household items". The implementation reads GlossaryAgreed on the translator's flag: "AI" → "KI" deserves a glossary entry, and I am tracking it for the next glossary refinement pass rather than adding it mid-story. The German copy here already uses "KI" consistently ("KI-Unterstützung aktivieren", "Mit KI generieren", "KI-Generierung fehlgeschlagen"), which is the correct convention and is exactly what the entry should codify. VerdictCHANGES_REQUIRED — Blocking items 1, 2, and 3. All three sit in the same two files ( Everything else across the six AC sections is met, and the baseline/override integration with #1900 is exactly right. |
… ADR-034 addenda Documents POST /api/source-reports/generate-content, the llmEnabled config field, and the baseline -> AI -> overrides content-layering model. Refs #1901 Co-Authored-By: Claude product-architect (Opus 4.6) <noreply@anthropic.com>
|
[product-architect] Architecture review — PR #1916 (story #1901)Verdict: CHANGES_REQUIRED — 1 blocking finding (2 linked defects at the same seam + the test gap that let them through), 9 non-blocking findings. Reviewed against the Wiki Architecture, API Contract and Schema pages, ADR-034, and the forward guidance I recorded when reviewing #1900. What is right (and worth keeping)
BLOCKING — B1: every monetary figure sent to the LLM is 100× too smallThe unit convention breaks across the service → prompt seam.
// reportContentGenerationService.ts:183
amount: inv.allocatedAmount, // e.g. 1234.56 (euros)
// …and totalAmount: includedTotal // e.g. 12345.67 (euros)but // prompts.ts:151
const amountFormatted = (input.totalAmount / 100).toFixed(2); // 12345.67 -> "123.46"
// prompts.ts:165
const invAmount = (inv.amount / 100).toFixed(2); // 1234.56 -> "12.35"The system prompt explicitly instructs the model to "Reference the source name, report type …, total amount and currency", so this lands in the headline figure of a bank-facing cover letter: a €12,345.67 claim is described to the bank as 123.46 EUR. This is the same failure class as ADR-034 finding B3, which is why the ADR now carries an explicit monetary-unit invariant. Linked defect B1b — includedTotal = Math.round(includedTotal); // comment says "Round to nearest cent"
Required fix (keeps the codebase's major-unit convention — do not convert to cents):
Linked test gap. Non-blocking (medium — please fold into refinement rather than this PR unless trivial)
Non-blocking (low)
E2EAgreed, unrelated. Shard 5's only non-recovering failure is Wiki (pushed)Wiki
The remaining ADR-034 corrections tracked under #1914 (width-assertion rule, stale module table) are deliberately not in this push. |
… layering Co-Authored-By: Claude product-architect (Opus 4.6) <noreply@anthropic.com>
…ort content prompt PO and architect review findings on PR #1916: - Prompt amounts were divided by 100 a second time even though report amounts are already major-unit euros, not cents — buildReportContentUserPrompt now renders totalAmount/per-invoice amounts verbatim - includedTotal rounded to the nearest whole euro instead of the nearest cent (Math.round(x) -> Math.round(x * 100) / 100); per-invoice LLM amounts previously ignored excludedLineIds entirely while the total honored them — a new exclusion-adjusted, cents-rounded per-invoice amount now feeds the LLM input - Reworded the four shared LLM_* error strings (en/de) to be feature-neutral ("AI assistance"/"AI service" / "KI-Unterstützung"/"KI-Dienst") now that they surface for both auto-itemize and AI report content generation, not just the former - Added direct buildReportContentUserPrompt/REPORT_CONTENT_SYSTEM_PROMPT coverage in prompts.test.ts (previously zero direct tests) including a permanent regression guard against the x100 unit bug Co-Authored-By: Claude dev-team-lead (Sonnet 4.6) <noreply@anthropic.com> Co-Authored-By: Claude backend-developer (Haiku 4.5) <noreply@anthropic.com> Co-Authored-By: Claude e2e-test-engineer (Sonnet 4.5) <noreply@anthropic.com> Co-Authored-By: Claude frontend-developer (Haiku 4.5) <noreply@anthropic.com> Co-Authored-By: Claude qa-integration-tester (Sonnet 4.5) <noreply@anthropic.com> Co-Authored-By: Claude translator (Sonnet 4.5) <noreply@anthropic.com>
Co-Authored-By: Claude product-architect (Opus 4.6) <noreply@anthropic.com>
|
[product-owner] Re-review of PR #1916 (story #1901) — round 2, against All five findings from my round-1 review are resolved. I verified each one against the code on disk rather than the commit message, and ran the new prompt tests locally. Blocking items — all fixed1. Currency scaling — FIXED. Both 2. Cents rounding — FIXED. 3. Per-invoice exclusion adjustment — FIXED. Worth calling out as a bonus: rounding each invoice to cents before summing makes the server's math identical to the client's Must-fix items — both done4. Error copy — FIXED, and better than I asked for. All four codes are feature-neutral in both locales:
The three transient ones also gained an explicit "Please try again." / "Bitte versuchen Sie es erneut.", which directly strengthens AC 4.3's "and I can retry". The German uses "KI-Unterstützung" and "KI-Dienst" — exactly the convention the translator flagged, so the pending glossary entry now has a second call site backing it. 5. Wiki — FIXED. Test-integrity checkI checked the 9 lines removed from CI
Remaining non-blocking notes (carry-forward, no action required here)
VerdictAPPROVED. All six AC sections of #1901 are met. The three numeric-accuracy defects are fixed at the root with permanent regression coverage, the error copy now speaks to the feature the user is actually in, and the API contract is published. Good, disciplined turnaround — particularly on the prompt test coverage, which went from zero direct tests to a purpose-built guard for the exact class of bug that slipped through. |
|
[product-architect] Re-review round 2 — PR #1916 (story #1901)Verdict: APPROVE. The blocking finding is fully resolved. Nothing remains blocking. B1 — verified fixed (
|
| Item | Status |
|---|---|
prompts.ts:151 (input.totalAmount / 100).toFixed(2) → input.totalAmount.toFixed(2) |
Fixed |
prompts.ts:165 (inv.amount / 100).toFixed(2) → inv.amount.toFixed(2) |
Fixed |
B1b whole-euro rounding → per-invoice Math.round(x * 100) / 100, summed, then rounded once more |
Fixed |
| Rendered-string regression tests | Added |
types.ts unit JSDoc |
Not done — see L5 below |
I re-read the diff rather than taking the summary on trust, and spot-ran both suites locally: prompts.test.ts 98/98 and reportContentGenerationService.test.ts 15/15 green.
The regression guard is the right shape — it asserts the rendered substring (Total Amount: 12345.67 EUR, Amount: 999.99 EUR) plus negative assertions against the divided-by-100 forms, and carries a comment explaining the class of bug. That is exactly the test that would have caught this, and it is now permanent. The multi-currency case (250.00 CHF) is a good addition I did not ask for.
Rounding parity re-derived and confirmed. Server: round per invoice → sum → round the sum. Client: applyLineExclusions rounds per invoice (only when an exclusion applies) → buildReportContent sums with no final round → formatCurrency renders at 2 dp. The server's extra unconditional per-invoice round is stricter than the client's conditional one, and the two can only produce different displayed totals for an allocatedAmount carrying more than 2 dp — which sourceReportService.ts:442 already excludes upstream (allocatedAmount: roundedAmount, and the shared type documents it as "rounded to 2dp"). So the extra rounding is defensive-only and parity holds in production; scenario 5's 333.335 fixture reaches it by direct DB insert, bypassing that invariant. No issue — worth recording so nobody "simplifies" the rounding away later.
PO's linked finding folded in correctly. Per-invoice amount is now the exclusion-adjusted figure from invoiceAmountsAdjusted, not raw allocatedAmount. This was the right call to fix together: sending raw per-invoice amounts alongside an exclusion-adjusted total gave the model a set of parts that did not sum to the stated whole, which is a plausible source of hallucinated reconciliation text in a bank-facing letter. Now recorded as a rule in ADR-034.
L4 addressed too (not required, but welcome): the four shared LLM_* strings are now feature-neutral in both locales. One follow-up for the PO rather than this PR — the German uses KI-Unterstützung / KI-Dienst, and "KI" still has no glossary.json entry. That was already an open question from the #1901 translator pass; now that the term is user-visible in four error strings it should be settled.
New low finding
- L5 — the root-cause annotation is still missing. Item 3 of the B1 fix list was not applied:
GenerateReportContentLlmInvoice.amount(budgetExtraction/types.ts:30) andGenerateReportContentLlmInput.totalAmount(:40) still carry no unit in their JSDoc. The behaviour is correct and pinned by tests, so this is not blocking — but the absent unit annotation is precisely what allowed the seam to break, and the next field added to these interfaces inherits the same silence. Two comment lines (/** Major currency units, 2 dp — not cents. */). - Informational —
invoiceAmountsAdjusted.get(inv.invoiceId) ?? inv.allocatedAmount: the fallback is unreachable, since the map is populated under exactly the same filter that guards the read. Harmless; noting it only so it is not later mistaken for a meaningful default.
Findings carried forward (still non-blocking, unchanged)
To answer the coordinator's question — yes, please track these. I'd suggest one follow-up refinement issue rather than five, since M1–M4 are all "the LLM integration outgrew its original naming/boundaries":
- M1 —
llmEnabled/autoItemizeEnabledduplication with no deprecation path;getProvider()still gates the report feature on the auto-itemize flag. (@deprecatedJSDoc + removal plan. The "no third alias; divergence needs an ADR" rule is already recorded on the API Contract page.) - M2 — included-total derivation duplicated client/server. B1b was the first drift; extracting
computeIncludedTotalinto@cornerstone/sharedis the structural fix. Highest value of the four. - M3 — route returns the server-internal
GenerateReportContentLlmResultinstead of the sharedGenerateReportContentResponse; bind the annotation so drift becomes a compile error. - M4 —
BudgetExtractionProvideris now a general LLM gateway under a budget-extraction name, with three unrelated capabilities and five unrelated test doubles stubbing all of them. Rename tollmGateway/(or split per capability) as a standalone refactor. - L1
sourceId!assertion · L2.includes()inside four loops where theSetalready exists · L3 the'German construction project' / 'Konstruktionsprojekt'ternary in the prompt (also not idiomatic —Bauprojekt) · L5 above.
CI
Quality Gates green on b70d821b — the required gate for a beta PR. E2E Gates red on shard 5 only, and it is the same pre-existing flake as the previous run: tests/invoices/invoices.spec.ts:841 ("Effective Amount" column, Issue #1876), unrelated to this PR and already on record as a beta → main promotion blocker to be cleared separately.
Wiki
master 905182d, submodule ref bumped in 4a610f31. Amended the ADR-034 monetary-unit invariant to match what actually shipped: the final round-the-sum step, and the "parts must add up to the stated whole" rule for per-invoice amounts handed to the generator. The API Contract page needed no change — this fix did not alter the request/response shape.
Nice work on the turnaround, and thank you for adding the negative assertions rather than just the positive ones.
… rule Co-Authored-By: Claude product-architect (Opus 4.6) <noreply@anthropic.com>
…n product-owner memory - Round-2 fix-verification patterns (deleted-test-line check, E2E flake attribution) - KI glossary decision and #1917 follow-up linkage - gh project item-list --query workaround via GraphQL projectItems Co-Authored-By: Claude product-owner (Opus 4.6) <noreply@anthropic.com>
|
🎉 This PR is included in version 2.13.0-beta.34 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
Summary
d8b47625): newPOST /api/source-reports/generate-contentendpoint issues one batched LLM call producing per-invoice usage descriptions and a cover letter subject/body in the requested report language. The server re-assembles all prompt context from the database itself (never trusts client-supplied invoice text), enforces a strict response schema with a validator (length caps, all-requested-ids-present), and persists nothing. Reuses the existing auto-itemize LLM client/config path end-to-end. Also fixes a pre-existingbuildRequestBodybug where the anthropic provider profile hardcoded the extraction response schema for every call type.59dda421): Step 4 gains an "Enable AI assistance" toggle, shown only whenGET /api/config's newllmEnabledflag is true (absent entirely otherwise — never presented as available when it can't work). Step 5 gains an explicit "Generate with AI" button (no auto-generation) with spinner + elapsed-seconds progress, mirroring the existing auto-itemize UX. Generated content becomes a new editable baseline (never a silent override) — manual edits on top of it behave exactly like edits on the derived baseline, survive to every export path, and regenerating over unsaved edits warns before overwriting.reportContentGenerationService.tsinitially imported non-existent schema exports (work_items/household_itemsinstead ofworkItems/householdItems), which brokebuildApp()compilation everywhere. While fixing it, linked-item descriptions fed to the LLM prompt were also corrected to read from the work/household item's owndescriptioncolumn (previously mistakenly re-reading the budget-record description already surfaced elsewhere in the report) —reportContentGenerationService.test.tsscenario 6c is the regression guard proving the two sources are independent.Fixes #1901
Fixes #1915
Test plan
Co-Authored-By: Claude dev-team-lead (Sonnet 4.6) noreply@anthropic.com
Co-Authored-By: Claude backend-developer (Haiku 4.5) noreply@anthropic.com
Co-Authored-By: Claude e2e-test-engineer (Sonnet 4.5) noreply@anthropic.com
Co-Authored-By: Claude frontend-developer (Haiku 4.5) noreply@anthropic.com
Co-Authored-By: Claude qa-integration-tester (Sonnet 4.5) noreply@anthropic.com
Co-Authored-By: Claude translator (Sonnet 4.5) noreply@anthropic.com
🤖 Generated with Claude Code