Repository navigation
Add aws.cost.undiscounted metric for credit-covered accounts - #14
amanpruthi wants to merge 3 commits into
Conversation
An AWS account under a 100% promotional/committed credit shows Credit == -Usage in Cost Explorer every day, so UnblendedCost/AmortizedCost read ~$0 while real infra consumption continues unseen. Adds a separate GetCostAndUsage call filtered to RECORD_TYPE IN (Usage, SavingsPlanCoveredUsage) — the only record types that carry positive consumption at on-demand/list-price value — exported as aws.cost.undiscounted. A positive filter (include only these types) rather than an exclude-list of every known discount/credit RECORD_TYPE, so a new AWS discount type added later is automatically excluded instead of silently leaking in. Ported from the validated KEPT_RECORD_TYPES set in pde_daily_glassbox_cost_reporter_clone/l9cost/aws_cost.py (measured across a 37-day, six-account audit). Verified live against a credit-covered test account: undiscounted rows export correctly, summing to a real per-day cost vs. the near-zero noise from UnblendedCost/AmortizedCost. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…imate drift Found while independently validating exported numbers against raw AWS CLI: Cost Explorer's daily cost is an ESTIMATE that keeps revising for ~2-3 days after the day passes. Measured live: a Marketplace line item for a given day was captured ~47% low the next day at DAYS_BACK=1, but the SAME day's live Cost Explorer value a few days later had settled to its final, higher value. Older days (5+ days old) matched exactly, confirming the estimate settles within a few days. Since each (tenant, service, region, cost_date) combination is written to Last9's TSDB exactly once, and the TSDB accepts neither backfilled nor overwritten samples for the same labels+timestamp, a day captured too early can never be corrected afterward — every prior day using DAYS_BACK=1/no offset is permanently understated by whatever the drift was at capture time. Adds CAPTURE_OFFSET_DAYS (default 3), which shifts the whole [start, end) fetch window back by this many days — "yesterday" becomes "CAPTURE_OFFSET_DAYS days ago", captured only after Cost Explorer's estimate has settled. Trade-off: the report is CAPTURE_OFFSET_DAYS days behind "today" instead of 1 — documented in README/deploy.sh/.env.example/cloudformation.yaml. New test verifies the fetch window's End date is actually offset from today by CAPTURE_OFFSET_DAYS, not just an unused env var. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
5b35e82 to
4fbd964
Compare
nishantmodak
left a comment
There was a problem hiding this comment.
Request changes at 4fbd9646d94863ae0ac9e69ebc0195a982d7adb9 for two reproduced P2 defects: CloudFormation ignores the new capture-offset override, and a collection crossing UTC midnight queries different billing windows for its metric families. Each inline thread includes the required fix and a complete executable regression.
Reviewed the complete six-file diff, collector/export paths, both deployment adapters, configuration, tests, CI and discussions. Local Python 3.13 validation: all 10 existing head tests and five added export/pagination/error controls pass, including 64 seeded amount/label cases. Both defect assertions fail twice on head and pass with local corrections. The same rollover assertion passes merge-base b59a2672f38a08a56d22c8395e2d8c7304d7a486; the capture option is new and has a passing direct-Lambda control.
Limits: external AWS/HTTP/package commands were mocked; no live billing, deployment or TSDB verification. Broader list-price/RI contracts and account-discovery pagination remain pending. GitHub tests pass, but both lint checks fail. No approval pending these fixes and remaining checks.
…tDays to CF - poll/lambda_handler compute the [start, end) period once and pass it to both fetch_costs and fetch_undiscounted_costs, so a run crossing UTC midnight can't query different billing days for different metric families. - deploy.sh: forward CAPTURE_OFFSET_DAYS as CaptureOffsetDays in the CloudFormation --parameter-overrides (direct-Lambda path already did). - Regression tests for both (midnight rollover; deploy.sh with stubbed aws in both CF and direct-Lambda modes). - ruff format + lint fixes; aws-cur lint (unused noqa, date.today) was also failing on main under current ruff. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nishantmodak
left a comment
There was a problem hiding this comment.
Approved at c45fc7475ecc463daeb15b7c77d3d510ba9d9b49. Both reported defects are fixed: CloudFormation receives the capture offset, and poll/Lambda compute one billing period for all metric families. The original attached regressions, recovered from these threads, pass unchanged. Finished the previously outstanding review scope: complete seven-file diff, collector/exporter and callers, configuration/deployment adapters, tests, account/filter semantics and failure paths. No remaining introduced blocker found.
Local Python 3.13.15: 13 repository tests and 9 regression/control/probe tests pass, including 64 seeded amount/label cases, independent export with empty net rows, cost-result pagination, Lambda window sharing and fetch/export errors. Current-head GitHub lint/tests pass, including the CUR fixture change. AWS, HTTP and deployment commands were synthetic boundaries; no cloud deployment, live billing or TSDB overwrite/backfill verification. The metric covers the documented Usage/SavingsPlanCoveredUsage filter; this review does not establish universal public-list-price equivalence for RI-covered or negotiated usage.
Summary
aws.cost.undiscounted— list-price cost per service/account/region, filtered toRECORD_TYPE IN (Usage, SavingsPlanCoveredUsage)only.Credit == -Usageevery day in Cost Explorer, soUnblendedCost/AmortizedCostread ~$0 while real infra consumption continues invisibly.RECORD_TYPEname — so a new AWS discount type Cost Explorer adds later is automatically excluded instead of silently leaking into the total. Ported from the validatedKEPT_RECORD_TYPESset inpde_daily_glassbox_cost_reporter_clone/l9cost/aws_cost.py(a 37-day, six-account measured audit).GetCostAndUsagecall (RECORD_TYPE can't be a thirdGroupBydimension alongside SERVICE+REGION — Cost Explorer capsGroupByat 2), merged into the same OTLP payload as an independent metric rather than joined onto the existing rows (a service can haveUsagewithout netUnblendedCostafter a credit, or vice versa).CAPTURE_OFFSET_DAYS(default 3): Cost Explorer's daily cost is an estimate that keeps revising for ~2-3 days after the day passes. Since each(tenant, service, region, cost_date)combination is written to Last9's TSDB exactly once, and the TSDB accepts neither backfilled nor overwritten samples for the same labels+timestamp, a day captured too early (at the oldDAYS_BACK=1/no-offset default) is permanently understated.CAPTURE_OFFSET_DAYSshifts the whole fetch window back so each day is only captured once its estimate has settled.Verified live
UnblendedCost/AmortizedCost.aws ce get-cost-and-usageCLI output (same filters/GroupBy) — matched to within floating-point precision onceCAPTURE_OFFSET_DAYSwas applied; confirmed the pre-fix (no offset) values were measurably low (one line item was ~47% understated one day after capture, matching exactly once re-checked 3+ days later).Test plan
python test_parse.py— 10/10 pass, including tests forfetch_undiscounted_costs(assertsRECORD_TYPEis aFilter, never a thirdGroupBydimension, and that zero-amount groups are skipped) and forCAPTURE_OFFSET_DAYS's window-shifting behavior.aws lambda invokeagainst a redeployed account, confirmed via direct Prometheus query thataws_cost_undiscounted_USDlanded with correct per-service values matching raw Cost Explorer output.🤖 Generated with Claude Code