Skip to content

Add aws.cost.undiscounted metric for credit-covered accounts - #14

Open
amanpruthi wants to merge 3 commits into
mainfrom
feat/undiscounted-cost
Open

amanpruthi wants to merge 3 commits into
mainfrom
feat/undiscounted-cost

Conversation

@amanpruthi

@amanpruthi amanpruthi commented Aug 31, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Adds aws.cost.undiscounted — list-price cost per service/account/region, filtered to RECORD_TYPE IN (Usage, SavingsPlanCoveredUsage) only.
  • Fixes a real blind spot: an account under a 100% promotional/committed AWS credit shows Credit == -Usage every day in Cost Explorer, so UnblendedCost/AmortizedCost read ~$0 while real infra consumption continues invisibly.
  • Positive filter (include only the two record types that carry on-demand consumption value), not an exclude-list of every known discount/credit RECORD_TYPE name — so a new AWS discount type Cost Explorer adds later is automatically excluded instead of silently leaking into the total. Ported from the validated KEPT_RECORD_TYPES set in pde_daily_glassbox_cost_reporter_clone/l9cost/aws_cost.py (a 37-day, six-account measured audit).
  • Separate GetCostAndUsage call (RECORD_TYPE can't be a third GroupBy dimension alongside SERVICE+REGION — Cost Explorer caps GroupBy at 2), merged into the same OTLP payload as an independent metric rather than joined onto the existing rows (a service can have Usage without net UnblendedCost after a credit, or vice versa).
  • Adds CAPTURE_OFFSET_DAYS (default 3): Cost Explorer's daily cost is an estimate that keeps revising for ~2-3 days after the day passes. Since each (tenant, service, region, cost_date) combination is written to Last9's TSDB exactly once, and the TSDB accepts neither backfilled nor overwritten samples for the same labels+timestamp, a day captured too early (at the old DAYS_BACK=1/no-offset default) is permanently understated. CAPTURE_OFFSET_DAYS shifts the whole fetch window back so each day is only captured once its estimate has settled.

Verified live

  • Deployed to a credit-covered test account: undiscounted rows export correctly, summing to a real per-day cost instead of the near-zero noise from UnblendedCost/AmortizedCost.
  • Independently cross-checked exported values against raw aws ce get-cost-and-usage CLI output (same filters/GroupBy) — matched to within floating-point precision once CAPTURE_OFFSET_DAYS was applied; confirmed the pre-fix (no offset) values were measurably low (one line item was ~47% understated one day after capture, matching exactly once re-checked 3+ days later).

Test plan

  • python test_parse.py — 10/10 pass, including tests for fetch_undiscounted_costs (asserts RECORD_TYPE is a Filter, never a third GroupBy dimension, and that zero-amount groups are skipped) and for CAPTURE_OFFSET_DAYS's window-shifting behavior.
  • Live aws lambda invoke against a redeployed account, confirmed via direct Prometheus query that aws_cost_undiscounted_USD landed with correct per-service values matching raw Cost Explorer output.

🤖 Generated with Claude Code

amanpruthi and others added 2 commits August 31, 2026 10:54
An AWS account under a 100% promotional/committed credit shows
Credit == -Usage in Cost Explorer every day, so UnblendedCost/AmortizedCost
read ~$0 while real infra consumption continues unseen.

Adds a separate GetCostAndUsage call filtered to RECORD_TYPE IN (Usage,
SavingsPlanCoveredUsage) — the only record types that carry positive
consumption at on-demand/list-price value — exported as aws.cost.undiscounted.
A positive filter (include only these types) rather than an exclude-list of
every known discount/credit RECORD_TYPE, so a new AWS discount type added
later is automatically excluded instead of silently leaking in. Ported from
the validated KEPT_RECORD_TYPES set in
pde_daily_glassbox_cost_reporter_clone/l9cost/aws_cost.py (measured across a
37-day, six-account audit).

Verified live against a credit-covered test account: undiscounted rows
export correctly, summing to a real per-day cost vs. the near-zero noise
from UnblendedCost/AmortizedCost.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…imate drift

Found while independently validating exported numbers against raw AWS CLI:
Cost Explorer's daily cost is an ESTIMATE that keeps revising for ~2-3 days
after the day passes. Measured live: a Marketplace line item for a given day
was captured ~47% low the next day at DAYS_BACK=1, but the SAME day's live
Cost Explorer value a few days later had settled to its final, higher value.
Older days (5+ days old) matched exactly, confirming the estimate settles
within a few days.

Since each (tenant, service, region, cost_date) combination is written to
Last9's TSDB exactly once, and the TSDB accepts neither backfilled nor
overwritten samples for the same labels+timestamp, a day captured too early
can never be corrected afterward — every prior day using DAYS_BACK=1/no
offset is permanently understated by whatever the drift was at capture time.

Adds CAPTURE_OFFSET_DAYS (default 3), which shifts the whole
[start, end) fetch window back by this many days — "yesterday" becomes
"CAPTURE_OFFSET_DAYS days ago", captured only after Cost Explorer's estimate
has settled. Trade-off: the report is CAPTURE_OFFSET_DAYS days behind "today"
instead of 1 — documented in README/deploy.sh/.env.example/cloudformation.yaml.

New test verifies the fetch window's End date is actually offset from today
by CAPTURE_OFFSET_DAYS, not just an unused env var.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@amanpruthi
amanpruthi force-pushed the feat/undiscounted-cost branch from 5b35e82 to 4fbd964 Compare August 31, 2026 05:25

@nishantmodak nishantmodak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes at 4fbd9646d94863ae0ac9e69ebc0195a982d7adb9 for two reproduced P2 defects: CloudFormation ignores the new capture-offset override, and a collection crossing UTC midnight queries different billing windows for its metric families. Each inline thread includes the required fix and a complete executable regression.

Reviewed the complete six-file diff, collector/export paths, both deployment adapters, configuration, tests, CI and discussions. Local Python 3.13 validation: all 10 existing head tests and five added export/pagination/error controls pass, including 64 seeded amount/label cases. Both defect assertions fail twice on head and pass with local corrections. The same rollover assertion passes merge-base b59a2672f38a08a56d22c8395e2d8c7304d7a486; the capture option is new and has a passing direct-Lambda control.

Limits: external AWS/HTTP/package commands were mocked; no live billing, deployment or TSDB verification. Broader list-price/RI contracts and account-discovery pagination remain pending. GitHub tests pass, but both lint checks fail. No approval pending these fixes and remaining checks.

Comment thread aws-cost-explorer/deploy.sh
Comment thread aws-cost-explorer/main.py Outdated
…tDays to CF

- poll/lambda_handler compute the [start, end) period once and pass it to
  both fetch_costs and fetch_undiscounted_costs, so a run crossing UTC
  midnight can't query different billing days for different metric families.
- deploy.sh: forward CAPTURE_OFFSET_DAYS as CaptureOffsetDays in the
  CloudFormation --parameter-overrides (direct-Lambda path already did).
- Regression tests for both (midnight rollover; deploy.sh with stubbed aws
  in both CF and direct-Lambda modes).
- ruff format + lint fixes; aws-cur lint (unused noqa, date.today) was also
  failing on main under current ruff.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@nishantmodak nishantmodak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved at c45fc7475ecc463daeb15b7c77d3d510ba9d9b49. Both reported defects are fixed: CloudFormation receives the capture offset, and poll/Lambda compute one billing period for all metric families. The original attached regressions, recovered from these threads, pass unchanged. Finished the previously outstanding review scope: complete seven-file diff, collector/exporter and callers, configuration/deployment adapters, tests, account/filter semantics and failure paths. No remaining introduced blocker found.

Local Python 3.13.15: 13 repository tests and 9 regression/control/probe tests pass, including 64 seeded amount/label cases, independent export with empty net rows, cost-result pagination, Lambda window sharing and fetch/export errors. Current-head GitHub lint/tests pass, including the CUR fixture change. AWS, HTTP and deployment commands were synthetic boundaries; no cloud deployment, live billing or TSDB overwrite/backfill verification. The metric covers the documented Usage/SavingsPlanCoveredUsage filter; this review does not establish universal public-list-price equivalence for RI-covered or negotiated usage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants