Add aws.cost.undiscounted metric for credit-covered accounts

aws-cost-explorer/.env.example

@@ -6,6 +6,13 @@ AWS_DEFAULT_REGION=us-east-1
 # Days of history to fetch per run. Use 1 for steady-state. Set to 30+ for initial backfill.
 DAYS_BACK=1
 
+# Shifts the fetch window this many days into the past. Cost Explorer's daily
+# estimate keeps revising for ~2-3 days after the day passes, and a day can
+# never be re-fetched/corrected once written to Last9 — this captures each
+# day only after its estimate has settled. See README "Why capture is
+# delayed" before lowering this.
+CAPTURE_OFFSET_DAYS=3
+
 # How often to re-poll (seconds). Cost Explorer data updates ~once/day.
 POLL_INTERVAL_SECONDS=86400
 

aws-cost-explorer/README.md

@@ -57,6 +57,7 @@ docker compose up
 | `AWS_SECRET_ACCESS_KEY` | No* | — | AWS secret key (Docker only) |
 | `AWS_DEFAULT_REGION` | No | `us-east-1` | AWS region |
 | `DAYS_BACK` | No | `1` | Days of history to fetch per run. Set to `30`+ for initial backfill, then drop to `1`. |
+| `CAPTURE_OFFSET_DAYS` | No | `3` | Shifts the fetch window this many days into the past — see "Why capture is delayed" below. |
 | `POLL_INTERVAL_SECONDS` | No | `86400` | Re-poll interval for Docker mode |
 | `OTEL_SERVICE_NAME` | No | `aws-cost-reporter` | Service name in Last9 |
 
@@ -68,11 +69,22 @@ docker compose up
 |---|---|---|
 | `aws.cost.unblended` | USD | `aws.service`, `aws.account.id`, `aws.region`, `cost.date` |
 | `aws.cost.amortized` | USD | same — RI and Savings Plan effective rates applied |
+| `aws.cost.undiscounted` | USD | same — list-price cost, ignoring credits/discounts/Savings-Plan fees |
 
 The CE API caps `GroupBy` at 2 dimensions. Each run calls `GetDimensionValues` to list linked accounts, then runs one `GetCostAndUsage` per account filtered by `LINKED_ACCOUNT` and grouped by `SERVICE + REGION`. This preserves all three dimensions in a single series without exceeding the API limit.
 
+### `aws.cost.undiscounted` — why a third metric
+
+An account under a promotional or committed AWS credit shows `Credit == -Usage` in Cost Explorer every day, so `UnblendedCost`/`AmortizedCost` read ~$0 while real infra consumption continues unseen. `aws.cost.undiscounted` is a separate `GetCostAndUsage` call filtered to `RECORD_TYPE IN (Usage, SavingsPlanCoveredUsage)` — the only record types that carry positive consumption at on-demand value — so it stays meaningful even when the credit fully offsets the standard metrics. It's a positive filter (include only these types) rather than an exclude-list of every known discount/credit RECORD_TYPE, so a new AWS discount type added later is automatically excluded instead of silently leaking into the total.
+
 `cost.date` (`YYYY-MM-DD`) encodes the billing date as a label so each day forms a distinct series. This matters when `DAYS_BACK > 1`: without it, all fetched days share identical labels and only the last sample survives per series. At the default `DAYS_BACK=1`, `cost.date` adds one unique label value per run with no cardinality overhead.
 
+### Why capture is delayed by `CAPTURE_OFFSET_DAYS`
+
+Cost Explorer's daily cost is an **estimate** that keeps revising for roughly 2-3 days after the day passes — measured live: a Marketplace line item read 47% low the day after capture (`DAYS_BACK=1`/no offset), then settled to its final value by day 3. Each `(tenant, service, region, cost_date)` combination is written to Last9 exactly once, and Last9's TSDB accepts neither backfilled nor overwritten samples for the same labels+timestamp — so a day captured too early can never be corrected afterward.
+
+`CAPTURE_OFFSET_DAYS` (default `3`) shifts the whole fetch window back by this many days, so "yesterday" (with `DAYS_BACK=1`) becomes "3 days ago" — captured only after Cost Explorer's estimate has settled. The tradeoff: the report is `CAPTURE_OFFSET_DAYS` days behind "today" instead of 1. Do not lower this to chase freshness without accepting that recent days will read low until they settle.
+
 ## Dashboard
 
 Import `last9-dashboard.json` into Last9 for a pre-built 14-panel cost dashboard covering spend totals, daily trends, top services, and per-account breakdown.

aws-cost-explorer/cloudformation.yaml

@@ -28,6 +28,11 @@ Parameters:
     Default: 1
     Description: "Days of cost history to fetch per run. Use 1 for steady-state daily runs. Set to 30+ for initial backfill, then reduce to 1."
 
+  CaptureOffsetDays:
+    Type: Number
+    Default: 3
+    Description: "Shifts the fetch window this many days into the past. Cost Explorer's daily estimate keeps revising for ~2-3 days after the day passes, and a day can never be re-fetched/corrected once written to Last9 — this captures each day only after its estimate has settled."
+
   Schedule:
     Type: String
     Default: rate(1 day)
@@ -84,6 +89,7 @@ Resources:
           OTEL_EXPORTER_OTLP_HEADERS: !Ref OtlpHeaders
           OTEL_SERVICE_NAME: !Ref ServiceName
           DAYS_BACK: !Ref DaysBack
+          CAPTURE_OFFSET_DAYS: !Ref CaptureOffsetDays
           CUSTOM_LABELS: !Ref CustomLabels
       Code:
         S3Bucket: !Ref LambdaS3Bucket

aws-cost-explorer/deploy.sh

@@ -18,6 +18,11 @@
 #   FUNCTION_NAME=aws-cost-reporter   (default)
 #   AWS_REGION=us-east-1              (default)
 #   DAYS_BACK=1                       (default; use 30+ for initial backfill)
+#   CAPTURE_OFFSET_DAYS=3             (default; Cost Explorer's daily estimate keeps
+#                                      revising for ~2-3 days after the day passes, and
+#                                      a day can never be re-fetched/corrected once
+#                                      written — this shifts the fetch window back so
+#                                      each day is captured only after it has settled)
 #   SCHEDULE=rate(1 day)              (default)
 #   OTEL_SERVICE_NAME=aws-cost-reporter
 #   CUSTOM_LABELS=env=prod,team=platform   (optional; extra labels on every exported metric)
@@ -29,6 +34,7 @@ set -euo pipefail
 FUNCTION_NAME="${FUNCTION_NAME:-aws-cost-reporter}"
 AWS_REGION="${AWS_REGION:-us-east-1}"
 DAYS_BACK="${DAYS_BACK:-1}"
+CAPTURE_OFFSET_DAYS="${CAPTURE_OFFSET_DAYS:-3}"
 SCHEDULE="${SCHEDULE:-rate(1 day)}"
 OTEL_SERVICE_NAME="${OTEL_SERVICE_NAME:-aws-cost-reporter}"
 CUSTOM_LABELS="${CUSTOM_LABELS:-}"
@@ -80,6 +86,7 @@ if [[ "${USE_CF}" == "1" ]]; then
       LambdaS3Bucket="${CF_S3_BUCKET}" \
       LambdaS3Key="${CF_S3_KEY}" \
       DaysBack="${DAYS_BACK}" \
+      CaptureOffsetDays="${CAPTURE_OFFSET_DAYS}" \
       Schedule="${SCHEDULE}" \
       ServiceName="${OTEL_SERVICE_NAME}" \
       CustomLabels="${CUSTOM_LABELS}"
@@ -142,14 +149,15 @@ else
   # JWTs, comma-bearing service names). JSON escapes everything correctly.
   ENV_FILE=$(mktemp /tmp/aws-cost-reporter-env-XXXXXX.json)
   TRAP_ENV_FILE="${ENV_FILE}"
-  export OTEL_EXPORTER_OTLP_ENDPOINT OTEL_EXPORTER_OTLP_HEADERS OTEL_SERVICE_NAME DAYS_BACK CUSTOM_LABELS
+  export OTEL_EXPORTER_OTLP_ENDPOINT OTEL_EXPORTER_OTLP_HEADERS OTEL_SERVICE_NAME DAYS_BACK CAPTURE_OFFSET_DAYS CUSTOM_LABELS
   python3 -c '
 import json, os
 print(json.dumps({"Variables": {
     "OTEL_EXPORTER_OTLP_ENDPOINT": os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"],
     "OTEL_EXPORTER_OTLP_HEADERS": os.environ["OTEL_EXPORTER_OTLP_HEADERS"],
     "OTEL_SERVICE_NAME": os.environ["OTEL_SERVICE_NAME"],
     "DAYS_BACK": os.environ["DAYS_BACK"],
+    "CAPTURE_OFFSET_DAYS": os.environ["CAPTURE_OFFSET_DAYS"],
     "CUSTOM_LABELS": os.environ["CUSTOM_LABELS"],
 }}))' > "${ENV_FILE}"
   ENV_VARS="file://${ENV_FILE}"