feat(vector): tiering M1 — one memory-accounting spine every ceiling reads

CHANGELOG.md

@@ -6,6 +6,52 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Changed — Memory: vector tiering accounting spine (M1, tiering-v2 D3/D9)
+
+- **`--maxmemory` now counts vector segment memory.** The background eviction
+  check compared only KV bytes against the per-shard budget, so a pure-vector
+  workload could drive RSS to OOM while eviction reported "under budget".
+  `timers::run_eviction` now gates on the shard AGGREGATE — Σ all dbs' KV +
+  the shard's published vector bytes, computed once per 100ms tick — and
+  evicts across dbs only until the aggregate is back under budget (adversarial
+  review caught the initial per-db formulation, which both under-detected with
+  KV spread across dbs and over-evicted sibling dbs). When the un-evictable
+  vector term alone exceeds the budget, KV drains then errors OOM
+  (shared-budget semantics; per-db quotas remain the tenant-isolation
+  mechanism); the pressure cascade (which shrinks vectors via offload) fires
+  earlier at `--disk-offload-threshold`, so with disk-offload enabled vectors
+  shed first. **Known limitation:** the on-write eviction gate still checks
+  KV-only, so under `noeviction` a vector-heavy shard over the vector-aware
+  budget does not yet reject client writes — RSS is bounded by the pressure
+  cascade and the RSS watchdog instead (write-gate consistency is a tracked
+  M1 follow-up).
+- **Elastic budget classification is vector-aware.** A vector-heavy/KV-light
+  shard was misclassified as an idle donor, lending headroom to siblings while
+  its true footprint was over base — and the pressure cascade compared a
+  vector-inclusive used-term against a budget inflated by that donation. The
+  donor/hot snapshot now sums KV + vector per shard; a vector-heavy shard is
+  classified hot and borrows instead of donating.
+- **`--vec-warm-mmap-budget` is now an instance-total cap divided across
+  shards** (matching `--maxmemory` semantics). Each shard previously applied
+  the full value — an N-shard instance silently allowed N× the configured WARM
+  memory. **Behavior change:** multi-shard deployments relying on the old
+  per-shard meaning should multiply their flag value by the shard count. A
+  nonzero total floors at 1 byte/shard ("0" still disables enforcement).
+- **IVF and DiskANN-cold segments report real `resident_bytes()`.** Both tiers
+  contributed a hardcoded 0 to the roll-up (untracked RAM: IVF centroids +
+  posting lists; DiskANN PQ codes + codebook). They now feed the pressure
+  trigger, MEMORY DOCTOR, and Prometheus like every other tier.
+- **`INFO reclamation_mmap_warm_bytes` reports the live WARM counter.** The
+  field read a never-incremented local static (permanent 0) while the real
+  `MmapBudget` counter was write-only. It now reads the live counter, and a
+  new `reclamation_mmap_budget_evictions_total` field exposes cumulative
+  byte-cap evictions.
+- **D9 (DiskANN retention) quarantine:** config docs now disambiguate
+  COLD-stub (`unloaded`, exact reload-on-touch default valve) vs COLD-ann
+  (`cold`, DiskANN serve-from-disk, inert behind `MOON_VEC_COLD_TIER`); the
+  dead knobs are marked `[reserved: M3/M5]`. Keep-vs-delete is decided at the
+  M3 exit-review on real per-index query-frequency telemetry.
+
 ### Fixed — Vector: memory-aware WARM offload with a real, reloadable ceiling (PR #252)
 
 - **Reloadable byte-cap eviction (A).** `MmapBudget::enforce_budget` (the

src/command/info_reclamation.rs

@@ -104,10 +104,6 @@ pub static RECL_MANIFEST_ACTIVE: AtomicU64 = AtomicU64::new(0);
 /// P1 wires this alongside RECL_MANIFEST_ACTIVE.
 pub static RECL_MANIFEST_TOMBSTONES: AtomicU64 = AtomicU64::new(0);
 
-/// Bytes mapped by mmap'd warm vector segment files (OS page-cache backed).
-/// TODO(P10→Wave2): wire from WarmSearchSegment on map/unmap.
-pub static RECL_MMAP_WARM_BYTES: AtomicU64 = AtomicU64::new(0);
-
 /// Number of committed transactions in the MVCC treemap (RoaringTreemap::len()).
 /// MA2 wires this; emits 0 until pruning is implemented.
 pub static RECL_MVCC_COMMITTED: AtomicU64 = AtomicU64::new(0);
@@ -273,10 +269,15 @@ pub fn write_reclamation_section(buf: &mut String) {
     );
 
     // -- Mmap warm bytes --
+    // Re-pointed at the LIVE warm-tier resident counter maintained by
+    // `MmapBudget` (add/sub on register/evict). The former local
+    // `RECL_MMAP_WARM_BYTES` static here was never incremented (permanent 0).
     let _ = write!(
         buf,
-        "reclamation_mmap_warm_bytes:{}\r\n",
-        RECL_MMAP_WARM_BYTES.load(Ordering::Relaxed)
+        "reclamation_mmap_warm_bytes:{}\r\n\
+         reclamation_mmap_budget_evictions_total:{}\r\n",
+        crate::admin::recl_atomics::warm_resident_bytes(),
+        crate::admin::recl_atomics::budget_evictions_total()
     );
 
     // -- MVCC --
@@ -494,6 +495,41 @@ mod tests {
             "RECL_WAL_BYTES store must be visible in section output"
         );
     }
+
+    /// The `reclamation_mmap_warm_bytes` INFO field must reflect the LIVE
+    /// warm-tier resident-bytes counter maintained by `MmapBudget`
+    /// (`admin::recl_atomics`), not a dead never-incremented local static.
+    ///
+    /// RED until the emit is re-pointed at `recl_atomics::warm_resident_bytes()`:
+    /// the pre-existing `info_reclamation::RECL_MMAP_WARM_BYTES` is never
+    /// incremented anywhere, so the field is a permanent `0`.
+    ///
+    /// Robust to parallel `MmapBudget` tests touching the same process-global
+    /// counter: we hold a known `DELTA` across the emit so the live counter has
+    /// a provable floor (`>= DELTA`); balanced add/sub in sibling tests cannot
+    /// erase our contribution before our own `sub`.
+    #[test]
+    fn info_reclamation_mmap_warm_bytes_reflects_live_counter() {
+        use crate::admin::recl_atomics;
+        const DELTA: u64 = 4_096;
+        recl_atomics::add_warm_resident(DELTA);
+        let mut buf = String::new();
+        write_reclamation_section(&mut buf);
+        recl_atomics::sub_warm_resident(DELTA); // restore
+
+        let emitted = buf
+            .lines()
+            .find_map(|l| l.strip_prefix("reclamation_mmap_warm_bytes:"))
+            .and_then(|v| v.trim().parse::<u64>().ok())
+            .expect("reclamation_mmap_warm_bytes field must be present and numeric");
+
+        assert!(
+            emitted >= DELTA,
+            "INFO reclamation_mmap_warm_bytes ({emitted}) must reflect the live \
+             warm-resident counter (>= {DELTA} while we hold a delta); the dead \
+             never-incremented static emits a permanent 0"
+        );
+    }
 }
 
 // ---------------------------------------------------------------------------

src/config.rs

@@ -429,28 +429,40 @@ pub struct ServerConfig {
     #[arg(long = "vec-codes-mlock", default_value = "enable")]
     pub vec_codes_mlock: String,
 
-    /// Maximum resident bytes allowed across all warm-tier vector segments on
-    /// this shard (e.g. "2gb", "512mb", "0"). When the total exceeds this
-    /// limit the budget enforcer drops LRU warm segments from memory; they
-    /// are reloaded from disk on next access. Set to "0" to disable.
+    /// Maximum resident bytes allowed across all warm-tier vector segments
+    /// on this INSTANCE (e.g. "2gb", "512mb", "0"), divided evenly across
+    /// shards — matching `--maxmemory` semantics (A5, tiering-v2 D3). When a
+    /// shard's share is exceeded the budget enforcer demotes LRU warm
+    /// segments to reloadable COLD stubs; they reload from disk on next
+    /// access. Set to "0" to disable.
     ///
     /// Default: "2gb". Tune down for cgroup-constrained containers.
     #[arg(long = "vec-warm-mmap-budget", default_value = "2gb")]
     pub vec_warm_mmap_budget: String,
 
     // ── Cold-tier / DiskANN (EXPERIMENTAL — gated by MOON_VEC_COLD_TIER) ──
+    // Two distinct COLD concepts exist (tiering-v2 decision D9):
+    //   COLD-stub (`SegmentList.unloaded`, `UnloadedSegment`) — the DEFAULT
+    //     valve: exact, ~0 RAM, reload-on-touch. Always available.
+    //   COLD-ann (`SegmentList.cold`, `DiskAnnSegment`) — THIS section:
+    //     approximate serve-from-disk (PQ in RAM + Vamana on NVMe), kept
+    //     inert behind MOON_VEC_COLD_TIER until the M5 production gate
+    //     (promote-back, delete, restart recovery, recall gate); an M3-exit
+    //     review decides productionize-vs-delete on real EWMA telemetry.
     // The DiskANN cold tier is incomplete (no cold-segment deletion, ADC-only
     // recall, restart reload of PQ codebooks unfinished). The WARM->COLD
     // transition is a NO-OP unless an operator sets `MOON_VEC_COLD_TIER=1`;
-    // warm-tier mmap + LRU eviction handles out-of-RAM indexes by default.
+    // warm-tier byte-budget LRU demotion to COLD-stub handles out-of-RAM
+    // indexes by default.
     /// Seconds after last access before a WARM segment is promoted to COLD.
     /// Consumed by the cold-transition timer ONLY when the experimental cold
     /// tier is enabled (`MOON_VEC_COLD_TIER=1`); otherwise inert.
     #[arg(long = "segment-cold-after", default_value_t = 86_400)]
     pub segment_cold_after: u64,
 
     /// Minimum queries-per-second threshold; segments below this are COLD candidates.
-    /// Not yet consumed — reserved for the experimental cold-tier heuristic.
+    /// [reserved: M3] — becomes the per-index EWMA boundary between COLD-stub
+    /// (idle) and COLD-ann (queried) demotion in the frequency classifier.
     #[arg(long = "segment-cold-min-qps", default_value_t = 0.1)]
     pub segment_cold_min_qps: f64,
 
@@ -463,14 +475,14 @@ pub struct ServerConfig {
     pub memory_arenas_cap: u32,
 
     /// DiskANN beam width for disk-resident vector search.
-    /// Not yet consumed — reserved for the experimental cold-tier search path
-    /// (gated by MOON_VEC_COLD_TIER).
+    /// [reserved: M5] — consumed when the COLD-ann search path is
+    /// productionized (gated by MOON_VEC_COLD_TIER until then).
     #[arg(long = "vec-diskann-beam-width", default_value_t = 8)]
     pub vec_diskann_beam_width: u32,
 
     /// Number of HNSW upper levels cached in memory for DiskANN hybrid search.
-    /// Not yet consumed — reserved for the experimental cold-tier cache layer
-    /// (gated by MOON_VEC_COLD_TIER).
+    /// [reserved: M5] — consumed when the COLD-ann cache layer is
+    /// productionized (gated by MOON_VEC_COLD_TIER until then).
     #[arg(long = "vec-diskann-cache-levels", default_value_t = 3)]
     pub vec_diskann_cache_levels: u32,
 
@@ -1030,6 +1042,22 @@ impl ServerConfig {
         Self::parse_size(&self.vec_warm_mmap_budget).unwrap_or(2 * 1024 * 1024 * 1024)
     }
 
+    /// Per-shard share of `--vec-warm-mmap-budget` (accounting-spine A5,
+    /// tiering-v2 D3): the flag is an INSTANCE-TOTAL cap divided across
+    /// shards, matching `maxmemory_per_shard` semantics. Previously each
+    /// shard applied the full value — an N-shard instance silently allowed
+    /// N× the configured WARM memory. `0` still disables enforcement; a
+    /// nonzero total floors at 1 byte per shard (0 would flip semantics to
+    /// "unlimited", the unsafe direction). Division floor is fine otherwise:
+    /// under-allocating a soft budget is the safe direction.
+    pub fn vec_warm_mmap_budget_bytes_per_shard(&self) -> u64 {
+        let total = self.vec_warm_mmap_budget_bytes();
+        if total == 0 {
+            return 0;
+        }
+        (total / self.shards.max(1) as u64).max(1)
+    }
+
     /// Returns the effective disk offload directory, falling back to --dir.
     pub fn effective_disk_offload_dir(&self) -> PathBuf {
         self.disk_offload_dir
@@ -2435,6 +2463,41 @@ mod tests {
         assert_eq!(config.vec_diskann_cache_levels, 3);
     }
 
+    /// Accounting-spine A5 (tiering-v2 D3): `--vec-warm-mmap-budget` is an
+    /// INSTANCE-TOTAL cap divided across shards, matching
+    /// `maxmemory_per_shard` semantics. Previously each shard applied the
+    /// full value — an N-shard instance silently allowed N× the configured
+    /// WARM memory. RED until the per-shard accessor exists and divides.
+    #[test]
+    fn test_vec_warm_budget_divided_per_shard() {
+        let mut config = ServerConfig::parse_from::<[&str; 0], &str>([]);
+
+        // 4 shards × default "2gb" ⇒ 512 MiB per shard.
+        config.shards = 4;
+        assert_eq!(
+            config.vec_warm_mmap_budget_bytes_per_shard(),
+            512 * 1024 * 1024
+        );
+
+        // Single shard: unchanged (full 2 GiB).
+        config.shards = 1;
+        assert_eq!(
+            config.vec_warm_mmap_budget_bytes_per_shard(),
+            2 * 1024 * 1024 * 1024
+        );
+
+        // "0" disables enforcement regardless of shard count.
+        config.vec_warm_mmap_budget = "0".to_string();
+        config.shards = 8;
+        assert_eq!(config.vec_warm_mmap_budget_bytes_per_shard(), 0);
+
+        // A tiny budget over many shards must floor at 1 byte, NOT 0 —
+        // 0 flips semantics to "unlimited", the unsafe direction.
+        config.vec_warm_mmap_budget = "100".to_string();
+        config.shards = 128;
+        assert_eq!(config.vec_warm_mmap_budget_bytes_per_shard(), 1);
+    }
+
     #[test]
     fn test_cold_tier_custom() {
         let config = ServerConfig::parse_from([

src/shard/event_loop.rs

@@ -659,8 +659,9 @@ impl super::Shard {
 
         // Per-shard warm-segment mmap budget enforcer.
         // Owned exclusively by this event-loop task; no locking needed.
+        // A5: the flag is an instance-total cap; each shard enforces its share.
         let mut warm_mmap_budget = crate::vector::persistence::mmap_budget::MmapBudget::new(
-            server_config.vec_warm_mmap_budget_bytes(),
+            server_config.vec_warm_mmap_budget_bytes_per_shard(),
         );
         // Tokio path doesn't take these into the spawn signatures; suppress warnings.
         let (_, _, _) = (&spill_sender, &spill_file_id, &disk_offload_dir);

src/shard/persistence_tick.rs

@@ -336,30 +336,13 @@ pub(crate) fn run_eviction_tick(
     // per 100ms tick. Siblings read the published snapshot on their own
     // ticks, so every budget is at most one tick stale — the same slack the
     // static scheme already has between eviction passes.
-    {
-        let rt = runtime_config.read();
-        // C5 / Phase 3: compute per-shard KV memory via ShardSlice without
-        // lock acquisitions (avoids per-DB read locks; estimated_memory() is
-        // an O(1) accumulator read). Published unconditionally: MEMORY DOCTOR
-        // and the Prometheus KV gauge read this atomic even when maxmemory is
-        // unlimited — gating it on maxmemory > 0 left them at a permanent 0.
-        let used = crate::shard::slice::with_shard(|s| {
-            s.databases
-                .iter()
-                .map(|db| db.estimated_memory())
-                .sum::<usize>()
-        });
-        shard_databases.publish_memory(shard_id, used);
-        // Elastic budgets only exist under a finite maxmemory cap.
-        if rt.maxmemory > 0 {
-            shard_databases.recompute_elastic_budget(shard_id, &rt);
-        }
-    }
-
     // C5 / M4: publish vector/text/graph store memory for lock-free observers.
     // Uses the existing lock path (Wave E collapses to slice). Runs every tick
     // so Prometheus and MEMORY DOCTOR never see stale zero values for long.
-    // C5 / M4: publish vector/text/graph store-memory atomics via thread-local slice.
+    // A4 review (LOW): published BEFORE the KV publish + elastic recompute
+    // below so the recompute's vector-aware donor/hot classification reads
+    // THIS tick's vector figure, not last tick's (siblings' figures remain
+    // ≤ 1 tick stale by design).
     let vector_resident_bytes = crate::shard::slice::with_shard(|s| {
         use std::sync::atomic::Ordering;
         let (mutable, immutable) = s.vector_store.resident_bytes();
@@ -385,6 +368,26 @@ pub(crate) fn run_eviction_tick(
         mutable + immutable
     });
 
+    {
+        let rt = runtime_config.read();
+        // C5 / Phase 3: compute per-shard KV memory via ShardSlice without
+        // lock acquisitions (avoids per-DB read locks; estimated_memory() is
+        // an O(1) accumulator read). Published unconditionally: MEMORY DOCTOR
+        // and the Prometheus KV gauge read this atomic even when maxmemory is
+        // unlimited — gating it on maxmemory > 0 left them at a permanent 0.
+        let used = crate::shard::slice::with_shard(|s| {
+            s.databases
+                .iter()
+                .map(|db| db.estimated_memory())
+                .sum::<usize>()
+        });
+        shard_databases.publish_memory(shard_id, used);
+        // Elastic budgets only exist under a finite maxmemory cap.
+        if rt.maxmemory > 0 {
+            shard_databases.recompute_elastic_budget(shard_id, &rt);
+        }
+    }
+
     if server_config.disk_offload_enabled()
         && should_run_pressure_cascade(
             runtime_config,
@@ -631,6 +634,16 @@ pub(crate) fn handle_memory_pressure(
     // budget (maxmemory/num_shards) so the summed eviction across shards bounds
     // aggregate RSS at the whole-instance maxmemory.
     //
+    // A3 review (MEDIUM): this used-term stays KV-only DELIBERATELY, unlike
+    // `timers::run_eviction` (the disk-offload-off path), which adds the
+    // shard's vector bytes. Inside the cascade the vector term has already
+    // pulled its weight: it fired the trigger (`should_run_pressure_cascade`
+    // is vector-inclusive) and step 2 sheds vector memory directly via
+    // offload-to-COLD. Adding it here too would evict KV to pay for memory
+    // that step 2 is already reclaiming more cheaply; the trigger refires
+    // every 100ms tick, so the cascade converges with vectors shedding
+    // first and KV eviction as the residual step.
+    //
     // When a SpillThread is available, use the async path: entries are removed
     // from DashTable immediately (freeing RAM) and pwrite is deferred to the
     // background thread. Otherwise, fall back to synchronous spill.