From 17f320a42d046ed372bdc7b5d7ec40dbde1e7521 Mon Sep 17 00:00:00 2001 From: Tin Dang Date: Wed, 8 Jul 2026 09:57:21 +0700 Subject: [PATCH] docs(tuning): vector bulk load & compaction section + fix stray CHANGELOG marker MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds a "Vector bulk load and compaction" section to docs/guides/tuning.md documenting the parallel-HNSW-build + insert-path-compaction behavior shipped in #237: vectors searchable immediately on the brute tier, HNSW build now overlapping ingest across cores, COMPACT_THRESHOLD as the time-to-serve vs recall lever, the MOON_VEC_COMPACT_WORKERS pool knob, the ~10K-vector parallel-build threshold, per-shard bulk-load parallelism, and the ingest-rate/recall trade-offs. Also removes a stray `<<<<<<< HEAD` conflict marker that #237's squash merge accidentally left in the CHANGELOG [Unreleased] heading — the merge resolution dropped the =======/>>>>>>> lines but not the <<<<<<< line, and the verification grep was $-anchored so it missed the trailing " HEAD". author: Tin Dang --- CHANGELOG.md | 12 +++++++++++- docs/guides/tuning.md | 37 +++++++++++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2c7d5a009..1350bc45b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] -<<<<<<< HEAD +### Docs — tuning guide: vector bulk load & compaction (PR #TBD) + +- `docs/guides/tuning.md`: new "Vector bulk load and compaction" section — documents + immediate-search-then-background-HNSW-build behavior, the `COMPACT_THRESHOLD` + time-to-serve vs recall lever, the `MOON_VEC_COMPACT_WORKERS` pool knob, the + ~10K-vector parallel-build threshold, per-shard bulk-load parallelism, and the + ingest-rate/recall trade-offs. Operator guidance for the parallel-build + + insert-path-compaction work shipped in #237. +- Removes a stray `<<<<<<< HEAD` conflict marker accidentally left in the + `[Unreleased]` heading by #237's squash merge. + ### Added — parallel HNSW build + insert-path compaction trigger: time-to-index-green 11× (PR #237) - **`src/vector/hnsw/parallel_build.rs`** (new): concurrent HNSW diff --git a/docs/guides/tuning.md b/docs/guides/tuning.md index bd9bf64eb..e8d559414 100644 --- a/docs/guides/tuning.md +++ b/docs/guides/tuning.md @@ -137,3 +137,40 @@ single-purpose benchmark rigs. embeddings; **TQ4** shines at 768-d and above. Validate recall with real embeddings, not random vectors. - Details: [Vector search guide](../vector-search-guide.md). + +## Vector bulk load and compaction + +Newly-inserted vectors are **searchable immediately** against the brute-force mutable +tier — exact results, but an O(N) scan per query. The HNSW graph that makes search +O(log N) is built by *compaction*, and Moon now builds it **concurrently with ingest** +across cores, so a bulk load reaches HNSW-tier serving shortly after the last insert +rather than on a later `FT.COMPACT`. On an 8-vCPU dedicated GCE instance, 50K × 384-d +vectors reach HNSW-quality serving in ≈ 9–10 s end to end. + +You usually don't need to touch anything — the defaults do the right thing. Reach for a +knob only in these cases: + +- **`COMPACT_THRESHOLD`** (per index, at `FT.CREATE`) sets when a mutable segment freezes + into an immutable HNSW segment. It's the main time-to-serve vs recall lever: + - *Streaming / continuous ingest* — leave it at the default. Segments compact in the + background as thresholds are crossed; queries stay fast throughout. + - *One-shot bulk load where recall matters most* — set it **at or above your dataset + size** and call `FT.COMPACT` once at the end. You get a single optimal segment (best + recall, no multi-segment beam split) at the cost of a later first-fast-query. + - Lower thresholds build more, smaller segments sooner (faster time-to-serve, ~0.001 + lower recall@10 from multi-segment search); higher thresholds do the opposite. +- **`MOON_VEC_COMPACT_WORKERS`** (env) sizes the background compaction thread pool. + Default is half the machine's cores, clamped to `[1, 8]`. Raise it on write-heavy + fleets that compact many indexes or shards at once; set `1` for strict shard-thread + isolation on latency-critical nodes. Segments of ~10K+ vectors additionally build with + a multi-core parallel HNSW builder; smaller segments use the single-threaded builder. + Both are automatic and correct on core-pinned deployments — no tuning required. +- **Trade-off to expect:** overlapping the HNSW build with ingest shares cores, so peak + *ingest* throughput drops while a build runs, and multi-segment serving costs about + 0.001 recall@10 versus a single fully-compacted segment. That buys a dramatically + faster time-to-first-fast-query. If you care about raw ingest rate and will query + later, prefer the high-`COMPACT_THRESHOLD` + single final `FT.COMPACT` recipe above. +- **Multi-shard:** each shard compacts its own segments independently and the trigger + fires per shard, so bulk loads parallelize across shards automatically. Co-locate + related vectors with hash tags only if you also do multi-key KV ops on them; vector + search itself scatter-gathers across shards regardless.