registry: fix five silent cue/BM25 cache decay paths (#563)

[t0k0sh1]

Summary

#532's audit found five paths where the registry's cue/BM25 caches degrade silently — no error, no warning, no metric, nothing to trigger on. All five are fixed here, one PR, matching #562's precedent of a compact fix set with test-per-item.

Closes #563

1. Empty cue vector cached for the process lifetime. cue_vector now treats a provider's Ok(vec![]) as a resolve failure — logged, metered, and never written to cue_cache. CueCache also tracks the resident vector width and clears itself on a width change, so a backend swap behind a stable model name (the same hazard the gloss refresh path already guards against) self-heals instead of leaving stale-width cues stuck at a silent 0.0 forever.
2. refresh_bm25 marked the sidecar dirty even when nothing changed. The dirty gate is now changed — whether any source in the batch actually altered the resident index — instead of !sources.is_empty(). Bm25Index::remove_source/tombstone now report whether they tombstoned anything live.
3. CueCache::insert didn't refresh recency on an existing key. A re-inserted cue now advances the LRU clock exactly like a get, so a cue that keeps getting resolved can no longer be evicted while genuinely hot.
4. Semaphore::acquire had no deadline and no fairness. Replaced with acquire_until(deadline): a FIFO ticket queue (fixing potential starvation under parking_lot::Condvar's non-FIFO wakeups) and a real timeout, both backed by three new metrics — taguru_embed_slot_waiters (gauge), taguru_embed_slot_waits_total / _timeouts_total (counters).
5. TAGURU_EMBED_PARALLEL=0 silently rewritten to 1 in two places that could disagree. Now floored loudly at the env boundary (resolve_embed_parallel, warns) and normalized once in boot_with before it reaches both the embed_parallel field and the semaphore's construction, so the two ceilings can never read different numbers.

Test plan

• cargo fmt
• cargo clippy --all-targets -- -D warnings
• cargo test — full suite, 0 failed
• Diff-scoped cargo mutants --in-diff — 55 mutants, 0 missed, 0 timeouts (2 initially-missed mutants around the new waiting()/embed_slot_waiters() gauges fixed by sampling them while a thread is still blocked, not only after it returns)
• New test per item, each pinned to the specific defect (empty-vector rejection, width-drift self-heal, recency-on-reinsert, no-op-retract dirty gate, FIFO fairness + timeout, zero-embed_parallel no-hang)

Summary by CodeRabbit

• 新機能

  • 埋め込み処理の同時実行数を制御し、待機・タイムアウト状況を監視できるようになりました。
  • 埋め込みスロットの待機数、待機回数、タイムアウト回数をメトリクスで確認できます。
  • 埋め込み設定が0の場合は自動的に1へ補正されます。

• 改善

  • 埋め込み処理の待機順序が安定し、期限切れ時も適切に処理されます。
  • 空の埋め込み結果をキャッシュせず、再試行できるようになりました。
  • キャッシュの利用状況やベクトル幅の変更時の動作を改善しました。
  • 実際に検索インデックスが変更された場合のみ更新状態として扱います。

[coderabbitai]

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: b62b350b-774c-4d4b-853b-59c6fa0faa92

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Walkthrough

BM25の変更判定、CueCacheの整合性、埋め込みスロットの期限付きFIFO制御、並列数の正規化、待機状態のメトリクス出力を追加・変更しました。

Changes

Registry correctness and embedding control

Layer / File(s)	Summary
BM25とCueCacheの状態更新 src/bm25.rs, src/registry.rs, src/registry/core_tests.rs, src/registry/embeddings/gloss_tests.rs, src/registry/retrieval_cache.rs	BM25削除が実際に変更したかを返すようになりました。CueCacheは再挿入時刻とベクトル幅を管理し、空ベクトルをキャッシュしません。
埋め込みスロット取得と正規化 src/env.rs, src/registry/concurrency.rs, src/registry/boot.rs, src/registry.rs, src/registry/embeddings/gloss_tests.rs	埋め込みスロットを期限付きFIFOキューで管理します。embed_parallel: 0を1へ正規化します。期限切れ時はプロバイダーを呼び出さず、更新を失敗として処理します。
埋め込みスロットの可観測性 src/metrics.rs, src/metrics/prometheus.rs, src/metrics/record.rs, src/metrics/taxonomy.rs, src/registry/embeddings.rs, src/registry/gauges.rs	待機数、待機回数、タイムアウト回数を記録し、Prometheusのゲージとカウンターとして出力します。

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant 更新処理
  participant セマフォ
  participant 埋め込みプロバイダー
  participant メトリクス
  更新処理->>セマフォ: 期限付きスロット取得
  セマフォ-->>更新処理: Acquisition
  alt スロット取得成功
    更新処理->>埋め込みプロバイダー: ベクトル生成
    埋め込みプロバイダー-->>更新処理: ベクトル
  else 待機期限切れ
    セマフォ-->>更新処理: SlotDeadlineExceeded
    更新処理->>メトリクス: 待機タイムアウトと更新失敗を記録
  end

Loading

Possibly related PRs

• t0k0sh1/taguru#473：BM25のソース撤回と削除動作に関連します。
• t0k0sh1/taguru#521：埋め込みレジストリの動作変更に関連します。

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	タイトルは、registry の cue/BM25 キャッシュ劣化に対する5件の修正という主変更を明確に示しています。
Linked Issues check	✅ Passed	[ #563 ] の5要件を、空ベクトル、BM25 dirty、LRU、セマフォ、並列数正規化の実装とテストで満たしています。
Out of Scope Changes check	✅ Passed	変更は [#563] のキャッシュ劣化、埋め込みスロット制御、設定正規化、および関連メトリクスとテストに限定されています。
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 563-registry-cue-bm25-cache-decay

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/registry.rs`:
- Around line 2477-2496: Update the `sources` loop to have `index.upsert_source`
report whether it actually changed the index, and combine that result into
`changed` instead of unconditionally setting it to true. Ensure empty
`PassageRecord` values with no existing slots return false, while real tombstone
creation or slot additions return true; preserve the existing `remove_source`
dirty tracking.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 56a42b44-04e6-41c0-bebc-a38e9ff9ed86

📥 Commits

Reviewing files that changed from the base of the PR and between 067de23 and 1fb6156.

📒 Files selected for processing (14)

• src/bm25.rs
• src/env.rs
• src/metrics.rs
• src/metrics/prometheus.rs
• src/metrics/record.rs
• src/metrics/taxonomy.rs
• src/registry.rs
• src/registry/boot.rs
• src/registry/concurrency.rs
• src/registry/core_tests.rs
• src/registry/embeddings.rs
• src/registry/embeddings/gloss_tests.rs
• src/registry/gauges.rs
• src/registry/retrieval_cache.rs