fix(transfer): preserve Knowledge Page search state - #3315
Closed
JiehoonKwak wants to merge 3 commits into
Closed
Conversation
Keep mental-model embeddings and backend-specific lexical projections aligned across create, update, clear, and Knowledge Page rename operations. Reconcile the current mental_models table instead of the retired reflections name and backfill missing VChord projections without weakening the fail-closed backend-switch guard. Context: - Fixes the derived-state prerequisite tracked in vectorize-io#3307. - Keeps PostgreSQL-specific projection SQL behind backend.ops. - Preserves native generated tsvector behavior and base-column backends. - Adds focused lifecycle, reconciliation, and backend abstraction coverage.
Whole-bank archives previously skipped knowledge_pages and restored mental models without rebuilding backend-specific derived state. That made blue-green backend migrations lose the page tree and leave carried mental models unsearchable.\n\nCarry Knowledge Page rows and counts, validate and restore their tree parent-first, re-embed mental models on the target, and rebuild the target text projection through the backend capability introduced by the mental-model lifecycle fix. Preflight validation and target-bank checks happen before writes or avoidable embedding cost.\n\nThe archive format remains schema version 1 because the new manifest count and knowledge_pages.json payload are additive; legacy archives still import with an empty page set. Tests cover corrupt trees, backend projection rebuilding, target-bank remapping, and an exact PostgreSQL round trip of pages and their backing models.\n\nContext:\n- Implements the transfer half of upstream issue vectorize-io#3308.\n- Intentionally depends on codex/fix-mental-model-search-lifecycle.\n- Keeps export/import LLM-free and does not fire webhooks or consolidation.\n- Production Compose and PostgreSQL were not changed.
Keep the generated Hindsight docs skill synchronized with the bank-transfer contract added in this branch. The mirror now documents Knowledge Page hierarchy, mental-model history, and target-side embedding/search-state regeneration, which is why verify-generated-files failed on PR vectorize-io#3315.
JiehoonKwak
marked this pull request as ready for review
August 10, 2026 09:46
Contributor
Author
|
Superseded by #3330, which implements the same transfer goal with a smaller typed Knowledge Page payload and target-derived mental-model search state. I left the only material remaining safety delta (pre-write tree/model validation for corrupt archives) as a focused review comment on #3330. Closing this stacked branch avoids duplicating the maintainer-owned lane. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Root cause
Whole-bank export skipped
knowledge_pagesand stripped mental-model derived state. Import restored mental-model rows without regenerating embeddings or the target backend projection, so a cross-instance migration could lose the page tree and leave its search state unusable.Closes #3308.
Dependency
Depends on #3313 for the backend capability used to rebuild mental-model search state. This branch contains that prerequisite commit; the transfer-specific review commit is
07d091b2a.Verification
The same-instance clone collision tracked in #3270 is a separate existing lane. This PR targets the documented cross-instance migration path and does not duplicate that work.