Skip to content

DOC-6854 Add redis-rb (Ruby) vector search docs: vecsearch page + ruby_home_query_vec [PARKED]#3652

Open
andy-stark-redis wants to merge 1 commit into
mainfrom
DOC-6854-redis-rb-vecsearch-docs
Open

DOC-6854 Add redis-rb (Ruby) vector search docs: vecsearch page + ruby_home_query_vec [PARKED]#3652
andy-stark-redis wants to merge 1 commit into
mainfrom
DOC-6854-redis-rb-vecsearch-docs

Conversation

@andy-stark-redis

@andy-stark-redis andy-stark-redis commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Preemptive docs for redis-rb's Query Engine vector search: a new "Index and query vectors" page for Ruby (content/develop/clients/ruby/vecsearch.md) and a run-verified ruby_home_query_vec example set. Follow-up to the parked Query Engine docs (DOC-6843 / #3641), which deliberately deferred vector search pending an embedding-library decision — now resolved by reusing the semantic-cache Ruby choice (informers running the ONNX all-MiniLM-L6-v2 encoder locally).

⛔ Do not merge yet

The Query Engine landed upstream in redis-rb#1356 (merged) but is not yet in a released gem — the latest is v5.4.1 (July 2025), which predates it. Merging now would document APIs (Redis::Commands::Search::*, create_index, vector_field, KNN search) that gem install redis cannot provide. The page carries a preview bannerText and this PR is labelled parked + do not merge yet.

Park manifest

Ticket: DOC-6854
Parked at: 2026-07-17
Trigger to pick up: redis-rb#1356 shipped in a released redis gem (a version later than v5.4.1 whose lib/redis/commands/modules/search is present)
Labels: parked, do not merge yet

Pinned sources (state observed at park time)

Source State at park Re-fetch
redis-rb#1356 — Query Engine support state: closed, merged: true, head SHA 659d885199d09b997172d753a2f67c39ee23769e, base master, milestone: none, updated 2026-07-14 (squash-merged to master as 71706ec) gh api repos/redis/redis-rb/pulls/1356 --jq '{state, merged, head_sha: .head.sha, base: .base.ref, milestone: .milestone.title, updated_at}'
redis-rb released gem latest released v5.4.1 (July 2025) — predates the Query Engine; no release yet contains #1356 gh api repos/redis/redis-rb/tags --jq '.[0:5] | map(.name)' (and check rubygems.org/gems/redis/versions)
informers gem (embedding lib) verified working at 1.3.0 (+ onnxruntime 0.11.3, tokenizers 0.7.0) on Ruby 4.0.6; ONNX all-MiniLM-L6-v2, 384-dim — already released, NOT a merge blocker (listed for provenance) gem list informers onnxruntime tokenizers

Observed shape the page assumes — confidence: MEDIUM

The example was run-verified on 2026-07-17 against the local redis-rb fork (master, PR #1356 merged, head 71706ec) on Redis 8 with informers 1.3.0: ruby -I <fork>/lib home_query_vec.rb exited 0, all REMOVE-block asserts passed, and the hash and JSON paths returned identical results. Confidence is MEDIUM rather than HIGH only because it was verified against fork master, not a released gem, so the surface could still shift before release. Key assumptions:

  • Schema DSL Search::Schema.build { text_field; tag_field; vector_field 'embedding', 'HNSW', type: 'FLOAT32', dim: 384, distance_metric: 'L2' }; vector_field extracts :as as a field-level option (verified in lib/redis/commands/modules/search/schema.rb), so the JSON path alias vector_field '$.embedding', 'HNSW', as: 'embedding', … works.
  • KNN query via index.search('*=>[KNN 3 @embedding $vec AS vector_distance]', sort_by:, return_fields:, params: { 'vec' => bytes }); sort_by defaults to ascending (nearest-first). Document#[] exposes doc['vector_distance'].
  • Search::Index strips the index key prefix from returned doc IDsdoc.id is 0, not doc:0 (unlike redis-py). The page/example call this out.
  • Vector encoding: hash fields store raw little-endian float32 (Array#pack('e*')); JSON stores the embedding as a plain Array<Float>, but the query parameter is packed bytes either way.
  • Embeddings from informers (model.(text, pooling: 'mean', normalize: true), flat 384-element Array) reproduce the canonical L2 distances to ~7 significant figures — 0.114169895649 / 0.610845208168 / 1.48624789715. Normalisation (unit vectors) is load-bearing for those distances (see commit Constraint: trailers).

Re-check checklist

  • (highest risk) Re-run local_examples/client-specific/ruby/home_query_vec.rb against the released gem on Redis 8.x; confirm all steps + asserts still pass and the API shape above is unchanged. (Note: the Redis TCP connection needs the sandbox OFF — Operation not permitted on localhost otherwise.)
  • Confirm the run-verified distances in both expected-output blocks of vecsearch.md still hold on the released gem; update if they shift.
  • Add a "requires redis-rb vX.Y+" note to vecsearch.md once the release version is known.
  • Remove the preview bannerText from vecsearch.md once released.
  • Confirm examples.json regenerates the Ruby tab from ruby_home_query_vec in a full CI build (not run locally).
  • When reconciling, preserve the commit Constraint: trailers — keep normalize on for every encode, and keep the embedding model/library aligned with the semantic-cache Ruby example — or stored vectors stop being comparable across examples.
  • No command-api-mapping work in this PR (the FT.* family is covered by DOC-6843 / DOC-6843 Add redis-rb (Ruby) Query Engine docs: queryjson [PARKED] #3641).

On unpark, then

When the trigger fires, run /unpark <this PR number>: it diffs each pinned source against the snapshot above, walks this checklist, then takes the PR through the normal /reflect/finalize pipeline to merge. The do not merge yet guard holds until /finalize completes.

🤖 Generated with Claude Code


Note

Low Risk
Documentation and example-only changes with no runtime or production code; main risk is documenting unreleased redis-rb APIs, which the page banner and park manifest already flag.

Overview
Adds a new Ruby client doc page Index and query vectors (vecsearch.md) that walks through indexing and KNN-searching text embeddings with redis-rb’s Query Engine, using the informers gem and all-MiniLM-L6-v2 locally.

The page is wired to a new run-verified example set ruby_home_query_vec (home_query_vec.rb): hash flow with to_bytes / FLOAT32 packing, then a JSON section covering path aliases, array vs binary storage, and packed query params. It also documents redis-rb behaviors such as default query dialect 2, stripped key prefixes on doc IDs, and a preview banner noting Query Engine APIs are not in a released gem yet.

Reviewed by Cursor Bugbot for commit 8aedcc0. Bugbot is set up for automated code reviews on this repo. Configure here.

…y_home_query_vec example

Adds the Ruby "Index and query vectors" page and its backing example set,
mirroring the redis-py/jedis vecsearch pages (hash + JSON). The embedding
library was the open question that got vector search deferred out of
DOC-6843; resolved it by reusing the semantic-cache Ruby choice — informers
running the ONNX all-MiniLM-L6-v2 encoder locally — the same 384-dim model
every other language's page uses, so vectors stay comparable across examples.
Normalising the embeddings is what makes the reported L2 distances line up
with the canonical page; run-verified against the fork (QE branch) and
Redis 8, where informers matched the reference distances to ~7 significant
figures, much closer than the ONNX drift I had expected.

Learned: informers with normalised embeddings reproduces the canonical L2 distances almost exactly; the deferred embedding-lib decision was already answered by the semantic-cache Ruby example
Constraint: keep normalize on for every encode — the L2 distances and cross-example comparability depend on unit vectors
Constraint: embedding model/library must stay aligned with the semantic-cache Ruby example (all-MiniLM-L6-v2 via informers) or stored vectors stop being comparable
Recheck: when redis-rb#1356 ships in a released redis gem (page is a parked preview against the fork)
Ticket: DOC-6854
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@andy-stark-redis andy-stark-redis added do not merge yet parked PR speculatively added based on pre-release info. Check validity when release goes ahead. labels Jul 17, 2026
@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

DOC-6854

@github-actions

Copy link
Copy Markdown
Contributor

andy-stark-redis added a commit that referenced this pull request Jul 22, 2026
Renamed the parked-PR thaw skill from `/pickup` to `/unpark` — directory, frontmatter, title, and prose — to free the `/pickup` name for another skill where it fits better; `park`/`unpark` also reads as a cleaner verb pair. Updated the two files that reference it, `/park` and the shared `_shared/park-manifest.md` contract, so the pairing stays consistent. The three parked PRs that instructed `/pickup <n>` (#3611, #3641, #3652) were updated out-of-band so no stale invocation survives. Approved by dwdougherty; Cursor Bugbot clean.

Directive: Two "pickup" strings are left un-renamed on purpose — the historical .claude/state/assess-comments.coverage.md audit log (it records the skill's old name as it was at the time; renaming falsifies history) and the "Trigger to pick up:" manifest field name in park-manifest.md (a header field, not the skill name). Don't "fix" them in a follow-up grep-and-replace.
Ticket: DOC-6876
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do not merge yet parked PR speculatively added based on pre-release info. Check validity when release goes ahead.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant