fix (redis): unEscapeSpecialChars now handles all RediSearch special c…#6598#6601
fix (redis): unEscapeSpecialChars now handles all RediSearch special c…#6598#6601eajajhossain wants to merge 1 commit into
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the Redis vector store utility functions to escape and unescape all RediSearch special characters instead of only hyphens, and introduces a comprehensive test suite. Feedback on the changes highlights a critical issue where unescaping all special characters (especially backslashes) on the entire JSON string can corrupt valid JSON escape sequences, potentially causing JSON parsing to fail. It is recommended to only unescape -, :, and " which are the characters escaped by @langchain/redis. Additionally, it is suggested to escape other characters like |, ?, and / to prevent query injection, and to add a regression test case for metadata containing backslashes (such as Windows file paths).
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
|
|
||
| export const unEscapeSpecialChars = (str: string) => { | ||
| return str.replaceAll('\\-', '-') | ||
| return str.replace(/\\([,.<>{}[\]"':;!@#$%^&*()\-+=~\\])/g, '$1') |
There was a problem hiding this comment.
Unescaping all RediSearch special characters (especially \\) on the entire JSON string will corrupt valid JSON escape sequences (such as \\\\ representing a single backslash in a path or string value). Since @langchain/redis only escapes -, :, and " when storing metadata, unEscapeSpecialChars should only unescape these three characters to avoid corrupting the JSON structure and causing JSON.parse to fail on metadata containing backslashes (e.g., Windows file paths).
| return str.replace(/\\([,.<>{}[\]"':;!@#$%^&*()\-+=~\\])/g, '$1') | |
| return str.replace(/\\([-:"])/g, '$1') |
| */ | ||
| export const escapeSpecialChars = (str: string) => { | ||
| return str.replaceAll('-', '\\-') | ||
| return str.replace(/([,.<>{}[\]"':;!@#$%^&*()\-+=~\\])/g, '\\$1') |
There was a problem hiding this comment.
In particular, unescaped | characters in filter values can be interpreted as OR operators, which can lead to query injection or filter bypasses. We should include |, ?, and / in the list of escaped characters to ensure robust query execution and security.
return str.replace(/([,.\u003c\u003e{}[\\]\"':;!@#$%^\u0026*()\\-+=~\\\\|?\\/])/g, '\\$1')| test('round-trips plain metadata with no special chars (sanity check)', () => { | ||
| const original = { source: 'simple_file.pdf', page: 4 } | ||
| const stored = simulateUpsert(original) | ||
| const retrieved = simulateRetrieval(stored) | ||
| expect(retrieved).toEqual(original) | ||
| }) |
There was a problem hiding this comment.
We should add a regression test case to verify that metadata containing backslashes (such as Windows file paths) can be successfully round-tripped without causing JSON parsing errors.
test('round-trips plain metadata with no special chars (sanity check)', () => {
const original = { source: 'simple_file.pdf', page: 4 }
const stored = simulateUpsert(original)
const retrieved = simulateRetrieval(stored)
expect(retrieved).toEqual(original)
})
test('round-trips metadata containing backslashes in a value', () => {
const original = { path: 'C:\\\\path\\\\to\\\\file', page: 5 }
const stored = simulateUpsert(original)
const retrieved = simulateRetrieval(stored)
expect(retrieved).toEqual(original)
})There was a problem hiding this comment.
Pull request overview
This PR updates the Redis vectorstore utilities to properly escape/unescape a broader set of RediSearch operator/delimiter characters, preventing malformed metadata round-trips (notably when JSON-serialized metadata includes reserved characters), and adds a dedicated test suite to lock in the behavior.
Changes:
- Expand
escapeSpecialCharsto escape a full set of RediSearch-reserved characters (instead of only-). - Expand
unEscapeSpecialCharsto reverse those escapes so metadata JSON can be parsed correctly on retrieval. - Add
utils.test.tswith unit tests plus a production-flow regression round-trip test.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| packages/components/nodes/vectorstores/Redis/utils.ts | Broadens RediSearch escaping/unescaping logic used by query-building and metadata retrieval. |
| packages/components/nodes/vectorstores/Redis/utils.test.ts | Adds coverage for escaping/unescaping and a regression test simulating the real storage/retrieval flow. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| * so they must be escaped when stored as metadata values: | ||
| * , . < > { } [ ] " ' : ; ! @ # $ % ^ & * ( ) - + = ~ \ | ||
| */ | ||
| export const escapeSpecialChars = (str: string) => { | ||
| return str.replaceAll('-', '\\-') | ||
| return str.replace(/([,.<>{}[\]"':;!@#$%^&*()\-+=~\\])/g, '\\$1') |
| export const unEscapeSpecialChars = (str: string) => { | ||
| return str.replaceAll('\\-', '-') | ||
| return str.replace(/\\([,.<>{}[\]"':;!@#$%^&*()\-+=~\\])/g, '$1') |
| '+', | ||
| '=', | ||
| '~', | ||
| '\\' | ||
| ] |
| '+', | ||
| '=', | ||
| '~', | ||
| '\\' | ||
| ] |
eajajhossain
changed the title
issue - 6598
Title:
fix(redis): unEscapeSpecialChars now handles all RediSearch special charaers
Description:
Bug:
Similarity search in the Retrieval Playground fails with
SyntaxError: Expected property name or '}' in JSONwhen a document's metadata contains RediSearch special characters such as:or"(e.g.gs://bucket:path/file.pdf).Root Cause:
@langchain/redis` stores metadata as escapeSpecialChars(JSON.stringify(metadata)), escaping -, :, and " in the full JSON string. On retrieval, Flowise calls unEscapeSpecialChars() before JSON.parse(). The old implementation only removed -, leaving " and : in the string, producing invalid JSON.
Fix:
Extended unEscapeSpecialChars regex in utils.ts to cover all RediSearch-reserved characters — matching the full set that RediSearch may escape.
Files Changed:
packages/components/nodes/vectorstores/Redis/utils.ts - fixed unEscapeSpecialChars
packages/components/nodes/vectorstores/Redis/utils.test.ts - added regression tests (15 tests, all passing)