Skip to content

test: assert the client parses execution plans, and fix a plan parser bug - #267

Merged
Naseem77 merged 11 commits into
mainfrom
naseem77-align-py-tests-with-engine
Aug 13, 2026
Merged

test: assert the client parses execution plans, and fix a plan parser bug#267
Naseem77 merged 11 commits into
mainfrom
naseem77-align-py-tests-with-engine

Conversation

@Naseem77

@Naseem77 Naseem77 commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Our CI runs against falkordb/falkordb:edge, which now resolves to the Rust engine, and the test suite failed 16 tests there before reaching any client code.

The tests asserted the exact operation tree that GRAPH.EXPLAIN / GRAPH.PROFILE returned — root op Results, then Project, then Unwind, and so on. That makes a planner change break the client's tests even when the client is fine, which is what happened: Results is gone, Join became Union, MERGE is rooted at Commit with a new Include Pending op, and a traverse renders as (t)<-(r:Rider).

Which operations a query compiles into is the engine's business. What the client owes us is that it issues the command and parses the reply. So that's what these tests assert now, and they pass unchanged on both engines.

falkordb:edge (Rust) falkordb:v4.20.1 (C)
main 16 failed, 84 passed passes
this branch 100 passed 100 passed

What the tests check now

tests/plan_helpers.py asserts, for any plan from any engine:

  • every non-blank line of the raw reply became exactly one operation
  • each operation sits as deep in the tree as its line was indented
  • the name and the | args half were split apart correctly
  • for profiles, statistics parsed, and records_produced matches the row count the query implies

The indentation comparison is the important one — it checks the parsed tree against the raw reply, so it fails if nesting is reconstructed wrongly.

It caught a real parser bug

Writing that assertion turned up a genuine client bug, reproducible on both engines. When three or more operations share a nesting level, the parser attached the third to the second:

reply:                          parsed as:
Project                         Project
    Cartesian Product               Cartesian Product
        All Node Scan | (a)             All Node Scan | (a)
        All Node Scan | (b)             All Node Scan | (b)
        All Node Scan | (c)                 All Node Scan | (c)   <- wrong parent

MATCH (a), (b), (c) RETURN * is enough to hit it. The cause: each new sibling was pushed onto the stack in place of its parent, so the next sibling popped a sibling instead. Fixed in the first commit by keeping the parent on the stack.

This is why the old assertions couldn't catch it — they only ever used two-way branches, and comparing a plan to a hardcoded tree never checks the reply itself.

Commits

  1. fix: attach every sibling operation to its parent when parsing a plan — the bug above.
  2. test: assert the client parses execution plans, not which plan the engine picked — the rework.
  3. test: use a slower query so the slowlog actually records itUNWIND range(0, 200000) runs in ~6ms on the Rust engine, below the ~10ms slowlog threshold, so nothing was recorded and the assertion indexed an empty list. Bumped to 1M rows (~25ms).

Verification

Full suite against live servers on both engines, plus ruff format --check, ruff check and mypy falkordb/. The parser fix was checked both ways: the new assertion fails against the old parser and passes against the fixed one.

Relationship to #266

#266 adapts the explain/profile tests to the Rust engine's new plans. This takes the other route — not asserting the plan at all — so it also covers the slowlog tests and the Join/Union drift in test_graph.py and test_async_graph.py, and it keeps passing on the C engine. Happy to close whichever we don't want.

Note on the codecov/project check

The repo has no codecov.yml, so it falls back to target: auto, threshold: 0% and any dip fails. test_query_timeout is marked xfail(strict=False) and its 1ms server-side timeout doesn't always fire; when it doesn't, the test stops early and a few lines go unexecuted. Running the same commit five times gives 4 xfailed / 1 xpassed. Unrelated to these changes — happy to fix it separately if it's worth pinning down.

Summary by CodeRabbit

  • Bug Fixes

    • Improved cluster detection by filtering unsupported connection settings before establishing a connection.
    • Corrected execution-plan parsing so sibling operations are associated with the correct parent.
  • Tests

    • Added broader validation for execution plans and profiling results.
    • Updated tests to remain reliable across engine-specific plan variations.
    • Added coverage for three-way Cartesian products and regression coverage for cluster detection.

The async client cannot run any query against redis >= 8.1.0:

    TypeError: Redis.__init__() got an unexpected keyword argument
    'himport_registry'

`Is_Cluster` copies the async pool's `connection_kwargs` and splats them
into a synchronous `redis.Redis(**kwargs)` probe. redis-py keeps internal
state in that dict alongside real connection parameters, and 8.1.0 added
six such entries (`himport_registry`, `maint_notifications_pool_handler`,
`maint_notifications_config`, `orig_host_address`, `orig_socket_timeout`,
`orig_socket_connect_timeout`). None are accepted by `Redis.__init__`.

Since the probe runs during connection setup, this fails before any
command is sent — every async query, not just cluster deployments.

Filter the kwargs against the constructor signature instead. This also
covers #235 (the same crash with `path`), and any future internal key,
without needing to enumerate them. The sync client is unaffected because
it already picks fields out by name rather than forwarding everything.

Fixes #261.
Builds a real async pool, injects an unknown key, and drives `Is_Cluster`
through both branches.

Only `info()` is stubbed — the real `Redis.__init__` runs, because
rejecting the extra kwargs is exactly what used to raise. Patching the
whole class would hide the regression, since the filter reads the
constructor signature.

The key is injected explicitly rather than relying on the installed redis
version, so the test still guards the behaviour on redis < 8.1.0.

Verified: reverting the previous commit fails this test with the reported
TypeError.
@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 9d505e53-b587-4884-9e87-2e04e400947a

📥 Commits

Reviewing files that changed from the base of the PR and between f7c3de7 and c0bdfb9.

📒 Files selected for processing (7)
  • tests/plan_helpers.py
  • tests/test_async_explain.py
  • tests/test_async_graph.py
  • tests/test_async_profile.py
  • tests/test_explain.py
  • tests/test_graph.py
  • tests/test_profile.py
🚧 Files skipped from review as they are similar to previous changes (5)
  • tests/test_async_explain.py
  • tests/test_async_graph.py
  • tests/test_profile.py
  • tests/test_graph.py
  • tests/test_async_profile.py

📝 Walkthrough

Walkthrough

The change adds shared execution-plan and profile validators, preserves parent nodes for sibling operations, filters unsupported cluster-probe parameters, and updates synchronous and asynchronous explain, graph, profile, and cluster tests.

Changes

Execution validation and cluster detection

Layer / File(s) Summary
Plan tree construction and shared validators
falkordb/execution_plan.py, tests/plan_helpers.py
Execution-tree construction preserves parent operations for siblings. Shared helpers validate parsed plan structure, arguments, nesting, profile statistics, execution times, and produced records.
Cluster probe parameter filtering
falkordb/asyncio/cluster.py, tests/test_async_db.py
Is_Cluster forwards only parameters accepted by the synchronous Redis constructor. Tests cover unsupported pool arguments and standalone or cluster detection.
Explain, graph, and profile test validation
tests/test_async_explain.py, tests/test_async_graph.py, tests/test_async_profile.py, tests/test_explain.py, tests/test_graph.py, tests/test_profile.py
Tests replace fixed operator-tree assertions with shared parsed-plan and profile checks. Slow-log query ranges increase from 200,000 to 1,000,000 where applicable.

Estimated code review effort: 3 (Moderate) | ~20 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 11.76% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main changes: execution-plan parsing tests and a parser bug fix.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch naseem77-align-py-tests-with-engine

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Aug 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.37398% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.84%. Comparing base (031eb04) to head (e78b3dd).

Files with missing lines Patch % Lines
tests/plan_helpers.py 96.87% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #267      +/-   ##
==========================================
- Coverage   92.98%   92.84%   -0.15%     
==========================================
  Files          38       39       +1     
  Lines        3152     3088      -64     
==========================================
- Hits         2931     2867      -64     
  Misses        221      221              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
tests/test_async_graph.py (1)

581-601: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Validate nested Operation children.

Operation.__eq__ compares only name and args. It does not compare children. The assertions therefore accept any structured subtree with the expected root operation. Use a recursive assertion helper that compares each operation name, args, child count, and child operations.

  • tests/test_async_graph.py#L581-L601: Validate both Union branches recursively.
  • tests/test_async_graph.py#L615-L618: Validate the Cartesian Product children recursively.
  • tests/test_graph.py#L543-L563: Validate both Union branches recursively.
  • tests/test_graph.py#L577-L580: Validate the Cartesian Product children recursively.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/test_async_graph.py` around lines 581 - 601, Operation equality ignores
children, so the tests can accept incorrect nested query plans. Add or reuse a
recursive assertion helper that checks each Operation’s name, args, child count,
and descendants, then apply it to both Union branches in
tests/test_async_graph.py:581-601 and tests/test_graph.py:543-563, and to the
Cartesian Product children in tests/test_async_graph.py:615-618 and
tests/test_graph.py:577-580.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@tests/test_async_graph.py`:
- Around line 581-601: Operation equality ignores children, so the tests can
accept incorrect nested query plans. Add or reuse a recursive assertion helper
that checks each Operation’s name, args, child count, and descendants, then
apply it to both Union branches in tests/test_async_graph.py:581-601 and
tests/test_graph.py:543-563, and to the Cartesian Product children in
tests/test_async_graph.py:615-618 and tests/test_graph.py:577-580.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 6c56f699-2935-46a5-b7ad-6ddd1af5ca14

📥 Commits

Reviewing files that changed from the base of the PR and between 5cb890f and db335e8.

📒 Files selected for processing (6)
  • tests/test_async_explain.py
  • tests/test_async_graph.py
  • tests/test_async_profile.py
  • tests/test_explain.py
  • tests/test_graph.py
  • tests/test_profile.py

When three or more operations share a nesting level, the parser pushed each
new sibling onto the stack in place of their parent, so the next sibling was
attached to the previous one instead. A plan like

    Project
        Cartesian Product
            All Node Scan | (a)
            All Node Scan | (b)
            All Node Scan | (c)

parsed with (c) nested under (b). Keep the parent on the stack instead.
…gine picked

The tests asserted the exact operation tree GRAPH.EXPLAIN and GRAPH.PROFILE
returned, so a planner change broke them even though the client was fine.
Which operations a query compiles into is the engine's business; the client
is responsible for parsing the reply.

Assert that instead: every line of the reply became exactly one operation,
nested as deep as that line was indented, with the name and arguments split
off correctly and the profile statistics parsed. These hold on both engines.
@Naseem77
Naseem77 force-pushed the naseem77-align-py-tests-with-engine branch from db335e8 to 49aa012 Compare August 10, 2026 13:32
@Naseem77 Naseem77 changed the title test: align test suite with the current engine test: assert the client parses execution plans, and fix a plan parser bug Aug 10, 2026
@Naseem77
Naseem77 requested a review from swilly22 August 11, 2026 09:37

plan = await g.profile("UNWIND range(0, 3) AS x RETURN x")

results_op = plan.structured_plan

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

even the number of operations might change from one implementation to another.
i prefer we use queries that have the same plan in both our C and Rust versions, instead of narrowing it down to just minimum number of operations, as this test is too weak.

…args

fix: only forward accepted kwargs from Is_Cluster to redis.Redis

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/plan_helpers.py`:
- Around line 79-82: Update the records_produced assertion in the relevant test
helper to compare plan.structured_plan.records_produced directly with
records_produced, instead of taking the maximum across parsed operations.
Preserve the existing conditional behavior when records_produced is None.

In `@tests/test_async_db.py`:
- Around line 354-388: Move the synchronous
test_is_cluster_filters_unknown_connection_kwargs from tests/test_async_db.py to
the corresponding synchronous test module, preserving its assertions and setup.
Keep tests/test_async_db.py limited to pytest-asyncio tests and maintain the
existing async/sync test mirroring convention.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 28cc287b-d70f-4f3d-96bf-e4a1f11c6dc2

📥 Commits

Reviewing files that changed from the base of the PR and between db335e8 and f7c3de7.

📒 Files selected for processing (10)
  • falkordb/asyncio/cluster.py
  • falkordb/execution_plan.py
  • tests/plan_helpers.py
  • tests/test_async_db.py
  • tests/test_async_explain.py
  • tests/test_async_graph.py
  • tests/test_async_profile.py
  • tests/test_explain.py
  • tests/test_graph.py
  • tests/test_profile.py
🚧 Files skipped from review as they are similar to previous changes (2)
  • tests/test_profile.py
  • tests/test_async_profile.py

Comment thread tests/plan_helpers.py
Comment thread tests/test_async_db.py
Naseem77 and others added 2 commits August 12, 2026 17:43
min_operations only checked that a plan had at least N operations, which
passes on almost any tree. Replace it with assert_plan_shape, which pins
the operation names and their nesting against an indented literal, and
optionally their arguments.

The C and Rust engines put a different driver operation at the root — C
wraps a read plan in Results, Rust wraps a write plan in Commit — while
the plan below it is identical, so no query matches byte for byte across
both. assert_plan_shape skips that root and asserts everything under it
exactly.

Two queries stay on assert_parsed_plan because the engines genuinely
compile them differently, each with a comment naming the divergence:
MERGE (Include Pending vs MergeCreate) and UNION (Union vs Join).

Add a three-way cartesian product test. Three operations sharing one
nesting level is the case the plan parser used to get wrong, so it guards
that fix directly.
@Naseem77
Naseem77 merged commit a9c7510 into main Aug 13, 2026
16 checks passed
@Naseem77
Naseem77 deleted the naseem77-align-py-tests-with-engine branch August 13, 2026 07:30
Naseem77 added a commit to SantoshDhaladhuli/falkordb-py that referenced this pull request Aug 13, 2026
Minor rather than patch: the release narrows the redis range, so it
changes which environments can install the client.

Since 1.6.2:
- fix: Is_Cluster() no longer forwards pool-only kwargs to redis.Redis,
  which broke every async client on redis 8.1 (FalkorDB#262)
- fix: the execution plan parser attaches every sibling operation to its
  parent instead of nesting siblings under each other (FalkorDB#267)
- build: redis and python-dateutil bounded to tested versions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants