valkey: exclude every subscription command from auto-pipelining - #38897
valkey: exclude every subscription command from auto-pipelining#38897robobun wants to merge 1 commit into
Conversation
The auto-pipelining disallow list spelled PUNSUBSCRIBE as UNPSUBSCRIBE, so
punsubscribe() and send("PUNSUBSCRIBE") were batched with the commands
issued alongside them while SUBSCRIBE, PSUBSCRIBE and UNSUBSCRIBE were not.
The list also lacked the sharded variants and only matched the uppercase
spelling, although is_subscription_command() already classifies all six
commands case-insensitively for SUBSCRIPTION_REQUEST.
Meta::check now clears SUPPORTS_AUTO_PIPELINING on every command it marks
as a subscription request, and the subscription entries are removed from
the disallow list so there is a single classification. The docs list is
corrected to match.
|
Warning Review limit reached
Next review available in: 13 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (3)
Comment |
|
Status: fix pushed; every CI lane that ran is green, the remaining red is two macOS jobs that never got an agent. Reproduced with CI on 8bffc7b: 177 of 179 jobs passed. On the debian x64 ASAN lane the new file is 15/15 and the docker-backed The debug-only |
There was a problem hiding this comment.
LGTM — focused typo fix that consolidates subscription-command classification onto a single source of truth.
What was reviewed:
- Traced
Meta::checkcallers: bothsend()invalkey.rs:1442and the raw-send()path injs_valkey_functions.rs:477route through it, andchecked_commandis what gets enqueued — the fix covers every path. - Confirmed the separate
if new.contains(SUBSCRIPTION_REQUEST)check (vs. nesting insideis_subscription_command) also handles callers that pre-set the flag (js_valkey_functions.rs:1701/1708/1825/1854). - Verified the three removed non-typo entries (
SUBSCRIBE/PSUBSCRIBE/UNSUBSCRIBE) are all matched case-insensitively byis_subscription_command(), so their behavior is unchanged. - Test file: hermetic mock server on
port: 0, buffers RESP frames across chunk boundaries,finally-cleanup,describe.concurrent, covers the full variant matrix plus a control.
Extended reasoning...
Overview
Three files: a 7-line net change to src/runtime/valkey_jsc/ValkeyCommand.rs (removes four subscription-command byte-string entries — one of which was the never-matching typo UNPSUBSCRIBE — from AUTO_PIPELINE_DISALLOWED_COMMANDS and adds three lines so Meta::check clears SUPPORTS_AUTO_PIPELINING whenever SUBSCRIPTION_REQUEST is set); the matching docs list in docs/runtime/redis.mdx; and a new docker-free test file that observes command-write ordering against a mock RESP3 server.
Security risks
None. This is scheduling metadata for an outbound Redis client. No parsing of untrusted input changes; no auth, crypto, or permissions surface.
Level of scrutiny
Low-to-medium. The Rust change is tiny and mechanical: it turns two divergent lists into one derived invariant, which is exactly the "one source of truth; derive one from the other" shape the review guide asks for. I traced the only two Meta::check call sites (valkey.rs:1442 for the internal send() all dedicated methods use, and js_valkey_functions.rs:477 for user-facing raw send()) and confirmed the checked meta is what reaches enqueue and every SUPPORTS_AUTO_PIPELINING consumer. The three correctly-spelled entries removed from the static set are all matched (case-insensitively) by is_subscription_command(), so their classification is unchanged; the newly-excluded cases (PUNSUBSCRIBE, sharded variants, lowercase raw spellings) were already flagged SUBSCRIPTION_REQUEST for reply-pairing, so this only brings the pipelining flag into agreement with existing intent.
Other factors
The test is well-constructed: hermetic (Bun.listen on 127.0.0.1, port: 0), no docker dependency, buffers socket reads to RESP frame boundaries, releases the client and server in finally, uses describe.concurrent, and covers the full matrix (dedicated punsubscribe()/psubscribe(), all six commands via raw send() in both casings, plus a positive control proving two PINGs do batch). The PR description documents fail-on-unfixed (10/15 fail on release 1.4.0 and unfixed debug) and pass-on-fixed (300/300 with --rerun-each=20 on ASAN debug), plus a real-server sanity probe. No CODEOWNERS entries cover valkey. No prior human review comments to address.
Problem
AUTO_PIPELINE_DISALLOWED_COMMANDSinsrc/runtime/valkey_jsc/ValkeyCommand.rs:170containedb"UNPSUBSCRIBE", which is not a Redis command. The intended entry isPUNSUBSCRIBE; its siblingsSUBSCRIBE,PSUBSCRIBEandUNSUBSCRIBEare listed correctly. The typo dates back to the originalBun.redisimplementation, so the entry never matched anything.client.punsubscribe(...)andclient.send("PUNSUBSCRIBE", ...)keptSUPPORTS_AUTO_PIPELININGafterMeta::check(ValkeyCommand.rs:177), so they were written to the socket in the same batch as the pipelined commands issued in the same tick, and were not held back until in-flight commands were answered. Every other subscription command takes the held-back path.is_subscription_command()(ValkeyCommand.rs:188) in two more ways: it had no entry for the shardedSSUBSCRIBE/SUNSUBSCRIBE, and it only matched the uppercase spelling, whileis_subscription_command()matches all six commands in any casing when it setsSUBSCRIPTION_REQUEST. So a rawsend("unsubscribe", ...)was classified as a subscription request for reply pairing but pipelined like a normal command.docs/runtime/redis.mdx:371documented the list with the same typo.Fix
Meta::checknow clearsSUPPORTS_AUTO_PIPELININGon any command that ends up flaggedSUBSCRIPTION_REQUEST, and the four subscription entries (including the misspelled one) are removed fromAUTO_PIPELINE_DISALLOWED_COMMANDS. There is now one classification of subscription commands,is_subscription_command(), driving both the reply-pairing flag and the pipelining exclusion, so the two cannot drift apart again.PUNSUBSCRIBEwas always meant to be excluded (the misspelled entry is that intent), and the fix gives it exactly the code pathUNSUBSCRIBEandPSUBSCRIBEalready take. Extending the exclusion to the sharded variants and to other spellings only affects rawsend()calls that the client already treats as subscription requests; the dedicated methods all send the canonical uppercase names, sosubscribe(),unsubscribe()andpsubscribe()behave exactly as before. No other command's classification changes.docs/runtime/redis.mdxnow readsPUNSUBSCRIBEand includesSSUBSCRIBE/SUNSUBSCRIBE.test/js/valkey/valkey-auto-pipelining.test.ts. A mock RESP3 server logs the order in which commands arrive and replies leave. Each test issues a pipelinedPINGand a subscription command in the same tick and expects> PING, < PING, > X, < X(X was held back until the PING was answered); a batched X shows up as> PING, > X, .... Rows:punsubscribe(),psubscribe(), rawsend()of all six commands in upper and lower case, plus a control showing twoPINGs do get batched. It needs no docker, unlike the rest of thetest/js/valkeysuite.bun1.4.0 and an unfixed debug build: 10 of 15 rows fail (punsubscribe(),PUNSUBSCRIBE, both sharded commands, all lowercase spellings); the 5 that pass are the control and the commands that were already excluded. 20 reruns give the same 200 failures / 100 passes on both builds.--rerun-each=20.valkey.test.ts, psubscribe plus punsubscribe in the same tick, pipelined commands queued on both sides of a punsubscribe, and raw lowercase / sharded spellings all resolve with their own replies (output below).resp-nesting-depth,valkey-incremental-scan,valkey-gcandconnection-failuresstill pass (31 pass, 13 skipped for docker).Background
RedisClientdoes not write each command to the socket immediately. A command whose meta hasSUPPORTS_AUTO_PIPELININGis queued and everything queued is written in one batch at the end of the current tick (on_auto_flushinvalkey.rs). A command without the flag is held until every in-flight command has been answered and is then written on its own (enqueue/drain/send_next_commandinvalkey.rs), andon_auto_flushstops a batch at the first such command.Meta::check(ValkeyCommand.rs) computes the final meta for every command right before it is sent, for the dedicated methods and for rawsend()alike. It is the only consumer ofAUTO_PIPELINE_DISALLOWED_COMMANDS.SUBSCRIPTION_REQUEST: subscription commands are answered with RESP3 push frames rather than ordinary replies. This flag tells the response handler (handle_responseinvalkey.rs) that the in-flight entry at the head of the queue is waiting for such a push.is_subscription_command()was added recently so that rawsend()of any of the six subscription commands, in any casing, gets the flag; the pipelining list predates it and was not updated.assertion failed: self.is_subscriber()that a second back-to-backpunsubscribe()trips is pre-existing (it reproduces identically on an unfixed debug build, and release builds are unaffected) and is being fixed separately in redis: accept unsubscribe acks that arrive outside subscriber mode #37340; the new test issues one unsubscribe per client, from subscriber mode, so it does not depend on that fix.Real-server probe output (fixed debug build)
Step 1 mirrors "commands issued alongside a multi-pattern psubscribe resolve with their own values" in
valkey.test.ts. Step 4 isPromise.all([send("PING",["before"]), punsubscribe(...), send("PING",["after"])]), i.e. pipelined commands queued on both sides of the now held-back command.Unfixed failure shape (released bun 1.4.0)