Skip to content

rpc: recover capacity from stalled connections - #1240

Open
niteshbalusu11 wants to merge 2 commits into
reardencode:masterfrom
smolcars:rpc/expire-stalled-connections
Open

niteshbalusu11 wants to merge 2 commits into
reardencode:masterfrom
smolcars:rpc/expire-stalled-connections

Conversation

@niteshbalusu11

@niteshbalusu11 niteshbalusu11 commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Opening 256 idle RPC connections could block every TCP and Unix-socket client indefinitely. Headers, request bodies, and idle keep-alive connections now expire after 30 seconds and release their slots. Complete RPC calls retain their existing execution deadlines, and in-flight replies finish gracefully when the listener stops.

Fixes #1237.

Proved with failing-then-passing socket regressions for connection expiry and shutdown replies. On a live regtest node, all 256 idle sockets closed and both listeners recovered within 31 seconds; incomplete requests expired; a 35-second long-poll returned successfully. Eight shutdown runs returned all eight stop replies and 64 waiting RPC replies.

Validation: workspace tests, formatting, Clippy, cargo-deny, ast-grep, unwrap ratchet, and OS smoke passed. Local tests used four test threads and two Tokio workers with the runner's artificial address-space cap disabled after resource exhaustion in unrelated tests.

Reviewed with sol-loop: 2 rounds with gpt-6.1-sol on high. Sol verdict: approved.

Created with GPT-6 Astra in Codex (T3 Code).

Bound HTTP header and body reads to 30 seconds so idle TCP and Unix clients release the shared connection cap. Preserve complete long-poll requests and verify recovery through the socket journeys.
@niteshbalusu11
niteshbalusu11 force-pushed the rpc/expire-stalled-connections branch from 4bdfe0f to cc27292 Compare October 27, 2026 18:57

@rearden-grok rearden-grok Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST_CHANGES

1 bug, 0 suggestions, 0 nits.

Shutdown aborts in-flight RPC replies. Inline on serve_rpc.

Comment thread crates/rbitcoin-rpc/src/server.rs Outdated
Detach connection tasks and signal Hyper graceful shutdown when the accept task exits. Keep shutdown independent of long-poll deadlines, close idle sockets, and retain the body read deadline while requests drain.

@rearden-grok rearden-grok Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE

The shutdown finding is fixed. Connection tasks are detached from the accept task, and dropping the watch sender when that task exits calls graceful_shutdown on the hyper connection and still awaits it. RpcHandle::shutdown does not wait on those tasks. long_poll_does_not_hold_the_work_queue returns from listener shutdown within 500ms, sees the idle socket close, and then reads the in-flight waitfornewblock body.

No new bugs. The rbitcoin-rpc server binary finished in 1.77s on this head versus 3.13s on master 5cc40ff0. The added waits are paused-time advances and 500ms guards.

0 bugs, 0 suggestions, 0 nits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RPC listener: 256 idle unauthenticated connections lock out all RPC clients (TCP and unix socket), no idle or read timeout

1 participant