Skip to content

feat(h3): HTTP/3 transport over quiche, with ECH on both ends - #21

Merged
madeye merged 2 commits into
mainfrom
feature/http3-quiche
Aug 20, 2026
Merged

madeye merged 2 commits into
mainfrom
feature/http3-quiche

Conversation

@madeye

@madeye madeye commented Aug 20, 2026 •

Copy link
Copy Markdown
Contributor

Adds an HTTP/3 (QUIC) transport alongside the existing TLS+WebSocket one: http3=true on the server, transport=h3 on the client. Both default to off, so existing deployments are untouched.

The payload path is unchanged — a WebSocket Extended CONNECT (RFC 9220) on the same secret path, bridged with copy_bidirectional.

What's here

src/quic.rs — shared plumbing: quiche::Config built from the project's existing BoringSSL cert/key/ECH wiring, per-stream send/receive state, and a QuicStream AsyncRead + AsyncWrite adapter so the tunnels reuse copy_bidirectional exactly like the WS path.

src/h3_server.rs — one task owns the UDP socket and every quiche::Connection (quiche isn't Sync); tunnels run as separate tasks over channels. Requests that aren't CONNECT + :protocol: websocket + the secret path get the same nginx-shaped 404 the TCP listener serves, so the QUIC port looks equally uninteresting to a scanner. CertSource keeps PEM behind an ArcSwap with a generation counter, so ACME renewals reach the QUIC listener and the expensive quiche::Config rebuild only happens when the cert actually rotates.

src/h3_client.rs — lazy connect, then every accepted sslocal connection becomes a stream on a single QUIC connection: only the first pays for a handshake, and a stalled stream can't head-of-line-block the others. A dead session reconnects on next demand.

ECH works on QUIC after all

The initial assumption was that the H3 path would have to leak the real SNI, because quiche exposes no ECH API. It doesn't have to:

  • Under its default boringssl-boring-crate feature, quiche::Connection implements AsMut<boring::ssl::SslRef>.
  • quiche doesn't start the TLS handshake until the first send().

That leaves a window after quiche::connect to call SSL_set1_ech_config_list — the same call the TCP path makes. quiche and tokio-boring share one boring 4.21.2 in the lockfile, so the SslRef types are identical. The QUIC Initial now carries an ECHClientHello whose outer SNI is only the cover public_name.

Second commit: StreamBlocked is retryable, not fatal

Caught while deploying this to a live server — under concurrency the client dropped roughly a fifth of connections with h3 send_request: StreamBlocked.

quiche returns StreamBlocked when the underlying QUIC stream has no send capacity yet, and TransportError(StreamLimit) when we're at the peer's concurrent-stream cap (MAX_STREAMS_BIDI, 256). Both clear as the connection makes progress — quiche's own docs say to retry once the stream is writable again — but open_stream treated every error as fatal and dropped the request, which the accept task reads as "no stream available" and closes the sslocal connection.

open_stream now hands the request back for those two cases and the session re-parks it, reusing the slot that already waits out the peer's extended-CONNECT SETTINGS. No busy-loop: while a request is parked the accept branch is disabled and the loop still blocks on the socket/timer, so the retry rides the next packet.

One caveat worth reviewing

reject_non_ech is genuinely weaker on H3, and is documented rather than papered over. The TCP listener peeks at the cleartext ClientHello before handshaking, so a probe never sees the production cert. A QUIC Initial is encrypted under keys derived from its own DCID, so h3_server can only check SSL_ech_accepted post-handshake — that hides the tunnel, not the cert.

Use the TCP transport when cert concealment is the threat model. This is called out in the quic module docs and the config option docs.

Also

  • Dedupes the client ECHConfigList loader into ech::load_client_config_list, shared with tls_client.
  • Ignores .docker-cache/, left behind by cross musl builds.

Testing

Five e2e tests cover the H3 path: plain round-trip, round-trip with ECH, and 400 concurrent streams (past the 256-stream limit). Both the ECH test and the concurrency test were sabotage-checked — stubbing out install_client_ech fails the ECH test on the server's reject_non_ech drop, and reverting the backpressure fix fails 82/400 streams.

cargo fmt --check, cargo clippy --all-targets -D warnings, and all 88 tests pass.

Validated in production

Deployed to a live server (http3=true) and driven from a real client. Confirmed genuinely on QUIC, not falling back: an 8 MB transfer moved with zero TCP connections on :443, and the server logged HTTP/3 ECH keys installed, which only fires when a real QUIC connection is accepted. ECH verified working over QUIC end to end.

Performance note for anyone considering this as a default: on a lossy long-haul link, HTTP/3 measured slower than TCP (1 MB: 3.4–17s over H3 vs 1.9–8.3s over TCP). The feature is correct, but it is not automatically a throughput win — leave it off unless you have a reason.

madeye and others added 2 commits August 20, 2026 10:50
Adds an HTTP/3 (QUIC) transport alongside the existing TLS+WebSocket
one: `http3=true` on the server, `transport=h3` on the client. The
payload path is unchanged — a WebSocket Extended CONNECT (RFC 9220) on
the same secret path, bridged with copy_bidirectional.

- quic: shared plumbing — quiche::Config from the project's BoringSSL
  cert/key/ECH wiring, per-stream state, and a QuicStream AsyncRead +
  AsyncWrite adapter so the tunnels reuse copy_bidirectional exactly
  like the WS path.
- h3_server: one task owns the UDP socket and every quiche::Connection
  (quiche isn't Sync); tunnels run as separate tasks over channels.
  Non-matching requests get the same nginx-shaped 404 as the TCP
  listener, so the QUIC port looks equally uninteresting to a scanner.
  CertSource keeps PEM behind an ArcSwap with a generation counter, so
  ACME renewals reach the QUIC listener and quiche::Config is only
  rebuilt when the cert actually rotates.
- h3_client: lazy connect, then every accepted sslocal connection
  becomes a stream on a single QUIC connection — only the first pays
  for a handshake, and a stalled stream can't head-of-line-block the
  others. A dead session reconnects on next demand.

ECH works on QUIC after all. quiche exposes no ECH API, but under its
default boringssl-boring-crate feature quiche::Connection implements
AsMut<boring::ssl::SslRef>, and it doesn't start the TLS handshake
until the first send() — so there's a window after quiche::connect to
apply SSL_set1_ech_config_list, the same call the TCP path makes.
quiche and tokio-boring share one boring 4.21.2, so the SslRef types
match. The QUIC Initial now carries an ECHClientHello whose outer SNI
is only the cover public_name.

reject_non_ech is weaker here and is documented rather than papered
over: the TCP listener peeks at the cleartext ClientHello before
handshaking, so a probe never sees the production cert, but a QUIC
Initial is encrypted under keys derived from its own DCID. h3_server
can only check SSL_ech_accepted post-handshake — that hides the
tunnel, not the cert. Use the TCP transport when cert concealment is
the threat model.

Also dedupes the client ECHConfigList loader into
ech::load_client_config_list (shared with tls_client) and ignores the
.docker-cache/ dir left by cross musl builds.

Verified with two new e2e tests covering the H3 path with and without
ECH. The ECH one was sabotage-checked: stubbing out install_client_ech
makes it fail on the server's reject_non_ech drop, so it isn't passing
vacuously. fmt, clippy -D warnings, and all 87 tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ection

Found deploying to a live server: under concurrency the client dropped
roughly a fifth of connections with "h3 send_request: StreamBlocked".

quiche returns StreamBlocked when the underlying QUIC stream has no
send capacity yet, and TransportError(StreamLimit) when we're at the
peer's concurrent-stream cap (MAX_STREAMS_BIDI, 256 here). Both clear
as the connection makes progress — quiche's own docs say to retry once
the stream is writable again — but open_stream treated every error as
fatal and dropped the OpenReq, which the accept task reads as "no
stream available" and closes the sslocal connection.

open_stream now returns the request back for those two cases and the
session re-parks it in `pending`, reusing the slot that already waits
out the peer's extended-CONNECT SETTINGS. No busy-loop: while a request
is parked the accept branch is disabled and the loop still blocks on
the socket/timer, so the retry rides the next packet.

Regression test drives 400 concurrent connections through one QUIC
connection, well past the 256-stream limit. Sabotage-checked: reverting
the fix fails 82/400 streams, with the fix 0 fail.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@madeye
madeye merged commit 266b7ef into main Aug 20, 2026
9 checks passed
@madeye
madeye deleted the feature/http3-quiche branch August 20, 2026 06:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant