electrum: answer read-only batch items concurrently

COMPAT.md

@@ -48,6 +48,8 @@ electrs-shaped explorers can verify and sync.
 `--max-sh-creates N` (default **10000**; **0** = unlimited) refuses an unpaged Electrum/Esplora SH
 joins with more than N creates: Esplora HTTP **503**, Electrum JSON-RPC error
 `scripthash join exceeds --max-sh-creates`. Stats stay full when under the cap.
+`--heavy-sh-creates N` (default **10000**; **0** = off) runs unpaged joins with
+more than N creates one at a time across the node; lighter joins stay parallel.
 
 `GET …/txs/summary` is a **dialect** route (not Blockstream Esplora `API.md`).
 Shape and owner row: [Esplora REST surface](#esplora-rest-surface).
@@ -136,7 +138,7 @@ Per-method notes, auth, and the shindex matrix live in
 | Method | Status | Notes |
 |--------|--------|-------|
 | server.version / banner / features | done | Banner: libre-relay-class. `server.version[0]` is `rbitcoin-electrs <workspace.package.version> electrumx` — **not electrs**; see below. `server.version` negotiates: omitted → `1.6`; `"1.4"` → `1.4`; `["1.4","1.4.2"]` → `1.4.2`; `["1.4","1.6"]` → `1.6`; `"1.4.2-asof"` (or a range containing it) → as-of dialect. First call wins. `features.protocol_max` is `1.6`; `features.asof_protocol` is `1.4.2-asof`. `features.genesis_hash` is display-order hex (refuse a wrong-chain server before a tweaks scan). `features.tweaks` / `silent_payments` advertise the method; they are not a substitute for the stream. |
-| JSON-RPC batch | done | A line holding a JSON array is a JSON-RPC 2.0 batch: one array line back, same order, each slot `{id, result}` or `{id, error}`. Items run in order on the connection, and each slot is written as soon as it is answered. A notification (no `id` member) is skipped: it is not run and gets no slot, and a batch of only notifications writes nothing. A non-object item gets a `-32600` `Invalid Request` slot. `[]` is one `-32600` `empty batch`; more than 345 items is one `-32600` `batch too large`. `blockchain.tweaks.subscribe` and `blockchain.silentpayments.subscribe` stream several lines, so inside a batch they get a `-32600` slot; send them alone. |
+| JSON-RPC batch | done | A line holding a JSON array is a JSON-RPC 2.0 batch: one array line back, same order, each slot `{id, result}` or `{id, error}`. Read-only items (scripthash `get_history` / `get_balance` / `listunspent` / `get_mempool`, `transaction.get` / `get_merkle` / `id_from_pos`, `block.header(s)`, `estimatefee`, `outpoint.get_status`, `mempool.get_fee_histogram` / `get_info`) run up to half the node's blocking pool at a time (at least 1; 2 on a 4-thread pool), and each reports only its own dispatch time; any other item waits for every earlier slot, then runs alone, so a `subscribe` is seen by the items after it. Slots are written in request order as soon as each is answered, and slot `i` is written before read `i + window` starts. A notification (no `id` member) is skipped: it is not run and gets no slot, and a batch of only notifications writes nothing. A non-object item gets a `-32600` `Invalid Request` slot. `[]` is one `-32600` `empty batch`; more than 345 items is one `-32600` `batch too large`. `blockchain.tweaks.subscribe` and `blockchain.silentpayments.subscribe` stream several lines, so inside a batch they get a `-32600` slot; send them alone. |
 | blockchain.tweaks.subscribe | done | **Stream**, not a one-shot result: JSON-RPC `result` is the **first** height only; remaining heights are unsolicited notifications; `{"message":"done"}` ends the **chunk** (60s wall at a wave boundary, or the requested `count` if sooner) — Cake electrs clamps `count` to 1000 instead. Cake Wallet resubscribes from `syncHeight+1` ([cake_wallet#3574](https://github.com/cake-tech/cake_wallet/issues/3574) persist still lags one event). [kiss-bdk](https://github.com/kkdao/kiss-bdk) today treats `done` as the asked range finished ([kiss-bdk#10](https://github.com/kkdao/kiss-bdk/issues/10)). Each height carries tweak + txid + taproot `output_pubkeys` (client scans locally; no block fetch). Naive walk, or `--sp-tweaks` thin index (`len:tweak` only; one `txout` span per wave; the first height shares that span when indexed). Pre-taproot: **one** notify with ≤1024 empty height keys (Cake `fromJson` last key is progress); probe `[0,1,false]` stays `{"0": {}}`. Param `[2]` (Cake `historicalMode`): `false` **cut-through** (omit confirmed-spent P2TR outs / txs); `true` keeps spent outs. Serve-time `--sp-tweaks-dust` (default **1000**) omits P2TR outs with `value <=` the floor (`0` = all; **546** matches Cake electrs `sp_min_dust`). Sparrow Silent Payments uses Frigate `blockchain.silentpayments.subscribe` (server-side scan) — **not** this method. Cake isolate may still hardcode `electrs.cakewallet.com`. |
 | headers / block headers | done | Tip push on subscribe |
 | scripthash history / balance / listunspent | done | Unconf when mempool attached; `get_history` optional BCH-style `from_height` / exclusive `to_height` (`-1` = tip + mempool); 1-arg = full history; a `from_height` above 0 scans that block window and is served above `--max-sh-creates` (open history, and a window that starts at height 0, still refuse above the cap); unconfirmed `get_history` rows include `fee` (same as `get_mempool`); `listunspent` mempool height is `0` or `-1` (unconfirmed parent); **subscribe status always full**; `blockchain.scripthash.unsubscribe` returns whether the connection was watching (frees a slot under `--electrum-max-subs`, default 10000 per connection). `listunspent` loads `txid.body` only for unspent creates; one TCP connection reuses the last SH outs+spent join until SH-view **hash** changes. Confirmed methods stamp live tip: a RAM SH head (pending write-behind) joins with durable SH so mempool can drop confirmed txs without a hole. Durable Class B seed waits until after tip announce. `get_history` skips mempool rows already in the confirmed list. `server.features.chain_tip = true`. Trailing **`asof:<blockhash>`** after the official args (`server.features.asof` / `asof_protocol = 1.4.2-asof`): confirmed rows as of that still-live ancestor **at or behind visible SH** (durable + pending), **no** mempool; stamp is the asof block; unknown hash or ahead of visible SH → `asof not on chain`. Prefix keeps it off the future positional-string landmine. Requires negotiated `1.4.2-asof` (first `server.version` only). Electrum `protocol_max` is **1.6**. |

changelog.d/electrum-batch-concurrency.md

@@ -0,0 +1,20 @@
+Changed
+
+- **Electrum batches answer read-only items concurrently.** A batch
+  line ran its items one after another, so a 100-address `get_history`
+  page cost the sum of 100 lookups. Read-only items (scripthash history,
+  balance, unspent and mempool, transaction and header reads, fee
+  estimates) now run up to half the node's blocking pool at once per
+  connection (2 on a 4-thread pool), so block decode and mempool accept
+  keep the other half, and still land in request order. Each reports its
+  own dispatch time to the API log and metrics. A subscribe, unsubscribe
+  or other stateful item waits for the reads before it and runs alone.
+  Slot `i` is written before read `i + window` starts, so a client that
+  stops reading still stops the work (#1174).
+- **One heavy scripthash join at a time.** An unpaged Electrum or Esplora
+  join with more than `--heavy-sh-creates` creates (conf
+  `heavy_sh_creates=`, NixOS `services.rbitcoin.heavyShCreates`, default
+  10000, `0` off) now waits until no other such join runs anywhere on the
+  node. One join near a raised `--max-sh-creates` can peak above 2 GiB,
+  and a batch or a few connections could start several at once. Lighter
+  joins stay parallel.

crates/rbitcoin-electrum/src/lib.rs

@@ -22,9 +22,9 @@ pub mod tweaks;
 pub mod unspent;
 
 pub use server::{
-    electrum_scripthash_hex, parse_electrum_request_line, perf_totals, run_electrum,
+    batch_window, electrum_scripthash_hex, parse_electrum_request_line, perf_totals, run_electrum,
     sample_reset_perf, ElectrumConfig, ElectrumHandle, ServeLimits, TipNotify,
-    DEFAULT_MAX_SCRIPTHASH_SUBS,
+    DEFAULT_MAX_SCRIPTHASH_SUBS, MIN_BLOCKING_POOL,
 };
 pub use tweaks::DEFAULT_TWEAKS_MIN_DUST;
 pub use unspent::{