fix: dealer reconnect and session loss recovery without playback interruption

[antoinecellerier]

When the dealer websocket connection drops or the session TCP connection dies,
librespot either hangs indefinitely or restarts spirc — stopping playback in
both cases. This PR fixes all observed failure modes so the process recovers
automatically without interrupting audio.

Symptoms before fix

Hang on dealer websocket drop (fixed in 4b7662f):
Spirc's tokio::select! loop blocked on subscription streams that would never
receive new messages. The mpsc senders in SubscriberMap weren't cleaned up on
reconnect, so spirc hung indefinitely — requiring a manual process restart.

Feb 17 01:12 — "Websocket peer does not respond."
[63.5 hour gap — process completely unresponsive]
Feb 19 16:44 — Manual restart

Unnecessary spirc restart on dealer reconnect (fixed in 29e814a):
After 4b7662f added reconnect notification, spirc broke out of its event loop
on every dealer websocket reconnect to "refresh subscriptions" — even though
the subscription streams survive reconnects on the shared DealerShared.

WARN  Dealer reconnected; restarting spirc to refresh subscriptions.
WARN  unexpected shutdown
WARN  Spirc shut down unexpectedly

Playback killed on session TCP loss (fixed in c87079b + c77d8cb):
When the session TCP connection dies, handle_disconnect() explicitly set
SpircPlayStatus::Stopped and tried to notify Spotify (which failed anyway).
The Player was still playing from its buffer, but the new SpircTask started
with a blank ConnectState and SpircPlayStatus::Stopped.

ERROR Connection to server closed.
WARN  unexpected shutdown
ERROR Broken pipe (os error 32)
ERROR Audio key response timeout
ERROR Unable to read audio file: end of stream
WARN  Spirc shut down unexpectedly

Commits

4b7662f — fix: dealer websocket reconnect leaving spirc hung on stale channels

Add a watch::Sender<u64> generation counter shared between the dealer and its
consumers. The dealer increments it on successful reconnect, get_url() timeout
(30s), or get_url() error. Spirc subscribes before dealer.start() and breaks
out of its event loop on change, triggering the existing auto-reconnect path in
main.rs.

Also fixes get_url() failures propagating via ? and terminating the dealer
background task entirely, rather than retrying.

Changes:

• core/src/dealer/mod.rs: watch channel plumbing, 30s timeout on get_url(),
  retry+signal on errors, signal consumers on reconnect
• core/src/dealer/manager.rs: Store watch::Sender, expose reconnect_receiver()
• connect/src/spirc.rs: Subscribe to reconnect watch, add select! branch

29e814a — fix: handle dealer reconnect in-place without restarting spirc

The subscription streams survive reconnects because they're registered on the
shared DealerShared — the new websocket dispatches through the same
message_handlers map. After reconnect, the server pushes a new connection_id
which handle_connection_id_update() already handles correctly.

Changes:

• reconnect_rx.changed(): log and continue instead of break
• handle_connection_id_update error: log instead of break

c87079b — fix: skip server cleanup on session loss to keep playback alive

When session.is_invalid(), skip handle_disconnect() (which sets
SpircPlayStatus::Stopped), delete_connect_state_request(), and
dealer().close() — all of which fail on a dead TCP connection anyway.
The Player continues playing from its buffer independently.

c77d8cb — fix: save and restore playback state across session reconnects

SpircTask saves its ConnectState, SpircPlayStatus, and play_request_id
into a SavedPlaybackState before exiting on session loss. main.rs captures
this and passes it to Spirc::with_saved_state() when creating the replacement.
The restored SpircTask updates the playback position on the first
connection_id_update and re-registers with Spotify showing the correct track.

Changes:

• connect/src/model.rs: Add SavedPlaybackState struct
• connect/src/spirc.rs: Spirc::with_saved_state(), save state on session
  loss, restore on creation, update position in handle_connection_id_update
• src/main.rs: Capture saved state from spirc_task, pass to new Spirc

Evidence after fix

4b7662f — 9 days of logs (Feb 28 - Mar 8):

• 0 manual restarts needed (vs 2 in 7 days before fix)
• 9 dealer reconnect events, all recovered in 2-91 seconds
• Process running continuously for 9+ days

29e814a + c87079b + c77d8cb — 6 days of logs (Mar 14-20):

Dealer reconnects handled in-place (~20 events, zero spirc restarts):

Mar 18 12:14 — "Dealer reconnected; awaiting new connection_id."
Mar 18 12:14 — "re-registering with active playback state: Playing { ... }"
[playback continued uninterrupted]

Session TCP losses recovered with state preserved (5 events):

Mar 18 11:49 — "Connection to server closed."
Mar 18 11:52 — "session lost, saving playback state for recovery:
                 Playing { nominal_start_time: 1773834169899, ... }"
Mar 18 11:52 — "Spirc shut down with saved playback state, reconnecting"
Mar 18 11:52 — "Spirc[1] restoring saved playback state"
Mar 18 11:52 — "re-registering with active playback state:
                 Playing { nominal_start_time: 1773834169899, ... }"
[3 second recovery, playback never stopped]

Summary (Mar 14-20):

• 0 "Spirc shut down unexpectedly" (vs ~2-3/day before)
• 0 process restarts needed
• 5 session TCP losses, all recovered with state preserved
• ~20 dealer reconnects, all handled in-place
• Process running continuously (Spirc counter reached Spirc[4])

Use of AI

This PR was created with GitHub Copilot CLI. Copilot assisted with root cause
analysis, implementation, code review, log analysis, and PR description. I'll
admit to not having any knowledge of Rust which means I'm not able to review
Rust specifics. At a high level the changes seem to make conceptual sense to me
and have proven to have positive effects. Do let me know if this is garbage.

[Copilot]

Pull request overview

This PR addresses a hang in the Connect “spirc” task after dealer websocket reconnects by adding an explicit reconnect notification mechanism so consumers can tear down and re-subscribe, and by bounding dealer URL resolution time during reconnect attempts.

Changes:

• Add a watch-based reconnect “generation” signal plumbed through dealer builder/manager and emitted on reconnect and get_url() failures/timeouts.
• Add a 30s timeout around get_url() during the dealer reconnect loop and retry instead of terminating the dealer task.
• Update SpircTask to subscribe to the reconnect signal before dealer.start() and break out of its select! loop when a reconnect is observed.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
core/src/dealer/mod.rs	Adds reconnect watch plumbing and get_url() timeout/handling in the reconnect loop; exposes a reconnect receiver.
core/src/dealer/manager.rs	Stores and exposes the reconnect watch sender/receiver; passes sender into dealer launch.
connect/src/spirc.rs	Subscribes to reconnect notifications and restarts spirc when dealer reconnects to avoid stale subscription hangs.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[antoinecellerier]

• Fixes Playlist playback stopped after a few hours with a broken pipe (os error 32) error / non recoverable websocket/TLS error (using Dealer) #1419 — Playlist playback stopped after broken pipe / non-recoverable websocket/TLS error
• Relates to Librespot stops working when switching networks #1627 — Librespot stops working when switching networks. Network switch kills the websocket. Our fix ensures spirc doesn't hang on stale channels after the dealer reconnects. The session TCP side (slow recovery) is still a separate issue.
• Relates to Connection to server closed #1492 — Connection to server closed. Session TCP death followed by spirc shutdown. Our get_url() timeout helps here but doesn't fully solve the session-layer problem.

[artenverho]

Works great for me thanks!. I switched ISP before the weekend and since then librespot would loose connection at least once every 15 minute. Very odd behavior that so far I have not been able to pinpoint to a specific cause (perhaps something to do with poor multicast support of the supplied router?). This PR fixed it!

[artenverho]

I think I’ve encountered an asymmetrical issue when switching between accounts that seems specific to this PR. If the device has credentials (username) defined in the config, it becomes impossible to switch back to that "Owner" account after a "Guest" (Discovery/Zeroconf) session has been active or is active (the "Owner" can join the jam but not take over the session). The error it shows is:

[ERROR librespot_core::dealer::manager] failed sending dealer request channel closed

to reproduce the error:

• Configure librespot with USERNAME and credentials saved (Account A).
• Start the service. It successfully logs in as Account A.
• Connect to the device using a different Spotify account (Account B) via Discovery/Zeroconf. (This works fine).
• Optional: Stop playback or Disconnect Account B.
• Attempt to trigger playback via Account A via the Spotify App.
• Result: Account A fails to connect.

The only way to recover is to manually restart the service. Curiously the "Guest" does not experience the same issues when the "Owner" is playing, it can happily take over.

It seems the new recovery logic might be preventing a clean shutdown of the Dealer task during an account handover? It is curious it only happens in this specific situation though. Other than this, I haven't experienced any problems (thanks again!)

[antoinecellerier]

@artenverho Thanks for the detailed report! Copilot has likely identified the root cause and I've pushed 3 fixup commits.

The problem: When Account B takes over via Discovery, spirc.shutdown() and session.shutdown() race. The session gets marked invalid before the SpircTask processes the Shutdown command, so our recovery code kicks in — it skips dealer().close() (thinking the session died unexpectedly) and saves playback state. This leaves a stale dealer with closed command channels, causing failed sending dealer request channel closed when Account A tries to reconnect.

Fixes pushed (as fixup commits, not yet squashed):

• 2ac494b — Only save state and skip dealer cleanup on unintentional session loss (&& !self.shutdown). When shutdown was explicitly requested (account handover), fall through to normal cleanup including dealer().close().

• 48adba0 — Clear saved playback state when Discovery provides new credentials, so stale state from a different account is never restored into the new Spirc.

• 303026b — Restore break on initial handle_connection_id_update failure (unrelated to your report, but another edge case found during review). After dealer reconnect, errors are tolerated since the next connection_id push will retry.

Could you test with these changes and let me know if the account switching issue is resolved?

[artenverho]

Sorry took a few days to find the time for testing. At first glance it seems to solve the issue! I will need to do some more long term testing but switching between users is now seamless again. Thanks!

[deGueux]

I wanted to chime in as a long-time librespot user who ran into a very weird, persistent issue where the connection would randomly drop, the music would stop and a reconnect would be needed. This persisted across various installations, standalone librespot or included in other pachages. It arrived a few months ago, but couldnt be pinpointed to any specific changes. Anyhow, this PR seems to have solved all of my issues, so thank you 👍

[antoinecellerier]

What's new since the previous revision

The branch has been soaking continuously on real devices since the last push. The soak surfaced four distinct field failures the previous revision couldn't recover from — each diagnosed live on the stuck device and fixed in its own commit, with the log evidence and diagnosis written into the commit message — plus one hang found by code review along the way. The earlier review fixups were squashed and the branch rebased onto current dev.

#	Failure mode	Fix	Seen in the field
1	spirc event loop frozen by an unbounded HTTP request	5adf966	×2 Pi Zero, ×4 Pi 3B
2	websocket handshake hang during reconnect	bb76d15	×3 Pi 3B
3	IPv6 blackhole defeating the reconnect timeout	5fb097a	×1 Pi Zero, ×6 Pi 3B
4	player events lost during the spirc restart window	a411289, bd93e18	×1 Pi Zero, ×3 Pi 3B
5	context-resolve failure wedging a transfer	6b84d8a, 999ffab, c3b1843, b08a5f7	unit tests only

The write-ups below state each problem and fix; the full field evidence is collapsed.

New failure modes

1. Spirc event loop frozen by an unbounded HTTP request (5adf966)
The device vanished from Spotify Connect with no crash: the dealer kept reconnecting, but the spirc select! loop had silently stopped — stuck on an await, most plausibly an spclient request over a half-open connection (hyper's legacy client ships without request timeouts, and the spclient retry loop only reacts to requests that return). Two defensive fixes that hold regardless of the exact trigger: a 30s timeout around the response-headers and body awaits in http_client (control-plane requests only; audio streaming untouched), and a spirc-side watchdog that forces the existing save/restore restart path if a dealer reconnect isn't followed by re-registration within the grace period.

Field evidence: Pi Zero ×2 (2026-07-28), Pi 3B (2026-08-18) — the hypothesized mechanism caught live

Pi Zero 2W, twice in one evening (22:04 and 23:45 UTC): a TCP reset tore down the dealer websocket and the spclient connection together, so the in-place reconnect succeeded but the re-registration PUT failed with ECONNRESET (failed handling connection id update: failed to put connect state for new device) — dealer alive, device not in the cluster, the exact silent-vanish state. The watchdog fired at +30s and the forced restart re-registered at +36s and +41s. Playback was never interrupted: the track playing across the first incident ran its full 260s.

The request-timeout half was validated 2026-08-18 on the Pi 3B: during an evening of ISP flaps, the re-registration PUT hung on a half-open connection — the 30s timeout cut it loose (Request to https://gew1-spclient.spotify.com:443/connect-state/v1/devices/… timed out after 30s, surfacing as hyper IncompleteMessage), the watchdog forced the restart 13 seconds later, and the forced restore had the device re-registered 67s after that (17:37:53 timeout → 17:38:06 watchdog → 17:39:13 registered, mid-outage). That is the exact mechanism hypothesized above — an spclient request stuck on an await over a half-open connection — finally observed live; without the timeout, that await had nothing to wake it. The watchdog fired four times on the Pi 3B that week (08-15, 08-18 ×2, 08-19), every one recovering unattended.

2. Websocket handshake hang during reconnect (bb76d15)
Found by review, later field-validated: connect() (TCP/TLS/websocket handshake) had no bound, so a blackholed host could stall reconnection forever with no signal to consumers. Each reconnect step is now bounded to 30s. (The IPv6 incident below validated the concern from an unexpected direction: at the TCP layer, the step bound alone turned out to be insufficient.)

Field evidence: Pi 3B ×3 (2026-08-18/19)

Three connect attempts during ISP outages hit the step bound (Timed out connecting to dealer.). TCP connects are 3s-bounded per address by the fix below, so these stalled past TCP — in the TLS or websocket handshake, exactly the layer this bound was written for. Each time the retry loop moved on and later reconnected.

3. IPv6 blackhole defeating the reconnect timeout (5fb097a)
A router fault left the device with an advertised-but-dead IPv6 route (SYNs silently dropped). The dealer's AAAA-first connect attempt sat in SYN-SENT, and the OS's sequential address fallback (~130s per address on Linux) never reached the working IPv4 address within the 30s step timeout — so reconnection looped forever: 42 failed attempts over 28 minutes until manual intervention, the device absent from Spotify Connect the whole time. Fixed by resolving addresses explicitly and bounding each address attempt to 3s in socket::connect (used by both the dealer and AP connections); verified by recreating the blackhole with an nftables drop rule against the fixed binary — recovery in 4 seconds, new websocket confirmed on IPv4.

Field evidence: Pi Zero (2026-08-03), Pi 3B ×6 — including a DNS-hijacking router incident

Pi Zero 2W, 2026-08-03: a ~90s WAN outage sent one dealer reconnect attempt at an address that blackholed it, and the per-address bound cut it off after exactly 3s (Error while connecting: Deadline expired before operation could complete { connection to 212.27.38.252:443 timed out }) instead of consuming the whole 30s step timeout. The retry loop kept cycling every 10s and reconnected 87s later, unattended.

It fired six more times on the Pi 3B (2026-08-16/18/19), most tellingly during a 2026-08-16 router incident that hijacked DNS: the dealer connect was first refused on a self-signed certificate (native-tls correctly declining the interception page), then a follow-up attempt was routed to a blackholed private address and bounded at 3s (connection to 172.31.255.254:443 timed out), then Network is unreachable — and the loop reconnected cleanly 73s into the incident. The remaining hits bounded blackholed Google-front addresses (35.186.224.x) during the 2026-08-18/19 ISP outages.

The original incident also validated the recovery machinery end-to-end: restoring connectivity made the stuck (unfixed) process recover by itself on its next retry cycle.

4. Player events lost during the spirc restart window → registered-but-silent (a411289, bd93e18)
A session died while a track load was in flight, and the doomed load failed a few seconds into the restored spirc's session setup — before with_saved_state had subscribed to player events, which it only did after the client_token/connect/login5 awaits. The Unavailable event went to no subscriber, and the restored spirc sat in LoadingPlay forever: registered, healthy, silent — clients showed the track "playing at 0:00" until they gave up. Two fixes: subscribe to player events before the session-setup awaits, so events emitted during the multi-second window are buffered (this also covers EndOfTrack firing in that window); and when restoring a LoadingPlay/LoadingPause state, don't trust the in-flight load at all — clear the stale play_request_id so the dead load's late events are ignored, and re-issue the load on the new authenticated session. Playing/Paused restores are untouched: running audio is never interrupted. (The late event subscription predates this PR; it only becomes observable with a multi-second restart window to lose events in.)

Field evidence: Pi Zero (2026-07-23), Pi 3B ×3 (2026-08-15/16)

The trigger pattern: the load's audio-key request rides the dying AP connection and times out (the player then "continues without decryption", upstream behavior), so the load is doomed before the restore begins.

Pi Zero 2W (different network), 2026-07-23: an AP connection died while the current track had ~20s left, and both decisive events — the doomed preload's failure and the current track's EndOfTrack — fired inside the ~13s restore window, before session setup finished. The early subscription buffered them; on loop start the restored spirc replayed them, skipped the loads that had died with the old session, and resumed playback on the new session ~10s after the old track ran out. Without the fix, the device would have re-registered as Playing a track that had already ended — registered, silent, stuck. Full log excerpt in a411289's commit message.

The second fix (bd93e18) validated three times on the Pi 3B (2026-08-15 ×2, 2026-08-16): each time a session died with a takeover load in flight (twice with the audio-key-timeout signature), the restored spirc logged restored mid-load state, re-issuing load, re-issued on the fresh session, and re-registered in the correct Paused state within 6–8s — each of these was previously the LoadingPause wedge. The 2026-08-16 instance also survived garbage from the cluster itself: the takeover's transfer state carried position_ms=10647017 for a 232s track (hours-stale paused cluster state, a Spotify-side artifact), which the re-issued load clamped (Invalid start position … starting track from the beginning) before registering cleanly.

Related: HTTP body reads now time out on stall rather than total transfer time (ceff4a8), so slow-but-progressing transfers aren't killed while genuine stalls still are.

5. Context-resolve failure during a transfer → wedged, unrecoverable for an hour (6b84d8a, 999ffab, c3b1843, b08a5f7)
During the July 14 Spotify platform outage, a takeover transfer hit two backend failures back-to-back and exposed three latent defects at once: the transferred track's Unavailable event arrived before the context had resolved and was dropped (could not dispatch player event: Invalid state), so the auto-skip never ran; the context resolve then got a 502, which — unlike 503 — failed after a single attempt; and the failed context was blacklisted for RETRY_UNAVAILABLE = 3600s with subsequent re-adds silently dropped. Net effect: device active in the cluster with healthy dealer pings, playing nothing, transfer_state pending forever, and retaking the same playlist couldn't recover it for an hour. All three defects are inherited upstream code (the touched files have zero changes on this branch); this PR increases exposure to them — devices now survive session loss and stay registered, so they receive transfers during exactly the flaky windows that used to just kill them. The fixes complete the reliability story:

• 6b84d8a — map 502 → Unavailable in http_client, giving it 503's existing bounded per-call retry (spclient TryTimes(10)). Severable, and deliberately global: flagging for review that this affects all spclient endpoints.
• 999ffab — user-initiated resolves (transfer ×2, Load) clear the unavailable-context blacklist entry before resolving; automatic re-adds (dealer updates, playlist modification, autoplay) still honor the full 1h blacklist, which is what prevents retry storms. The silent drop is now logged at info with the remaining blacklist time. Unit tests added to context_resolver.rs.
• c3b1843 — when a resolve fails and won't be retried, clean up loudly: take the pending transfer_state, stop, and push Stopped to the cluster instead of showing "loading" forever. Failures of automatic context updates while music is playing remain warn-and-continue — healthy playback is never interrupted. The terminal state is deliberately Stopped-but-active (registration and volume kept, a phone tap recovers) rather than became_inactive.
• b08a5f7 — an Unavailable that races the in-flight resolve is remembered and replayed once the context arrives; bounded by that single resolve's lifetime, no timers.

A design note for reviewers: the absence of automatic retry/backoff here is a product decision, not an oversight. Failing while Spotify is down is acceptable; what these commits guarantee is that the device never ends up in a locally unrecoverable state and that every fresh user action gets a full, honest attempt. (Known, accepted: a session-lost restart still drops queued resolves and transfer_state — upstream-shaped gap, recoverable by user action. Synthesizing degraded playback from transfer-embedded track data was considered and dropped; possible follow-up.)

Field soak

Two devices on different networks: the Pi Zero 2W provided the long steady-state baseline, while the Pi 3B caught a week of rolling ISP/router failure — DNS outages, TLS connections cut mid-handshake, certificate interception, blackholed routes — and became the stress test the Zero never provided.

	Pi Zero 2W	Pi 3B
Window	2026-07-23 → 08-11 (19 days; one process up 14 of them)	2026-08-13 → 08-20 (7 days)
Track loads	~800	~370
Dealer reconnects	21, all absorbed in place	113, ~70 absorbed in place
Session-lost save/restore restarts	27, plus 2 watchdog-forced	41, with 4 watchdog firings
Re-registered after every recovery	yes, within 1–15s	yes, within 3–114s
Stalls	none	none

("Stall" = a track not followed by its end-of-track, the next load, or an explicit stop. Zero across both soaks.)

• The 3B's longest absence from Connect in any single episode was ~9 minutes — bounded by the outage itself rather than by any wedge; reconnection succeeded on the first retry cycle after connectivity returned, every time. One episode (2026-08-15 22:00) chained three of the fixes inside 80 seconds: in-place dealer reconnect → session death mid-load → bd93e18 re-issue → the new dealer's registration PUT reset → watchdog restart → registered.
• On the Zero, 14 audio-key timeouts produced the doomed-load pattern from 4; every one resolved by skipping to the next track or through a restore, none left the device silent. Never hit there: the restart rate limit (e8a6f62), the dealer connect/handshake timeouts, and any context-resolve failure.
• The 3B ran d92c96d — this branch's tip before the rebase onto current dev; the rebase only added three unrelated upstream commits.
• After both soaks, the only code paths still resting on unit tests alone are the four context-resolve commits under 5 and the restart-rate backoff (e8a6f62).

Known gaps (documented, not fixed here)

Both are upstream-inherited: nothing on this branch touches the code involved.

1. Idle session death is detected late. Session's keepalive notices a dead AP session within ~80s and calls shutdown(), but spirc's while !self.session.is_invalid() is a loop condition, not a select! arm — an idle loop stays parked until some unrelated event arrives. Sessions lost during playback are caught in 5–150s because the player touches the session; only idle devices are affected. The condition is unchanged from dev (from daf7ecd): upstream exits the loop for good once it notices, this branch restores and re-registers in about a second. Closing the detection latency needs a shutdown signal on Session (currently a plain bool) — a separable follow-up rather than more surface area on this PR.

Occurrence data

Seen 23 times in 19 days on the Zero; 15 episodes over 5 minutes, the longest 10h19m, ~64h cumulative — dealer connected, session dead underneath. The 3B shows the same pattern, e.g. 2026-08-15: session dead from 03:03 until a dealer ping failure surfaced it at 05:00, the device idling registered throughout.

2. Silent server-side eviction with every local signal green (found 2026-08-22; needs one more captured occurrence to design the fix). The Pi Zero dropped out of the Connect picker while everything looked healthy locally: AP session and dealer websocket ESTABLISHED, pings answered every 30s, three successful re-registrations since — each returning a server-parsed Cluster — yet inbound cluster updates had stopped: real pushes were being dropped upstream of us. A process restart (same device_id) brought the device back instantly; the stuck state had survived a full Session rebuild, so it is not merely a stale dealer connection. None of this PR's detectors can see it — the watchdog disarms on each successful re-registration, the session is genuinely valid, and no HTTP request is in flight to time out. The next step is instrumentation, not a fix: log whether our own device_id is present in the registration response's device map (a server fact already in hand) plus the resolved dealer host, so any future recovery trigger acts on server state rather than a silence timer — consistent with the no-automatic-retry stance above. dev would sit in the same state, with fewer re-registration attempts.

Forensics from the live process

Cluster updates ran 6–275/day through 2026-08-20, then none for 42 hours — spanning a 27-hour websocket during which the cluster's player session visibly changed, so pushes were provably being dropped before they reached the device. Byte counts on the live socket showed nothing but 24-byte pongs, including while a phone opened the app. The three ineffective re-registrations: two dealer reconnects and one full Session rebuild via the save/restore path. A SIGKILL restart re-registered in ~2s and the device appeared in the picker immediately; device_id is SHA1(device name), identical across the restart, so the eviction was process-scoped, not device-scoped.

One path stays fatal by design. could not initialize spirc: Service unavailable { client error (Connect) } still exits the process — at startup (twice on 2026-07-27, when the wrapper launched librespot before wifi was up) and when a recovery restore cannot build a fresh session at all (five times during the 2026-08-19 outage evening). Every exit was absorbed by the supervisor and the device was back registered 29–88s later. The boundary: everything recoverable in-process is recovered; "no session can be built" is handed to the supervisor, which drops the saved playback state (all five instances were Stopped, so nothing was lost — a restore interrupted mid-music would come back registered but stopped).

Hardening and review follow-ups

• Close the abandoned session's dealer when spirc restarts with saved state, so its reconnect task can't keep the old session alive forever (891c730)
• Arm the watchdog when re-registration fails after a reconnect, instead of waiting for another connection-id push that may never come (d70aba0)
• Make watchdog suppression race-free with a reconnect generation counter (ace64f7)
• Back off instead of exiting when recovery restarts hit the rate limit (e8a6f62)
• Cleanups: remove now-unused Dealer::reconnect_receiver (7460f73), mark the Spirc::new return-type change as breaking in the changelog (9433753), rustfmt (fa07cc5)

Temporary

da004f1 adds DIAG-prefixed logging (controller identity, transfer contents, cluster updates) used to diagnose the incidents above. It will be reverted before merge, per its commit message.

CI

Both failures on the current run pre-date this branch:

• clippy — GitHub's runner image moved stable to Rust 1.98 (2026-08-18), and clippy::needless_late_init now fires on dev code this branch doesn't touch (src/main.rs); other open PRs fail the same job.
• cargo +1.85 test (ubuntu) — shuffle_vec::test_shuffle_with_first, a ~1/200 seed-dependent flake (it fails whenever the randomly chosen "first" track happens to shuffle to index 0); it also failed the repo's own update-protos run on dev on 08-21, and passes 50/50 local runs on this branch. The other three test jobs are fail-fast cancellations of that one failure, not independent failures.

---

Rebase bookkeeping (2026-08-22)

Rebased onto dev @ d4494f5; pre-rebase tip d92c96d (tag pre-rebase-2026-08-22). Hash map applied: d92c96d→b08a5f7, 03a85ce→c3b1843, 664ffda→999ffab, 3dbfba5→6b84d8a, 4104fce→bd93e18, 42a3abf→a411289, 2014614→5fb097a, 9c6ac39→fa07cc5, 9af3ac8→9433753, da031d1→7460f73, adfc9a1→ceff4a8, d52b57f→e8a6f62, cbbba30→ace64f7, d311cca→d70aba0, cd2c4e9→891c730, 3368264→bb76d15, 144be7d→5adf966, a724582→da004f1.

[michaelherger]

@antoinecellerier I have to add a personal comment, as this is not really my PR to review nor anything. But as a maintainer of another project I feel the urge to speak up.

Posting these pages of AI generated documents is a major problem for open source projects. It's not helpful, because you overwhelm the poor guys who are already struggling to find spare time to work on their side projects. A lot of these reports are just noise. Reading what you just posted will take anyone several minutes - I bet you didn't read it yourself. Now imagine they have other, similar reports to review. Sometimes there are several of them in a day. Then the devs have easily spent an hour just reading and digesting all that stuff, before they even try to get to the code or the suggested changes. Hey, and many of them actually have other things in their life to deal with on the side. Like a paid job, a family, pets, or (imagine!) other hobbies!

So please keep things short. Read everything before you post. If you don't understand what you're about to post, don't post. If it takes you several minutes just to read it, shorten it. Do yourself and all of the community a favour and stop posting books nobody wants to or can read.

[antoinecellerier]

@antoinecellerier I have to add a personal comment, as this is not really my PR to review nor anything. But as a maintainer of another project I feel the urge to speak up.

Fair enough. Thanks for the feedback. This ended up being more of a mouthful than I initially anticipated it would be when starting a few months back. If there is interest in merging any of it I will do work to chunk it up in smaller pieces. Else it'll just sit here as a research log in case it's useful to anyone else.

[michaelherger]

Fair enough. Thanks for the feedback. This ended up being more of a mouthful than I initially anticipated it would be when starting a few months back. If there is interest in merging any of it I will do work to chunk it up in smaller pieces. Else it'll just sit here as a research log in case it's useful to anyone else.

The problem is that just the discussion alone is so verbose (20 pages A4 and counting!), you risk that nobody will actually read it. Some will merge the PR and figure it helped their issue. But maintainers have to be more careful.