Health check restarts the entire daemon on `session_valid=false` — upstream librespot PRs might allow in-place recovery instead

[woorszt]

Problem Statement

Observed a full daemon restart triggered purely by a stale live session, credentials themselves were never the issue:

Health check restart for b8:27:eb:b3:15:27: session_valid=false
Shutting down Unified daemon for b8:27:eb:b3:15:27 (pid: 202032)
Quitting SpotOn Unified daemon for b8:27:eb:b3:15:27
daemon_stop: mac=b8:27:eb:b3:15:27 pid=202032 uptime=10150.7s
Need to create Unified daemon for b8:27:eb:b3:15:27
daemon_start: mac=b8:27:eb:b3:15:27 pid=210209 [...]

Confirmed via the Status page that this was purely a live-session issue, not a credentials problem — Auth Health showed PKCE/sp_dc both Valid for the account throughout, only the daemon process itself got recycled (old PID 202032 → new PID 210209).

Likely trigger: the player had been switched to internet radio for a while beforehand, so the Connect session sat idle — Spotify's own servers plausibly dropped the dealer/session server-side after enough inactivity. Spirc itself doesn't appear to have a built-in way to detect and recover from a server-initiated disconnect like this, so SpotOn's health check has to fall back to the bluntest available tool: restarting the whole process.

This restart was clean/fast in this instance (~130ms between stop and new port announce, no lingering errors), so not filing this as urgent — more flagging the pattern, since "switch away and back" seems like a fairly normal, recurring scenario rather than a rare edge case.

Proposed Solution

Not proposing a specific fix, since this seems to be a librespot-level limitation rather than something SpotOn's own code could easily solve alone. A few open librespot PRs target exactly this class of problem — recovering a stale/lost session without restarting the whole process:

• librespot-org/librespot#1692 — "fix: dealer reconnect and session loss recovery without playback interruption"
• librespot-org/librespot#1690 — "Add session recovery with automatic playback resume"
• librespot-org/librespot#1716 — "fix(connect): keep spirc running after transient connection-id update failure"

All three are open/unmerged upstream, so not something actionable right now — just noting them alongside this observed case in case they're useful context if this pattern shows up again or gets more disruptive. No strong opinion on whether it's worth pursuing before they land upstream.

Alternatives Considered

No response

[woorszt]

Found some independent, informational context for this while looking into something unrelated — not proposing any change, just adding it in case it's useful.

Ralph (the same contributor behind librespot-org/librespot#1676, add_to_queue) recently did a 49-commit deep-dive into Connect session recovery for his own iOS app, Spotifly: ralph/Spotifly#43

Two things from it seem relevant here:

1. Validates the "full restart" approach. After trying to keep sessions alive across reconnects ("soft reconnect"), he deliberately moved to full rebuild instead: "A brief gap during an outage is the better trade [...] The rebuild path is not new code — it was already the fallback whenever soft reconnect failed." That's basically what SpotOn's health check already does here (full daemon restart rather than trying to patch a stale session in place).

Worth being precise about what this does and doesn't confirm, though: he tested against upstream commit 9c7d756, and none of librespot-org/librespot#1690, #1692, #1716, or #1727 were merged yet at that point — so he almost certainly wasn't evaluating and rejecting those specific PRs. He built his own generation-safe reconnect/rebuild logic entirely at the Spotifly app layer (Swift + Rust FFI glue), independent of that upstream work. So the more accurate takeaway isn't "an experienced dev looked at the upstream fixes and still chose to build his own" — it's that he had to independently solve the same underlying gap (librespot itself not handling this robustly) that this daemon-restart workaround also exists for. Same practical conclusion either way: this isn't a workaround to defer in favor of relying on librespot getting more robust upstream — it's filling a real, still-open gap that another serious project had to fill independently too.

2. A possible partial explanation for some of the DEVICES_DISAPPEARED/cluster-flapping behavior I ran into separately while testing AirPlay/Connect device switching (not part of this issue, just related territory): "librespot emits SessionDisconnected when the local Connect device becomes INACTIVE, not when the network session fails [...] This is upstream behavior, not part of our patch — official dev has the identical emission." So a normal device handoff (e.g. playback moving to a phone) can apparently look like a disconnect at the librespot event level, even though the session itself is fine. Doesn't affect the session_valid check itself, which he confirms stays a genuine signal — just a possible source of some of the noisier cluster-update logs.

He also retired his own librespot fork entirely in that same PR, building against unpatched upstream dev instead — for what it's worth, another data point in the same direction as the earlier discussion about not investing in upstream/fork maintenance for #144.

[woorszt]

Closing the loop here. The quick fix (idle guard) landed via #149 rather than this issue, and covers the immediate pain — no more mid-playback restarts on transient AP blips.

This issue's actual ask (in-place session recovery instead of a full restart, via librespot-org/librespot#1692 / #1690 / #1716) was deliberately left untouched once #149's fix shipped — not urgent anymore, and per your comment there it's on the roadmap via the deeper approach flagged in the Ralph/Spotifly reference above.

Worth noting: those three librespot PRs might end up moot if the Spotifly-style approach is what gets adopted. They're about improving librespot's own internal soft-reconnect behavior — but Spotifly's fix deliberately went the other way, removing soft reconnect entirely in favor of a full rebuild + rehydration handled outside librespot, in their own app layer (their PR: "Soft reconnect is gone... keeping the Player alive across sessions is what forced the fork patches"). If SpotOn ends up building resilience the same way — in its own Rust/Perl orchestration rather than depending on librespot's internal reconnect improving — there'd be no need to wait on those upstream PRs landing at all.

No action needed from me here, just leaving the connection in case it's useful whenever this gets picked up.

[woorszt]

Another example of this pattern, for reference — clean heartbeat activity right up until an abrupt, unwarned server-side close:

06:50:39Z  INFO  librespot_core::session] Authenticated as '<redacted>' !
06:50:39Z  INFO  librespot_connect::spirc] active device is <> with session <6Ukz7DYEmabkOGzDWlNfXD>
...
06:58:39Z  DEBUG Session strong=6 weak=7
07:00:39Z  DEBUG Session strong=6 weak=7
07:00:39Z  ERROR librespot_core::session] Connection to server closed.
07:00:39Z  DEBUG Shutdown: Invalidating session

No prior early eof / Connection reset warning — goes straight from a normal heartbeat to ERROR ... Connection to server closed. Health check correctly identified session_valid=false and restarted cleanly (nothing was playing at the time, so no interruption).

Not reopening this — the way ralph does it in Spotifly remains the long-term preferred approach here.