fix(voice): signal the interruption before un-gating the audio sink

[toubatbrian]

Stack 5/5 — split out of #2126. Base: #2133.

Review only the top commit. The diff against main includes #2130, #2131, #2132 and #2133.

The defect

cancelSpeechPause() reopens the sink's pause gate to admit the next speech. But the frames parked at that gate belong to the speech it has just interrupted, and they are released first — so audio the user has already barged in over goes back on the wire.

The fix clears the sink before reopening the gate, and only when that same call is what interrupted the paused speech; an interrupt: false cancel still resumes without clearing.

Python parity

This is parity, not a divergence. _cancel_speech_pause (agent_activity.py:4230) awaits _wait_for_generation() before resume(), so by the time Python reopens the gate the interrupted reply task has already run its own clear_buffer(). JS wraps the same await in waitIfNotInterrupted, which races it against the interrupt future handle.interrupt() just resolved — so it returns immediately, deliberately, per #1124. JS therefore has to signal the sink explicitly to land in the same state.

What this PR does not do

An earlier version of this change also had onInterruption call cancelSpeechPause(), making the model's verdict commit the turn outright. @chenghao-mou pointed out that leaving the speech parked is by design, and Python agrees — on_interruption (agent_activity.py:2066-2076) does restore → interrupt-by-audio-activity → _on_end_of_agent_speech and stops. That half has been dropped, so JS matches Python line for line.

The cost of dropping it, measured. confirmed_interruption_pause_and_commit.test.ts drives a real AgentSession against a real ParticipantAudioOutput and counts frames at the AudioSource boundary; only the model verdict is synthesized. The reply is 40 frames and the barge-in lands after 3, so 38 is "the entire rest of the reply":

After a confirmed barge-in verdict…	origin/main	this stack
…an STT final transcript arrives	interrupted=true deliveredAfterVerdict=1	interrupted=true deliveredAfterVerdict=0
…nothing arrives before the timeout	interrupted=false deliveredAfterVerdict=38 falseInterruption=[true]	interrupted=false deliveredAfterVerdict=38 falseInterruption=[true]

Read the second row loudly: on the no-transcript path this stack is identical to main. The whole tail of the reply goes back on the wire and the session reports agent_false_interruption(resumed: true). So the "agent pauses, then resumes mid-sentence" symptom is only prevented by an STT final transcript landing inside falseInterruptionTimeout (default 2s) — which is precisely Python's contract, and precisely the dead-air trade-off being defended. Merging this stack means accepting it.

The first row is what this PR buys: on the path a real barge-in takes, the turn commits and not one frame of the interrupted reply reaches the wire, where main leaks 20ms. Removing the clearBuffer() call turns that 0 back into a 1.

Closes #2122.

Made with Cursor

[changeset-bot]

🦋 Changeset detected

Latest commit: 5887cb6

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 39 packages

Name	Type
@livekit/agents	Patch
@livekit/agents-plugin-anam	Patch
@livekit/agents-plugin-anthropic	Patch
@livekit/agents-plugin-assemblyai	Patch
@livekit/agents-plugin-azure	Patch
@livekit/agents-plugin-baseten	Patch
@livekit/agents-plugin-bey	Patch
@livekit/agents-plugin-cartesia	Patch
@livekit/agents-plugin-cerebras	Patch
@livekit/agents-plugin-deepgram	Patch
@livekit/agents-plugin-did	Patch
@livekit/agents-plugin-elevenlabs	Patch
@livekit/agents-plugin-fishaudio	Patch
@livekit/agents-plugin-google	Patch
@livekit/agents-plugin-hedra	Patch
@livekit/agents-plugin-hume	Patch
@livekit/agents-plugin-inworld	Patch
@livekit/agents-plugin-krisp	Patch
@livekit/agents-plugin-lemonslice	Patch
@livekit/agents-plugin-liveavatar	Patch
@livekit/agents-plugin-livekit	Patch
@livekit/agents-plugin-minimax	Patch
@livekit/agents-plugin-mistral	Patch
@livekit/agents-plugin-mistralai	Patch
@livekit/agents-plugin-neuphonic	Patch
@livekit/agents-plugin-openai	Patch
@livekit/agents-plugin-perplexity	Patch
@livekit/agents-plugin-phonic	Patch
@livekit/agents-plugin-protoface	Patch
@livekit/agents-plugin-resemble	Patch
@livekit/agents-plugin-rime	Patch
@livekit/agents-plugin-runway	Patch
@livekit/agents-plugin-sarvam	Patch
@livekit/agents-plugin-silero	Patch
@livekit/agents-plugin-soniox	Patch
@livekit/agents-plugin-tavus	Patch
@livekit/agents-plugins-test	Patch
@livekit/agents-plugin-trugen	Patch
@livekit/agents-plugin-xai	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

[toubatbrian]

Live verification with cue-cli

The ordering this PR is about is directly observable in a live barge-in, and it turns out to be the change that makes #2133 do anything at all.

Same scripted scenario throughout (long story, 6s, then "Sorry, stop, stop. I changed my mind. What is two plus two?"), three runs per cell, driven against LiveKit Cloud with real STT/TTS and adaptive interruption. The marks below are the order in which the sink observed clearBuffer() and resume() within the interrupted reply's segment.

	#2133 absent	#2133 present
this fix absent	resume,resume — no clear at all; 0 frames dropped (3/3)	resume,resume — no clear at all; 0 frames dropped (3/3)
this fix present	resume,clear,resume; 1 frame / 98.5 ms dropped (3/3)	resume,clear,resume; 2 frames / 115.5–118.5 ms dropped (3/3)

The top row is the finding. Without this change, clearBuffer() is never called on the sink at all before the segment flushes — the interrupted reply's own teardown does not get there first in a live barge-in. That is precisely the argument in the changeset: Python reaches resume() only after awaiting the interrupted generation, by which time the reply task has already run its own clear_buffer(); JS wraps that await in waitIfNotInterrupted, which returns immediately against the interrupt future handle.interrupt() just resolved (deliberately, per #1124). So JS arrives at the gate with nothing having signalled the sink, and this call is what supplies it.

Consequence for the stack: #2133 sits below this PR and is inert until this lands — both cells in the top row are 0 drops regardless of whether its hoisted check is present. The two are jointly necessary, and this is the half that arms the other.

Control: the follow-up reply delivered every frame in all 6 runs of the bottom row (104/104, 24/24, 101/101 with #2133; 24/24, 101/101, 102/102 without). The frames dropped belong to the interrupted reply.

Still unchanged: what the revert costs

None of this touches the no-transcript path. agent_false_interruption did not fire in any live run here, and in a separate 6-run scenario built specifically to force it — a short filler barge-in plus 7s of slack for the timer — it still never fired, because real Deepgram STT produced a final transcript every time the model ruled a barge-in. So the live evidence says nothing either way about the row in the description where nothing arrives before the timeout; that row is still main-identical by construction, and merging this stack still means accepting that trade-off.

[toubatbrian]

Live verification: reproduced, reproducibly, and now with a measured duration

This is the one change in the interruption half of the stack whose effect I could see in a live session as a frame-level difference, and it reproduces on every run.

Method. A throwaway probe in ParticipantAudioOutput records every frame entering captureFrame and every frame reaching audioSource.captureFrame; drops fall out as the difference, so the same probe is valid across builds. cue-cli drives a real room with the plain barge-in scenario — long story, barge in 6 s later with "Sorry, stop, stop. I changed my mind. What is two plus two?" — against Deepgram nova-3, Inworld TTS, openai/gpt-4.1-mini and adaptive interruption. Three runs per build.

build	frames dropped at the sink	segment-0 markers around the barge-in
without this commit	0	resume … resume — no clear at the barge-in at all
with this commit, #2133 absent	1  (98.5 ms)	resume … clear, resume
with this commit and #2133	2  (114.0 ms / 118.5 ms)	resume … clear, resume

The marker column is the fix itself showing up in the trace: without this commit there is no clearBuffer() before the sink is un-gated, so the parked frames are released and go to the wire. With it, the clear appears immediately before the resume and the frames are stopped instead. Zero dropped frames in every run without it, and a non-zero count in every run with it.

The duration is measured, not inferred. The probe now records samplesPerChannel and sampleRate on each frame, so a frame count converts honestly. Inworld delivers 2364-sample frames at 24 kHz, i.e. 98.5 ms each, plus a short tail frame per segment. So this commit alone keeps one 98.5 ms frame off the wire, and together with #2133 it keeps 114.0 ms and 118.5 ms off the wire in the two runs where geometry was recorded — one full frame plus the segment's short tail frame (15.5 ms and 20.0 ms). I deliberately did not back-compute this from probe timestamps: TTS hands the sink several seconds of audio ahead of realtime, so wall-clock spacing between frames is not frame duration.

What this establishes, and what it does not. It confirms the ordering defect is real on the live path and that the fix stops the audio the user has already barged in over — and it confirms the scale. Around 100–120 ms, not seconds. That is consistent with the unit test in the description measuring a 1-frame leak on main, and it is emphatically not an explanation for the multi-second audio/transcript desync report. It also says nothing about the no-transcript path, which the description already flags as identical to main; every run here took the committed-barge-in path, with a final transcript arriving and no agent_false_interruption firing.

Coupling with #2133, worth flagging before merge. The middle row is the interesting one. With this commit but without #2133's hoisted check, one frame is still dropped — caught by the pre-existing check inside the closed-gate branch, which this commit's clearBuffer() activates by waking the parked frame there. The reverse is also true and I have measured it: without this commit, #2133's hoisted check drops nothing at all, because nothing advances interruptCount while the segment is still open. The two changes are mutually dependent, and this one has to land first for the other to reach anything. I have written the same finding up on #2133.

I noticed the branches now appear reordered so that this one sits directly on brian/interruption-stale-pause-gate with #2133 on top, while the PR base refs still describe the old order. If that is intentional it matches what the measurements argue for, and the middle row above is then exactly this PR's own pre/post pair.

The probe is throwaway instrumentation, not committed and not pushed; all five branch tips still match origin.

[toubatbrian]

This PR was closed as merged by mistake. During an attempted reorder of the stack, a force-push moved this PR's base branch (brian/interrupted-reply-backlog) so that it contained this PR's head commit, and GitHub resolved that as a merge with an empty +0/-0 diff.

Nothing was merged into main. origin/main is unchanged at 7b6c9f71 and contains no part of this stack. The branches have been restored to their pre-push SHAs.

Merged PRs cannot be reopened, so the work continues at #2136, which carries the same commit (dc8e1da4) on a new head branch with the same base. The live-verification writeups above have been reposted there.