fix(sip): fix one-way audio after long hold#3640
Conversation
|
Thanks for your contribution, @edospadoni! Please make sure you sign our CLA, as it's a required step before we can merge this. |
|
Thanks for your contribution!
I'm not sure what you mean by that. The relay thread in the SIP plugin never goes to "sleep", since it's always actively monitoring incoming sockets. The fact some packets will be ignored doesn't mean they're not received by the thread first. As such, "Relay thread not notified when only SDP direction changes" doesn't make much sense to me: no one notifies the thread, the thread is always alive, and simply monitors The SRTP issue is indeed a known one, instead, as it appeared in a few other plugins too (for different use cases that may lead to paused media that is then resumed). We fixed those issues in those other plugins, but not in the SIP plugin. Which is why I think that:
is actually not the way to do that. We already have ways to tell the RTP header function to reset things because the stream was paused for a while. It's a property called https://github.com/meetecho/janus-gateway/blob/master/src/plugins/janus_videoroom.c#L11592-L11593 As such, you shouldn't track sequence numbers at all or change things manually, or you'll mess with the way the RTP context works, which could potentially cause issues. Just set that property to |
| old_video_send != session->media.video_send)) { | ||
| *changed = TRUE; | ||
| } | ||
| if(*changed) { |
There was a problem hiding this comment.
I see now what you mean by "not waking up the thread". You meant that a change is not detected, since direction changed but the address didn't, and so the if(session->media.updated) block in janus_sip_relay_thread isn't invoked. I still don't see why that should be an issue, since the media direction would be updated in the session media object anyway, and so the thread would simply start referring to those instead. If address and port didn't change, that block in janus_sip_relay_thread doesn't need invoking, since its only purpose is to reconnect the sockets. Can you clarify why you think it's needed instead?
|
@lminiero thanks for the review. On fix #1 ( On fix #2: thanks for the pointer to I'll push the updated patch shortly. |
7818ffd to
276fbb0
Compare
|
@lminiero done. Patch is now down to a single change in |
|
Thanks for the quick turnaround! Have you checked if this does indeed still work as expected for you? |
|
Did a quick local test and it behaves the same as the previous patch (which makes sense since |
|
Thanks! As soon as you say it's ok I'll merge, backport to |
|
@lminiero we've kept it running on our production box for the past 5 days and monitored the logs across many hold/unhold cycles, including long ones. The fix looks solid. From our side it's good to merge whenever you want. |
|
@edospadoni thanks for taking the time to validate this, and thanks again for your contribution! Merging right away, and I'll then backport to |
|
Actually the NoSIP plugin doesn't have an interface for |
|
Do you have a rough idea of when this will land in a tagged release, and which version it'll be in (1.4.2)? Asking so we can plan when to move our production box off our fork and back onto an official build. |
|
It will be in 1.4.2 (and 0.16.2), but I don't know yet when a new version will be tagged, sorry. |
|
No worries, thanks anyway. I'll keep an eye on the releases for 1.4.2 / 0.16.2. Thanks again for the quick review and merge! |
Summary
This PR fixes a one-way audio bug in the SIP plugin that occurs after a call is on hold for a long time (roughly 11+ minutes).
When the call resumes, the remote party can still hear the Janus side, but the Janus side no longer receives audio from the peer. The RTP packets arrive at Janus and the SIP relay thread forwards them, but
srtp_protectrejects them withsrtp_err_status_replay_oldbefore they reach the WebRTC PeerConnection, so the browser'spacketsReceivedcounter stays frozen whilepacketsSentkeeps growing.Root cause
Two separate issues in
plugins/janus_sip.c:1. Relay thread not notified when only SDP direction changes
janus_sip_sdp_process()only setssession->media.updated = TRUE(and writes to the wake-up pipe) when thechangedflag is set, which currently only happens on IP / port differences. When the peer answers a hold/unhold re-INVITE with the same IP and port but a changeda=sendrecv/a=recvonlydirection, the relay thread is never woken up, sosession->media.audio_recvstays out of date from the thread's point of view.2. Outbound audio seq number can regress after a long hold
During hold, packets are dropped by the relay thread (
audio_recv == FALSE) and never reachjanus_rtp_header_update. The peer's RTP sequence number keeps advancing (~50 pps for G.711). After ~11 minutes of hold the peer's 16-bitseq_numberwraps past 65535 and comes back to a lower value.When the first packet after unhold is processed,
janus_rtp_header_updatemaps the peer's (now wrapped) seq throughacontextand produces an output seq that is lower than the last seq that was already successfully protected bypc->dtls->srtp_out. libsrtp's replay database treats the new index as "old" andsrtp_protectreturnssrtp_err_status_replay_old; the encrypted packet is never sent to the browser.The bug is intermittent because it only manifests when the combined active time + hold duration causes the peer's seq to wrap uint16 into a range 128–32768 below the last protected seq (SRTP's "dead zone" where ROC is not automatically adjusted).
Fix
Two minimal changes in
plugins/janus_sip.c:In
janus_sip_sdp_process(), remember the previousaudio_recv/audio_send/video_recv/video_sendvalues and force*changed = TRUEwhenever any of them differs after processing the SDP. This guarantees the relay thread is woken up on every hold/unhold transition, even when IP and port are unchanged.In
janus_sip_relay_thread(), introduce a localguint16 audio_out_seqthat is incremented once per relayed audio packet, and overwrite the RTP header seq number with it right afterjanus_rtp_header_update. The timestamp continues to come fromacontext(so silence/gap handling is preserved). This makes the outbound audio seq strictly monotonic regardless of what the peer does during hold, sosrtp_protectcan never see a regression and libsrtp's natural 65535→0 wrap continues to be handled transparently via ROC.Total change: 1 file, 38 insertions, 9 deletions. No public API change.
Verification
Deployed in production on a NethVoice/Asterisk + Janus installation on 2026-04-17. Monitored for 7 days:
srtp_err_status_replay_oldon SIP audio handlesAcross ~250+ unhold events we observed ~26 concrete cases where
audio_out_seqproduced a strictly monotonic value while the pre-fix code path (acontext.last_seq) would have produced a regressed value in SRTP's dead zone. All of these would have been silent audio on the browser side before the fix.