Skip to content

Add two-way audio to the door station camera - #117

Draft
FWeinb wants to merge 8 commits into
mainfrom
feat/2wayaudio
Draft

FWeinb wants to merge 8 commits into
mainfrom
feat/2wayaudio

Conversation

@FWeinb

@FWeinb FWeinb commented Oct 28, 2026 •

Copy link
Copy Markdown
Owner

Warning

Do not merge yet. This waits for an aiortc release on PyPI that accepts av>=14,<20. Home Assistant pins av==19.0.0, and the current aiortc release (1.15.0) requires av<18, so it cannot be installed alongside Home Assistant. aiortc main already has the relaxed pin, and this branch is built and tested against a commit of it. Until the release is out, the manifest requirement aiortc>=1.16.0 cannot be installed anywhere.

Before merging:

  • aiortc released with av>=14,<20; check that aiortc>=1.16.0 in manifest.json names that release
  • requirements.txt: replace the aiortc commit pin with the release
  • requirements.txt: homeassistant==2026.11.0b0 → 2026.11.0
  • requirements_test.txt: pytest-homeassistant from PyPI instead of the core 2026.11.0b0 tag archive
  • CI green

Summary

Talk to the person at the door from the camera's live view, using Home Assistant 2026.11's two-way audio preview (Labs). It's opt-in with the new Two-way audio option; without it, the camera stays MJPEG as before.

4ff2443  Write out the config flow strings that used common references   (existing bug)
efe7a00  Test with Home Assistant's pytest-home-assistant on 2026.11
8fbb832  Add two-way audio to the door station camera
7c822e5  Add a header and a HACS badge to the README, list actions in a table

With SIP Auto Answer off and Two-way audio on, Home Assistant rings along with the indoor stations and answers when talk is pressed:

sequenceDiagram
    participant Door as Door station / SG-150
    participant HA as Home Assistant
    participant Browser as Live view
    Door->>HA: INVITE
    HA-->>Door: 180 Ringing (ring along)
    Browser->>HA: WebRTC offer with microphone (talk pressed)
    HA-->>Door: 200 OK, then ACK
    Door-)HA: RTP (G.711)
    HA-)Browser: door audio + VP8 video
    Browser-)HA: microphone
    HA-)Door: RTP (G.711)
    Browser->>HA: close live view (last talker)
    HA->>Door: BYE
Loading
custom_components/sg_150/
├── sip/rtp.py      # new: RTP session per call, G.711, jitter buffer, clip + live mixing
├── sip/minisip.py  # sendrecv both ways, PCMU, CANCEL, resend 200 OK until ACK, ring then answer later
├── line.py         # ring along, answer(), ring timeout
├── talkback.py     # new: aiortc WebRTC sessions, one shared VP8 encoder per camera
├── camera.py       # native WebRTC + two_way_audio capability when the option is on
├── button.py       # Hang Up next to each camera; errors when there is no call
└── media_source.py # image view registered in async_setup (2026.11 loads platforms lazily)

Also in this PR:

  • Tests move from pytest-homeassistant-custom-component, which doesn't load on 2026.11, to Home Assistant's own pytest-home-assistant (Clarify pytest-home-assistant docstrings and README home-assistant/core#187272).
  • The config flow's [%key:common::…%] references are written out. Custom integrations ship translations/*.json as they are, so users saw the raw reference text.
  • New options: Two-way audio, and First/Last RTP Port (default 5004–5013).

Evidence

  • Before: on HA 2026.11.0b0, the test suite doesn't start (RuntimeError: httpx was already imported). Stored-image URLs return 404 after a restart, until the media browser has been opened.
    After: scripts/test gives 138 passed, 84% coverage. This includes an in-process WebRTC test with a second aiortc peer as the browser:
    ringing call + browser offer without microphone  → video arrives, call keeps ringing
    ringing call + browser offer with microphone     → 200 OK, mic tone reaches the gateway's RTP,
                                                       door audio reaches the browser
    close the last talking session                   → BYE
    
  • Hardware: tested by hand on a real SG-150 with a local HA 2026.11.0b0 instance. Ring-along, talking both ways, hanging up, and play_audio all work. The live view shows video while the call is still ringing.

Merge Danger

Door: two-way

Two-way audio is opt-in and off by default. Reverting the PR restores the previous behaviour, with no config migration. Saved options gain three keys, which older versions ignore.

Blast Radius: calls

  • Every call changes, not only two-way audio calls. Call audio now goes through a per-call RTP port (5004-5013 by default), not the SIP port. The SG-150 must be able to reach those ports, so firewalls may need updating. Outgoing calls offer sendrecv instead of sendonly.
  • Minimum Home Assistant becomes 2026.11.0 (hacs.json). HACS won't offer this version to older installs. Installed by hand on older HA, the camera platform fails to load, because CameraEntityCapabilities doesn't exist there.
  • The camera live view becomes WebRTC once the option is on. Encoding costs about a third of a CPU core per camera while someone watches (VP8, 768×576, 15 fps, measured on the HA host).
  • The two-way audio API is a Labs preview and may change in later Home Assistant releases.

Custom integrations ship their translations/*.json as they are: only
core's compile step replaces [%key:...%] references, so users saw the
raw reference for unknown errors, an already configured gateway and a
successful reauth. Write the text out, using the same wording as core's
common strings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FWeinb and others added 3 commits October 28, 2026 23:42
pytest-homeassistant-custom-component does not load on Home Assistant
2026.11, which now ships its own test package, pytest-home-assistant
(home-assistant/core#187272). Switch to it and target 2026.11.0b0:

- Import the test helpers from pytest_homeassistant, and drop the
  enable_custom_integrations wrapper; custom integrations are enabled
  by default now.
- Disable the respx pytest plugin, so Home Assistant aliases httpx to
  httpx2 before respx imports it.
- Install pytest-home-assistant from the 2026.11.0b0 core tag until it
  is published on PyPI.

media_source now loads platforms lazily, so the image proxy view was
only registered once media had been browsed, and stored image URLs
returned 404 after a restart. Register it in async_setup instead.

Declare pydantic as a requirement. api.py and types.py use it, but it was
never declared: Home Assistant only constrains its version and doesn't
install it, and the old test package happened to depend on it, while
pytest-home-assistant doesn't. Require it in the manifest for installs,
and pin Home Assistant's constrained version in requirements.txt for CI.

Add pytest-cov to the test requirements for the same reason: scripts/test
uses its --cov options and CI's coverage report needs coverage, which the
old test package brought along and pytest-home-assistant doesn't.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Talk to the person at the door from the camera's live view, using Home
Assistant 2026.11's two-way audio preview feature. Opt in with the new
Two-way audio option; without it the camera stays MJPEG.

- sip/rtp.py: an RtpSession per call on a configurable UDP port range
  (default 5004-5013). It sends a continuous, clock-paced G.711 stream
  from the advertised port, plays received audio out through a small
  jitter buffer, and mixes live audio over play_audio clips.
- minisip: offer sendrecv on incoming and outgoing calls, accept PCMU,
  handle CANCEL, resend our 200 OK until the ACK, and let a call ring
  until it is answered or rejected later.
- GatewayLine: with auto answer off and two-way audio on, ring along
  with the handsets and answer when talk is pressed; calls nobody
  answers are rejected as busy after a minute.
- talkback.py: native WebRTC through aiortc. One VP8 encoder per camera
  feeds every viewer, door audio goes to every session, and the newest
  talking session's microphone goes into the call. The call is hung up
  when the last session that talked closes.
- A Hang Up button next to each camera, ending the calls its device
  is on. Pressing it with no call shows an error.

aiortc needs an av<20 release (main has it; PyPI 1.15.0 pins av<18),
so requirements.txt pins an aiortc commit until then.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A header image in Home Assistant's style, with a light and a dark
  version picked by GitHub's theme.
- The HACS default badge.
- The actions get their own section, as a table with their fields and
  responses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Setup the gateway device in init and link all the entities via get_gateway_identifier()
Still need to wait until a proper release of
* pytest-homeassistant==2026.11.0
* aiortc==1.16.0 (including aioice>=0.12.1)
was made.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant