Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
c3202c8
clarifies video session cleanup ownership
chanderlud Jul 25, 2026
16ebdbf
documents generic video lifecycle tracing
chanderlud Jul 25, 2026
551e4fb
defines generic video protocol domain
chanderlud Jul 25, 2026
b21e999
extracts static video platform adapters
chanderlud Jul 25, 2026
4749b24
adds bounded video transport framing
chanderlud Jul 25, 2026
c599fc3
joins video workers during teardown
chanderlud Jul 25, 2026
2113145
routes calls through video lifecycle
chanderlud Jul 25, 2026
773c00e
generalizes video capabilities
chanderlud Jul 25, 2026
ada6893
adds video integration harness support
chanderlud Jul 25, 2026
67750d1
covers video session lifecycle
chanderlud Jul 25, 2026
8267681
publishes typed video lifecycle APIs
chanderlud Jul 25, 2026
9ba77bd
adapts CLI video lifecycle events
chanderlud Jul 25, 2026
9edbb7a
regenerates Rust bridge bindings
chanderlud Jul 25, 2026
28e0221
regenerates Dart video bindings
chanderlud Jul 25, 2026
4e796d7
migrates Flutter video state
chanderlud Jul 25, 2026
d50913c
updates generic video call controls
chanderlud Jul 25, 2026
ce5c379
updates Flutter video lifecycle tests
chanderlud Jul 25, 2026
f732ba7
documents joined video session teardown
chanderlud Jul 25, 2026
3a00dcb
centralizes video session vocabulary
chanderlud Jul 25, 2026
6756805
fixes generated bridge lint checks
chanderlud Jul 25, 2026
062a307
Merge remote-tracking branch 'origin/master' into feature/generic-vid…
chanderlud Jul 26, 2026
c87eb58
fix video session lifecycle
chanderlud Jul 26, 2026
232ed07
fix wasm helper imports
chanderlud Jul 28, 2026
3c2ec03
Merge remote-tracking branch 'origin/master' into feature/generic-vid…
chanderlud Jul 29, 2026
000c26e
Merge master into feature/generic-video-sessions
chanderlud Jul 30, 2026
fa78a42
code cleanup
chanderlud Jul 30, 2026
cb47688
fix(core): make video adapter capability-safe
chanderlud Jul 30, 2026
1157273
Merge branch 'refactor/generic-video-sessions-cleanup' into feature/g…
chanderlud Jul 30, 2026
36b8ff8
docs(core): document adapter-safe video capabilities
chanderlud Jul 30, 2026
f3a100c
refactor(core): remove stale video helpers
chanderlud Jul 31, 2026
3ebeaea
fix(core): gate unsupported video helpers
chanderlud Jul 31, 2026
a876256
Merge remote-tracking branch 'origin/master' into feature/generic-vid…
chanderlud Aug 1, 2026
cae29dd
feat: implement generic video sessions and update generated bindings
chanderlud Aug 1, 2026
d90df81
refactor: simplify video imports
chanderlud Aug 1, 2026
b737111
Merge remote-tracking branch 'origin/master' into feature/generic-vid…
chanderlud Aug 1, 2026
44fa6ea
fix(cli): resolve strict warning lints
chanderlud Aug 1, 2026
e7e404e
fix: resolve CI build and test failures
chanderlud Aug 2, 2026
5f26f03
fix(core): suppress dead-code on unsupported platforms via lint expec…
chanderlud Aug 2, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
* System-test CLI: `rust/telepathy-cli`
* Flutter: `lib/`
* Docs: `docs/`
* Documented solutions: `docs/solutions/` stores searchable architecture and troubleshooting learnings with YAML metadata; `docs/CONCEPTS.md` defines shared project vocabulary. Both are relevant when implementing or debugging in documented areas.
* System tests: `system-tests/`
* Generated, never read/edit: `lib/core/rust/*`, `frb_generated.rs`

Expand All @@ -33,6 +34,10 @@ dart format .

Format only after cleanup is complete.

## Rust Style

- Import `VideoWorkerStartup` at file top instead of spelling inline paths such as `crate::internal::video::VideoWorkerStartup::Failed`; rename the import only when it conflicts with another name.

## Rust Tests

Prefer nextest. Use `cargo test` only when required, and state why.
Expand Down
1 change: 1 addition & 0 deletions analysis_options.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ include: package:flutter_lints/flutter.yaml
analyzer:
exclude:
- lib/src/**
- lib/core/rust/lib.dart

linter:
# The lint rules applied to this project can be customized in the
Expand Down
21 changes: 21 additions & 0 deletions docs/CONCEPTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,3 +29,24 @@ An attempt remains active while direct dialing or an associated inbound candidat
The observable per-peer state that tells a call request whether a direct session already exists, a direct session attempt may still publish one, or no session can be expected.

Availability changes wake waiting call requests; a request only acquires call ownership after the session is published.

## Video Sessions

### Video Session
A peer-scoped exchange that coordinates one display-media source, its lifecycle controls, and its media transport without owning the underlying call.

### Video Attempt
One incarnation of a Video Session, scoped so late asynchronous work from an earlier incarnation cannot affect a later use of the same peer slot.

### Video Slot
The per-peer lifecycle boundary that admits at most one Video Attempt and remains occupied until that attempt is fully finished.

### Joined Teardown
The terminal process that keeps a Video Slot unavailable until all work for its current Video Attempt has finished.

### Runtime Video Capability
The current adapter-probed set of video sources and media formats that the running target can send or receive. It is distinct from serializable video configuration, which may name a device or encoder that is no longer available or has no implemented command path.

## Relationships

A Video Slot owns one Video Attempt at a time. Joined Teardown preserves that ownership until the attempt has fully finished, after which the slot may admit a replacement attempt.
144 changes: 82 additions & 62 deletions docs/plans/2026-07-17-001-refactor-generic-video-sessions-plan.md

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
---
title: Adapter-Safe Video Capabilities Separate Runtime Truth From Configuration
date: 2026-07-30
category: docs/solutions/architecture-patterns/
module: telepathy-core video platform
problem_type: architecture_pattern
component: tooling
severity: high
applies_when:
- A target-specific adapter exposes serializable device or codec settings
- A compiled adapter has target-dependent command implementations
- Receiver playback must match a negotiated media format
related_components:
- flutter-rust-bridge
- testing-framework
tags: [video-sessions, capabilities, platform-adapters, ffmpeg, configuration]
---

# Adapter-Safe Video Capabilities Separate Runtime Truth From Configuration

## Context

Generic video sessions retain serializable internal `Device`, `Encoder`, and `Decoder` values, but a stored value does not prove that the current target can run it. The selected adapter must report what this runtime can actually start or receive.

## Guidance

Select one private adapter at compile time in `rust/telepathy-core/src/internal/video/platform.rs`: the desktop FFmpeg adapter for Windows, macOS, and Linux, or the unsupported adapter elsewhere. Keep the coordinator and public contract independent of that selection.

Treat the fresh adapter probe as the source of capability truth. `desktop_ffmpeg::video_capabilities` advertises a display source only when it has both a device and an encoder result. `Device::devices` currently supplies capture devices only on Windows, while `Device::to_args` returns a typed platform-unavailable error for unimplemented paths. Therefore a desktop build on macOS or Linux must not advertise display capture merely because the adapter compiled or a `RecordingConfig` can deserialize.

Validate the selected configuration again in `desktop_ffmpeg::prepare_sender`. Missing current devices or encoders return `VideoUnavailable::ConfigurationUnavailable`; missing source formats return the corresponding typed unavailability. The unsupported adapter returns `VideoUnavailable::PlatformUnsupported` for capabilities and sender preparation.

For playback, `desktop_ffmpeg::run_receiver` probes locally immediately before startup. `select_decoder` chooses the first decoder from that fresh local probe list whose codec matches the negotiated `VideoMediaDescriptor`. Decoder preference is local probe order, not sender configuration or a global cross-platform order.

## Why This Matters

Configuration is stable enough to save and present later. Capability is a statement about the current binary, target, installed FFmpeg components, and implemented command paths. Combining them can advertise a mode that cannot start, then turn a user action or restored setting into a panic or a false success.

Keeping the distinction inside the adapter lets the generic session lifecycle remain target-neutral while still reporting precise typed outcomes to native and Flutter callers.

## When to Apply

- A target-specific implementation compiles on more targets than it fully supports.
- Device, encoder, decoder, permission, or binary availability can change after settings are stored.
- A receiver must choose a local implementation compatible with peer-negotiated media.

## Examples

The adapter keeps unavailable configuration on the typed path rather than starting a process:

```rust
if !capabilities.encoders.contains(&config.encoder)
|| !capabilities.devices.contains(&config.device)
{
return Err(VideoUnavailable::ConfigurationUnavailable);
}
```

The verified adapter tests cover the same boundary: `unimplemented_device_returns_typed_error_without_panicking`, `sender_start_rejects_encoder_removed_after_preflight`, `sender_start_rejects_device_removed_after_preflight`, and `decoder_selection_uses_first_compatible_local_decoder` in `rust/telepathy-core/src/internal/video/platform/desktop_ffmpeg.rs`. `unsupported_adapter_query_and_start_report_typed_unavailable` in `rust/telepathy-core/src/internal/video/platform.rs` covers the selected unsupported contract.

## Related

- [Joined Video Session Teardown Keeps Slots Safe for Reuse](joined-video-session-teardown.md) covers lifecycle ownership after a session has started.
- The generic video-session implementation plan is `docs/plans/2026-07-17-001-refactor-generic-video-sessions-plan.md`.
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
---
title: Joined Video Session Teardown Keeps Slots Safe for Reuse
date: 2026-07-25
category: docs/solutions/architecture-patterns/
module: telepathy-core video sessions
problem_type: architecture_pattern
component: tooling
severity: high
applies_when:
- A peer-scoped session owns transport workers or platform resources
- Multiple asynchronous paths can terminate the same session
- A slot may be reused after cancellation
related_components:
- session-manager
- flutter-rust-bridge
- testing-framework
tags: [video-sessions, teardown, cancellation, task-ownership, iroh]
---

# Joined Video Session Teardown Keeps Slots Safe for Reuse

## Context

A video session can finish through local stop, remote control, a timeout, transport failure, or session teardown. Cancellation alone does not release the transport worker or its platform I/O. Reusing the peer slot before that worker joins lets stale work overlap a new session.

## Guidance

Make the per-peer `VideoSlot` the single owner of each `VideoAttempt` and its worker. A local start records a fresh session identity and generation in a reservation before it sends an offer. The worker installs only while that exact attempt is still `Starting`; a stale installation cancels and joins itself instead of attaching to a replacement reservation.

Terminal paths must claim the reservation by moving it to `Stopping`, cancel its token, take and join the worker, then clear the reservation and notify idle. A second terminal path waits for idle instead of joining or clearing the same worker again. `VideoSlot::cancel_and_join` implements this ordering in `rust/telepathy-core/src/internal/video.rs`.

Session teardown must invoke `cancel_current_and_join` after signalling call and session cancellation. `SessionState::teardown` does this before returning in `rust/telepathy-core/src/internal/state.rs`.

## Why This Matters

The corrected teardown regression showed that cancelling only the session token leaves an installed video worker blocked on its own token. The session could appear torn down while the worker still held transport or platform resources.

The reservation remains occupied until the worker join completes, so a new generation cannot reuse the slot early. The full attempt identity also prevents a late worker result from mutating or terminating a replacement session.

## When to Apply

- A logical slot owns sockets, streams, subprocesses, device handles, or long-lived tasks.
- More than one event can end that work.
- A stale task could act after its slot has been reused.

## Examples

The installation guard requires matching attempt identity, `Starting` phase, and no installed worker before storing the handle. Otherwise it cancels the launch and awaits the worker.

```rust
if reservation.attempt == launch.attempt
&& reservation.phase == VideoPhase::Starting
&& reservation.worker.is_none()
{
reservation.phase = VideoPhase::Active;
reservation.worker = Some(worker);
} else {
launch.cancellation.cancel();
let _ = worker.await;
}
```

The sender and receiver race stream creation against cancellation and reset or stop interrupted streams in `rust/telepathy-core/src/internal/video/transport.rs`. Integration coverage verifies that teardown does not make the slot idle before a blocked worker is released and joined in `rust/telepathy-core/tests/core_integration_test/video_sessions/lifecycle.rs`.

## Related

- [Prepared Identity Switching Requires Runtime Readiness and Token-Owned Commit](prepared-identity-switch-runtime-readiness.md) applies the same ownership principle to a prepared identity operation.
- The generic video-session implementation plan is `docs/plans/2026-07-17-001-refactor-generic-video-sessions-plan.md`.
7 changes: 7 additions & 0 deletions lib/controllers/network_settings_controller.dart
Original file line number Diff line number Diff line change
Expand Up @@ -211,6 +211,13 @@ class NetworkSettingsController with ChangeNotifier {
'screenshareConfigBuffer', base64Encode(screenshareConfig.toBytes()));
}

Future<bool> isVideoSourceConfigured(VideoSource source) async {
switch (source) {
case VideoSource.display:
return (await screenshareConfig.recordingConfig()) != null;
}
}

Future<CodecConfig> loadCodecConfig() async {
return CodecConfig(
enabled: await options.getBool('codecEnabled') ?? true,
Expand Down
102 changes: 68 additions & 34 deletions lib/controllers/state_controller.dart
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,11 @@ class StateController extends ChangeNotifier {
final Map<String, SessionStatus> sessions = {};

ManagerState _sessionManagerState = ManagerState.stopped;
FrontendNotify? _stopSendingScreenshare;
FrontendNotify? _stopReceivingScreenshare;
VideoSessionIdentity? _sendingScreenshareIdentity;
VideoSessionIdentity? _receivingScreenshareIdentity;
VideoSessionIdentity? _stoppedSendingScreenshareIdentity;
final Set<VideoSessionIdentity> _terminalSendingScreenshareIdentities = {};
final Set<VideoSessionIdentity> _terminalReceivingScreenshareIdentities = {};
bool isSendingScreenshare = false;
bool isReceivingScreenshare = false;

Expand Down Expand Up @@ -345,45 +348,77 @@ class StateController extends ChangeNotifier {
});
}

void screenshareStarted((FrontendNotify stop, bool sending) record) {
if (record.$2) {
DebugConsole.log('Sending screenshare started');
_stopSendingScreenshare = record.$1;
isSendingScreenshare = true;

// this catches the sending screenshare being closed by the receiver
Future.microtask(() async {
await record.$1.notified();
// if the screen share is still sending, stop the screenshare
if (isSendingScreenshare) {
stopScreenshare(true, true);
void handleVideoLifecycle(VideoLifecycleEvent event) {
if (event.source != VideoSource.display) return;

if (event.phase == VideoPhase.active) {
if (!isCallActive) return;
if (event.role == VideoRole.sender) {
if (event.identity == _stoppedSendingScreenshareIdentity) return;
if (_terminalSendingScreenshareIdentities.contains(event.identity)) {
return;
}
_terminalSendingScreenshareIdentities.remove(event.identity);
_sendingScreenshareIdentity = event.identity;
isSendingScreenshare = true;
} else {
if (_terminalReceivingScreenshareIdentities.contains(event.identity)) {
return;
}
});
_terminalReceivingScreenshareIdentities.remove(event.identity);
_receivingScreenshareIdentity = event.identity;
isReceivingScreenshare = true;
}
notifyListeners();
return;
}

if (event.phase != VideoPhase.terminal) return;

var handled = false;
if (event.role == VideoRole.sender) {
_terminalSendingScreenshareIdentities.add(event.identity);
if (event.identity == _sendingScreenshareIdentity) {
_sendingScreenshareIdentity = null;
isSendingScreenshare = false;
handled = true;
}
if (event.identity == _stoppedSendingScreenshareIdentity) {
_stoppedSendingScreenshareIdentity = null;
handled = true;
}
} else {
DebugConsole.log('Receiving screenshare started');
_stopReceivingScreenshare = record.$1;
isReceivingScreenshare = true;
_terminalReceivingScreenshareIdentities.add(event.identity);
if (event.identity == _receivingScreenshareIdentity) {
_receivingScreenshareIdentity = null;
isReceivingScreenshare = false;
handled = true;
}
}

if (!handled) return;
notifyListeners();
}

void stopScreenshare(bool sending, bool notify) {
DebugConsole.log('Stopping screenshare sending: $sending');
VideoSessionIdentity? stopSendingScreenshare() {
final identity = _sendingScreenshareIdentity;
if (identity == null) return null;

if (sending) {
_stopSendingScreenshare?.notify();
_stopSendingScreenshare = null;
isSendingScreenshare = false;
} else {
_stopReceivingScreenshare?.notify();
_stopReceivingScreenshare = null;
isReceivingScreenshare = false;
}
_sendingScreenshareIdentity = null;
_stoppedSendingScreenshareIdentity = identity;
isSendingScreenshare = false;
notifyListeners();
return identity;
}

if (notify) {
notifyListeners();
}
void clearScreenshares() {
_sendingScreenshareIdentity = null;
_receivingScreenshareIdentity = null;
_stoppedSendingScreenshareIdentity = null;
_terminalSendingScreenshareIdentities.clear();
_terminalReceivingScreenshareIdentities.clear();
isSendingScreenshare = false;
isReceivingScreenshare = false;
}

/// A group of actions run when the call ends.
Expand All @@ -393,8 +428,7 @@ class StateController extends ChangeNotifier {
_activeRoom = null;
_callTimer.stop();
_callTimer.reset();
stopScreenshare(true, false);
stopScreenshare(false, false);
clearScreenshares();

if (_startRequestPending) {
// Keep the target, attempt, and operation until the original start future
Expand Down
21 changes: 15 additions & 6 deletions lib/core/rust/flutter.dart
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ abstract class FlutterCallbacks implements RustOpaqueInterface {
required FutureOr<void> Function(Statistics) statistics,
required FutureOr<void> Function(ChatMessage) messageReceived,
required FutureOr<void> Function(ManagerState) managerActive,
required FutureOr<void> Function((FrontendNotify, bool))
screenshareStarted}) =>
required FutureOr<void> Function(VideoLifecycleEvent)
videoLifecycle}) =>
RustLib.instance.api.crateFlutterFlutterCallbacksNew(
acceptCall: acceptCall,
getContact: getContact,
Expand All @@ -38,7 +38,7 @@ abstract class FlutterCallbacks implements RustOpaqueInterface {
statistics: statistics,
messageReceived: messageReceived,
managerActive: managerActive,
screenshareStarted: screenshareStarted);
videoLifecycle: videoLifecycle);
}

// Rust type: RustOpaqueMoi<flutter_rust_bridge::for_generated::RustAutoOpaqueInner<PreparedIdentitySwitch>>
Expand Down Expand Up @@ -94,6 +94,9 @@ abstract class Telepathy implements RustOpaqueInterface {
Future<PreparedIdentitySwitch> prepareIdentitySwitch(
{required List<int> targetKey, required List<Contact> targetContacts});

Future<VideoStartOutcome> requestVideoSource(
{required Contact contact, required VideoSource source});

/// Restarts the session manager
Future<void> restartManager();

Expand Down Expand Up @@ -140,14 +143,20 @@ abstract class Telepathy implements RustOpaqueInterface {
{required Contact contact, required StartOperation operation});

/// Non-blocking: spawns the manager task and returns. The Dart side observes
/// the eventual `Active` transition via the `managerActive` callback.
/// the eventual `Active` transition via the `managerActive` callback. The
/// non-blocking contract is validated by the CLI system test
/// `test_start_manager_ack_precedes_active_event`; the `()` return type
/// prevents silent reintroduction of blocking semantics.
Future<void> startManager();

Future<void> startScreenshare({required Contact contact});

/// Tries to start a session for a contact
Future<void> startSession({required Contact contact});

/// Stops a specific session (called when a contact is deleted)
Future<void> stopSession({required Contact contact});

Future<VideoStopOutcome> stopVideoSource(
{required VideoSessionIdentity identity});

Future<VideoCapabilities> videoCapabilities();
}
Loading
Loading