From 7b339caa43fe122e538d3c5637ba1d284e57f9cd Mon Sep 17 00:00:00 2001 From: nishitha-burman Date: Mon, 24 Aug 2026 14:54:00 -0700 Subject: [PATCH 01/20] Add playback audio capture explainer Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- PlaybackAudioCapture/README.md | 397 ++++++++++++++++++ .../audio-only-access-UX-flows.md | 372 ++++++++++++++++ 2 files changed, 769 insertions(+) create mode 100644 PlaybackAudioCapture/README.md create mode 100644 PlaybackAudioCapture/audio-only-access-UX-flows.md diff --git a/PlaybackAudioCapture/README.md b/PlaybackAudioCapture/README.md new file mode 100644 index 00000000..4290bef5 --- /dev/null +++ b/PlaybackAudioCapture/README.md @@ -0,0 +1,397 @@ +# Playback Audio Capture Without Screen Sharing + +## Author + +- [Nishitha Dey](https://github.com/nishitha-burman) + +## Participate + +- [Proposal issue and discussion](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12) +- [MSEdgeExplainers issue tracker](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues) + +## Introduction + +Today, websites cannot capture audio playing in a browser tab, application, or +across a device without also asking the user to share their screen. + +`getUserMedia()` captures microphone and camera input, while +`getDisplayMedia()` is designed for sharing a screen, window, or browser tab +and requires video. As a workaround, a website must start screen sharing, use +the audio if it is available, and discard the video track. Developers have +reported that this can add +[unnecessary processing](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12#issuecomment-1960941133), +[hurt performance](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12#issuecomment-1960941077), +and +[confuse users](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12#issuecomment-1960941098) +who are asked to share visual content for an audio-only feature. + +Even if the website immediately stops the video track, the user must approve +screen sharing first. This exposes more information than an audio-only feature +needs and makes the permission request harder to understand. + +This document proposes allowing users to share **playback audio**—audio played +by a browser tab, application, or the device—without sharing screen pixels. A +website can request system audio or ask the browser to let the user select a +source. The browser controls permission and source selection, and the website +receives a standard audio track that it can record, process, or share. + +This document considers two API options: + +1. Add a new `getPlaybackMedia()` method. +2. Extend `getUserMedia()` with a playback-audio option. + +Both options return an audio track that works with existing web APIs. + +## User scenarios + +### Note taking, captions, and translation + +A note-taking application could record audio from a +[meeting or lecture](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12#issuecomment-1960941133) +playing in another browser tab or application, without also capturing video. +The same audio could be transcribed to create captions, translations, or +searchable notes. +A standalone note-taking application could also combine audio from a selected +meeting source with the user's microphone for +[AI transcription](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12#issuecomment-4449563056). + +### Live production and audio sharing + +A +[production application](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12#issuecomment-1960941143) +could mix audio from guests, music players, games, soundboards, and other +applications for a livestream or recording. It could also share an +application's audio during a call without sharing its window. + +Native tools already support these workflows. +[OBS Studio](https://obsproject.com/kb/application-audio-capture-guide) +captures individual applications as separate audio sources, while +[Voicemeeter](https://vb-audio.com/Voicemeeter/) mixes and routes audio from +multiple sources. +[Rocket Broadcaster](https://www.rocketbroadcaster.com/) captures and sends +live audio to a streaming service. These tools show an established need for +capturing and combining audio without requiring video. + +### Audio analysis and processing + +A website could use playback audio to create visualizations or apply audio +effects chosen by the user. + +The W3C discussion includes a +[developer building an audio visualizer](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12#issuecomment-1960941124) +who found that screen capture hurt performance. + +### Real-time communication + +In a call, audio played by the device—such as far-end participants, shared +media, or notifications—can leak into the microphone. Other participants may +hear this as echo, and the extra audio can interfere with the application's +voice processing. + +The same problem affects people who use a screen reader or Narrator. Assistive +audio played by the device can be picked up by the microphone and sent to +other participants as if it came from the user. + +Call-center or webphone software could +[record call audio](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12#issuecomment-1960941133) +together with the operator's microphone without capturing video or integrating +with the calling application. + +With a separate playback track, the communication application can compare what +the device played with what the microphone captured, apply its own echo +cancellation and voice processing, and send only the processed microphone +track to other participants. Camera capture remains separate and continues to +use `getUserMedia()`. + +### AI voice chat + +An AI voice-chat application could use playback audio to avoid treating its +own spoken response or Narrator as a new user request and to detect when the +user interrupts. A related +[W3C comment](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12#issuecomment-4668855578) +argues that AI agents should not require video access when they need only +audio. + +## Goals + +- Enable users to share audio from a supported tab, application, or device + without sharing screen. +- Use browser UI to let users authorize playback capture, select a source when + needed, see when capture is active, and stop sharing. +- Return a standard audio `MediaStreamTrack` that websites can record, process, + or share using existing web APIs. +- Support applications that process playback and microphone audio together. + +## Non-goals + +- Allowing playback audio to be captured without the user's permission. +- Guaranteeing support for tab, application, and device audio on every + platform. +- Bypassing restrictions that prevent protected music or video from being + captured. + +## Proposed Approach + +Both options let a website request system audio or a source selected by the +user without requiring video. Option 1 returns a playback-only `MediaStream`. +Option 2 adds a playback track alongside any other requested tracks. + +### Option 1: Add `getPlaybackMedia()` + +Add a method for requesting playback audio: + +```js +const playbackStream = + await navigator.mediaDevices.getPlaybackMedia({ + audio: { + source: "user-selected", + }, + }); + +const playbackTrack = playbackStream.getAudioTracks()[0]; +``` + +A note-taking app could use this to let the user select the tab where a +meeting or lecture is playing. + +This option clearly separates playback capture from microphone, camera, and +screen capture. + +Microphone and camera capture continue to use `getUserMedia()`, including its +existing device and track constraints. + +#### Use with a microphone + +A note-taking, communication, or AI voice-chat application first gets the +microphone: + +```js +const microphoneStream = + await navigator.mediaDevices.getUserMedia({ + audio: true, + }); + +const microphoneTrack = microphoneStream.getAudioTracks()[0]; +``` + +A note-taking application could then let the user select the meeting or +lecture audio: + +```js +const playbackStream = + await navigator.mediaDevices.getPlaybackMedia({ + audio: { + source: "user-selected", + }, + }); +``` + +A communication or AI voice-chat application could instead request system +audio: + +```js +const playbackStream = + await navigator.mediaDevices.getPlaybackMedia({ + audio: { + source: "system", + }, + }); +``` + +The two requests are independent. If the user declines playback capture, the +application can continue using the microphone. + +### Option 2: Extend `getUserMedia()` + +Add a `playbackAudio` option to `getUserMedia()`. + +A note-taking website that only needs playback audio could let the user choose +the tab or application where a meeting or lecture is playing: + +```js +const playbackStream = + await navigator.mediaDevices.getUserMedia({ + playbackAudio: { + source: "user-selected", + }, + }); +``` + +A communication or AI voice-chat application could request microphone and +playback audio together: + +```js +const stream = + await navigator.mediaDevices.getUserMedia({ + audio: true, + playbackAudio: { + source: "system", + }, + }); +``` + +The existing `audio` and `video` members and their constraints remain +unchanged. A website that also needs a camera can request all three: + +```js +const stream = + await navigator.mediaDevices.getUserMedia({ + audio: true, + video: true, + playbackAudio: { + source: "system", + }, + }); +``` + +As with `getUserMedia()` today, every requested media type is required. If +microphone, camera, or playback access fails, the request is rejected. A +website that can use either microphone or playback audio can request them +separately and handle each result independently. Whether a future version +should support an explicit best-effort playback request remains open. + +Both tracks are audio tracks, so the proposal must give developers a way to +tell them apart. One possible design is a `sourceType` value: + +```js +const microphoneTrack = + stream.getAudioTracks().find(track => + track.getSettings().sourceType === "microphone"); + +const playbackTrack = + stream.getAudioTracks().find(track => + track.getSettings().sourceType === "playback"); +``` + +Applications that process microphone and playback audio together need timing +information that can be compared reliably. The exact clock, timestamp, drift, +and latency requirements remain open. + +This option reuses an existing capture method, but changes `getUserMedia()` +from an API for microphone and camera input into one that can also capture +audio being played by the device. + +### Comparison + +| Option | Benefit | Tradeoff | +| --- | --- | --- | +| New `getPlaybackMedia()` | Clearly describes playback capture and works independently | A communication application makes separate microphone and playback requests | +| Extend `getUserMedia()` | Can request microphone and playback audio together | Changes the meaning of `getUserMedia()`, may combine a permission prompt with a source picker, and requires a way to identify the two audio tracks | + +### Source selection + +Both options can support two source modes: + +| Source value | Browser behavior | +| --- | --- | +| `"system"` | The combined audio rendered by the device that the browser and platform can expose | +| `"user-selected"` | Let the user choose a tab or application through browser-controlled UI | + +The source names are illustrative and open to discussion. The website does not +receive a list of running applications or choose a tab or application without +the user. The exact sources included in system audio may vary by browser and +operating system. + +### Use with existing APIs + +The website can use the returned track with existing APIs. + +Record it: + +```js +const recorder = new MediaRecorder(playbackStream); +recorder.start(); +``` + +Process or analyze it: + +```js +const context = new AudioContext(); +const source = context.createMediaStreamSource(playbackStream); +``` + +Share it through an existing WebRTC connection: + +```js +peerConnection.addTrack(playbackTrack, playbackStream); +``` + +## Privacy and security + +Playback audio may include sensitive content from tabs, applications, or the +device. + +See the +[permission UX flow mockups](audio-only-access-UX-flows.md). + +- Access requires a user action and browser-controlled permission. +- A user-selected request uses browser UI; the website cannot enumerate or + silently choose applications. +- The browser clearly indicates when playback audio is being captured and lets + the user stop it. +- Browsers may exclude protected content or sources the platform cannot + capture. + +Playback capture ends when the user stops it, permission is revoked, the source +becomes unavailable, or the website stops the track. + +## Open questions + +- What timing guarantees are needed when microphone and playback audio are + processed together? +- What sources and processing stages are included in system audio on each + platform? +- How should playback permission integrate with existing browser permission + controls and capability elements? + +## Alternatives considered + +### Allow audio-only `getDisplayMedia()` + +The platform could allow: + +```js +const stream = await navigator.mediaDevices.getDisplayMedia({ + video: false, + audio: true, +}); +``` + +This reuses an existing method. However, `getDisplayMedia()` is designed around +selecting and sharing visible content. Video is required and audio is optional. +The WebRTC Working Group discussion identified audio-only capture as a large +change to that model. For that reason, this is not one of the proposed +options. + +### Use `systemAudio` or `audioPreference` with `getDisplayMedia()` + +`systemAudio: "include"` asks the browser to offer system audio during display +capture. The proposed `audioPreference: "high"` value, discussed in the +July 2026 WebRTC Working Group meeting ([minutes](https://www.w3.org/2026/07/14-webrtc-minutes.html), +[slides](https://docs.google.com/presentation/d/1aivu0RWAtlHpJ4dFyTPblPXGMn-ZdK9GG2xuqgcg9MQ/edit?slide=id.g2bb12bc23cb_0_0#slide=id.g2bb12bc23cb_0_0)), +tells the browser that receiving audio is important: + +```js +const stream = await navigator.mediaDevices.getDisplayMedia({ + video: true, + audio: true, + systemAudio: "include", + audioPreference: "high", +}); +``` + +These options can improve requests that need both video and audio, but neither +provides audio-only capture. + +### Stop the video track after requesting display capture + +A website can request display capture and immediately stop the video track. +The user must still approve screen sharing, the website may initially receive +video, and an audio track is not guaranteed. + +### Improve browser echo cancellation + +Better browser echo cancellation would improve calls and should be explored +separately. It would not support audio-only recording, note taking, +broadcasting, visualization, translation, processing, or sharing scenarios in +the browser. diff --git a/PlaybackAudioCapture/audio-only-access-UX-flows.md b/PlaybackAudioCapture/audio-only-access-UX-flows.md new file mode 100644 index 00000000..540df3e2 --- /dev/null +++ b/PlaybackAudioCapture/audio-only-access-UX-flows.md @@ -0,0 +1,372 @@ +# Audio-only access UX flows + +These flows illustrate how playback-audio access could work. The exact +permission and selection UI is controlled by the browser. + +```text +[Website UI] Explains why the feature needs audio and starts the request. +[Browser UI] Authorizes access, selects a source, and controls ongoing use. +``` + +## Option 1: `getPlaybackMedia()` + +### Microphone and system audio + +```js +const microphoneStream = + await navigator.mediaDevices.getUserMedia({ + audio: true, + }); + +const playbackStream = + await navigator.mediaDevices.getPlaybackMedia({ + audio: { + source: "system", + }, + }); +``` + +```text ++---------------------------------------------------+ +| WEBSITE UI | +| | +| Start voice chat | +| | +| [Start] | ++--------------------------+------------------------+ + | + v + Website calls getUserMedia() + | + v ++---------------------------------------------------+ +| BROWSER UI | +| | +| example.com wants to use available microphones | +| | +| [Default microphone v] | +| | +| [ Allow while visiting the site ] | +| [ Allow this time ] | +| [ Never allow ] | ++--------------------------+------------------------+ + | + v + Microphone track returned + | + v ++---------------------------------------------------+ +| WEBSITE UI | +| | +| Improve voice quality | +| | +| Use audio playing on this device to reduce echo | +| and improve voice quality. | +| | +| [Enable] | ++--------------------------+------------------------+ + | + v + Website calls getPlaybackMedia() + | + v ++---------------------------------------------------+ +| BROWSER UI | +| | +| example.com wants to access audio playing on | +| this device. This may include audio from other | +| applications and websites. | +| | +| [ Allow while visiting the site ] | +| [ Allow this time ] | +| [ Never allow ] | ++--------------------------+------------------------+ + | + v + System playback audio track + | + v + Microphone + playback reference processing + | + v + Cleaned microphone output +``` + +The microphone and playback requests have separate permission decisions. If +the user declines playback access, the application can continue using the +microphone. No picker is needed because the playback request already identifies +system audio. + +### User-selected audio + +```js +const playbackStream = + await navigator.mediaDevices.getPlaybackMedia({ + audio: { + source: "user-selected", + }, + }); +``` + +```text ++---------------------------------------------------+ +| WEBSITE UI | +| | +| Transcribe meeting audio | +| | +| Choose the tab or application where your meeting | +| or lecture is playing. Only audio will be shared. | +| | +| [Choose audio source] | ++--------------------------+------------------------+ + | + v + Website calls getPlaybackMedia() + | + v ++---------------------------------------------------+ +| BROWSER UI | +| | +| Choose audio to share with notes.example | +| | +| Tabs | +| ( ) Google Meet | +| ( ) Lecture livestream | +| | +| Applications | +| ( ) Microsoft Teams | +| ( ) Media Player | +| | +| [Cancel] [Share audio] | ++--------------------------+------------------------+ + | + v + Selected playback audio track + | + v + Browser shows an ongoing indicator +``` + +The website explains the request but cannot see the available sources or grant +access. The browser owns the picker and returns a track only after the user +selects a source. + +## Option 2: extend `getUserMedia()` + +When a website requests only playback audio, the UX can follow the same system +or user-selected flow shown for `getPlaybackMedia()`. + +### Microphone and system audio + +```js +const stream = + await navigator.mediaDevices.getUserMedia({ + audio: true, + playbackAudio: { + source: "system", + }, + }); +``` + +Both microphone and playback audio are required by this combined request. The +browser could use one combined panel: + +```text ++---------------------------------------------------+ +| WEBSITE UI | +| | +| Use your microphone for the call and audio playing | +| on this device to improve voice quality. | +| | +| [Continue] | ++--------------------------+------------------------+ + | + v + Website calls getUserMedia() + | + v ++---------------------------------------------------+ +| BROWSER UI | +| | +| example.com wants to: | +| | +| Use available microphones (1) | +| Access audio playing on this device | +| | +| Microphone | +| [Default microphone v] | +| | +| This may include audio from other applications | +| and websites. | +| | +| [ Allow while visiting the site ] | +| [ Allow this time ] | +| [ Never allow ] | ++--------------------------+------------------------+ + | + +------------+------------+ + Never allow Allowed + | | + v v + Request denied Microphone track + + playback track +``` + +Because this is one combined request, both microphone and playback permission +must be granted for the request to succeed. + +### Microphone and user-selected audio + +```js +const stream = + await navigator.mediaDevices.getUserMedia({ + audio: true, + playbackAudio: { + source: "user-selected", + }, + }); +``` + +```text ++---------------------------------------------------+ +| WEBSITE UI | +| | +| Create notes from your microphone and meeting | +| audio. You will choose the meeting source next. | +| | +| [Continue] | ++--------------------------+------------------------+ + | + v + Website calls getUserMedia() + | + v ++---------------------------------------------------+ +| BROWSER UI | +| | +| example.com wants to use available microphones | +| | +| [Default microphone v] | +| | +| [ Allow while visiting the site ] | +| [ Allow this time ] | +| [ Never allow ] | ++--------------------------+------------------------+ + | + v ++---------------------------------------------------+ +| BROWSER UI | +| | +| Choose a tab or application whose audio to share | +| | +| [Browser tabs] [Applications] | +| | +| ( ) Team meeting - Video call | +| (o) Product demo - example.org | +| ( ) Presentation - Slides | +| | +| [Cancel] [Share] | ++--------------------------+------------------------+ + | + +------------+------------+ + | | + Picker cancelled Source selected + | | + v v + Request rejected Microphone track + + playback track +``` + +Because microphone and playback audio were requested together, cancelling the +picker rejects the combined request. + +### Independent microphone and playback requests + +A note-taking website may need only the selected meeting audio. If it also +wants microphone audio, it can request the two tracks independently so that +declining microphone access does not block transcription. + +```text ++---------------------------------------------------+ +| WEBSITE UI | +| | +| Create notes from your meeting audio. | +| You can also include your microphone. | +| | +| [Continue] | ++--------------------------+------------------------+ + | + v + Website calls getUserMedia() for microphone + | + v ++---------------------------------------------------+ +| BROWSER UI | +| | +| notes.example wants to use available microphones | +| | +| [Default microphone v] | +| | +| [ Allow while visiting the site ] | +| [ Allow this time ] | +| [ Never allow ] | ++--------------------------+------------------------+ + | + +------------+------------+ + | | + Permission denied Permission granted + | | + v v + Continue without Microphone track + microphone available + | | + +------------+------------+ + | + v + Website calls getUserMedia() for user-selected + playback audio + | + v ++---------------------------------------------------+ +| BROWSER UI | +| | +| Choose the tab or application where your meeting | +| is playing. Only audio will be shared. | +| | +| [Cancel] [Share audio] | ++--------------------------+------------------------+ + | + +------------+------------+ + | | + Cancel Share audio + | | + v v + No meeting audio Playback track + is available available + | + v + Website transcribes + playback audio and + includes the microphone + if it was allowed +``` + +Each request keeps the existing all-or-nothing behavior, but one request +failing does not affect the other. + +## Ongoing controls + +All successful flows end with browser-controlled ongoing-use UI: + +```text +Capture begins + | + v +Browser indicates that playback audio is being captured + | + v +User can stop capture +``` + +Playback capture ends when the user stops it, permission is revoked, the source +becomes unavailable, or the website stops the track. From 3002790be1254d96a9f19976cb1dbc018b01cff5 Mon Sep 17 00:00:00 2001 From: nishitha-burman Date: Mon, 24 Aug 2026 15:14:38 -0700 Subject: [PATCH 02/20] Rename playback audio explainer file Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- PlaybackAudioCapture/{README.md => explainer.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename PlaybackAudioCapture/{README.md => explainer.md} (100%) diff --git a/PlaybackAudioCapture/README.md b/PlaybackAudioCapture/explainer.md similarity index 100% rename from PlaybackAudioCapture/README.md rename to PlaybackAudioCapture/explainer.md From 9017327cbb63da58fe89d4e8a93759d972d901d5 Mon Sep 17 00:00:00 2001 From: nishitha-burman Date: Mon, 24 Aug 2026 15:16:03 -0700 Subject: [PATCH 03/20] Rename playback permission UX flows Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- PlaybackAudioCapture/explainer.md | 2 +- ...o-only-access-UX-flows.md => permission-ux-flows-mockups.md} | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) rename PlaybackAudioCapture/{audio-only-access-UX-flows.md => permission-ux-flows-mockups.md} (99%) diff --git a/PlaybackAudioCapture/explainer.md b/PlaybackAudioCapture/explainer.md index 4290bef5..806d8a64 100644 --- a/PlaybackAudioCapture/explainer.md +++ b/PlaybackAudioCapture/explainer.md @@ -322,7 +322,7 @@ Playback audio may include sensitive content from tabs, applications, or the device. See the -[permission UX flow mockups](audio-only-access-UX-flows.md). +[permission UX flow mockups](permission-ux-flows-mockups.md). - Access requires a user action and browser-controlled permission. - A user-selected request uses browser UI; the website cannot enumerate or diff --git a/PlaybackAudioCapture/audio-only-access-UX-flows.md b/PlaybackAudioCapture/permission-ux-flows-mockups.md similarity index 99% rename from PlaybackAudioCapture/audio-only-access-UX-flows.md rename to PlaybackAudioCapture/permission-ux-flows-mockups.md index 540df3e2..5da1806e 100644 --- a/PlaybackAudioCapture/audio-only-access-UX-flows.md +++ b/PlaybackAudioCapture/permission-ux-flows-mockups.md @@ -1,4 +1,4 @@ -# Audio-only access UX flows +# Permission UX Flow Mockups These flows illustrate how playback-audio access could work. The exact permission and selection UI is controlled by the browser. From 34ed498ab5bdc94f64e9a5ad692f305f327c16e8 Mon Sep 17 00:00:00 2001 From: nishitha-burman Date: Mon, 24 Aug 2026 15:17:12 -0700 Subject: [PATCH 04/20] Use singular UX flow filename Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- PlaybackAudioCapture/explainer.md | 2 +- ...ission-ux-flows-mockups.md => permission-ux-flow-mockups.md} | 0 2 files changed, 1 insertion(+), 1 deletion(-) rename PlaybackAudioCapture/{permission-ux-flows-mockups.md => permission-ux-flow-mockups.md} (100%) diff --git a/PlaybackAudioCapture/explainer.md b/PlaybackAudioCapture/explainer.md index 806d8a64..385bae26 100644 --- a/PlaybackAudioCapture/explainer.md +++ b/PlaybackAudioCapture/explainer.md @@ -322,7 +322,7 @@ Playback audio may include sensitive content from tabs, applications, or the device. See the -[permission UX flow mockups](permission-ux-flows-mockups.md). +[permission UX flow mockups](permission-ux-flow-mockups.md). - Access requires a user action and browser-controlled permission. - A user-selected request uses browser UI; the website cannot enumerate or diff --git a/PlaybackAudioCapture/permission-ux-flows-mockups.md b/PlaybackAudioCapture/permission-ux-flow-mockups.md similarity index 100% rename from PlaybackAudioCapture/permission-ux-flows-mockups.md rename to PlaybackAudioCapture/permission-ux-flow-mockups.md From 017304403f50f3cc23e201bac718b85d8c25ec43 Mon Sep 17 00:00:00 2001 From: nishitha-burman Date: Wed, 26 Aug 2026 22:27:48 -0700 Subject: [PATCH 05/20] Link playback audio UX mockups Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be1a166-cbd8-492b-8a72-521f78e49b86 --- PlaybackAudioCapture/explainer.md | 6 ++++-- PlaybackAudioCapture/permission-ux-flow-mockups.md | 4 ++++ 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/PlaybackAudioCapture/explainer.md b/PlaybackAudioCapture/explainer.md index 385bae26..68777fc1 100644 --- a/PlaybackAudioCapture/explainer.md +++ b/PlaybackAudioCapture/explainer.md @@ -321,8 +321,10 @@ peerConnection.addTrack(playbackTrack, playbackStream); Playback audio may include sensitive content from tabs, applications, or the device. -See the -[permission UX flow mockups](permission-ux-flow-mockups.md). +See the illustrative +[permission and source-selection flow mockups](permission-ux-flow-mockups.md) +and +[ongoing capture indicator and control mockups](https://nishitha-burman.github.io/playback-audio-ongoing-use-mockups/). - Access requires a user action and browser-controlled permission. - A user-selected request uses browser UI; the website cannot enumerate or diff --git a/PlaybackAudioCapture/permission-ux-flow-mockups.md b/PlaybackAudioCapture/permission-ux-flow-mockups.md index 5da1806e..7411dd53 100644 --- a/PlaybackAudioCapture/permission-ux-flow-mockups.md +++ b/PlaybackAudioCapture/permission-ux-flow-mockups.md @@ -356,6 +356,10 @@ failing does not affect the other. ## Ongoing controls +See the +[ongoing capture UX mockups](https://nishitha-burman.github.io/playback-audio-ongoing-use-mockups/) +for illustrative ideas showing how users could monitor and stop active capture. + All successful flows end with browser-controlled ongoing-use UI: ```text From d4de6b56a81791b993a7c1954c20619349a845d1 Mon Sep 17 00:00:00 2001 From: nishitha-burman Date: Fri, 28 Aug 2026 13:38:06 -0700 Subject: [PATCH 06/20] Host playback capture mockups on Edge Explainers Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be1a166-cbd8-492b-8a72-521f78e49b86 --- PlaybackAudioCapture/explainer.md | 4 +- .../ongoing-use-mockups/index.html | 1007 +++++++++++++++++ .../permission-ux-flow-mockups.md | 2 +- 3 files changed, 1010 insertions(+), 3 deletions(-) create mode 100644 PlaybackAudioCapture/ongoing-use-mockups/index.html diff --git a/PlaybackAudioCapture/explainer.md b/PlaybackAudioCapture/explainer.md index 68777fc1..d450706b 100644 --- a/PlaybackAudioCapture/explainer.md +++ b/PlaybackAudioCapture/explainer.md @@ -322,9 +322,9 @@ Playback audio may include sensitive content from tabs, applications, or the device. See the illustrative -[permission and source-selection flow mockups](permission-ux-flow-mockups.md) +[permission and source-selection flow mockups](https://microsoftedge.github.io/MSEdgeExplainers/PlaybackAudioCapture/permission-ux-flow-mockups.html) and -[ongoing capture indicator and control mockups](https://nishitha-burman.github.io/playback-audio-ongoing-use-mockups/). +[ongoing capture indicator and control mockups](https://microsoftedge.github.io/MSEdgeExplainers/PlaybackAudioCapture/ongoing-use-mockups/). - Access requires a user action and browser-controlled permission. - A user-selected request uses browser UI; the website cannot enumerate or diff --git a/PlaybackAudioCapture/ongoing-use-mockups/index.html b/PlaybackAudioCapture/ongoing-use-mockups/index.html new file mode 100644 index 00000000..aaedad00 --- /dev/null +++ b/PlaybackAudioCapture/ongoing-use-mockups/index.html @@ -0,0 +1,1007 @@ + + + + + + Playback Audio Capture - Ongoing Use Mockups + + + +
+

Ongoing playback-audio capture concepts

+

+ These mockups explore how browsers and operating systems could help users + remain aware of and stop playback-audio capture after permission is + granted. They are illustrative rather than proposed final UI and draw on + existing microphone, camera, and screen-sharing affordances in browsers. +

+ +
+

Existing screen-sharing UX patterns

+

+ Browser implementations vary. Current Microsoft Edge screen-sharing UI + is one example that informed the concepts below: +

+

+ If a browser tab is shared: the browser can identify + sharing in both places. The captured tab can say which site is receiving + it, while the capturing tab can identify the selected tab and provide a + stop control. +

+

+ If audio from an application or device is captured: a + persistent browser-controlled sharing indicator could keep the capture + visible and let the user stop it, even when another application is + active. +

+
+ +
+
+
+

Capturing browser tab

+

1. Capturing-site sharing bar

+

+ If audio from a tab, application, or device is captured, the + capturing tab could identify the source and provide a stop control. +

+
+
+
+
+
+ Meeting notes +
+
+ Meeting app + +
+
+
+ +
notes.example
+ +
+
+ + Sharing audio from meetingapp.example to this + tab + + +
+
+ Meeting notes + Listening to the selected audio source +
+
+
+
+ +
+
+

Captured browser tab

+

2. Captured-tab sharing bar

+

+ If a browser tab is the selected source, that tab could identify + which site is receiving its audio and provide a stop control. +

+
+
+
+
+
+ Meeting app + +
+
+ Meeting notes +
+
+
+ +
meetingapp.example
+ +
+
+ + This tab's audio is being shared with + notes.example + + +
+
+
Team meeting
+
+
+
+
+ +
+
+

Another active browser tab

+

3. Sharing status outside either site

+

+ If Site A captures Site B while the user views Site C, the active + tab could still identify the source and destination, provide sharing + controls, and show the browser's playback-capture indicator. +

+
+
+
+
+
+ Meeting notes +
+
+ Meeting app + +
+
+ Research +
+
+
+ +
research.example
+ +
+
+ + Sharing meetingapp.example to + notes.example + + +
+
+ Research + Another site is open while audio sharing continues +
+
+
+
+ +
+
+

Outside the browser

+

4. Persistent system-audio sharing control

+

+ If a site captures system audio, a browser-owned floating control + could remain visible when the user switches to another application. +

+
+
+
+
+ Document + +
+
+ Another application is active + + + +
+
+ + + notes.example is sharing your system audio. + + + Hide +
+
+
+
+ +
+
+

Browser address bar

+

5. Permission indicator and controls

+

+ Extend Edge's microphone permission panel to show active + playback-audio access and let the user manage future access. +

+
+
+
+
+
+ Meeting notes +
+
+
+ +
+ notes.example +
+ +
+
+ +
+
+ Meeting notes + Transcription in progress +
+
+

Playback audio allowed

+

+ This page is accessing audio playing on your device. +

+ + +
+ + +
+
+
+
+
+ +
+
+

Possible OS integration

+

6. System taskbar indicator

+

+ The OS could eventually expose playback capture similarly to + microphone use, with a distinct icon for audio being captured. +

+
+
+
+
+
+ + + + + + 2:15 PM
8/25/2026 +
+
+
+
+
+
+ + +
+ + diff --git a/PlaybackAudioCapture/permission-ux-flow-mockups.md b/PlaybackAudioCapture/permission-ux-flow-mockups.md index 7411dd53..e06c146a 100644 --- a/PlaybackAudioCapture/permission-ux-flow-mockups.md +++ b/PlaybackAudioCapture/permission-ux-flow-mockups.md @@ -357,7 +357,7 @@ failing does not affect the other. ## Ongoing controls See the -[ongoing capture UX mockups](https://nishitha-burman.github.io/playback-audio-ongoing-use-mockups/) +[ongoing capture UX mockups](https://microsoftedge.github.io/MSEdgeExplainers/PlaybackAudioCapture/ongoing-use-mockups/) for illustrative ideas showing how users could monitor and stop active capture. All successful flows end with browser-controlled ongoing-use UI: From 6f3bb0aa068684b73c8940e6627ab4814eb86157 Mon Sep 17 00:00:00 2001 From: nishitha-burman Date: Fri, 28 Aug 2026 13:45:18 -0700 Subject: [PATCH 07/20] Keep permission UX flow link relative Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be1a166-cbd8-492b-8a72-521f78e49b86 --- PlaybackAudioCapture/explainer.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/PlaybackAudioCapture/explainer.md b/PlaybackAudioCapture/explainer.md index d450706b..de554807 100644 --- a/PlaybackAudioCapture/explainer.md +++ b/PlaybackAudioCapture/explainer.md @@ -322,7 +322,7 @@ Playback audio may include sensitive content from tabs, applications, or the device. See the illustrative -[permission and source-selection flow mockups](https://microsoftedge.github.io/MSEdgeExplainers/PlaybackAudioCapture/permission-ux-flow-mockups.html) +[permission and source-selection flow mockups](permission-ux-flow-mockups.md) and [ongoing capture indicator and control mockups](https://microsoftedge.github.io/MSEdgeExplainers/PlaybackAudioCapture/ongoing-use-mockups/). From a5406dce1f6b65e74378c21c4b1ce4e45aeb5b44 Mon Sep 17 00:00:00 2001 From: nishitha-burman Date: Mon, 31 Aug 2026 13:50:04 -0700 Subject: [PATCH 08/20] List playback audio capture explainer in root README Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be1a166-cbd8-492b-8a72-521f78e49b86 --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index ea499038..7db4fc3e 100644 --- a/README.md +++ b/README.md @@ -82,6 +82,7 @@ we move them into the [Alumni section](#alumni-) below. | [Calling Notifications](Notifications/notifications_actions_customization.md) | ![GitHub issues by-label](https://img.shields.io/github/issues/MicrosoftEdge/MSEdgeExplainers/Calling%20Notifications?label=issues) | [New issue...](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/new?template=calling-notifications.md) | Web Applications | | [Split Tab Navigation](SplitTab/NavigationInSplitTab/explainer.md) | ![GitHub issues by-label](https://img.shields.io/github/issues/MicrosoftEdge/MSEdgeExplainers/Split%20Tab?label=issues) | [New Issue...](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/new?template=split-tab.md) | Web Applications | | [Set Default Audio Output Device](SetPreferredSinkId/explainer.md) | ![GitHub issues by-label](https://img.shields.io/github/issues/MicrosoftEdge/MSEdgeExplainers/SetPreferredSinkId?label=issues) | [New issue...](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/new?assignees=sunggch&labels=SetPreferredSinkId&title=%5BSetPreferredSinkId%5D+Issue) | WebRTC | +| [Playback Audio Capture Without Screen Sharing](PlaybackAudioCapture/explainer.md) | [Discussion](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12) | [New issue...](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/new?title=%5BPlaybackAudioCapture%5D+) | WebRTC | | [Handwriting attribute](Handwriting/explainer.md) | ![GitHub issues by-label](https://img.shields.io/github/issues/MicrosoftEdge/MSEdgeExplainers/Handwriting?label=issues) | [New issue...](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/new?assignees=adettenb&labels=Handwriting&title=%5BHandwriting%5D+Issue) | HTML | | [Mulitple Stylesheets Per File (@sheet)](AtSheet/explainer.md) | ![GitHub issues by-label](https://img.shields.io/github/issues/MicrosoftEdge/MSEdgeExplainers/AtSheet?label=issues) | [New Issue...](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/new?assignees=aluhrs13&labels=AtSheet&title=%5B%40sheet%5D+%3CTITLE+HERE%3E) | CSS | | [Link Rel Local Reference](LocalReferenceLinkRel/explainer.md) | ![GitHub issues by-label](https://img.shields.io/github/issues/MicrosoftEdge/MSEdgeExplainers/LocalReferenceLinkRel?label=issues) | [New Issue...](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/new?assignees=KurtCattiSchmidt&labels=LRLR&title=%5BLRLR%5D+%3CTITLE+HERE%3E) | HTML | From b7325b453a83678f092ce7e1c35377325e6e9e8a Mon Sep 17 00:00:00 2001 From: nishitha-burman Date: Mon, 31 Aug 2026 15:09:32 -0700 Subject: [PATCH 09/20] Add Status of This Document section Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be1a166-cbd8-492b-8a72-521f78e49b86 --- PlaybackAudioCapture/explainer.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/PlaybackAudioCapture/explainer.md b/PlaybackAudioCapture/explainer.md index de554807..2f723ab1 100644 --- a/PlaybackAudioCapture/explainer.md +++ b/PlaybackAudioCapture/explainer.md @@ -9,6 +9,18 @@ - [Proposal issue and discussion](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12) - [MSEdgeExplainers issue tracker](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues) +## Status of this Document + +This document is a starting point for engaging the community and standards +bodies in developing collaborative solutions fit for standardization. As the +solutions described here progress along the standards track, we will retain +this document as an archive and update this section with the current standards +venue and content location. + +- This document status: **Active** +- Expected venue: [WebRTC Working Group](https://www.w3.org/groups/wg/webrtc/) +- Current version: this document + ## Introduction Today, websites cannot capture audio playing in a browser tab, application, or From 4a4b1fe8d531e7e61224061dbc88e1a0f7a94c4f Mon Sep 17 00:00:00 2001 From: nishitha-burman Date: Mon, 31 Aug 2026 15:13:57 -0700 Subject: [PATCH 10/20] Update explainer authors and acknowledgements Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be1a166-cbd8-492b-8a72-521f78e49b86 --- PlaybackAudioCapture/explainer.md | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/PlaybackAudioCapture/explainer.md b/PlaybackAudioCapture/explainer.md index 2f723ab1..1d2884ed 100644 --- a/PlaybackAudioCapture/explainer.md +++ b/PlaybackAudioCapture/explainer.md @@ -1,8 +1,9 @@ # Playback Audio Capture Without Screen Sharing -## Author +## Authors - [Nishitha Dey](https://github.com/nishitha-burman) +- [Steve Becker](https://github.com/SteveBeckerMSFT) ## Participate @@ -409,3 +410,13 @@ Better browser echo cancellation would improve calls and should be explored separately. It would not support audio-only recording, note taking, broadcasting, visualization, translation, processing, or sharing scenarios in the browser. + +## Acknowledgements + +This proposal builds on an +[earlier exploration of playback-reference audio](https://github.com/MicrosoftEdge/MSEdgeExplainers/pull/1377) +by [Huseyin Ozcan](https://github.com/huozcan-ms), +[Max Solodovnikov](https://github.com/solmaks), Hong Sodoma, and Vinod Prakash. + +Thanks to [Erik Anderson](https://github.com/erik-anderson) for reviewing and +providing feedback on this explainer. From d96b7c9f8ea8cd4472bca71e196fbad992bc0234 Mon Sep 17 00:00:00 2001 From: nishitha-burman Date: Mon, 31 Aug 2026 15:28:06 -0700 Subject: [PATCH 11/20] Add Stakeholder Feedback / Opposition section Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be1a166-cbd8-492b-8a72-521f78e49b86 --- PlaybackAudioCapture/explainer.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/PlaybackAudioCapture/explainer.md b/PlaybackAudioCapture/explainer.md index 1d2884ed..6bdadf23 100644 --- a/PlaybackAudioCapture/explainer.md +++ b/PlaybackAudioCapture/explainer.md @@ -411,6 +411,20 @@ separately. It would not support audio-only recording, note taking, broadcasting, visualization, translation, processing, or sharing scenarios in the browser. +## Stakeholder Feedback / Opposition + +This proposal is being shared to gather feedback from browser vendors, web +developers, and standards participants. + +Developer interest and use cases are documented in +[the existing W3C discussion](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12). +No browser vendor has yet expressed a formal position on either API option. + +Prior discussions have raised questions about platform support, privacy and +security, source scope, permission UX, and whether playback-audio capture +should extend an existing API or use a new API. These remain open areas for +feedback and discussion. + ## Acknowledgements This proposal builds on an From f4e532af2c8587a57e7e1023b1fb057156c42a1b Mon Sep 17 00:00:00 2001 From: nishitha-burman Date: Tue, 1 Sep 2026 10:21:06 -0700 Subject: [PATCH 12/20] Add explainer references Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be1a166-cbd8-492b-8a72-521f78e49b86 --- PlaybackAudioCapture/explainer.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/PlaybackAudioCapture/explainer.md b/PlaybackAudioCapture/explainer.md index 6bdadf23..4ba7b69f 100644 --- a/PlaybackAudioCapture/explainer.md +++ b/PlaybackAudioCapture/explainer.md @@ -425,7 +425,11 @@ security, source scope, permission UX, and whether playback-audio capture should extend an existing API or use a new API. These remain open areas for feedback and discussion. -## Acknowledgements +## References & Acknowledgements + +- [Media Capture and Streams](https://w3c.github.io/mediacapture-main/) +- [Screen Capture](https://w3c.github.io/mediacapture-screen-share/) +- [Media Capture Screen Share Extensions issue #12](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12) This proposal builds on an [earlier exploration of playback-reference audio](https://github.com/MicrosoftEdge/MSEdgeExplainers/pull/1377) From e52a28bfb141bfe447f189e79e5865a679945229 Mon Sep 17 00:00:00 2001 From: nishitha-burman Date: Tue, 1 Sep 2026 11:28:25 -0700 Subject: [PATCH 13/20] Add source model alignment question Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be1a166-cbd8-492b-8a72-521f78e49b86 --- PlaybackAudioCapture/explainer.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/PlaybackAudioCapture/explainer.md b/PlaybackAudioCapture/explainer.md index 4ba7b69f..8c24b11e 100644 --- a/PlaybackAudioCapture/explainer.md +++ b/PlaybackAudioCapture/explainer.md @@ -352,6 +352,8 @@ becomes unavailable, or the website stops the track. ## Open questions +- How should the playback-audio source model align with the existing + `systemAudio` and `windowAudio` options in `getDisplayMedia()`? - What timing guarantees are needed when microphone and playback audio are processed together? - What sources and processing stages are included in system audio on each From e145296d47abe2ca064de65c2f40bca1e08ec39e Mon Sep 17 00:00:00 2001 From: nishitha-burman Date: Tue, 1 Sep 2026 11:49:00 -0700 Subject: [PATCH 14/20] Clarify source model alignment question Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be1a166-cbd8-492b-8a72-521f78e49b86 --- PlaybackAudioCapture/explainer.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/PlaybackAudioCapture/explainer.md b/PlaybackAudioCapture/explainer.md index 8c24b11e..871f87e7 100644 --- a/PlaybackAudioCapture/explainer.md +++ b/PlaybackAudioCapture/explainer.md @@ -352,8 +352,13 @@ becomes unavailable, or the website stops the track. ## Open questions -- How should the playback-audio source model align with the existing - `systemAudio` and `windowAudio` options in `getDisplayMedia()`? +- The existing `systemAudio` and `windowAudio` options in `getDisplayMedia()` + express preferences about which audio choices the browser offers alongside + display capture (see + [Screen Capture](https://w3c.github.io/mediacapture-screen-share/#displaymediastreamoptions)), + while the source values proposed here identify the type of audio-only source + being requested. Given these different roles, how, if at all, should the + playback-audio source model align with their terminology or semantics? - What timing guarantees are needed when microphone and playback audio are processed together? - What sources and processing stages are included in system audio on each From 58b79a0034be34a29d9dbfb668c1c9dfa477ce25 Mon Sep 17 00:00:00 2001 From: nishitha-burman Date: Tue, 1 Sep 2026 12:15:31 -0700 Subject: [PATCH 15/20] Clarify playback track retrieval options Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be1a166-cbd8-492b-8a72-521f78e49b86 --- PlaybackAudioCapture/explainer.md | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/PlaybackAudioCapture/explainer.md b/PlaybackAudioCapture/explainer.md index 871f87e7..10ddf351 100644 --- a/PlaybackAudioCapture/explainer.md +++ b/PlaybackAudioCapture/explainer.md @@ -263,18 +263,11 @@ website that can use either microphone or playback audio can request them separately and handle each result independently. Whether a future version should support an explicit best-effort playback request remains open. -Both tracks are audio tracks, so the proposal must give developers a way to -tell them apart. One possible design is a `sourceType` value: - -```js -const microphoneTrack = - stream.getAudioTracks().find(track => - track.getSettings().sourceType === "microphone"); - -const playbackTrack = - stream.getAudioTracks().find(track => - track.getSettings().sourceType === "playback"); -``` +Option 2 returns microphone and playback audio tracks in the same +`MediaStream`, so the application needs a reliable way to retrieve the +playback track. Possible approaches include playback-specific track metadata +(such as a setting or attribute), a specialized track subtype, or a dedicated +`MediaStream` getter. The appropriate mechanism remains open. Applications that process microphone and playback audio together need timing information that can be compared reliably. The exact clock, timestamp, drift, @@ -289,7 +282,7 @@ audio being played by the device. | Option | Benefit | Tradeoff | | --- | --- | --- | | New `getPlaybackMedia()` | Clearly describes playback capture and works independently | A communication application makes separate microphone and playback requests | -| Extend `getUserMedia()` | Can request microphone and playback audio together | Changes the meaning of `getUserMedia()`, may combine a permission prompt with a source picker, and requires a way to identify the two audio tracks | +| Extend `getUserMedia()` | Can request microphone and playback audio together | Changes the meaning of `getUserMedia()`, may combine a permission prompt with a source picker, and requires a reliable way to retrieve the playback track | ### Source selection @@ -359,6 +352,9 @@ becomes unavailable, or the website stops the track. while the source values proposed here identify the type of audio-only source being requested. Given these different roles, how, if at all, should the playback-audio source model align with their terminology or semantics? +- How should an application reliably retrieve a playback track returned + alongside a microphone track—for example, through playback-specific track + metadata, a specialized track subtype, or a dedicated `MediaStream` getter? - What timing guarantees are needed when microphone and playback audio are processed together? - What sources and processing stages are included in system audio on each From 79d652d66e65634e50dcfcdd1c8f4dd0df23ceca Mon Sep 17 00:00:00 2001 From: nishitha-burman Date: Tue, 1 Sep 2026 13:20:23 -0700 Subject: [PATCH 16/20] Add playback track constraints question Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be1a166-cbd8-492b-8a72-521f78e49b86 --- PlaybackAudioCapture/explainer.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/PlaybackAudioCapture/explainer.md b/PlaybackAudioCapture/explainer.md index 10ddf351..3a160482 100644 --- a/PlaybackAudioCapture/explainer.md +++ b/PlaybackAudioCapture/explainer.md @@ -355,6 +355,9 @@ becomes unavailable, or the website stops the track. - How should an application reliably retrieve a playback track returned alongside a microphone track—for example, through playback-specific track metadata, a specialized track subtype, or a dedicated `MediaStream` getter? +- Which audio constraints and capabilities should playback tracks support, and + how should `getCapabilities()`, `getSettings()`, `getConstraints()`, and + `applyConstraints()` behave for them? - What timing guarantees are needed when microphone and playback audio are processed together? - What sources and processing stages are included in system audio on each From 831f7f96aa57d511b7c562a5ebc62e35121e97f3 Mon Sep 17 00:00:00 2001 From: nishitha-burman Date: Tue, 1 Sep 2026 13:23:32 -0700 Subject: [PATCH 17/20] Clarify playback permission questions Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be1a166-cbd8-492b-8a72-521f78e49b86 --- PlaybackAudioCapture/explainer.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/PlaybackAudioCapture/explainer.md b/PlaybackAudioCapture/explainer.md index 3a160482..42224b8e 100644 --- a/PlaybackAudioCapture/explainer.md +++ b/PlaybackAudioCapture/explainer.md @@ -362,8 +362,9 @@ becomes unavailable, or the website stops the track. processed together? - What sources and processing stages are included in system audio on each platform? -- How should playback permission integrate with existing browser permission - controls and capability elements? +- Should playback-audio permission persist or require a new user decision for + every request? Should permission behavior differ between system audio and a + user-selected source? ## Alternatives considered From d55af10eb2123343d91db8b23c2866cfc2266a84 Mon Sep 17 00:00:00 2001 From: nishitha-burman Date: Tue, 1 Sep 2026 13:50:05 -0700 Subject: [PATCH 18/20] Expand playback audio open questions Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be1a166-cbd8-492b-8a72-521f78e49b86 --- PlaybackAudioCapture/explainer.md | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/PlaybackAudioCapture/explainer.md b/PlaybackAudioCapture/explainer.md index 42224b8e..5037394b 100644 --- a/PlaybackAudioCapture/explainer.md +++ b/PlaybackAudioCapture/explainer.md @@ -355,9 +355,19 @@ becomes unavailable, or the website stops the track. - How should an application reliably retrieve a playback track returned alongside a microphone track—for example, through playback-specific track metadata, a specialized track subtype, or a dedicated `MediaStream` getter? -- Which audio constraints and capabilities should playback tracks support, and - how should `getCapabilities()`, `getSettings()`, `getConstraints()`, and - `applyConstraints()` behave for them? +- Which audio constraints and capabilities (see + [Screen Capture](https://www.w3.org/TR/screen-capture/#dom-displaymediastreamoptions-audio)) + should playback tracks support, and how should `getCapabilities()`, + `getSettings()`, `getConstraints()`, and `applyConstraints()` behave for + them? +- How should self-capture work for playback audio? Should the requesting tab + appear in the source picker, should system capture include audio produced by + the requesting site, and should the proposal align with existing mechanisms + such as + [`selfBrowserSurface`](https://w3c.github.io/mediacapture-screen-share/#dom-displaymediastreamoptions-selfbrowsersurface), + [`restrictOwnAudio`](https://w3c.github.io/mediacapture-screen-share/#def-constraint-restrictOwnAudio), + or + [`suppressLocalAudioPlayback`](https://w3c.github.io/mediacapture-screen-share/#def-constraint-suppressLocalAudioPlayback)? - What timing guarantees are needed when microphone and playback audio are processed together? - What sources and processing stages are included in system audio on each @@ -365,6 +375,9 @@ becomes unavailable, or the website stops the track. - Should playback-audio permission persist or require a new user decision for every request? Should permission behavior differ between system audio and a user-selected source? +- How should playback-audio permission integrate with existing browser + permission controls and the proposed + [``, ``, and `` capability elements](https://chromestatus.com/feature/5153829504024576)? ## Alternatives considered From 3549d97e53b6016f79055c02f169869a4940672b Mon Sep 17 00:00:00 2001 From: nishitha-burman Date: Tue, 1 Sep 2026 13:57:57 -0700 Subject: [PATCH 19/20] Organize playback audio open questions Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be1a166-cbd8-492b-8a72-521f78e49b86 --- PlaybackAudioCapture/explainer.md | 45 ++++++++++++++++++++----------- 1 file changed, 29 insertions(+), 16 deletions(-) diff --git a/PlaybackAudioCapture/explainer.md b/PlaybackAudioCapture/explainer.md index 5037394b..fdd75f8d 100644 --- a/PlaybackAudioCapture/explainer.md +++ b/PlaybackAudioCapture/explainer.md @@ -345,21 +345,30 @@ becomes unavailable, or the website stops the track. ## Open questions +### API design and integration + - The existing `systemAudio` and `windowAudio` options in `getDisplayMedia()` express preferences about which audio choices the browser offers alongside - display capture (see - [Screen Capture](https://w3c.github.io/mediacapture-screen-share/#displaymediastreamoptions)), + [display capture](https://w3c.github.io/mediacapture-screen-share/#displaymediastreamoptions), while the source values proposed here identify the type of audio-only source being requested. Given these different roles, how, if at all, should the playback-audio source model align with their terminology or semantics? - How should an application reliably retrieve a playback track returned alongside a microphone track—for example, through playback-specific track - metadata, a specialized track subtype, or a dedicated `MediaStream` getter? -- Which audio constraints and capabilities (see - [Screen Capture](https://www.w3.org/TR/screen-capture/#dom-displaymediastreamoptions-audio)) - should playback tracks support, and how should `getCapabilities()`, - `getSettings()`, `getConstraints()`, and `applyConstraints()` behave for - them? + metadata, a specialized track subtype, or a dedicated + [`MediaStream`](https://w3c.github.io/mediacapture-main/#dom-mediastream) + getter? +- Which audio constraints and capabilities should playback tracks support, and + how should the + [constrainable pattern](https://w3c.github.io/mediacapture-main/#constrainable-interface), + including `getCapabilities()`, `getSettings()`, `getConstraints()`, and + `applyConstraints()`, apply to them? +- How should playback-audio permission integrate with existing browser + permission controls and the proposed + [``, ``, and `` capability elements](https://chromestatus.com/feature/5153829504024576)? + +### Permission and privacy + - How should self-capture work for playback audio? Should the requesting tab appear in the source picker, should system capture include audio produced by the requesting site, and should the proposal align with existing mechanisms @@ -368,16 +377,20 @@ becomes unavailable, or the website stops the track. [`restrictOwnAudio`](https://w3c.github.io/mediacapture-screen-share/#def-constraint-restrictOwnAudio), or [`suppressLocalAudioPlayback`](https://w3c.github.io/mediacapture-screen-share/#def-constraint-suppressLocalAudioPlayback)? +- Should playback-audio permission persist like + [`getUserMedia()`](https://w3c.github.io/mediacapture-main/#dom-mediadevices-getusermedia) + or require a new user decision for every request like + [`getDisplayMedia()`](https://w3c.github.io/mediacapture-screen-share/#dom-mediadevices-getdisplaymedia)? + Should permission behavior differ between system audio and a user-selected + source? + +### Media and platform behavior + - What timing guarantees are needed when microphone and playback audio are processed together? -- What sources and processing stages are included in system audio on each - platform? -- Should playback-audio permission persist or require a new user decision for - every request? Should permission behavior differ between system audio and a - user-selected source? -- How should playback-audio permission integrate with existing browser - permission controls and the proposed - [``, ``, and `` capability elements](https://chromestatus.com/feature/5153829504024576)? +- What sources and processing stages should be included in + [system audio](https://w3c.github.io/mediacapture-screen-share/#dom-displaymediastreamoptions-systemaudio) + on each platform? ## Alternatives considered From 85a433e8112b796d9401f39161099f9ed4e753b1 Mon Sep 17 00:00:00 2001 From: nishitha-burman Date: Tue, 1 Sep 2026 14:01:58 -0700 Subject: [PATCH 20/20] Refine open question categories Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3be1a166-cbd8-492b-8a72-521f78e49b86 --- PlaybackAudioCapture/explainer.md | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/PlaybackAudioCapture/explainer.md b/PlaybackAudioCapture/explainer.md index fdd75f8d..a746c499 100644 --- a/PlaybackAudioCapture/explainer.md +++ b/PlaybackAudioCapture/explainer.md @@ -345,7 +345,7 @@ becomes unavailable, or the website stops the track. ## Open questions -### API design and integration +### API and media behavior - The existing `systemAudio` and `windowAudio` options in `getDisplayMedia()` express preferences about which audio choices the browser offers alongside @@ -363,12 +363,17 @@ becomes unavailable, or the website stops the track. [constrainable pattern](https://w3c.github.io/mediacapture-main/#constrainable-interface), including `getCapabilities()`, `getSettings()`, `getConstraints()`, and `applyConstraints()`, apply to them? -- How should playback-audio permission integrate with existing browser - permission controls and the proposed - [``, ``, and `` capability elements](https://chromestatus.com/feature/5153829504024576)? +- What timing guarantees are needed when microphone and playback audio are + processed together? +- What sources and processing stages should be included in + [system audio](https://w3c.github.io/mediacapture-screen-share/#dom-displaymediastreamoptions-systemaudio) + on each platform? ### Permission and privacy +- How should playback-audio permission integrate with existing browser + permission controls and the proposed + [``, ``, and `` capability elements](https://chromestatus.com/feature/5153829504024576)? - How should self-capture work for playback audio? Should the requesting tab appear in the source picker, should system capture include audio produced by the requesting site, and should the proposal align with existing mechanisms @@ -384,14 +389,6 @@ becomes unavailable, or the website stops the track. Should permission behavior differ between system audio and a user-selected source? -### Media and platform behavior - -- What timing guarantees are needed when microphone and playback audio are - processed together? -- What sources and processing stages should be included in - [system audio](https://w3c.github.io/mediacapture-screen-share/#dom-displaymediastreamoptions-systemaudio) - on each platform? - ## Alternatives considered ### Allow audio-only `getDisplayMedia()`