diff --git a/PlaybackAudioCapture/explainer.md b/PlaybackAudioCapture/explainer.md new file mode 100644 index 00000000..a746c499 --- /dev/null +++ b/PlaybackAudioCapture/explainer.md @@ -0,0 +1,470 @@ +# Playback Audio Capture Without Screen Sharing + +## Authors + +- [Nishitha Dey](https://github.com/nishitha-burman) +- [Steve Becker](https://github.com/SteveBeckerMSFT) + +## Participate + +- [Proposal issue and discussion](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12) +- [MSEdgeExplainers issue tracker](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues) + +## Status of this Document + +This document is a starting point for engaging the community and standards +bodies in developing collaborative solutions fit for standardization. As the +solutions described here progress along the standards track, we will retain +this document as an archive and update this section with the current standards +venue and content location. + +- This document status: **Active** +- Expected venue: [WebRTC Working Group](https://www.w3.org/groups/wg/webrtc/) +- Current version: this document + +## Introduction + +Today, websites cannot capture audio playing in a browser tab, application, or +across a device without also asking the user to share their screen. + +`getUserMedia()` captures microphone and camera input, while +`getDisplayMedia()` is designed for sharing a screen, window, or browser tab +and requires video. As a workaround, a website must start screen sharing, use +the audio if it is available, and discard the video track. Developers have +reported that this can add +[unnecessary processing](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12#issuecomment-1960941133), +[hurt performance](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12#issuecomment-1960941077), +and +[confuse users](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12#issuecomment-1960941098) +who are asked to share visual content for an audio-only feature. + +Even if the website immediately stops the video track, the user must approve +screen sharing first. This exposes more information than an audio-only feature +needs and makes the permission request harder to understand. + +This document proposes allowing users to share **playback audio**—audio played +by a browser tab, application, or the device—without sharing screen pixels. A +website can request system audio or ask the browser to let the user select a +source. The browser controls permission and source selection, and the website +receives a standard audio track that it can record, process, or share. + +This document considers two API options: + +1. Add a new `getPlaybackMedia()` method. +2. Extend `getUserMedia()` with a playback-audio option. + +Both options return an audio track that works with existing web APIs. + +## User scenarios + +### Note taking, captions, and translation + +A note-taking application could record audio from a +[meeting or lecture](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12#issuecomment-1960941133) +playing in another browser tab or application, without also capturing video. +The same audio could be transcribed to create captions, translations, or +searchable notes. +A standalone note-taking application could also combine audio from a selected +meeting source with the user's microphone for +[AI transcription](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12#issuecomment-4449563056). + +### Live production and audio sharing + +A +[production application](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12#issuecomment-1960941143) +could mix audio from guests, music players, games, soundboards, and other +applications for a livestream or recording. It could also share an +application's audio during a call without sharing its window. + +Native tools already support these workflows. +[OBS Studio](https://obsproject.com/kb/application-audio-capture-guide) +captures individual applications as separate audio sources, while +[Voicemeeter](https://vb-audio.com/Voicemeeter/) mixes and routes audio from +multiple sources. +[Rocket Broadcaster](https://www.rocketbroadcaster.com/) captures and sends +live audio to a streaming service. These tools show an established need for +capturing and combining audio without requiring video. + +### Audio analysis and processing + +A website could use playback audio to create visualizations or apply audio +effects chosen by the user. + +The W3C discussion includes a +[developer building an audio visualizer](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12#issuecomment-1960941124) +who found that screen capture hurt performance. + +### Real-time communication + +In a call, audio played by the device—such as far-end participants, shared +media, or notifications—can leak into the microphone. Other participants may +hear this as echo, and the extra audio can interfere with the application's +voice processing. + +The same problem affects people who use a screen reader or Narrator. Assistive +audio played by the device can be picked up by the microphone and sent to +other participants as if it came from the user. + +Call-center or webphone software could +[record call audio](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12#issuecomment-1960941133) +together with the operator's microphone without capturing video or integrating +with the calling application. + +With a separate playback track, the communication application can compare what +the device played with what the microphone captured, apply its own echo +cancellation and voice processing, and send only the processed microphone +track to other participants. Camera capture remains separate and continues to +use `getUserMedia()`. + +### AI voice chat + +An AI voice-chat application could use playback audio to avoid treating its +own spoken response or Narrator as a new user request and to detect when the +user interrupts. A related +[W3C comment](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12#issuecomment-4668855578) +argues that AI agents should not require video access when they need only +audio. + +## Goals + +- Enable users to share audio from a supported tab, application, or device + without sharing screen. +- Use browser UI to let users authorize playback capture, select a source when + needed, see when capture is active, and stop sharing. +- Return a standard audio `MediaStreamTrack` that websites can record, process, + or share using existing web APIs. +- Support applications that process playback and microphone audio together. + +## Non-goals + +- Allowing playback audio to be captured without the user's permission. +- Guaranteeing support for tab, application, and device audio on every + platform. +- Bypassing restrictions that prevent protected music or video from being + captured. + +## Proposed Approach + +Both options let a website request system audio or a source selected by the +user without requiring video. Option 1 returns a playback-only `MediaStream`. +Option 2 adds a playback track alongside any other requested tracks. + +### Option 1: Add `getPlaybackMedia()` + +Add a method for requesting playback audio: + +```js +const playbackStream = + await navigator.mediaDevices.getPlaybackMedia({ + audio: { + source: "user-selected", + }, + }); + +const playbackTrack = playbackStream.getAudioTracks()[0]; +``` + +A note-taking app could use this to let the user select the tab where a +meeting or lecture is playing. + +This option clearly separates playback capture from microphone, camera, and +screen capture. + +Microphone and camera capture continue to use `getUserMedia()`, including its +existing device and track constraints. + +#### Use with a microphone + +A note-taking, communication, or AI voice-chat application first gets the +microphone: + +```js +const microphoneStream = + await navigator.mediaDevices.getUserMedia({ + audio: true, + }); + +const microphoneTrack = microphoneStream.getAudioTracks()[0]; +``` + +A note-taking application could then let the user select the meeting or +lecture audio: + +```js +const playbackStream = + await navigator.mediaDevices.getPlaybackMedia({ + audio: { + source: "user-selected", + }, + }); +``` + +A communication or AI voice-chat application could instead request system +audio: + +```js +const playbackStream = + await navigator.mediaDevices.getPlaybackMedia({ + audio: { + source: "system", + }, + }); +``` + +The two requests are independent. If the user declines playback capture, the +application can continue using the microphone. + +### Option 2: Extend `getUserMedia()` + +Add a `playbackAudio` option to `getUserMedia()`. + +A note-taking website that only needs playback audio could let the user choose +the tab or application where a meeting or lecture is playing: + +```js +const playbackStream = + await navigator.mediaDevices.getUserMedia({ + playbackAudio: { + source: "user-selected", + }, + }); +``` + +A communication or AI voice-chat application could request microphone and +playback audio together: + +```js +const stream = + await navigator.mediaDevices.getUserMedia({ + audio: true, + playbackAudio: { + source: "system", + }, + }); +``` + +The existing `audio` and `video` members and their constraints remain +unchanged. A website that also needs a camera can request all three: + +```js +const stream = + await navigator.mediaDevices.getUserMedia({ + audio: true, + video: true, + playbackAudio: { + source: "system", + }, + }); +``` + +As with `getUserMedia()` today, every requested media type is required. If +microphone, camera, or playback access fails, the request is rejected. A +website that can use either microphone or playback audio can request them +separately and handle each result independently. Whether a future version +should support an explicit best-effort playback request remains open. + +Option 2 returns microphone and playback audio tracks in the same +`MediaStream`, so the application needs a reliable way to retrieve the +playback track. Possible approaches include playback-specific track metadata +(such as a setting or attribute), a specialized track subtype, or a dedicated +`MediaStream` getter. The appropriate mechanism remains open. + +Applications that process microphone and playback audio together need timing +information that can be compared reliably. The exact clock, timestamp, drift, +and latency requirements remain open. + +This option reuses an existing capture method, but changes `getUserMedia()` +from an API for microphone and camera input into one that can also capture +audio being played by the device. + +### Comparison + +| Option | Benefit | Tradeoff | +| --- | --- | --- | +| New `getPlaybackMedia()` | Clearly describes playback capture and works independently | A communication application makes separate microphone and playback requests | +| Extend `getUserMedia()` | Can request microphone and playback audio together | Changes the meaning of `getUserMedia()`, may combine a permission prompt with a source picker, and requires a reliable way to retrieve the playback track | + +### Source selection + +Both options can support two source modes: + +| Source value | Browser behavior | +| --- | --- | +| `"system"` | The combined audio rendered by the device that the browser and platform can expose | +| `"user-selected"` | Let the user choose a tab or application through browser-controlled UI | + +The source names are illustrative and open to discussion. The website does not +receive a list of running applications or choose a tab or application without +the user. The exact sources included in system audio may vary by browser and +operating system. + +### Use with existing APIs + +The website can use the returned track with existing APIs. + +Record it: + +```js +const recorder = new MediaRecorder(playbackStream); +recorder.start(); +``` + +Process or analyze it: + +```js +const context = new AudioContext(); +const source = context.createMediaStreamSource(playbackStream); +``` + +Share it through an existing WebRTC connection: + +```js +peerConnection.addTrack(playbackTrack, playbackStream); +``` + +## Privacy and security + +Playback audio may include sensitive content from tabs, applications, or the +device. + +See the illustrative +[permission and source-selection flow mockups](permission-ux-flow-mockups.md) +and +[ongoing capture indicator and control mockups](https://microsoftedge.github.io/MSEdgeExplainers/PlaybackAudioCapture/ongoing-use-mockups/). + +- Access requires a user action and browser-controlled permission. +- A user-selected request uses browser UI; the website cannot enumerate or + silently choose applications. +- The browser clearly indicates when playback audio is being captured and lets + the user stop it. +- Browsers may exclude protected content or sources the platform cannot + capture. + +Playback capture ends when the user stops it, permission is revoked, the source +becomes unavailable, or the website stops the track. + +## Open questions + +### API and media behavior + +- The existing `systemAudio` and `windowAudio` options in `getDisplayMedia()` + express preferences about which audio choices the browser offers alongside + [display capture](https://w3c.github.io/mediacapture-screen-share/#displaymediastreamoptions), + while the source values proposed here identify the type of audio-only source + being requested. Given these different roles, how, if at all, should the + playback-audio source model align with their terminology or semantics? +- How should an application reliably retrieve a playback track returned + alongside a microphone track—for example, through playback-specific track + metadata, a specialized track subtype, or a dedicated + [`MediaStream`](https://w3c.github.io/mediacapture-main/#dom-mediastream) + getter? +- Which audio constraints and capabilities should playback tracks support, and + how should the + [constrainable pattern](https://w3c.github.io/mediacapture-main/#constrainable-interface), + including `getCapabilities()`, `getSettings()`, `getConstraints()`, and + `applyConstraints()`, apply to them? +- What timing guarantees are needed when microphone and playback audio are + processed together? +- What sources and processing stages should be included in + [system audio](https://w3c.github.io/mediacapture-screen-share/#dom-displaymediastreamoptions-systemaudio) + on each platform? + +### Permission and privacy + +- How should playback-audio permission integrate with existing browser + permission controls and the proposed + [``, ``, and `` capability elements](https://chromestatus.com/feature/5153829504024576)? +- How should self-capture work for playback audio? Should the requesting tab + appear in the source picker, should system capture include audio produced by + the requesting site, and should the proposal align with existing mechanisms + such as + [`selfBrowserSurface`](https://w3c.github.io/mediacapture-screen-share/#dom-displaymediastreamoptions-selfbrowsersurface), + [`restrictOwnAudio`](https://w3c.github.io/mediacapture-screen-share/#def-constraint-restrictOwnAudio), + or + [`suppressLocalAudioPlayback`](https://w3c.github.io/mediacapture-screen-share/#def-constraint-suppressLocalAudioPlayback)? +- Should playback-audio permission persist like + [`getUserMedia()`](https://w3c.github.io/mediacapture-main/#dom-mediadevices-getusermedia) + or require a new user decision for every request like + [`getDisplayMedia()`](https://w3c.github.io/mediacapture-screen-share/#dom-mediadevices-getdisplaymedia)? + Should permission behavior differ between system audio and a user-selected + source? + +## Alternatives considered + +### Allow audio-only `getDisplayMedia()` + +The platform could allow: + +```js +const stream = await navigator.mediaDevices.getDisplayMedia({ + video: false, + audio: true, +}); +``` + +This reuses an existing method. However, `getDisplayMedia()` is designed around +selecting and sharing visible content. Video is required and audio is optional. +The WebRTC Working Group discussion identified audio-only capture as a large +change to that model. For that reason, this is not one of the proposed +options. + +### Use `systemAudio` or `audioPreference` with `getDisplayMedia()` + +`systemAudio: "include"` asks the browser to offer system audio during display +capture. The proposed `audioPreference: "high"` value, discussed in the +July 2026 WebRTC Working Group meeting ([minutes](https://www.w3.org/2026/07/14-webrtc-minutes.html), +[slides](https://docs.google.com/presentation/d/1aivu0RWAtlHpJ4dFyTPblPXGMn-ZdK9GG2xuqgcg9MQ/edit?slide=id.g2bb12bc23cb_0_0#slide=id.g2bb12bc23cb_0_0)), +tells the browser that receiving audio is important: + +```js +const stream = await navigator.mediaDevices.getDisplayMedia({ + video: true, + audio: true, + systemAudio: "include", + audioPreference: "high", +}); +``` + +These options can improve requests that need both video and audio, but neither +provides audio-only capture. + +### Stop the video track after requesting display capture + +A website can request display capture and immediately stop the video track. +The user must still approve screen sharing, the website may initially receive +video, and an audio track is not guaranteed. + +### Improve browser echo cancellation + +Better browser echo cancellation would improve calls and should be explored +separately. It would not support audio-only recording, note taking, +broadcasting, visualization, translation, processing, or sharing scenarios in +the browser. + +## Stakeholder Feedback / Opposition + +This proposal is being shared to gather feedback from browser vendors, web +developers, and standards participants. + +Developer interest and use cases are documented in +[the existing W3C discussion](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12). +No browser vendor has yet expressed a formal position on either API option. + +Prior discussions have raised questions about platform support, privacy and +security, source scope, permission UX, and whether playback-audio capture +should extend an existing API or use a new API. These remain open areas for +feedback and discussion. + +## References & Acknowledgements + +- [Media Capture and Streams](https://w3c.github.io/mediacapture-main/) +- [Screen Capture](https://w3c.github.io/mediacapture-screen-share/) +- [Media Capture Screen Share Extensions issue #12](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12) + +This proposal builds on an +[earlier exploration of playback-reference audio](https://github.com/MicrosoftEdge/MSEdgeExplainers/pull/1377) +by [Huseyin Ozcan](https://github.com/huozcan-ms), +[Max Solodovnikov](https://github.com/solmaks), Hong Sodoma, and Vinod Prakash. + +Thanks to [Erik Anderson](https://github.com/erik-anderson) for reviewing and +providing feedback on this explainer. diff --git a/PlaybackAudioCapture/ongoing-use-mockups/index.html b/PlaybackAudioCapture/ongoing-use-mockups/index.html new file mode 100644 index 00000000..aaedad00 --- /dev/null +++ b/PlaybackAudioCapture/ongoing-use-mockups/index.html @@ -0,0 +1,1007 @@ + + + + + + Playback Audio Capture - Ongoing Use Mockups + + + +
+

Ongoing playback-audio capture concepts

+

+ These mockups explore how browsers and operating systems could help users + remain aware of and stop playback-audio capture after permission is + granted. They are illustrative rather than proposed final UI and draw on + existing microphone, camera, and screen-sharing affordances in browsers. +

+ +
+

Existing screen-sharing UX patterns

+

+ Browser implementations vary. Current Microsoft Edge screen-sharing UI + is one example that informed the concepts below: +

+

+ If a browser tab is shared: the browser can identify + sharing in both places. The captured tab can say which site is receiving + it, while the capturing tab can identify the selected tab and provide a + stop control. +

+

+ If audio from an application or device is captured: a + persistent browser-controlled sharing indicator could keep the capture + visible and let the user stop it, even when another application is + active. +

+
+ +
+
+
+

Capturing browser tab

+

1. Capturing-site sharing bar

+

+ If audio from a tab, application, or device is captured, the + capturing tab could identify the source and provide a stop control. +

+
+
+
+
+
+ Meeting notes +
+
+ Meeting app + +
+
+
+ +
notes.example
+ +
+
+ + Sharing audio from meetingapp.example to this + tab + + +
+
+ Meeting notes + Listening to the selected audio source +
+
+
+
+ +
+
+

Captured browser tab

+

2. Captured-tab sharing bar

+

+ If a browser tab is the selected source, that tab could identify + which site is receiving its audio and provide a stop control. +

+
+
+
+
+
+ Meeting app + +
+
+ Meeting notes +
+
+
+ +
meetingapp.example
+ +
+
+ + This tab's audio is being shared with + notes.example + + +
+
+
Team meeting
+
+
+
+
+ +
+
+

Another active browser tab

+

3. Sharing status outside either site

+

+ If Site A captures Site B while the user views Site C, the active + tab could still identify the source and destination, provide sharing + controls, and show the browser's playback-capture indicator. +

+
+
+
+
+
+ Meeting notes +
+
+ Meeting app + +
+
+ Research +
+
+
+ +
research.example
+ +
+
+ + Sharing meetingapp.example to + notes.example + + +
+
+ Research + Another site is open while audio sharing continues +
+
+
+
+ +
+
+

Outside the browser

+

4. Persistent system-audio sharing control

+

+ If a site captures system audio, a browser-owned floating control + could remain visible when the user switches to another application. +

+
+
+
+
+ Document + +
+
+ Another application is active + + + +
+
+ + + notes.example is sharing your system audio. + + + Hide +
+
+
+
+ +
+
+

Browser address bar

+

5. Permission indicator and controls

+

+ Extend Edge's microphone permission panel to show active + playback-audio access and let the user manage future access. +

+
+
+
+
+
+ Meeting notes +
+
+
+ +
+ notes.example +
+ +
+
+ +
+
+ Meeting notes + Transcription in progress +
+
+

Playback audio allowed

+

+ This page is accessing audio playing on your device. +

+ + +
+ + +
+
+
+
+
+ +
+
+

Possible OS integration

+

6. System taskbar indicator

+

+ The OS could eventually expose playback capture similarly to + microphone use, with a distinct icon for audio being captured. +

+
+
+
+
+
+ + + + + + 2:15 PM
8/25/2026 +
+
+
+
+
+
+ + +
+ + diff --git a/PlaybackAudioCapture/permission-ux-flow-mockups.md b/PlaybackAudioCapture/permission-ux-flow-mockups.md new file mode 100644 index 00000000..e06c146a --- /dev/null +++ b/PlaybackAudioCapture/permission-ux-flow-mockups.md @@ -0,0 +1,376 @@ +# Permission UX Flow Mockups + +These flows illustrate how playback-audio access could work. The exact +permission and selection UI is controlled by the browser. + +```text +[Website UI] Explains why the feature needs audio and starts the request. +[Browser UI] Authorizes access, selects a source, and controls ongoing use. +``` + +## Option 1: `getPlaybackMedia()` + +### Microphone and system audio + +```js +const microphoneStream = + await navigator.mediaDevices.getUserMedia({ + audio: true, + }); + +const playbackStream = + await navigator.mediaDevices.getPlaybackMedia({ + audio: { + source: "system", + }, + }); +``` + +```text ++---------------------------------------------------+ +| WEBSITE UI | +| | +| Start voice chat | +| | +| [Start] | ++--------------------------+------------------------+ + | + v + Website calls getUserMedia() + | + v ++---------------------------------------------------+ +| BROWSER UI | +| | +| example.com wants to use available microphones | +| | +| [Default microphone v] | +| | +| [ Allow while visiting the site ] | +| [ Allow this time ] | +| [ Never allow ] | ++--------------------------+------------------------+ + | + v + Microphone track returned + | + v ++---------------------------------------------------+ +| WEBSITE UI | +| | +| Improve voice quality | +| | +| Use audio playing on this device to reduce echo | +| and improve voice quality. | +| | +| [Enable] | ++--------------------------+------------------------+ + | + v + Website calls getPlaybackMedia() + | + v ++---------------------------------------------------+ +| BROWSER UI | +| | +| example.com wants to access audio playing on | +| this device. This may include audio from other | +| applications and websites. | +| | +| [ Allow while visiting the site ] | +| [ Allow this time ] | +| [ Never allow ] | ++--------------------------+------------------------+ + | + v + System playback audio track + | + v + Microphone + playback reference processing + | + v + Cleaned microphone output +``` + +The microphone and playback requests have separate permission decisions. If +the user declines playback access, the application can continue using the +microphone. No picker is needed because the playback request already identifies +system audio. + +### User-selected audio + +```js +const playbackStream = + await navigator.mediaDevices.getPlaybackMedia({ + audio: { + source: "user-selected", + }, + }); +``` + +```text ++---------------------------------------------------+ +| WEBSITE UI | +| | +| Transcribe meeting audio | +| | +| Choose the tab or application where your meeting | +| or lecture is playing. Only audio will be shared. | +| | +| [Choose audio source] | ++--------------------------+------------------------+ + | + v + Website calls getPlaybackMedia() + | + v ++---------------------------------------------------+ +| BROWSER UI | +| | +| Choose audio to share with notes.example | +| | +| Tabs | +| ( ) Google Meet | +| ( ) Lecture livestream | +| | +| Applications | +| ( ) Microsoft Teams | +| ( ) Media Player | +| | +| [Cancel] [Share audio] | ++--------------------------+------------------------+ + | + v + Selected playback audio track + | + v + Browser shows an ongoing indicator +``` + +The website explains the request but cannot see the available sources or grant +access. The browser owns the picker and returns a track only after the user +selects a source. + +## Option 2: extend `getUserMedia()` + +When a website requests only playback audio, the UX can follow the same system +or user-selected flow shown for `getPlaybackMedia()`. + +### Microphone and system audio + +```js +const stream = + await navigator.mediaDevices.getUserMedia({ + audio: true, + playbackAudio: { + source: "system", + }, + }); +``` + +Both microphone and playback audio are required by this combined request. The +browser could use one combined panel: + +```text ++---------------------------------------------------+ +| WEBSITE UI | +| | +| Use your microphone for the call and audio playing | +| on this device to improve voice quality. | +| | +| [Continue] | ++--------------------------+------------------------+ + | + v + Website calls getUserMedia() + | + v ++---------------------------------------------------+ +| BROWSER UI | +| | +| example.com wants to: | +| | +| Use available microphones (1) | +| Access audio playing on this device | +| | +| Microphone | +| [Default microphone v] | +| | +| This may include audio from other applications | +| and websites. | +| | +| [ Allow while visiting the site ] | +| [ Allow this time ] | +| [ Never allow ] | ++--------------------------+------------------------+ + | + +------------+------------+ + Never allow Allowed + | | + v v + Request denied Microphone track + + playback track +``` + +Because this is one combined request, both microphone and playback permission +must be granted for the request to succeed. + +### Microphone and user-selected audio + +```js +const stream = + await navigator.mediaDevices.getUserMedia({ + audio: true, + playbackAudio: { + source: "user-selected", + }, + }); +``` + +```text ++---------------------------------------------------+ +| WEBSITE UI | +| | +| Create notes from your microphone and meeting | +| audio. You will choose the meeting source next. | +| | +| [Continue] | ++--------------------------+------------------------+ + | + v + Website calls getUserMedia() + | + v ++---------------------------------------------------+ +| BROWSER UI | +| | +| example.com wants to use available microphones | +| | +| [Default microphone v] | +| | +| [ Allow while visiting the site ] | +| [ Allow this time ] | +| [ Never allow ] | ++--------------------------+------------------------+ + | + v ++---------------------------------------------------+ +| BROWSER UI | +| | +| Choose a tab or application whose audio to share | +| | +| [Browser tabs] [Applications] | +| | +| ( ) Team meeting - Video call | +| (o) Product demo - example.org | +| ( ) Presentation - Slides | +| | +| [Cancel] [Share] | ++--------------------------+------------------------+ + | + +------------+------------+ + | | + Picker cancelled Source selected + | | + v v + Request rejected Microphone track + + playback track +``` + +Because microphone and playback audio were requested together, cancelling the +picker rejects the combined request. + +### Independent microphone and playback requests + +A note-taking website may need only the selected meeting audio. If it also +wants microphone audio, it can request the two tracks independently so that +declining microphone access does not block transcription. + +```text ++---------------------------------------------------+ +| WEBSITE UI | +| | +| Create notes from your meeting audio. | +| You can also include your microphone. | +| | +| [Continue] | ++--------------------------+------------------------+ + | + v + Website calls getUserMedia() for microphone + | + v ++---------------------------------------------------+ +| BROWSER UI | +| | +| notes.example wants to use available microphones | +| | +| [Default microphone v] | +| | +| [ Allow while visiting the site ] | +| [ Allow this time ] | +| [ Never allow ] | ++--------------------------+------------------------+ + | + +------------+------------+ + | | + Permission denied Permission granted + | | + v v + Continue without Microphone track + microphone available + | | + +------------+------------+ + | + v + Website calls getUserMedia() for user-selected + playback audio + | + v ++---------------------------------------------------+ +| BROWSER UI | +| | +| Choose the tab or application where your meeting | +| is playing. Only audio will be shared. | +| | +| [Cancel] [Share audio] | ++--------------------------+------------------------+ + | + +------------+------------+ + | | + Cancel Share audio + | | + v v + No meeting audio Playback track + is available available + | + v + Website transcribes + playback audio and + includes the microphone + if it was allowed +``` + +Each request keeps the existing all-or-nothing behavior, but one request +failing does not affect the other. + +## Ongoing controls + +See the +[ongoing capture UX mockups](https://microsoftedge.github.io/MSEdgeExplainers/PlaybackAudioCapture/ongoing-use-mockups/) +for illustrative ideas showing how users could monitor and stop active capture. + +All successful flows end with browser-controlled ongoing-use UI: + +```text +Capture begins + | + v +Browser indicates that playback audio is being captured + | + v +User can stop capture +``` + +Playback capture ends when the user stops it, permission is revoked, the source +becomes unavailable, or the website stops the track. diff --git a/README.md b/README.md index ea499038..7db4fc3e 100644 --- a/README.md +++ b/README.md @@ -82,6 +82,7 @@ we move them into the [Alumni section](#alumni-) below. | [Calling Notifications](Notifications/notifications_actions_customization.md) | ![GitHub issues by-label](https://img.shields.io/github/issues/MicrosoftEdge/MSEdgeExplainers/Calling%20Notifications?label=issues) | [New issue...](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/new?template=calling-notifications.md) | Web Applications | | [Split Tab Navigation](SplitTab/NavigationInSplitTab/explainer.md) | ![GitHub issues by-label](https://img.shields.io/github/issues/MicrosoftEdge/MSEdgeExplainers/Split%20Tab?label=issues) | [New Issue...](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/new?template=split-tab.md) | Web Applications | | [Set Default Audio Output Device](SetPreferredSinkId/explainer.md) | ![GitHub issues by-label](https://img.shields.io/github/issues/MicrosoftEdge/MSEdgeExplainers/SetPreferredSinkId?label=issues) | [New issue...](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/new?assignees=sunggch&labels=SetPreferredSinkId&title=%5BSetPreferredSinkId%5D+Issue) | WebRTC | +| [Playback Audio Capture Without Screen Sharing](PlaybackAudioCapture/explainer.md) | [Discussion](https://github.com/w3c/mediacapture-screen-share-extensions/issues/12) | [New issue...](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/new?title=%5BPlaybackAudioCapture%5D+) | WebRTC | | [Handwriting attribute](Handwriting/explainer.md) | ![GitHub issues by-label](https://img.shields.io/github/issues/MicrosoftEdge/MSEdgeExplainers/Handwriting?label=issues) | [New issue...](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/new?assignees=adettenb&labels=Handwriting&title=%5BHandwriting%5D+Issue) | HTML | | [Mulitple Stylesheets Per File (@sheet)](AtSheet/explainer.md) | ![GitHub issues by-label](https://img.shields.io/github/issues/MicrosoftEdge/MSEdgeExplainers/AtSheet?label=issues) | [New Issue...](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/new?assignees=aluhrs13&labels=AtSheet&title=%5B%40sheet%5D+%3CTITLE+HERE%3E) | CSS | | [Link Rel Local Reference](LocalReferenceLinkRel/explainer.md) | ![GitHub issues by-label](https://img.shields.io/github/issues/MicrosoftEdge/MSEdgeExplainers/LocalReferenceLinkRel?label=issues) | [New Issue...](https://github.com/MicrosoftEdge/MSEdgeExplainers/issues/new?assignees=KurtCattiSchmidt&labels=LRLR&title=%5BLRLR%5D+%3CTITLE+HERE%3E) | HTML |