Media Capture and Devices

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production.

See the section bibliography for the reference material consulted while preparing these pages.

Every videoconference starts before any connection exists: the browser has to obtain a camera and a microphone, let the user choose between several of each, and keep working when a headset is plugged in half way through a call. This page covers that capture half of WebRTC, the Media Capture and Streams API, and builds the media layer that Huddle uses. The resulting tracks are handed to an RTCPeerConnection, which is the subject of Peer Connections and SDP. The overall picture of how capture, signalling and transport fit together is in WebRTC Architecture.

Versions and specifications were checked on 2026-10-24. Media Capture and Streams is a W3C Candidate Recommendation Draft of 2025-10-09, and Audio Output Devices (setSinkId, selectAudioOutput) is a Candidate Recommendation Draft of the same date. The examples were run in headless Chrome with fake devices (Puppeteer 25 with a locally served page).

Of the three scenarios, Huddle depends on all of this, Pixel Arena needs none of it (an in-game voice channel would reuse the same calls), and Chatterbox only needs it for voice notes recorded through the APIs in Recording and Media Processing.

Streams, tracks and sources

The API has three nouns. A source is the physical or virtual thing producing media (a camera, a microphone, a screen, a canvas). A MediaStreamTrack carries one kind of media (audio or video) from a source and is the unit you mute, constrain, clone and send. A MediaStream is only a labelled bag of tracks: it exists so that audio and video that belong together can be played and recorded together. WebRTC senders work with tracks; the stream matters for the media element, for MediaRecorder and for grouping remote tracks.

navigator.mediaDevices.getUserMedia(constraints) returns a promise of a MediaStream. The browser shows a permission prompt unless permission has been remembered, and the promise settles only after the user answers, which can be never. Passing { audio: true, video: true } asks for the defaults; passing an empty object rejects with TypeError.

Screen capture uses a different entry point with different rules, described in Screen Sharing.

Secure contexts and permissions

getUserMedia, device labels and setSinkId exist only in a secure context: a page served over HTTPS, or from localhost or 127.0.0.1 during development. On a plain http:// page on another host navigator.mediaDevices is undefined, which is the classic cause of a "cannot read properties of undefined" error when a developer tests a Huddle build from a phone over the local network. Serve the development build over HTTPS (a local certificate authority or a tunnel) instead of working around it.

Permission is granted per origin and per device kind. Three layers can say no, and each produces a different error:

  • the user (prompt dismissed or denied, or a remembered denial) rejects with NotAllowedError;

  • the operating system (privacy switches, or a device in use by another application) rejects with NotReadableError or NotAllowedError depending on the platform;

  • the page’s policy: an embedded page whose embedder did not delegate the feature gets NotAllowedError without any prompt.

The Permissions API can ask for the state without prompting: navigator.permissions.query() with the name camera or microphone resolves to a status object whose state is granted, denied or prompt, and which fires change events. Support for those two names differs between browsers, so treat the query as a hint used to decide which UI to show, never as a substitute for handling the errors of getUserMedia.

Permissions-Policy and iframes

The camera, microphone and speaker-selection policy-controlled features default to the page’s own origin. If Huddle is embedded in a customer’s site as an iframe, the embedder must delegate them with the allow attribute, and the embedded document can additionally be restricted by its own response header:

Permissions-Policy: camera=(self "https://huddle.example.com"), microphone=(self "https://huddle.example.com"), speaker-selection=(self "https://huddle.example.com")
<iframe src="https://huddle.example.com/room/standup"
        allow="camera; microphone; speaker-selection; autoplay; display-capture"></iframe>

Source: MDN: Permissions-Policy camera directive

A page that must never use the camera (the Chatterbox client, for example) should send camera=(), microphone=() so that an injected script cannot even raise a prompt.

Constraints

A constraints dictionary describes what you want, not what you will get. Each of audio and video is either a boolean or a MediaTrackConstraints object whose properties are constrainable: width, height, frameRate, aspectRatio, facingMode, deviceId, groupId, resizeMode, and for audio echoCancellation, noiseSuppression, autoGainControl, channelCount, sampleRate, sampleSize and latency. Each value is expressed in one of four ways:

Form Meaning

plain value

Shorthand for a preference. The browser tries to match it and settles for the closest value.

ideal

A preference. It never causes failure; the browser picks the setting whose distance from the ideal is smallest.

min and max

Hard bounds. If no device mode fits, getUserMedia rejects with OverconstrainedError.

exact

A single required value. Same failure as min and max; it also makes deviceId a strict selection.

The practical rule for Huddle: use ideal for quality (it adapts to whatever camera the user owns), max to protect your bandwidth budget, and exact only for a deviceId chosen from the picker. An exact resolution on a laptop camera that tops out at 1280 by 720 simply fails. OverconstrainedError.constraint names the offending property, which is what turns a generic failure into a precise message in the UI.

Resolution constraints interact with orientation. A phone held upright can report a portrait stream, so ask for sizes as preferences and read getSettings() instead of assuming. facingMode (user, environment, left, right) matters on phones; desktop cameras report no facing mode.

For audio, the three processing flags are on by default in browsers and do real work in a call: echoCancellation removes the remote participants' voices that the microphone picks up from the speakers, noiseSuppression attenuates steady background noise and autoGainControl evens out the level. Turn them off (and ask for stereo with channelCount) only for a music-sharing mode, and tell users to wear headphones, since the echo canceller is what keeps an open microphone from feeding back.

Three introspection calls give you the truth afterwards:

  • navigator.mediaDevices.getSupportedConstraints() lists the constraint names this browser understands; an unsupported name is ignored silently rather than rejected.

  • track.getCapabilities() lists what this device can do (ranges and enumerations).

  • track.getSettings() reports what the track is doing now.

// Report which constraints this browser understands and what the device can do.
export async function describeSupport() {
  const supported = navigator.mediaDevices.getSupportedConstraints();
  const wanted = ['width', 'height', 'frameRate', 'facingMode', 'aspectRatio',
    'echoCancellation', 'noiseSuppression', 'autoGainControl', 'deviceId', 'groupId'];
  const missing = wanted.filter((name) => !supported[name]);
  const stream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true });
  const [video] = stream.getVideoTracks();
  const [audio] = stream.getAudioTracks();
  const report = {
    missing,
    videoCapabilities: video.getCapabilities(),
    videoSettings: video.getSettings(),
    audioSettings: audio.getSettings(),
  };
  stream.getTracks().forEach((track) => track.stop());
  return report;
}

Source: MDN: MediaDevices.getSupportedConstraints()

In the verification run, a request for 1280 by 720 as ideal against the fake camera (which advertises up to 3840 by 2160 and 20 frames per second) produced exactly 1280 by 720 at 20 frames per second: the frame-rate preference of 30 was reduced to the device maximum without an error. Always read the settings back before showing an "HD" badge.

Listing devices

navigator.mediaDevices.enumerateDevices() returns MediaDeviceInfo objects with kind (audioinput, videoinput, audiooutput), deviceId, groupId and label. Three behaviours surprise newcomers:

  1. Labels can be empty until permission exists. Before the first successful getUserMedia, browsers return empty label strings and often a single entry per kind, to limit fingerprinting. This is why Huddle’s lobby asks for permission first and builds the picker afterwards.

  2. Device identifiers are per origin. A deviceId is stable for a given origin until site data is cleared; do not send it to the server as a user identifier.

  3. default and communications entries appear in Chromium as aliases for the operating-system defaults. Use groupId to pair a microphone with its matching speaker (the same physical headset shares a group).

Hot-plugging with devicechange

navigator.mediaDevices fires devicechange whenever a device appears or disappears. The event carries no detail, so the handler calls enumerateDevices() again. If the active device vanished (the headset was unplugged), its track fires ended and the application must react: tell the user, fall back to the default device or show the picker. Tracks are not switched automatically. When a new headset appears, Huddle does not switch silently; it offers the choice in a toast, because moving the audio of a live call unexpectedly is worse than ignoring a plug event.

Choosing speakers

Playing remote audio through a chosen output uses HTMLMediaElement.setSinkId(deviceId) on the audio or video element that plays the remote stream. Chrome has supported it since version 49, Firefox since 116 and Safari since 18.4. Rejections are NotAllowedError (blocked by the speaker-selection policy or no permission), NotFoundError (unknown identifier) and AbortError.

Because non-default outputs need their own consent, the specification adds navigator.mediaDevices.selectAudioOutput(): a prompt that must be triggered by a user gesture, adds the chosen speaker to the list and grants it. Firefox implements it; Chrome and Safari did not at the time of checking, so a robust picker fills a select from enumerateDevices(), hides it when setSinkId is missing, and calls selectAudioOutput() behind a "Choose speakers" button where it exists.

The Huddle media layer

The layer below is plain ES modules with no build step. HuddleMedia owns the local stream and exposes the operations the call UI needs; device-picker.js connects the device lists to the page. The three modules were run together in headless Chrome with fake devices.

// Huddle media layer: constraints, device picker, mute, camera off, quality changes.
export const CALL_PROFILE = {
  audio: { echoCancellation: true, noiseSuppression: true, autoGainControl: true },
  video: {
    width: { ideal: 1280, max: 1920 },
    height: { ideal: 720, max: 1080 },
    frameRate: { ideal: 30, max: 30 },
    facingMode: 'user',
  },
};

export class HuddleMedia extends EventTarget {
  #stream = null;

  get stream() { return this.#stream; }

  async open(deviceIds = {}) {
    const constraints = structuredClone(CALL_PROFILE);
    if (deviceIds.audio) constraints.audio.deviceId = { exact: deviceIds.audio };
    if (deviceIds.video) constraints.video.deviceId = { exact: deviceIds.video };
    try {
      this.#stream = await navigator.mediaDevices.getUserMedia(constraints);
    } catch (err) {
      throw new Error(explain(err));
    }
    for (const track of this.#stream.getTracks()) {
      track.addEventListener('ended', () => this.dispatchEvent(
        new CustomEvent('trackended', { detail: track.kind })));
    }
    navigator.mediaDevices.addEventListener('devicechange', () => this.refresh());
    return this.#stream;
  }

  async refresh() {
    const devices = await navigator.mediaDevices.enumerateDevices();
    const byKind = { audioinput: [], videoinput: [], audiooutput: [] };
    for (const d of devices) byKind[d.kind]?.push({ id: d.deviceId, label: d.label || '(unnamed)' });
    this.dispatchEvent(new CustomEvent('devices', { detail: byKind }));
    return byKind;
  }

  // Muting keeps the sender and the RTP stream alive: silence flows, nothing renegotiates.
  setMuted(muted) {
    this.#stream.getAudioTracks().forEach((t) => { t.enabled = !muted; });
  }

  // Turning the camera off for real releases the device (the LED goes out).
  async setCameraOff(off) {
    const [old] = this.#stream.getVideoTracks();
    if (off) {
      old.stop();
      this.#stream.removeTrack(old);
      return null;
    }
    const fresh = await navigator.mediaDevices.getUserMedia({ video: CALL_PROFILE.video });
    const [track] = fresh.getVideoTracks();
    this.#stream.addTrack(track);
    return track; // the caller passes it to sender.replaceTrack(track)
  }

  async setQuality(level) {
    const presets = {
      low: { width: 320, height: 180, frameRate: 15 },
      high: { width: 1280, height: 720, frameRate: 30 },
    };
    const [track] = this.#stream.getVideoTracks();
    await track.applyConstraints(presets[level]);
    return track.getSettings();
  }

  // Camera feed is motion; slides are detail. Hints steer degradation under load.
  setContentHint(hint) {
    const [track] = this.#stream.getVideoTracks();
    track.contentHint = hint; // '', 'motion', 'detail', 'text'
    return track.contentHint;
  }
}

export function explain(err) {
  switch (err.name) {
    case 'NotAllowedError': return 'Permission denied by the user, the browser or a Permissions-Policy.';
    case 'NotFoundError': return 'No camera or microphone matches the request.';
    case 'NotReadableError': return 'The device is busy or the OS blocked it.';
    case 'OverconstrainedError': return `Constraint ${err.constraint} cannot be satisfied.`;
    case 'SecurityError': return 'getUserMedia needs a secure context.';
    case 'TypeError': return 'Empty constraints: ask for audio, video or both.';
    default: return `${err.name}: ${err.message}`;
  }
}

Source: MDN: MediaDevices.getUserMedia()

// Device picker: fill three <select> elements and apply the choices.
export function wirePicker(media, els, remoteAudio) {
  const fill = (select, list) => {
    select.replaceChildren(...list.map((d) => new Option(d.label, d.id)));
  };
  media.addEventListener('devices', (e) => {
    fill(els.mic, e.detail.audioinput);
    fill(els.cam, e.detail.videoinput);
    fill(els.speaker, e.detail.audiooutput);
    els.speaker.disabled = !('setSinkId' in HTMLMediaElement.prototype);
  });
  els.speaker.addEventListener('change', async () => {
    try {
      await remoteAudio.setSinkId(els.speaker.value);
    } catch (err) {
      console.warn('Speaker not changed:', err.name);
    }
  });
}

export function attachRemote(video, stream) {
  video.playsInline = true; // iOS: do not take over the screen
  video.srcObject = stream;
  video.muted = true;       // allows autoplay; unmute after a user gesture
  return video.play();
}

Source: MDN: HTMLMediaElement.setSinkId()

The page that loads them needs only a secure origin (a local server is enough):

<video id="remote" playsinline></video>
<audio id="remote-audio"></audio>
<select id="mic"></select> <select id="cam"></select> <select id="speaker"></select>
<script type="module">
  import { HuddleMedia } from './huddle-media.js';
  import { wirePicker, attachRemote } from './device-picker.js';

  const media = new HuddleMedia();
  wirePicker(media, {
    mic: document.getElementById('mic'),
    cam: document.getElementById('cam'),
    speaker: document.getElementById('speaker'),
  }, document.getElementById('remote-audio'));
  const stream = await media.open();   // permission prompt + first getUserMedia
  await media.refresh();               // labels are available now
  await attachRemote(document.getElementById('remote'), stream);
</script>

Source: MDN: MediaDevices.enumerateDevices()

Mute, camera off and stopping tracks

Huddle has several controls that look alike and behave differently on the wire:

Operation What it does Effect on the call

track.enabled = false

The track stays live and produces silence (audio) or black frames (video).

The sender keeps transmitting and nothing is renegotiated; the camera light stays on. Instant and reversible.

track.stop()

Ends the track and releases the device.

The camera light goes out. The track cannot be restarted: capture again and call RTCRtpSender.replaceTrack().

sender.replaceTrack(null)

Detaches the track from the sender but leaves it running.

The remote side stops receiving media; the local preview continues.

transceiver.direction set to inactive

Changes the negotiated direction.

Requires renegotiation; used for a long pause, not for a mute button.

The rule used by Huddle: microphone mute uses enabled, because users toggle it constantly and expect no delay; camera off uses stop() and a fresh getUserMedia on the way back, because a lit camera indicator on a call that looks switched off is a privacy bug. Remote participants see the video freeze or go black either way, so Huddle also signals the state in an application message (see Signaling and Perfect Negotiation) to show an avatar instead of a black tile. In the verification run, muting left the track live with enabled false, and stopping the camera moved it to ended, after which a new track was live.

Tracks also report their own lifecycle: muted (the browser is not receiving data, which happens on remote tracks when packets stop and on local tracks when the system suspends capture) with mute and unmute events, and ended. Do not confuse muted, a state you cannot set, with enabled, a switch you can.

Changing a live track

track.applyConstraints(constraints) changes the track in place without a new prompt, so Huddle can adapt capture to the network or to the number of tiles on screen. The result is visible in getSettings() afterwards. In the verification run, applying a 320 by 180 preset at 15 frames per second changed the settings to exactly those values, with resizeMode switching to crop-and-scale. A failing request rejects with OverconstrainedError and leaves the previous configuration in place.

Lowering capture resolution is not the only way to save bandwidth. Encoders also scale and limit the frame rate per sender through RTCRtpSender.setParameters(), which is covered in Stats, Quality and Bandwidth and Codecs, Simulcast and SVC. Use applyConstraints when the capture itself should become cheaper (a weak mobile CPU), and sender parameters when only the network leg should.

contentHint

track.contentHint tells the encoder what the pixels mean, so that when bandwidth drops it knows what to sacrifice. For video, motion keeps the frame rate and lets resolution fall (a talking head), while detail and text keep resolution and lower the frame rate (slides, code, a whiteboard). For audio, speech and music select tuning. The default is the empty string. Chrome and Safari implement it and Firefox does not, so treat it as a progressive enhancement. Screen sharing relies on it heavily; the camera path rarely needs it.

Playing media: autoplay and playsinline

Browsers restrict sound that starts without user activity. A video element with the muted attribute plays automatically in every major browser; unmuted playback needs a prior user gesture (or, in Chrome, enough media engagement), and otherwise play() rejects with NotAllowedError. Huddle’s rule is simple: the user always presses "Join" to enter the room, which counts as the gesture, and the remote media elements are created or resumed in that click handler. attachRemote in the code above shows the fallback: start muted, then unmute after a gesture.

On iOS Safari a video element plays full screen unless it has the playsinline attribute (or the playsInline property); a call view without it hijacks the screen. Give the element that shows the local preview the muted attribute too, otherwise users hear their own voice with a delay.

A related restriction applies to Web Audio: an AudioContext created before a gesture starts suspended, so resume it inside the click handler, as active-speaker detection in Huddle does when it analyses remote audio levels.

Source: MDN: Autoplay guide for media and Web Audio APIs

Common errors

Error Usual cause and fix

NotAllowedError

The user denied, the denial was remembered, the page is in an iframe without allow, or play() was blocked by the autoplay policy. Show a help screen explaining how to re-enable the camera; a page cannot re-prompt after a denial.

NotFoundError

No device of that kind (a desktop without a camera). Offer audio-only entry instead of failing the join.

NotReadableError

The device is held by another application or the system blocked access; on some platforms two tabs cannot open the same camera. Close the other user and retry.

OverconstrainedError

An exact, min or max constraint cannot be met. Read constraint, then retry with ideal only.

TypeError

An empty constraints object, or both audio and video false.

Missing navigator.mediaDevices

Not a secure context.

AbortError

A failure that is not explained by the others. Retrying once often works.

In the verification run, an unknown deviceId given as exact produced OverconstrainedError naming deviceId, an impossible width produced OverconstrainedError naming width, and an empty constraints object produced TypeError.

The recommended join flow is: request audio and video together; if that fails with NotFoundError or OverconstrainedError, retry audio only, then video only, and report what is missing. A single catch-all that gives up entirely makes the product useless for people with a broken camera.

Security and privacy

  • Browsers show a recording indicator while a track is live, and stopping the track is the only way to turn it off. Never keep a stream open "for later" in a lobby.

  • Permissions are per origin, so any script on the page, including third-party scripts, can use a granted camera. A strict Content Security Policy and the Permissions-Policy header limit the damage of an injected script.

  • Device identifiers and labels are fingerprinting surface, and browsers hide them until permission exists. Do not log them or send them to analytics.

  • Capture is unavailable in cross-origin frames unless delegated. Treat a request to widen allow as a security review item.

Testing capture

Automated tests cannot click a real prompt. Chrome offers --use-fake-ui-for-media-stream (auto-accept the prompt) and --use-fake-device-for-media-stream (a synthetic camera and microphone, with --use-file-for-fake-video-capture for a recorded clip). Firefox has the media.navigator.streams.fake and media.navigator.permission.disabled preferences. This is how the examples above were verified, and E2E Testing in the Browser shows the general approach. More on real-time tests is in Testing Real-Time Apps.

import puppeteer from 'puppeteer-core';

const browser = await puppeteer.launch({
  executablePath: process.env.CHROME_PATH,
  headless: true,
  args: ['--use-fake-device-for-media-stream', '--use-fake-ui-for-media-stream'],
});
const page = await browser.newPage();
await page.goto('http://localhost:8080/');   // localhost is a secure context
const labels = await page.evaluate(async () => {
  const s = await navigator.mediaDevices.getUserMedia({ video: true, audio: true });
  s.getTracks().forEach((t) => t.stop());
  return (await navigator.mediaDevices.enumerateDevices()).map((d) => d.label);
});
console.log(labels);
await browser.close();

Source: MDN: getUserMedia, security

Version and specification notes

  • The promise-based navigator.mediaDevices.getUserMedia() replaced the callback-based navigator.getUserMedia(), which is deprecated. Do not use the old one or its prefixed variants.

  • The addtrack and removetrack events of MediaStream fire only for changes made by the browser, not for your own calls.

  • Audio output selection lives in its own specification, Audio Output Devices, which defines setSinkId, selectAudioOutput and the speaker-selection policy.

  • contentHint lives in its own specification, MediaStreamTrack Content Hints, and is not implemented everywhere.

  • adapter.js is not needed for any call on this page.

Related pages

References