Codecs, Simulcast and SVC

This content was generated with the assistance of AI and should be verified against the official documentation before being relied on in production.

See the section bibliography for the reference material consulted while preparing these pages.

This page was written against the W3C WebRTC 1.0 Recommendation (2025-03-13 edition), the W3C WebRTC Scalable Video Coding (SVC) Extension Working Draft of 2026-09-14, the Media Capabilities specification, RFC 7742 and RFC 7874 for the mandatory codecs, and RFC 8851 and RFC 8853 for simulcast. The browser examples were run in Chrome 155 (headless) and everything was checked on 2026-10-24. Codec availability changes with every browser release, platform and even GPU, so the page teaches how to ask the browser at run time instead of listing support tables that would be stale within months.

Every example is part of the media layer of Huddle, the videoconference application of this section: a mesh mode for up to four participants and an SFU mode (see Topologies: Mesh, SFU and MCU and mediasoup). Codecs decide how many bits a call needs and which devices can take part, simulcast and SVC let one sender serve receivers with very different downlinks, and setParameters is how the application reacts when the network gets worse. The Chatterbox chat and the Pixel Arena game do not use audio or video codecs; Pixel Arena only uses data channels.

Where codecs enter the connection

A codec is never chosen by the application directly. It is negotiated: each side puts the codecs it supports into the SDP of its offer or answer, in order of preference, and the answerer keeps the subset it also supports. The Peer Connections and SDP page shows the lines involved; the parts that matter here are:

  • The m=video line lists RTP payload types in preference order (for example 96 97 98). The first payload type that both sides support is what the sender normally uses.

  • a=rtpmap:96 VP8/90000 binds a payload type to a codec name and clock rate, and a=fmtp: lines carry codec parameters such as the H.264 profile-level-id or the VP9 profile-id.

  • Companion payload types such as rtx (retransmission, RFC 4588), red and ulpfec (redundancy and forward error correction) are not codecs of their own, but they appear in the same lists and must be kept when you reorder them.

  • The API surface is small: RTCRtpSender.getCapabilities(kind) and RTCRtpReceiver.getCapabilities(kind) tell you what the browser can encode and decode, RTCRtpTransceiver.setCodecPreferences() reorders or filters the list that goes into the next offer or answer, and RTCRtpSender.getParameters() reports what was negotiated.

  • Call setCodecPreferences() before createOffer() or createAnswer(), on the side that creates the offer and, if you want a say in the answer, on the answerer too. Changing codecs in an established call needs a new offer/answer round (the negotiationneeded event does not fire for it by itself).

Mandatory and common codecs

The IETF decided which codecs every WebRTC endpoint must implement so that two browsers always find a common one. The current statements are RFC 7874 for audio and RFC 7742 for video.

Codec Role in WebRTC Notes for Huddle

Opus (RFC 6716, payload RFC 7587)

Mandatory audio codec, 48 kHz clock, mono or stereo, 6 to 510 kbit/s, built-in forward error correction and DTX.

Always the first choice for voice. Use useinbandfec=1 and, for speech, a bitrate cap of about 32 kbit/s.

G.711 (PCMU and PCMA)

Mandatory audio fallback, 8 kHz.

Needed for interop with telephony gateways; never preferable to Opus between browsers.

VP8 (RFC 7741 payload)

Mandatory video codec together with H.264 (constrained baseline).

Software-encoded and available everywhere; the safest default for simulcast.

H.264 (constrained baseline, profile-level-id=42e01f)

Mandatory video codec; often hardware accelerated on phones and laptops.

Best battery life on mobile. Several H.264 profiles are advertised, so match sdpFmtpLine when you filter.

VP9

Optional; adds SVC (L1T3, L3T3_KEY and others) and better compression than VP8.

First choice for SVC when every client in the room can decode it.

AV1

Optional; best compression, scalable modes, heavy to encode without hardware support.

Good for screen sharing and for rooms of recent devices; check powerEfficient before using it for camera video.

H.265 (HEVC)

Optional and appearing only in some browsers and on some hardware; the payload format is described by an IETF draft (draft-ietf-avtcore-hevc-webrtc) and licensing differs from the royalty-free codecs.

Treat it as an optimisation negotiated opportunistically, never as the only codec.

The mandatory set guarantees that a call can always be established. Everything else is optional and its presence in getCapabilities() depends on the browser version, the operating system and sometimes the GPU. This is the reason the first example inventories the codecs rather than assuming them.

Asking the browser what it supports

RTCRtpSender.getCapabilities(kind) and RTCRtpReceiver.getCapabilities(kind) return an object with a codecs array (each entry has mimeType, clockRate, optionally channels and sdpFmtpLine) and a headerExtensions array. They are static methods: no connection is needed, and they work before getUserMedia(). The sending and receiving lists differ in some browsers; in the Chrome 155 run below, the receive list additionally contained flexfec-03.

// huddle-media.js: codec, simulcast and SVC helpers for the Huddle client

/** Codec inventory of this browser, grouped by kind and direction. */
export function codecInventory() {
  const names = (caps) => [...new Set(caps.codecs.map((c) => c.mimeType))];
  return {
    send: {
      audio: names(RTCRtpSender.getCapabilities('audio')),
      video: names(RTCRtpSender.getCapabilities('video')),
    },
    receive: {
      audio: names(RTCRtpReceiver.getCapabilities('audio')),
      video: names(RTCRtpReceiver.getCapabilities('video')),
    },
  };
}

Run in headless Chrome 155 on macOS, the codec names of the send direction were:

{
  "audio": ["audio/opus", "audio/red", "audio/G722", "audio/PCMU", "audio/PCMA", "audio/CN", "audio/telephone-event"],
  "video": ["video/VP8", "video/rtx", "video/H264", "video/AV1", "video/VP9", "video/H265", "video/red", "video/ulpfec"]
}

The full entries are more detailed. Chrome reported the Opus entry as audio/opus/48000/2 with minptime=10;useinbandfec=1, and several H.264 entries that differ only in packetization-mode and profile-level-id (42001f, 42e01f, 4d001f and 640034). The header extensions list contained, among others, urn:ietf:params:rtp-hdrext:sdes:mid, urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id and urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id, which are the extensions simulcast relies on (see below).

Source: MDN: RTCRtpSender.getCapabilities()

The list is a capability, not a guarantee. A codec listed here can still fail to start when the machine runs out of hardware encoder sessions, and the browser then falls back or reports an error through RTCRtpSender statistics. To ask whether an encode at a given resolution would be smooth and power efficient, use Media Capabilities, covered in the section on SVC below.

Choosing codecs with setCodecPreferences

transceiver.setCodecPreferences(codecs) takes an array of codec objects taken from getCapabilities() and replaces the codec list that the next offer or answer advertises for that transceiver. The rules, verified in Chrome 155:

  • The array must come from RTCRtpReceiver.getCapabilities(kind) (not the sender’s list), because it controls what the local side is willing to receive and therefore what the peer is allowed to send. Passing a codec the browser does not support throws InvalidModificationError (the specification lists InvalidAccessError for an unsupported codec; Chrome reported InvalidModificationError for an unknown MIME type, so catch DOMException and inspect name).

  • The list must contain at least one real media codec; a list of only rtx, red, ulpfec or CN entries is rejected.

  • An empty array resets the transceiver to the browser defaults.

  • Order is a preference, not a command. The answerer decides the final order of the answer, and a codec missing from the other side is skipped.

  • Keep the rtx entries for any codec you keep. Without them a lost packet cannot be retransmitted.

  • Filtering removes a codec from the offer completely. To merely prefer, sort and keep the others after the wanted ones.

/**
 * Put the wanted media codecs first. Companion entries (rtx, red, ulpfec)
 * and codecs that are not listed stay in the array, after the wanted ones.
 */
export function preferCodecs(transceiver, kind, wanted) {
  const { codecs } = RTCRtpReceiver.getCapabilities(kind);
  const rank = (c) => {
    const i = wanted.findIndex((m) => m.toLowerCase() === c.mimeType.toLowerCase());
    return i === -1 ? wanted.length : i;
  };
  // Array.prototype.sort is stable, so the browser's order is kept inside a rank.
  const sorted = [...codecs].sort((a, b) => rank(a) - rank(b));
  transceiver.setCodecPreferences(sorted);
  return sorted.filter((c) => rank(c) < wanted.length).map((c) => c.mimeType);
}

The sort is stable, so the browser’s own order (for example its H.264 profile order) is kept inside each rank. The helper is used for the camera and for the microphone; for audio the call is preferCodecs(audioTransceiver, 'audio', ['audio/opus']). In the Chrome 155 test the video offer’s m=video line started with the VP9 payload types after preferCodecs(transceiver, 'video', ['video/VP9', 'video/AV1']), and getParameters().codecs of the sender then listed video/VP9 first, then video/AV1, then the remaining codecs.

Source: MDN: RTCRtpTransceiver.setCodecPreferences()

A few practical notes on codec choice in Huddle:

  • Do not force a single codec in a room where you do not control the clients. Prefer, do not filter, so that a device without VP9 still joins.

  • Mixed rooms in an SFU. The SFU receives what each sender publishes and forwards packets without transcoding, so every subscriber must be able to decode the codec the publisher uses. SFUs therefore usually restrict the room to codecs everyone supports, which is why mediasoup and LiveKit take a codec list in their router or room configuration (see LiveKit).

  • H.264 profiles. Two H.264 entries with different profile-level-id are different codecs for negotiation. A mobile app that offers only 42e01f will not match a browser that was told to prefer 640034.

  • Audio. Opus has two parameters worth setting in SDP: useinbandfec=1 (on by default in browsers) and usedtx=1 for discontinuous transmission, which saves bandwidth in large rooms where most people are silent. Setting maxBitrate on the audio encoding through setParameters() is simpler and needs no SDP editing.

Simulcast: one camera, several qualities

In an SFU room, each participant uploads once and the server forwards to many receivers, whose downlinks differ. If the sender uploads only 720p, a receiver on a poor connection either gets nothing usable or forces everyone to a lower quality. Simulcast solves it by encoding the same camera frame several times at different resolutions and bitrates and sending them as separate RTP streams. The SFU picks, per receiver, the highest layer that fits the receiver’s bandwidth and forwards only that layer.

On the wire, each layer is an independent RTP stream with its own SSRC. Because SSRCs are assigned by the browser and change between negotiations, the layers are identified in SDP by a restriction identifier (rid), defined in RFC 8851, and the set of layers is announced with the a=simulcast attribute, defined in RFC 8853. The rid value is also sent in an RTP header extension (urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id) so that a receiver can map incoming packets to layers before it has seen the SDP mapping for the SSRC; the companion repaired-rtp-stream-id extension does the same for retransmissions. In an offer from a Huddle client, the relevant lines look like this (from the Chrome 155 test):

a=rid:q send
a=rid:h send
a=rid:f send
a=simulcast:send q;h;f

The answerer must mirror them. A receiver or SFU that accepts simulcast replies with a=rid:q recv, a=rid:h recv, a=rid:f recv and a=simulcast:recv q;h;f. If the answer does not contain the a=simulcast line, the browser does not send the extra layers: in the test, a plain browser answerer made Chrome send a single layer, and getParameters().encodings contained only the first rid (q, the smallest layer) after negotiation. This is the most common reason that simulcast "does nothing" when you test two browsers against each other without an SFU.

Requesting simulcast from the browser

The layers are requested when the transceiver is created, with sendEncodings. Each entry may carry rid, active, maxBitrate, maxFramerate, scaleResolutionDownBy, priority, networkPriority and, for video, scalabilityMode. The rid can only be set at creation: it cannot be changed afterwards and the number and order of encodings is fixed.

/** Three simulcast layers for a 720p camera: quarter, half and full size. */
export const HUDDLE_LAYERS = [
  { rid: 'q', scaleResolutionDownBy: 4, maxBitrate: 150_000, maxFramerate: 15 },
  { rid: 'h', scaleResolutionDownBy: 2, maxBitrate: 500_000, maxFramerate: 30 },
  { rid: 'f', scaleResolutionDownBy: 1, maxBitrate: 1_500_000, maxFramerate: 30 },
];

/** Publish a camera track as three simulcast layers (SFU mode). */
export function addSimulcastCamera(pc, track, stream, codecs = ['video/VP8']) {
  const transceiver = pc.addTransceiver(track, {
    direction: 'sendonly',
    streams: [stream],
    sendEncodings: HUDDLE_LAYERS.map((l) => ({ ...l })),
  });
  preferCodecs(transceiver, 'video', codecs);
  return transceiver;
}

In Chrome 155, after negotiation getParameters().encodings reported all three layers with their scaleResolutionDownBy values (4, 2 and 1), and getStats() showed three outbound-rtp reports with rid q, h and f, of 320x180, 640x360 and 1280x720 pixels, each with scalabilityMode L1T3 (Chrome applies three temporal layers per spatial layer for VP8 simulcast by default).

Source: MDN: RTCPeerConnection.addTransceiver()

Order the encodings from the lowest to the highest quality, as RFC 8853 and the SFUs expect, and give the lowest layer a maxBitrate that fits a weak mobile uplink. The scaleResolutionDownBy values are divisors of the capture resolution, so with a 1280x720 camera 4, 2 and 1 give 320x180, 640x360 and 1280x720. If the camera delivers a smaller resolution than requested, the layers scale proportionally; a layer can also be missing when the browser decides the CPU cannot sustain it (see qualityLimitationReason in Stats, Quality and Bandwidth).

Testing simulcast without an SFU

To test the code above on one page, the answer has to carry the recv lines that an SFU would add. The test harness below mirrors the offered rids; it exists only for tests and is not something a production client does.

async function loopback(pcA, pcB) {
  pcA.onicecandidate = (e) => e.candidate && pcB.addIceCandidate(e.candidate);
  pcB.onicecandidate = (e) => e.candidate && pcA.addIceCandidate(e.candidate);
  await pcA.setLocalDescription();
  await pcB.setRemoteDescription(pcA.localDescription);
  const answer = await pcB.createAnswer();
  await pcB.setLocalDescription(answer);
  await pcA.setRemoteDescription({ type: 'answer', sdp: acceptSimulcast(pcA.localDescription.sdp, answer.sdp) });
}
// An SFU answers a simulcast offer by mirroring the rids as "recv".
function acceptSimulcast(offerSdp, answerSdp) {
  const rids = [...offerSdp.matchAll(/^a=rid:(\S+) send/gm)].map((m) => m[1]);
  if (rids.length === 0) return answerSdp;
  const extra = rids.map((r) => `a=rid:${r} recv`).concat(`a=simulcast:recv ${rids.join(';')}`);
  return answerSdp.replace(/(a=mid:[^\r\n]+\r\n)/, `$1${extra.join('\r\n')}\r\n`);
}

const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

Run in headless Chrome 155, this negotiation produced the three expected outbound-rtp reports, all active.

Source: RFC 8853: Using Simulcast in SDP and RTP Sessions

Simulcast limitations

  • Codecs. Simulcast is codec independent in the specification, but browsers and hardware encoders differ in which codecs and how many layers they support; check by running the test above on the devices you target.

  • Uplink cost. Three layers cost about the sum of their bitrates. At the values above, the upload is up to 2.15 Mbit/s, and the browser’s congestion controller drops the highest layers first when the uplink cannot carry them.

  • Active layers. A layer set to active: false stops sending but keeps its SSRC and rid, so it can be turned back on without renegotiation. This is how Huddle saves uplink when nobody subscribes to the top layer.

  • Keyframes. When an SFU switches a receiver to another layer, it needs a keyframe of that layer. The SFU asks for it with an RTCP PLI or FIR message; the browser answers automatically.

Scalable video coding (SVC)

Simulcast sends several independent streams. Scalable video coding sends one stream that contains several nested layers: a base layer and enhancement layers that depend on it. An SFU can drop packets of the upper layers for one receiver without decoding or re-encoding anything, and all layers share one SSRC. The advantage over simulcast is efficiency (the layers are not independent copies of the same picture, so the total bitrate is lower) and finer adaptation; the cost is more complex encoders and dependence on the codec (VP9 and AV1 support it, VP8 supports temporal layers only, H.264 only temporal layers in WebRTC).

A layer is identified by a scalability mode string, defined by the W3C WebRTC-SVC specification and based on the AV1 and VP9 conventions. LxTy means x spatial layers (resolutions, a 2:1 ratio between layers) and y temporal layers (frame-rate layers). Related forms:

  • L1T2, L1T3: one spatial layer, two or three temporal layers. Frame rate can be reduced without changing the resolution; this is useful for noisy links and works with VP8, VP9, AV1 and H.264.

  • L2T3, L3T3: spatial layers with inter-layer prediction. Efficient, but a lost packet in a lower layer corrupts the upper layers until the next keyframe.

  • L3T3_KEY: K-SVC, where the spatial layers depend on lower layers only at keyframes. It is more robust to loss and is what most SFUs expect for VP9 and AV1 video conferences.

  • S2T1, S3T3: "simulcast on one SSRC" modes, which the specification defines but which addTransceiver() does not accept in sendEncodings.

  • A suffix h (for example L2T1h) means a 1.5:1 ratio between spatial layers instead of 2:1.

The dependency information travels in the RTP packets themselves. In the Chrome 155 test, the offers contained the RTP dependency descriptor header extension, which lets an SFU decide which packets to forward without understanding the codec payload.

Request SVC by putting scalabilityMode in a single encoding of sendEncodings:

/** Pick the first scalability mode that the browser can encode smoothly. */
export async function pickScalabilityMode(mimeType, candidates, width = 1280, height = 720) {
  for (const scalabilityMode of candidates) {
    const info = await navigator.mediaCapabilities.encodingInfo({
      type: 'webrtc',
      video: { contentType: mimeType, width, height, bitrate: 1_500_000, framerate: 30, scalabilityMode },
    });
    if (info.supported && info.smooth) return scalabilityMode;
  }
  return undefined;
}

/** Publish a camera track as one SVC stream (SFU mode, VP9 or AV1). */
export async function addSvcCamera(pc, track, stream, mimeType = 'video/VP9') {
  const mode = await pickScalabilityMode(mimeType, ['L3T3_KEY', 'L1T3', 'L1T2']);
  const transceiver = pc.addTransceiver(track, {
    direction: 'sendonly',
    streams: [stream],
    sendEncodings: [mode ? { scalabilityMode: mode, maxBitrate: 1_500_000 } : { maxBitrate: 1_500_000 }],
  });
  preferCodecs(transceiver, 'video', [mimeType]);
  return { transceiver, mode };
}

Test results in Chrome 155 (headless, macOS):

Codec and requested mode Result in outbound-rtp.scalabilityMode Remarks

VP9, L1T3

L1T3

Software libvpx

VP9, L3T3_KEY

L3T3_KEY

Software libvpx

AV1, L1T3

L1T3

Software libaom

AV1, L3T3_KEY

L3T3_KEY

Software libaom

VP8, L1T3

L1T3

Temporal layers only

VP8, L3T3

L1T2

Silently downgraded; spatial layers are not available for VP8

H.264, L1T3

L1T3

OpenH264 in this build

VP9, L9T9

error

addTransceiver() throws OperationError for a mode the codec cannot produce

Two things in the table deserve emphasis. First, an unknown mode is an exception, but a known mode the codec cannot do (L3T3 with VP8) is silently replaced by the nearest mode the encoder supports, so always read back getParameters().encodings[0].scalabilityMode or outbound-rtp.scalabilityMode and do not trust the request. Second, the support check for a mode with a given codec and resolution is navigator.mediaCapabilities.encodingInfo(). With type: 'webrtc' and a scalabilityMode in the video configuration, Chrome 155 answered that VP9 L1T3 and L3T3_KEY and AV1 L3T3_KEY are supported and smooth, VP8 L3T3 is not supported, and VP8 L1T2 is. The smooth and powerEfficient flags reflect whether the encode is expected to be real time and whether it is hardware accelerated; in this headless run none of the encodes was reported as power efficient, while decodes of VP9, AV1, H.265 and H.264 were.

Source: W3C: Scalable Video Coding (SVC) Extension for WebRTC

Choosing between simulcast and SVC

Simulcast SVC (K-SVC or temporal only)

Streams

One RTP stream per layer, each decodable alone

One RTP stream with dependent layers

Uplink

Sum of layers

Lower than the sum of independent layers, because the layers are not copies of the same picture

Codec support

Any codec, including H.264

VP9 and AV1 for spatial layers; temporal layers for VP8, VP9, AV1 and H.264

SFU support

Universal

Needs an SFU that understands the layer dependencies of the codec; check the SFU documentation

Failure mode

A lost layer affects only that layer

A lost base-layer packet affects all layers until a keyframe

Huddle uses simulcast with VP8 as the default because it works with every SFU and every device, and enables VP9 L3T3_KEY as an opt-in when pickScalabilityMode() reports support and the SFU advertises VP9.

Adapting quality with setParameters

sender.getParameters() returns the current RTCRtpSendParameters, and sender.setParameters(params) applies a modified copy. The object has a transactionId: you must pass the object you received, with only allowed fields changed, otherwise the call rejects. The allowed changes, verified in Chrome 155:

  • encodings[].active: stop or start a layer without renegotiation.

  • encodings[].maxBitrate (bits per second), maxFramerate, scaleResolutionDownBy (a value of 1 or more; a smaller value throws RangeError).

  • encodings[].priority and networkPriority (very-low, low, medium, high), which tell the browser how to share the bandwidth between senders.

  • degradationPreference: balanced, maintain-framerate, maintain-resolution or, in newer browsers, maintain-framerate-and-resolution. It tells the encoder what to give up first when it is short of CPU or bandwidth.

  • encodings[].dtx for audio.

Things that cannot change after the transceiver exists: the number and order of encodings, the rid values, the codecs and the header extensions. Trying to add an encoding (encodings.push({ rid: 'x' })) rejected with InvalidModificationError, and deleting the transactionId rejected with TypeError in the test. The call is asynchronous: await it, and do not issue overlapping getParameters()/setParameters() pairs, because the second one can fail when the first has changed the transaction id.

/** Quality tiers for the simulcast sender, applied with one setParameters call. */
const TIERS = {
  high: { active: ['q', 'h', 'f'] },
  medium: { active: ['q', 'h'] },
  low: { active: ['q'] },
};

export async function setTier(sender, tier) {
  const params = sender.getParameters(); // keeps the transactionId
  for (const enc of params.encodings) {
    enc.active = TIERS[tier].active.includes(enc.rid);
  }
  params.degradationPreference = 'maintain-framerate';
  await sender.setParameters(params);
  return params.encodings.map((e) => `${e.rid}:${e.active ? 'on' : 'off'}`).join(' ');
}

In the test, setTier(sender, 'medium') returned q:on h:on f:off, and after low the stats reported the h and f reports as active: false while the q layer kept sending at about 14 frames per second (capped by its maxFramerate of 15). maxFramerate: 15 set on a single-layer sender produced 14 fps in the stats.

Source: MDN: RTCRtpSender.setParameters()

Screen sharing, mesh and voice

Different content needs different trade-offs, and degradationPreference expresses them. Camera video prefers maintain-framerate, so motion stays smooth and resolution drops. Screen sharing prefers maintain-resolution, so text stays readable while the frame rate falls, and the track’s contentHint ('detail' or 'text' for screen content, 'motion' for video) lets the browser choose sensible defaults. In mesh mode a participant uploads once per remote peer, so the application must split the uplink budget:

/** Screen sharing: keep text sharp, let the frame rate drop. */
export async function tuneScreenShare(sender, track) {
  track.contentHint = 'detail';
  const params = sender.getParameters();
  params.degradationPreference = 'maintain-resolution';
  params.encodings[0].maxBitrate = 2_500_000;
  params.encodings[0].maxFramerate = 15;
  await sender.setParameters(params);
}

/** Mesh mode: split an upload budget between the remote peers. */
export async function capMeshUpload(senders, totalBitsPerSecond) {
  const each = Math.floor(totalBitsPerSecond / Math.max(senders.length, 1));
  for (const sender of senders) {
    const params = sender.getParameters();
    params.encodings[0].maxBitrate = each;
    await sender.setParameters(params);
  }
}

/** Opus tuning that does not need SDP munging: a bitrate ceiling. */
export async function tuneVoice(sender, maxBitrate = 32_000) {
  const params = sender.getParameters();
  params.encodings[0].maxBitrate = maxBitrate;
  await sender.setParameters(params);
}

Test: tuneScreenShare produced contentHint detail, degradationPreference maintain-resolution, maxBitrate 2,500,000 and maxFramerate 15, and a following capMeshUpload call with a single sender replaced the bitrate with 1,800,000. tuneVoice set an Opus encoding maxBitrate of 32,000.

Source: W3C: Media Capture and Streams Extensions: Content Hints

Per-layer statistics

Whatever you change, read the result from getStats(). The outbound-rtp report has one entry per encoding with rid, active, frameWidth, frameHeight, framesPerSecond, scalabilityMode and qualityLimitationReason (none, cpu, bandwidth or other):

/** Per-layer outbound report. */
export async function layerReport(pc) {
  const rows = [];
  (await pc.getStats()).forEach((s) => {
    if (s.type === 'outbound-rtp' && s.kind === 'video') {
      rows.push({ rid: s.rid, active: s.active, size: `${s.frameWidth}x${s.frameHeight}`, fps: s.framesPerSecond, mode: s.scalabilityMode, limit: s.qualityLimitationReason });
    }
  });
  return rows.sort((a, b) => String(a.rid).localeCompare(String(b.rid)));
}

In the simulcast test this printed the f, h and q layers at 1280x720, 640x360 and 320x180 and about 20 frames per second for the larger ones; the numbers in a real call depend on the camera, the encoder and the uplink. The report fields are described in Stats, Quality and Bandwidth.

Source: W3C: Identifiers for WebRTC’s Statistics API, RTCOutboundRtpStreamStats

The Huddle publisher, assembled

The module is consumed from the call code. In SFU mode, the session starts like this (the signalling is covered in Signaling and Perfect Negotiation):

import { addSimulcastCamera, addSvcCamera, preferCodecs, setTier, tuneVoice } from './huddle-media.js';

const stream = await navigator.mediaDevices.getUserMedia({ audio: true, video: { width: 1280, height: 720 } });
const pc = new RTCPeerConnection({ iceServers });

const audio = pc.addTransceiver(stream.getAudioTracks()[0], { direction: 'sendonly', streams: [stream] });
preferCodecs(audio, 'audio', ['audio/opus']);
await tuneVoice(audio.sender);

const useSvc = room.codecs.includes('video/VP9') && room.svc === true;
const video = useSvc
  ? (await addSvcCamera(pc, stream.getVideoTracks()[0], stream, 'video/VP9')).transceiver
  : addSimulcastCamera(pc, stream.getVideoTracks()[0], stream, ['video/VP8']);

// When the SFU reports that nobody watches the top layer, release the uplink:
signalling.on('layer-demand', ({ highest }) => setTier(video.sender, highest === 'f' ? 'high' : 'medium'));

Source: MDN: RTCPeerConnection.addTransceiver()

This sketch relies on a signalling channel (signalling), an iceServers list and a room object that the application defines; they are not part of the verified module. With mediasoup, the equivalent calls are the producer’s encodings and codecOptions, and LiveKit’s client takes videoSimulcastLayers and videoCodec publish options.

Common errors and pitfalls

  • Simulcast sends one layer. The answer lacks a=simulcast:recv. An SFU or the test harness above must add it.

  • setCodecPreferences() throws. The array contains an entry that is not in the receiver capabilities, or contains only rtx/red/ulpfec. Always build the array from getCapabilities().

  • The codec does not change. The call happened after createOffer(), or no new negotiation followed. Preferences apply to the next offer/answer.

  • scalabilityMode silently ignored or replaced. Read it back from getParameters() and stats.

  • InvalidModificationError on setParameters(). You tried to change a fixed field (rid, number of encodings, codecs). Only change the fields listed above.

  • TypeError or InvalidStateError. The parameters object was constructed by hand or reused after another setParameters() call; always start from a fresh getParameters().

  • H.264 mismatch. Two endpoints with different profile-level-id or packetization-mode values will not agree on H.264 even though both list it.

  • Encoders limit simulcast. An encoder may produce fewer streams than requested; fall back to another codec such as VP8 when the stats show fewer outbound-rtp reports than requested layers.

  • Safari and Firefox differences. scaleResolutionDownBy and some sendEncodings options have historically lagged in Safari and Firefox. MDN suggests applying the resolution with track.applyConstraints() when scaleResolutionDownBy stays at 1. Test the matrix you support.

  • Overlapping setParameters() calls. Serialise them in a queue per sender.

Security and privacy considerations

  • Fingerprinting. The codec list, the H.264 profiles, the header extensions and the hardware encoder presence are a fairly stable device fingerprint available without any permission. Do not log the full capabilities together with user identifiers, and do not build features that depend on the exact list.

  • Resource exhaustion. A hostile or buggy remote peer cannot choose your codec, but in an SFU the publisher’s settings determine server load. Validate the codecs, rids, bitrates and number of layers the publisher asks for on the server side, and cap them in the router or room configuration.

  • Bandwidth abuse. maxBitrate is a request made by the client. Enforce limits in the SFU, not in the page.

  • Encrypted media. Codecs and layers are visible to the SFU because it needs the RTP header extensions and dependency descriptors to forward packets. With end-to-end encryption, the payload is opaque but these headers must stay readable; see End-to-End Encryption.

Version and specification notes

  • RFC 7742 (video) and RFC 7874 (audio) define the mandatory codecs, and they date from 2016. VP8 and H.264 remain the two mandatory video codecs; VP9 and AV1 are widely deployed optional codecs.

  • setCodecPreferences() is listed by MDN as Baseline 2024, so it is available in all current browsers.

  • Simulcast through sendEncodings and rid is specified in WebRTC 1.0 together with RFC 8851 and RFC 8853 (both published in January 2021). Firefox and Safari implement it with differing coverage, which is why SFU vendors still document per-browser limitations.

  • The W3C WebRTC-SVC extension is a Working Draft (2026-09-14) and Chrome ships its scalabilityMode support; the S modes are defined but not accepted by addTransceiver().

  • H.265 in WebRTC is governed by an IETF draft (draft-ietf-avtcore-hevc-webrtc) and its availability depends on hardware, the browser and licensing.

  • AV1 in WebRTC is specified by the AV1 RTP payload format and the dependency descriptor header extension.

See also

References