On this pageOPENCLOSE
Media transport: DTLS, SRTP & codecs
Follow media through encryption, congestion control, simulcast, and SVC, and understand the cost of every quality knob.
Goal
- 01Follow a frame from capture through encoder, RTP, congestion control, jitter buffer, and decoder.
- 02Explain what DTLS-SRTP protects and which parts of a WebRTC session are not encrypted end to end.
- 03Use simulcast and codec controls without assuming they fix a bad network.
The concept, plainly
Media does not travel as a `MediaStream` after it leaves the browser. The encoder produces encoded frames, RTP packetizes them, and SRTP encrypts the packets. The remote jitter buffer reorders packets by sequence number and timestamps, then the decoder produces frames for the viewer. Each stage can reduce quality independently: capture resolution, encoder target, network loss, or receiver decode capacity.
creates an authenticated, encrypted transport between the peers. uses keys derived from that DTLS handshake to encrypt RTP and SRTCP. This protects media in transit against passive observers, but the signaling server and SFU may still see metadata or decrypted media depending on the topology. WebRTC's built-in encryption is hop-by-hop through an SFU, not automatically end-to-end between users.
RTP carries sequence numbers, timestamps, payload type, and the encoded frame fragments. RTCP carries sender and receiver reports: packets lost, jitter, round-trip time, and receiver-estimated bandwidth. Congestion control responds to those reports with sender-side bandwidth estimation, adjusting resolution, frame rate, and bitrate. The browser may also reduce quality because of CPU pressure, not just network conditions.
Codecs are a negotiated trade-off. VP8, VP9, H.264, and AV1 differ in CPU cost, bandwidth efficiency, and hardware support. Screen content benefits from a codec and content hint that preserve sharp text. A codec that looks efficient in a file comparison can be disastrous on a low-end phone if it lacks hardware decode. Choose from the devices and networks you actually serve.
Simulcast sends multiple independently encoded spatial layers (for example 180p, 360p, 720p) and lets an SFU forward the right layer to each receiver. SVC encodes layers within one stream and can adapt more finely, but support varies. Both are topology-dependent: peer-to-peer receives one encoding unless you renegotiate; an SFU can choose per subscriber. Do not enable simulcast and assume the browser automatically uses the best layer without a server.
See it move
Interactive diagram
The media pipeline and its feedback loop
Capture produces raw media
Camera and microphone deliver frames and samples. Resolution and frame rate set the upper bound on everything downstream.
Encoding, packetization, encryption, congestion control, and jitter buffering are separate stages with separate failure signals.
Behind the scenes
A connected call can still be one-way
Encryption and transport can be healthy while a transceiver direction is wrong, a sender has no track, or the remote element renders a muted stream. Inspect sender parameters, transceiver direction, and negotiated codecs rather than assuming the network is at fault.
SFUs change the encryption boundary
In a mesh, DTLS-SRTP terminates at the peer. With an SFU, each participant has a DTLS-SRTP hop to the server, and the server forwards or re-encrypts media to others. That is transport encryption, not necessarily end-to-end encryption. If the product promises E2EE, it needs a separate frame-level scheme and a clear key management story.
Simulcast is a publisher and subscriber decision
The publisher encodes multiple resolutions. The SFU decides which layer each subscriber receives and can switch when bandwidth changes. A peer-to-peer call has no SFU to choose layers, so enabling simulcast alone does not upgrade quality. The topology and the server policy matter as much as the browser API.
Code walkthrough
Simulcast works when an SFU can select layers. The sender produces multiple encodings; the server decides who receives which.
const transceiver = pc.addTransceiver(track, {direction: "sendonly",streams: [stream],sendEncodings: [{ rid: "q", scaleResolutionDownBy: 4, maxBitrate: 150_000 },{ rid: "h", scaleResolutionDownBy: 2, maxBitrate: 500_000 },{ rid: "f", scaleResolutionDownBy: 1, maxBitrate: 1_500_000 },],});const offer = await pc.createOffer();await pc.setLocalDescription(offer);// The SFU must support and select simulcast layers. A P2P peer// receives only the encoding negotiation it can actually use.void transceiver;
Do and don't
Do
Measure the whole pipeline
Capture constraints, sender target, network loss, receiver jitter, and decode frame rate are different numbers.
Choose codecs for your audience
Hardware decode on the devices you support matters more than a benchmark on your laptop.
Treat SFU encryption honestly
Say hop-by-hop when media is decrypted at the SFU. Use a separate E2EE architecture if privacy requires it.
Let congestion control work
Set sensible maximum bitrates and avoid forcing tiny buffers or exact resolutions that fight adaptation.
Don't
Don't blame packet loss for every stutter
The sender may be CPU-limited, the receiver may be decode-limited, or the video element may be the bottleneck.
Don't assume simulcast works everywhere
It requires browser support and an SFU that subscribes to layers. P2P does not get the same benefit.
Don't force a single codec without fallback
A codec unsupported by the other side or the SFU can eliminate media entirely. Negotiate and verify.
Don't call DTLS-SRTP end-to-end by default
Without a specific frame-level E2EE design, an SFU can see decrypted media.
Practice
Hands-on lab
Watch adaptation in real time
The 1:1 lab shows outbound and inbound RTP stats, bitrate, resolution, frame rate, jitter, loss, and the selected path. Change the test pattern and observe the numbers move.
- 1Connect two tabs with the synthetic feed and find FPS, frame size, and bitrate.
- 2Move one tab to a throttled network or pause the animation; compare sender and receiver metrics.
- 3Find the qualityLimitationReason after CPU stress or bandwidth limits.
- 4Explain the difference between outbound bytes and inbound bytes for the same call.
Solo? Loopback gives both sender and receiver stats in one tab. It is ideal for learning the field names without network variability.
Quick quiz
Knowledge check
0 of 4 answered
- 01
What does DTLS-SRTP protect?
- 02
What does RTCP feedback to the sender?
- 03
Why can a jitter buffer increase latency?
- 04
What is the main advantage of simulcast in an SFU topology?
Key takeaways
- Media quality is a chain: capture, encode, packetize, encrypt, network, buffer, decode, render.01
- DTLS-SRTP secures each hop, not automatically every participant end to end.02
- RTCP feedback drives bitrate and resolution adaptation.03
- Codec and simulcast choices only help when the devices and topology support them.04
Common errors and fixes
Terms in this lesson
Done with this lesson?
Mark it complete, then keep the thread.