webrtc_frame_buffer (webrtc v0.1.0)
View SourceFrame assembly and jitter buffer for one received RTP stream. No I/O: packets go in with push/4, frames come out as actions, and the caller runs timeout/2 at next_timeout/1.
Packets are put in sequence order and duplicates dropped (RFC 3550 §A.1). The packets of one timestamp form a frame, complete once the packets from its first one to the one with the marker bit are all there. The first packet is the one the payload format marks (VP8 S bit and partition 0, VP9 B bit, AV1 N bit), or the one after a packet of another frame. An H.264 packet that can start an access unit (ITU-T H.264 §7.4.1.2.3) is taken as the first once the packets missing before it are given up.
Timestamps must rise with sequence numbers: H.264 with B frames, which WebRTC encoders do not send, is not supported.
A frame leaves in order once it is complete and decodable:
- a key frame always is;
- VP8, and VP9 in non-flexible mode, with picture IDs: a frame of the base temporal layer when the previous base frame left (TL0PICIDX), a higher layer frame when its base frame left and every frame between them did, or it is a layer sync frame;
- VP9 in flexible mode: when every picture it references left;
- VP8 and VP9 with picture IDs only: the picture after the last one that left;
- other video (H.264, AV1, and VP8 and VP9 without picture IDs): when it follows the last frame that left with no packet missing;
- audio: every frame.
It leaves at its playout time: its timestamp, plus the smallest transit time of the last 10 s, plus the target delay, three times the interarrival jitter of frames (RFC 3550 §6.4.1) within min_delay and max_delay (and the playout-delay extension when the sender sets it). A complete frame that comes later leaves at once.
A frame that misses packets is waited for until a later frame is complete and its playout time has passed by 100 ms, for packets that come out of order, or by three round trip times when the stream is repaired by NACK (two retransmissions); until lost/3 says a missing packet will not come; or for 3 s. It is then dropped with the frames that depend on it, and a key frame is requested (at most once per round trip and never more than five times a second).
Actions: {frame, Frame} and request_keyframe.
Summary
Functions
The sender will not repeat these packets (NACK gave up on them).
A buffer for one stream. Options: kind (audio or video), clock_rate, codecs ([#{pt, name}]), nack (the stream is repaired by retransmission), min_delay and max_delay in ms (default 0 and 10000).
A packet of the stream that carries no media (an FEC packet, padding): its sequence number is no gap.
Add a packet received at Now (ms). Opts: playout_delay, the {Min, Max} of the playout-delay extension it carried.
The round trip time of the path, in ms.
Counters (W3C RTCInboundRtpStreamStats names, in snake case; times in seconds) and the current frame size, rate and target delay.
Types
-type action() :: {frame, frame()} | request_keyframe.
-type frame() :: #{timestamp := 0..4294967295, keyframe := boolean(), data := binary(), codec := binary(), width => undefined | pos_integer(), height => undefined | pos_integer()}.
-opaque state()
Functions
The sender will not repeat these packets (NACK gave up on them).
A buffer for one stream. Options: kind (audio or video), clock_rate, codecs ([#{pt, name}]), nack (the stream is repaired by retransmission), min_delay and max_delay in ms (default 0 and 10000).
A packet of the stream that carries no media (an FEC packet, padding): its sequence number is no gap.
-spec push(#rtp{version :: 2, padding :: 0..255, marker :: boolean(), pt :: 0..127, seq :: 0..65535, ts :: 0..4294967295, ssrc :: 0..4294967295, csrcs :: [0..4294967295], ext :: undefined | {Profile :: 0..65535, Data :: binary()} | [webrtc_rtp_ext:element()], payload :: binary()}, integer(), map(), state()) -> {[action()], state()}.
Add a packet received at Now (ms). Opts: playout_delay, the {Min, Max} of the playout-delay extension it carried.
-spec set_rtt(non_neg_integer(), state()) -> state().
The round trip time of the path, in ms.
Counters (W3C RTCInboundRtpStreamStats names, in snake case; times in seconds) and the current frame size, rate and target delay.