webrtc_media_codec behaviour (webrtc v0.1.0)
View SourceA codec plugin decodes the frames a transceiver receives and encodes what the application sends on it, and reports the W3C decoder and encoder statistics. This library stays pure Erlang; a plugin is where native code (FFmpeg, libvpx, Opus) comes in, in a separate package.
Set it on a transceiver that delivers frames:
webrtc:add_transceiver(Pc, video, #{frames => true, codec_plugin => {my_codec, Opts}})Each frame received goes to decode/2; every output comes as {webrtc, Pc, {decoded, Mid, Out}}, after the {frame, Mid, Frame} event. webrtc:send_media/3 hands raw media to encode/2 and sends the frames it returns. The plugin runs in the transport process: keep each call short, or hand the work to a process of your own.
Stats are W3C members of the stream (framesDecoded, totalDecodeTime, keyFramesEncoded, ...): counters add up across calls, the others replace the previous value, as with webrtc:report_media_stats/4. info/1 gives decoderImplementation or encoderImplementation.
Loss indications (RFC 8834 §5.1.3, §5.1.4), when the transceiver negotiated nack sli or nack rpsi:
decode/2may return{ok, Outs, Stats, State, Feedback}, with{slice_loss, [{First, Number, PictureId}]},{reference_picture, Pt, Bits}orkeyframe_requestinFeedback, for the library to send to the remote encoder.- The optional
feedback/2gets what the remote decoder reports about the stream it encodes,{slice_loss, Entries}or{reference_picture, Pt, Bits}, to repair the slices or change its reference picture.
A plugin that raises or returns an error is turned off for that transceiver and the owner gets {codec_plugin_error, Mid, Reason}; frames keep coming.
Summary
Types
Callbacks
-callback feedback(webrtc_media_fb:indication(), State :: term()) -> {ok, State :: term()} | {error, term()}.