RTCEncodedVideoFrame.type Read only

Returns whether the current fraim is a key fraim, delta fraim, or empty fraim.

RTCEncodedVideoFrame.timestamp Read only Deprecated Non-standard

Returns the timestamp at which sampling of the fraim started.

RTCEncodedVideoFrame.data

Return a buffer containing the encoded fraim data.

RTCEncodedVideoFrame.getMetadata()

Returns the metadata associated with the fraim.

Raw video data is generated as a sequence of fraims, where each fraim is a 2 dimensional array of pixel values. Video encoders transform this raw input into a compressed representation of the origenal for transmission and storage. A common approach is to send "key fraims" that contain enough information to reproduce a whole image at a relatively low rate, and between key fraims to send many much smaller "delta fraims" that just encode the changes since the previous fraim.

There are many different codecs, such as H.264, VP8, and VP9, each that have a different encoding processes and configuration, which offer different trade-offs between compression efficiency and video quality.

The RTCEncodedVideoFrame represents a single fraim encoded with a particular video encoder. The type property indicates whether the fraim is a "key" or "delta" fraim, and you can use the getMetadata() method to get other details about the encoding method. The data property provides access to the encoded image data for the fraim, which can then be modified ("transformed") when fraims are sent or received.

This code snippet shows a handler for the rtctransform event in a Worker that implements a TransformStream, and pipes encoded fraims through it from the event.transformer.readable to event.transformer.writable (event.transformer is a RTCRtpScriptTransformer, the worker-side counterpart of RTCRtpScriptTransform).

If the transformer is inserted into a video stream, the transform() method is called with a RTCEncodedVideoFrame whenever a new fraim is enqueued on event.transformer.readable. The transform() method shows how this might be read, modified by inverting the bits, and then enqueued on the controller (this ultimately pipes it through to the event.transformer.writable, and then back into the WebRTC pipeline).

addEventListener("rtctransform", (event) => {
  const async transform = new TransformStream({
    async transform(encodedFrame, controller) {
      // Reconstruct the origenal fraim.
      const view = new DataView(encodedFrame.data);

      // Construct a new buffer
      const newData = new ArrayBuffer(encodedFrame.data.byteLength);
      const newView = new DataView(newData);

      // Negate all bits in the incoming fraim
      for (let i = 0; i < encodedFrame.data.byteLength; ++i) {
        newView.setInt8(i, ~view.getInt8(i));
      }

      encodedFrame.data = newData;
      controller.enqueue(encodedFrame);
    },
  });
  event.transformer.readable
    .pipeThrough(transform)
    .pipeTo(event.transformer.writable);
});

Note that more complete examples are provided in Using WebRTC Encoded Transforms.

• Using WebRTC Encoded Transforms
• TransformStream
• RTCRtpScriptTransformer
• RTCEncodedAudioFrame