draft-ietf-payload-vp9-13.txt   draft-ietf-payload-vp9-14.txt 
AVTCore Working Group J. Uberti AVTCore Working Group J. Uberti
Internet-Draft S. Holmer Internet-Draft S. Holmer
Intended status: Standards Track M. Flodman Intended status: Standards Track M. Flodman
Expires: 8 November 2021 D. Hong Expires: 5 December 2021 D. Hong
Google Google
J. Lennox J. Lennox
8x8 / Jitsi 8x8 / Jitsi
7 May 2021 3 June 2021
RTP Payload Format for VP9 Video RTP Payload Format for VP9 Video
draft-ietf-payload-vp9-13 draft-ietf-payload-vp9-14
Abstract Abstract
This specification describes an RTP payload format for the VP9 video This specification describes an RTP payload format for the VP9 video
codec. The payload format has wide applicability, as it supports codec. The payload format has wide applicability, as it supports
applications from low bit-rate peer-to-peer usage, to high bit-rate applications from low bit-rate peer-to-peer usage, to high bit-rate
video conferences. It includes provisions for temporal and spatial video conferences. It includes provisions for temporal and spatial
scalability. scalability.
Status of This Memo Status of This Memo
skipping to change at page 1, line 38 skipping to change at page 1, line 38
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on 8 November 2021. This Internet-Draft will expire on 5 December 2021.
Copyright Notice Copyright Notice
Copyright (c) 2021 IETF Trust and the persons identified as the Copyright (c) 2021 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/ Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document. license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights Please review these documents carefully, as they describe your rights
skipping to change at page 2, line 17 skipping to change at page 2, line 17
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Conventions, Definitions and Acronyms . . . . . . . . . . . . 3 2. Conventions, Definitions and Acronyms . . . . . . . . . . . . 3
3. Media Format Description . . . . . . . . . . . . . . . . . . 3 3. Media Format Description . . . . . . . . . . . . . . . . . . 3
4. Payload Format . . . . . . . . . . . . . . . . . . . . . . . 5 4. Payload Format . . . . . . . . . . . . . . . . . . . . . . . 5
4.1. RTP Header Usage . . . . . . . . . . . . . . . . . . . . 5 4.1. RTP Header Usage . . . . . . . . . . . . . . . . . . . . 5
4.2. VP9 Payload Descriptor . . . . . . . . . . . . . . . . . 6 4.2. VP9 Payload Descriptor . . . . . . . . . . . . . . . . . 6
4.2.1. Scalability Structure (SS): . . . . . . . . . . . . . 11 4.2.1. Scalability Structure (SS): . . . . . . . . . . . . . 11
4.3. Frame Fragmentation . . . . . . . . . . . . . . . . . . . 12 4.3. Frame Fragmentation . . . . . . . . . . . . . . . . . . . 13
4.4. Scalable encoding considerations . . . . . . . . . . . . 13 4.4. Scalable encoding considerations . . . . . . . . . . . . 13
4.5. Examples of VP9 RTP Stream . . . . . . . . . . . . . . . 13 4.5. Examples of VP9 RTP Stream . . . . . . . . . . . . . . . 13
4.5.1. Reference picture use for scalable structure . . . . 13 4.5.1. Reference picture use for scalable structure . . . . 14
5. Feedback Messages and Header Extensions . . . . . . . . . . . 14 5. Feedback Messages and Header Extensions . . . . . . . . . . . 14
5.1. Reference Picture Selection Indication (RPSI) . . . . . . 14 5.1. Reference Picture Selection Indication (RPSI) . . . . . . 15
5.2. Full Intra Request (FIR) . . . . . . . . . . . . . . . . 15 5.2. Full Intra Request (FIR) . . . . . . . . . . . . . . . . 15
5.3. Layer Refresh Request (LRR) . . . . . . . . . . . . . . . 15 5.3. Layer Refresh Request (LRR) . . . . . . . . . . . . . . . 15
6. Payload Format Parameters . . . . . . . . . . . . . . . . . . 16 6. Payload Format Parameters . . . . . . . . . . . . . . . . . . 16
6.1. SDP Parameters . . . . . . . . . . . . . . . . . . . . . 17 6.1. SDP Parameters . . . . . . . . . . . . . . . . . . . . . 18
6.1.1. Mapping of Media Subtype Parameters to SDP . . . . . 18 6.1.1. Mapping of Media Subtype Parameters to SDP . . . . . 18
6.1.2. Offer/Answer Considerations . . . . . . . . . . . . . 18 6.1.2. Offer/Answer Considerations . . . . . . . . . . . . . 19
7. Media Type Definition . . . . . . . . . . . . . . . . . . . . 19 7. Media Type Definition . . . . . . . . . . . . . . . . . . . . 19
8. Security Considerations . . . . . . . . . . . . . . . . . . . 21 8. Security Considerations . . . . . . . . . . . . . . . . . . . 21
9. Congestion Control . . . . . . . . . . . . . . . . . . . . . 21 9. Congestion Control . . . . . . . . . . . . . . . . . . . . . 21
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 21 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22
11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 22 11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 22
12. References . . . . . . . . . . . . . . . . . . . . . . . . . 22 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 22
12.1. Normative References . . . . . . . . . . . . . . . . . . 22 12.1. Normative References . . . . . . . . . . . . . . . . . . 22
12.2. Informative References . . . . . . . . . . . . . . . . . 23 12.2. Informative References . . . . . . . . . . . . . . . . . 23
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24
1. Introduction 1. Introduction
This specification describes an RTP payload specification applicable This specification describes an RTP [RFC3550] payload specification
to the transmission of video streams encoded using the VP9 video applicable to the transmission of video streams encoded using the VP9
codec [VP9-BITSTREAM]. The format described in this document can be video codec [VP9-BITSTREAM]. The format described in this document
used both in peer-to-peer and video conferencing applications. can be used both in peer-to-peer and video conferencing applications.
The VP9 video codec was developed by Google, and is the successor to The VP9 video codec was developed by Google, and is the successor to
its earlier VP8 [RFC6386] codec. Above the compression improvements its earlier VP8 [RFC6386] codec. Above the compression improvements
and other general enhancements above VP8, VP9 is also designed in a and other general enhancements above VP8, VP9 is also designed in a
way that allows spatially-scalable video encoding. way that allows spatially-scalable video encoding.
2. Conventions, Definitions and Acronyms 2. Conventions, Definitions and Acronyms
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
skipping to change at page 3, line 48 skipping to change at page 3, line 48
both spatial and quality layers. both spatial and quality layers.
This payload format specification defines how such temporal and This payload format specification defines how such temporal and
spatial scalability layers can be described and communicated. spatial scalability layers can be described and communicated.
Temporal and spatial scalability layers are associated with non- Temporal and spatial scalability layers are associated with non-
negative integer IDs. The lowest layer of either type has an ID of negative integer IDs. The lowest layer of either type has an ID of
0, and is sometimes referred to as the "base" temporal or spatial 0, and is sometimes referred to as the "base" temporal or spatial
layer. layer.
Layers are designed (and MUST be encoded) such that if any layer, and Layers are designed, and MUST be encoded, such that if any layer, and
all higher layers, are removed from the bitstream along either of the all higher layers, are removed from the bitstream along either the
two dimensions, the remaining bitstream is still correctly decodable. spatial or temporal dimension, the remaining bitstream is still
correctly decodable.
For terminology, this document uses the term "frame" to refer to a For terminology, this document uses the term "frame" to refer to a
single encoded VP9 frame for a particular resolution/quality, and single encoded VP9 frame for a particular resolution/quality, and
"picture" to refer to all the representations (frames) at a single "picture" to refer to all the representations (frames) at a single
instant in time. A picture thus consists of one or more frames, instant in time. A picture thus consists of one or more frames,
encoding different spatial layers. encoding different spatial layers.
Within a picture, a frame with spatial layer ID equal to SID, where Within a picture, a frame with spatial layer ID equal to SID, where
SID > 0, can depend on a frame of the same picture with a lower SID > 0, can depend on a frame of the same picture with a lower
spatial layer ID. This "inter-layer" dependency can result in spatial layer ID. This "inter-layer" dependency can result in
skipping to change at page 4, line 40 skipping to change at page 4, line 40
"flexible mode" and "non-flexible mode". An encoder can only switch "flexible mode" and "non-flexible mode". An encoder can only switch
between the two on the first packet of a key frame with temporal between the two on the first packet of a key frame with temporal
layer ID equal to 0. layer ID equal to 0.
In flexible mode, each packet can contain up to 3 reference indices, In flexible mode, each packet can contain up to 3 reference indices,
which identify all frames referenced by the frame transmitted in the which identify all frames referenced by the frame transmitted in the
current packet for inter-picture prediction. This (along with the D current packet for inter-picture prediction. This (along with the D
bit) enables a receiver to identify if a frame is decodable or not bit) enables a receiver to identify if a frame is decodable or not
and helps it understand the temporal layer structure. Since this is and helps it understand the temporal layer structure. Since this is
signaled in each packet it makes it possible to have very flexible signaled in each packet it makes it possible to have very flexible
temporal layer hierarchies and patterns which are changing temporal layer hierarchies, and scalability structures which are
dynamically. changing dynamically.
In non-flexible mode, the inter-picture dependency (the reference In non-flexible mode, frames are encoded using a fixed, recurring
indices) of a Picture Group (PG) MUST be pre-specified as part of the pattern of dependencies; the set of pictures that recur in this
scalability structure (SS) data. In this mode, each packet has an pattern is known as a Picture Group (PG). In this mode, the inter-
index to refer to one of the described pictures in the PG, from which picture dependencies (the reference indices) of the Picture Group
the pictures referenced by the picture transmitted in the current MUST be pre-specified as part of the scalability structure (SS) data.
packet for inter-picture prediction can be identified. A Picture Group is a recurring pattern of spatial and temporal
dependencies which In this mode, each packet has an index to refer to
one of the described pictures in the PG, from which the pictures
referenced by the picture transmitted in the current packet for
inter-picture prediction can be identified.
(Note: A "Picture Group", as used in this document, is not the same (Note: A "Picture Group", as used in this document, is not the same
thing as a the term "Group of Pictures" as it is traditionally used thing as the term "Group of Pictures" as it is traditionally used in
in video coding, i.e. to mean an independently-decoadable run of video coding, i.e. to mean an independently-decoadable run of
pictures beginning with a keyframe.) pictures beginning with a keyframe.)
The SS data can also be used to specify the resolution of each The SS data can also be used to specify the resolution of each
spatial layer present in the VP9 stream for both flexible and non- spatial layer present in the VP9 stream for both flexible and non-
flexible modes. flexible modes.
4. Payload Format 4. Payload Format
This section describes how the encoded VP9 bitstream is encapsulated This section describes how the encoded VP9 bitstream is encapsulated
in RTP. To handle network losses usage of RTP/AVPF [RFC4585] is in RTP. To handle network losses usage of RTP/AVPF [RFC4585] is
skipping to change at page 5, line 40 skipping to change at page 5, line 40
| timestamp | | timestamp |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| synchronization source (SSRC) identifier | | synchronization source (SSRC) identifier |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| contributing source (CSRC) identifiers | | contributing source (CSRC) identifiers |
| .... | | .... |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| VP9 payload descriptor (integer #octets) | | VP9 payload descriptor (integer #octets) |
: : : :
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| : VP9 pyld hdr | | | : |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| | | |
+ | + |
: Bytes 2..N of VP9 payload : : VP9 payload :
| | | |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| : OPTIONAL RTP padding | | : OPTIONAL RTP padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 1 Figure 1
The VP9 payload descriptor will be described in Section 4.2; the VP9 The VP9 payload descriptor will be described in Section 4.2; the VP9
payload header is described in [VP9-BITSTREAM]. OPTIONAL RTP padding payload is described in [VP9-BITSTREAM]. OPTIONAL RTP padding MUST
MUST NOT be included unless the P bit is set. The figure NOT be included unless the P bit is set.
specifically shows the format for the first packet in a frame.
Subsequent packets will not contain the VP9 payload header, and will
have later octets in the frame payload.
Marker bit (M): MUST be set to 1 for the final packet of the highest Marker bit (M): MUST be set to 1 for the final packet of the highest
spatial layer frame (the final packet of the picture), and 0 spatial layer frame (the final packet of the picture), and 0
otherwise. Unless spatial scalability is in use for this picture, otherwise. Unless spatial scalability is in use for this picture,
this will have the same value as the E bit described below. Note this will have the same value as the E bit described below. Note
this bit MUST be set to 1 for the target spatial layer frame if a this bit MUST be set to 1 for the target spatial layer frame if a
stream is being rewritten to remove higher spatial layers. stream is being rewritten to remove higher spatial layers.
Payload Type (PT): In line with the policy in Section 3 of Payload Type (PT): In line with the policy in Section 3 of
[RFC3551], applications using the VP9 RTP payload profile MUST [RFC3551], applications using the VP9 RTP payload profile MUST
assign a dynamic payload type number to be used in each RTP assign a dynamic payload type number to be used in each RTP
session and provide a mechanism to indicate the mapping. See session and provide a mechanism to indicate the mapping. See
Section 6.1 for the mechanism to be used with the Session Section 6.1 for the mechanism to be used with the Session
Description Protocol (SDP) [RFC8866]. Description Protocol (SDP) [RFC8866].
Timestamp: The RTP timestamp indicates the time when the input frame Timestamp: The RTP timestamp [RFC3550] indicates the time when the
was sampled, at a clock rate of 90 kHz. If the input picture is input frame was sampled, at a clock rate of 90 kHz. If the input
encoded with multiple layer frames, all of the frames of the picture is encoded with multiple layer frames, all of the frames
picture MUST have the same timestamp. of the picture MUST have the same timestamp.
If a frame has the VP9 show_frame field set to 0 (i.e., it is If a frame has the VP9 show_frame field set to 0 (i.e., it is
meant only to populate a reference buffer, without being output) meant only to populate a reference buffer, without being output)
its timestamp MAY alternatively be set to be the same as the its timestamp MAY alternatively be set to be the same as the
subsequent frame with show_frame equal to 1. (This will be subsequent frame with show_frame equal to 1. (This will be
convenient for playing out pre-encoded content packaged with VP9 convenient for playing out pre-encoded content packaged with VP9
"superframes", which typically bundle show_frame==0 frames with a "superframes", which typically bundle show_frame==0 frames with a
subsequent show_frame==1 frame.) Every frame with show_frame==1, subsequent show_frame==1 frame.) Every frame with show_frame==1,
however, MUST have a unique timestamp modulo the 2^32 wrap of the however, MUST have a unique timestamp modulo the 2^32 wrap of the
field. field.
skipping to change at page 7, line 47 skipping to change at page 7, line 47
| TL0PICIDX | (Conditionally REQUIRED) | TL0PICIDX | (Conditionally REQUIRED)
+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+
V: | SS | V: | SS |
| .. | | .. |
+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+
Figure 3 Figure 3
I: Picture ID (PID) present. When set to one, the OPTIONAL PID MUST I: Picture ID (PID) present. When set to one, the OPTIONAL PID MUST
be present after the mandatory first octet and specified as below. be present after the mandatory first octet and specified as below.
Otherwise, PID MUST NOT be present. If the SS field was present Otherwise, PID MUST NOT be present. If the V bit was set in the
in the stream's most recent start of a keyframe (i.e., non- stream's most recent start of a keyframe (i.e. the SS field was
flexible scalability mode is in use), then the PID MUST also be present, and non-flexible scalability mode is in use), then this
present in every packet. bit MUST be set on every packet.
P: Inter-picture predicted frame. When set to zero, the frame does P: Inter-picture predicted frame. When set to zero, the frame does
not utilize inter-picture prediction. In this case, up-switching not utilize inter-picture prediction. In this case, up-switching
to a current spatial layer's frame is possible from directly lower to a current spatial layer's frame is possible from directly lower
spatial layer frame. P SHOULD also be set to zero when encoding a spatial layer frame. P SHOULD also be set to zero when encoding a
layer synchronization frame in response to an LRR layer synchronization frame in response to an LRR
[I-D.ietf-avtext-lrr] message (see Section 5.3). When P is set to [I-D.ietf-avtext-lrr] message (see Section 5.3). When P is set to
zero, the TID field (described below) MUST also be set to 0 (if zero, the TID field (described below) MUST also be set to 0 (if
present). Note that the P bit does not forbid intra-picture, present). Note that the P bit does not forbid intra-picture,
inter-layer prediction from earlier frames of the same picture, if inter-layer prediction from earlier frames of the same picture, if
skipping to change at page 8, line 29 skipping to change at page 8, line 29
below) is set to 1 (indicating flexible mode), then only one octet below) is set to 1 (indicating flexible mode), then only one octet
is present for the layer indices. Otherwise if the F bit is set is present for the layer indices. Otherwise if the F bit is set
to 0 (indicating non-flexible mode), then two octets are present to 0 (indicating non-flexible mode), then two octets are present
for the layer indices. for the layer indices.
F: Flexible mode. F set to one indicates flexible mode and if the P F: Flexible mode. F set to one indicates flexible mode and if the P
bit is also set to one, then the octets following the mandatory bit is also set to one, then the octets following the mandatory
first octet, the PID, and layer indices (if present) are as first octet, the PID, and layer indices (if present) are as
described by "Reference indices" below. This MUST only be set to described by "Reference indices" below. This MUST only be set to
1 if the I bit is also set to one; if the I bit is set to zero, 1 if the I bit is also set to one; if the I bit is set to zero,
then this MUST also be set to zero and ignored by receivers. The then this MUST also be set to zero and ignored by receivers.
value of this F bit MUST only change on the first packet of a key (Flexible mode's Reference indices are defined as offsets from the
picture. A key picture is a picture whose base spatial layer Picture ID field, so they would have no meaning if I were not
frame is a key frame, and which thus completely resets the encoder set.) The value of this F bit MUST only change on the first
state. This packet will have its P bit equal to zero, SID or D packet of a key picture. A key picture is a picture whose base
bit (described below) equal to zero, and B bit (described below) spatial layer frame is a key frame, and which thus completely
equal to 1. resets the encoder state. This packet will have its P bit equal
to zero, SID or L bit (described below) equal to zero, and B bit
(described below) equal to 1.
B: Start of a frame. MUST be set to 1 if the first payload octet of B: Start of a frame. MUST be set to 1 if the first payload octet of
the RTP packet is the beginning of a new VP9 frame, and MUST NOT the RTP packet is the beginning of a new VP9 frame, and MUST NOT
be 1 otherwise. Note that this frame might not be the first frame be 1 otherwise. Note that this frame might not be the first frame
of a picture. of a picture.
E: End of a frame. MUST be set to 1 for the final RTP packet of a E: End of a frame. MUST be set to 1 for the final RTP packet of a
VP9 frame, and 0 otherwise. This enables a decoder to finish VP9 frame, and 0 otherwise. This enables a decoder to finish
decoding the frame, where it otherwise may need to wait for the decoding the frame, where it otherwise may need to wait for the
next packet to explicitly know that the frame is complete. Note next packet to explicitly know that the frame is complete. Note
that, if spatial scalability is in use, more frames from the same that, if spatial scalability is in use, more frames from the same
picture may follow; see the description of the M bit above. picture may follow; see the description of the M bit above.
V: Scalability structure (SS) data present. When set to one, the V: Scalability structure (SS) data present. When set to one, the
OPTIONAL SS data MUST be present in the payload descriptor. OPTIONAL SS data MUST be present in the payload descriptor.
Otherwise, the SS data MUST NOT be present. Otherwise, the SS data MUST NOT be present.
Z: Not a reference frame for upper spatial layers. If set to 1, Z: Not a reference frame for upper spatial layers. If set to 1,
indicates that frames with higher spatial layers SID+1 of the indicates that frames with higher spatial layers SID+1 and greater
current and following pictures do not depend on the current of the current and following pictures do not depend on the current
spatial layer SID frame. This enables a decoder which is spatial layer SID frame. This enables a decoder which is
targeting a higher spatial layer to know that it can safely targeting a higher spatial layer to know that it can safely
discard this packet's frame without processing it, without having discard this packet's frame without processing it, without having
to wait for the "D" bit in the higher-layer frame (see below). to wait for the "D" bit in the higher-layer frame (see below).
The mandatory first octet is followed by the extension data fields The mandatory first octet is followed by the extension data fields
that are enabled: that are enabled:
M: The most significant bit of the first octet is an extension flag. M: The most significant bit of the first octet is an extension flag.
The field MUST be present if the I bit is equal to one. If set, The field MUST be present if the I bit is equal to one. If M is
the PID field MUST contain 15 bits; otherwise, it MUST contain 7 set, the PID field MUST contain 15 bits; otherwise, it MUST
bits. See PID below. contain 7 bits. See PID below.
Picture ID (PID): Picture ID represented in 7 or 15 bits, depending Picture ID (PID): Picture ID represented in 7 or 15 bits, depending
on the M bit. This is a running index of the pictures. The field on the M bit. This is a running index of the pictures, where the
MUST be present if the I bit is equal to one. If M is set to sender increments the value by 1 for each picture it sends. (Note
zero, 7 bits carry the PID; else if M is set to one, 15 bits carry however that because a middlebox can discard pictures where
the PID in network byte order. The sender may choose between a 7- permitted by the scalability structure, Picture IDs as received by
or 15-bit index. The PID SHOULD start on a random number, and a receiver might not be contiguous.) This field MUST be present
MUST wrap after reaching the maximum ID (0x7f or 0x7fff depending if the I bit is equal to one. If M is set to zero, 7 bits carry
on the index size chosen). The receiver MUST NOT assume that the the PID; else if M is set to one, 15 bits carry the PID in network
number of bits in PID stay the same through the session. byte order. The sender may choose between a 7- or 15-bit index.
The PID SHOULD start on a random number, and MUST wrap after
reaching the maximum ID (0x7f or 0x7fff depending on the index
size chosen). The receiver MUST NOT assume that the number of
bits in PID stay the same through the session. If this field
transitions from 7-bits to 15-bits, the value is zero-extended
(i.e. the value after 0x6e is 0x006f); if the field transitions
from 15 bits to 7 bits, it is truncated (i.e. the value after
0x1bbe is 0xbf).
In the non-flexible mode (when the F bit is set to 0), this PID is In the non-flexible mode (when the F bit is set to 0), this PID is
used as an index to the picture group (PG) specified in the SS used as an index to the picture group (PG) specified in the SS
data below. In this mode, the PID of the key frame corresponds to data below. In this mode, the PID of the key frame corresponds to
the first specified frame in the PG. Then subsequent PIDs are the first specified frame in the PG. Then subsequent PIDs are
mapped to subsequently specified frames in the PG (modulo N_G, mapped to subsequently specified frames in the PG (modulo N_G,
specified in the SS data below), respectively. specified in the SS data below), respectively.
All frames of the same picture MUST have the same PID value. All frames of the same picture MUST have the same PID value.
skipping to change at page 10, line 39 skipping to change at page 11, line 5
SID: The spatial layer ID of current frame. Note that frames SID: The spatial layer ID of current frame. Note that frames
with spatial layer SID > 0 may be dependent on decoded spatial with spatial layer SID > 0 may be dependent on decoded spatial
layer SID-1 frame within the same picture. Different frames of layer SID-1 frame within the same picture. Different frames of
the same picture MUST have distinct spatial layer IDs, and the same picture MUST have distinct spatial layer IDs, and
frames' spatial layers MUST appear in increasing order within frames' spatial layers MUST appear in increasing order within
the frame. the frame.
D: Inter-layer dependency used. MUST be set to one if and only D: Inter-layer dependency used. MUST be set to one if and only
if the current spatial layer SID frame depends on spatial layer if the current spatial layer SID frame depends on spatial layer
SID-1 frame of the same picture, otherwise MUST set to zero. SID-1 frame of the same picture, otherwise MUST be set to zero.
For the base layer frame (with SID equal to 0), this D bit MUST For the base layer frame (with SID equal to 0), this D bit MUST
be set to zero. be set to zero.
TL0PICIDX: 8 bits temporal layer zero index. TL0PICIDX is only TL0PICIDX: 8 bits temporal layer zero index. TL0PICIDX is only
present in the non-flexible mode (F = 0). This is a running present in the non-flexible mode (F = 0). This is a running
index for the temporal base layer pictures, i.e., the pictures index for the temporal base layer pictures, i.e., the pictures
with TID set to 0. If TID is larger than 0, TL0PICIDX with TID set to 0. If TID is larger than 0, TL0PICIDX
indicates which temporal base layer picture the current picture indicates which temporal base layer picture the current picture
depends on. TL0PICIDX MUST be incremented when TID is equal to depends on. TL0PICIDX MUST be incremented by 1 when TID is
0. The index SHOULD start on a random number, and MUST restart equal to 0. The index SHOULD start on a random number, and
at 0 after reaching the maximum number 255. MUST restart at 0 after reaching the maximum number 255.
Reference indices: When P and F are both set to one, indicating a Reference indices: When P and F are both set to one, indicating a
non-key frame in flexible mode, then at least one reference index non-key frame in flexible mode, then at least one reference index
MUST be specified as below. Additional reference indices (total MUST be specified as below. Additional reference indices (total
of up to 3 reference indices are allowed) may be specified using of up to 3 reference indices are allowed) may be specified using
the N bit below. When either P or F is set to zero, then no the N bit below. When either P or F is set to zero, then no
reference index is specified. reference index is specified.
P_DIFF: The reference index (in 7 bits) specified as the relative P_DIFF: The reference index (in 7 bits) specified as the relative
PID from the current picture. For example, when P_DIFF=3 on a PID from the current picture. For example, when P_DIFF=3 on a
packet containing the picture with PID 112 means that the packet containing the picture with PID 112 means that the
picture refers back to the picture with PID 109. This picture refers back to the picture with PID 109. This
calculation is done modulo the size of the PID field, i.e., calculation is done modulo the size of the PID field, i.e.,
either 7 or 15 bits. either 7 or 15 bits. A P_DIFF value of 0 is invalid.
N: 1 if there is additional P_DIFF following the current P_DIFF. N: 1 if there is additional P_DIFF following the current P_DIFF.
4.2.1. Scalability Structure (SS): 4.2.1. Scalability Structure (SS):
The scalability structure (SS) data describes the resolution of each The scalability structure (SS) data describes the resolution of each
frame within a picture as well as the inter-picture dependencies for frame within a picture as well as the inter-picture dependencies for
a picture group (PG). If the VP9 payload descriptor's "V" bit is a picture group (PG). If the VP9 payload descriptor's "V" bit is
set, the SS data is present in the position indicated in Figure 2 and set, the SS data is present in the position indicated in Figure 2 and
Figure 3. Figure 3.
skipping to change at page 12, line 18 skipping to change at page 12, line 43
G: PG description present flag. G: PG description present flag.
-: Bit reserved for future use. MUST be set to zero and MUST be -: Bit reserved for future use. MUST be set to zero and MUST be
ignored by the receiver. ignored by the receiver.
N_G: N_G indicates the number of pictures in a Picture Group (PG). N_G: N_G indicates the number of pictures in a Picture Group (PG).
If N_G is greater than 0, then the SS data allows the inter- If N_G is greater than 0, then the SS data allows the inter-
picture dependency structure of the VP9 stream to be pre-declared, picture dependency structure of the VP9 stream to be pre-declared,
rather than indicating it on the fly with every packet. If N_G is rather than indicating it on the fly with every packet. If N_G is
greater than 0, then for N_G pictures in the PG, each picture's greater than 0, then for N_G pictures in the PG, each picture's
temporal layer ID (TID), switch up point (U), and the R reference temporal layer ID (TID), switch up point (U), and the Reference
indices (P_DIFFs) are specified. indices (P_DIFFs) are specified.
The first picture specified in the PG MUST have TID set to 0. The first picture specified in the PG MUST have TID set to 0.
G set to 0 or N_G set to 0 indicates that either there is only one G set to 0 or N_G set to 0 indicates that either there is only one
temporal layer or no fixed inter-picture dependency information is temporal layer (for non-flexible mode) or no fixed inter-picture
present going forward in the bitstream. dependency information is present (for flexible mode) going
forward in the bitstream.
Note that for a given picture, all frames follow the same inter- Note that for a given picture, all frames follow the same inter-
picture dependency structure. However, the frame rate of each picture dependency structure. However, the frame rate of each
spatial layer can be different from each other and this can be spatial layer can be different from each other and this can be
controlled with the use of the D bit described above. The described with the use of the D bit described above. The
specified dependency structure in the SS data MUST be for the specified dependency structure in the SS data MUST be for the
highest frame rate layer. highest frame rate layer.
In a scalable stream sent with a fixed pattern, the SS data SHOULD be In a scalable stream sent with a fixed pattern, the SS data SHOULD be
included in the first packet of every key frame. This is a packet included in the first packet of every key frame. This is a packet
with P bit equal to zero, SID or D bit equal to zero, and B bit equal with P bit equal to zero, SID or Lis not the bit equal to zero, and B
to 1. The SS data MUST only be changed on the picture that bit equal to 1. The SS data MUST only be changed on the picture that
corresponds to the first picture specified in the previous SS data's corresponds to the first picture specified in the previous SS data's
PG (if the previous SS data's N_G was greater than 0). PG (if the previous SS data's N_G was greater than 0).
4.3. Frame Fragmentation 4.3. Frame Fragmentation
VP9 frames are fragmented into packets, in RTP sequence number order, VP9 frames are fragmented into packets, in RTP sequence number order,
beginning with a packet with the B bit set, and ending with a packet beginning with a packet with the B bit set, and ending with a packet
with the E bit set. There is no mechanism for finer-grained access with the E bit set. There is no mechanism for finer-grained access
to parts of a VP9 frame. to parts of a VP9 frame.
skipping to change at page 13, line 18 skipping to change at page 13, line 39
additional forms of inter-frame dependencies, largely involving additional forms of inter-frame dependencies, largely involving
probability tables for the entropy and tree encoders. In VP9 syntax, probability tables for the entropy and tree encoders. In VP9 syntax,
the syntax element "error_resilient_mode" resets this additional the syntax element "error_resilient_mode" resets this additional
inter-frame data, allowing a frame's syntax to be decoded inter-frame data, allowing a frame's syntax to be decoded
independently. independently.
Due to the requirements of scalable streams, a VP9 encoder producing Due to the requirements of scalable streams, a VP9 encoder producing
a scalable stream needs to ensure that a frame does not depend on a a scalable stream needs to ensure that a frame does not depend on a
previous frame (of the same or a previous picture) that can previous frame (of the same or a previous picture) that can
legitimately be removed from the stream. Thus, a frame that follows legitimately be removed from the stream. Thus, a frame that follows
a removable frame (in full decode order) MUST be encoded with a frame that might be removed (in full decode order) MUST be encoded
"error_resilient_mode" set to true. with "error_resilient_mode" set to true.
For spatially-scalable streams, this means that For spatially-scalable streams, this means that
"error_resilient_mode" needs to be turned on for the base spatial "error_resilient_mode" needs to be turned on for the base spatial
layer; it can however be turned off for higher spatial layers, layer; it can however be turned off for higher spatial layers,
assuming they are sent with inter-layer dependency (i.e. with the "D" assuming they are sent with inter-layer dependency (i.e. with the "D"
bit set). For streams that are only temporally-scalable without bit set). For streams that are only temporally-scalable without
spatial scalability, "error_resilient_mode" can additionally be spatial scalability, "error_resilient_mode" can additionally be
turned off for any picture that immediately follows a temporal layer turned off for any picture that immediately follows a temporal layer
0 frame. 0 frame.
skipping to change at page 15, line 6 skipping to change at page 15, line 19
Either it can signal a preferred reference picture when a loss has Either it can signal a preferred reference picture when a loss has
been detected by the decoder -- preferably then a reference that the been detected by the decoder -- preferably then a reference that the
decoder knows is perfect -- or, it can be used as positive feedback decoder knows is perfect -- or, it can be used as positive feedback
information to acknowledge correct decoding of certain reference information to acknowledge correct decoding of certain reference
pictures. The positive feedback method is useful for VP9 used for pictures. The positive feedback method is useful for VP9 used for
point to point (unicast) communication. The use of RPSI for VP9 is point to point (unicast) communication. The use of RPSI for VP9 is
preferably combined with a special update pattern of the codec's two preferably combined with a special update pattern of the codec's two
special reference frames -- the golden frame and the altref frame -- special reference frames -- the golden frame and the altref frame --
in which they are updated in an alternating leapfrog fashion. When a in which they are updated in an alternating leapfrog fashion. When a
receiver has received and correctly decoded a golden or altref frame, receiver has received and correctly decoded a golden or altref frame,
and that frame had a PictureID in the payload descriptor, the and that frame had a Picture ID in the payload descriptor, the
receiver can acknowledge this simply by sending an RPSI message back receiver can acknowledge this simply by sending an RPSI message back
to the sender. The message body (i.e., the "native RPSI bit string" to the sender. The message body (i.e., the "native RPSI bit string"
in [RFC4585]) is simply the PictureID of the received frame. in [RFC4585]) is simply the (7 or 15 bit) Picture ID of the received
frame.
Note: because all frames of the same picture must have the same Note: because all frames of the same picture must have the same
inter-picture reference structure, there is no need for a message to inter-picture reference structure, there is no need for a message to
specify which frame is being selected. specify which frame is being selected.
5.2. Full Intra Request (FIR) 5.2. Full Intra Request (FIR)
The Full Intra Request (FIR) [RFC5104] RTCP feedback message allows a The Full Intra Request (FIR) [RFC5104] RTCP feedback message allows a
receiver to request a full state refresh of an encoded stream. receiver to request a full state refresh of an encoded stream.
Upon receipt of an FIR request, a VP9 sender MUST send a picture with Upon receipt of an FIR request, a VP9 sender MUST send a picture with
a keyframe for its spatial layer 0 layer frame, and then send frames a keyframe for its spatial layer 0 layer frame, and then send frames
without inter-picture prediction (P=0) for any higher layer frames. without inter-picture prediction (P=0) for any higher layer frames.
5.3. Layer Refresh Request (LRR) 5.3. Layer Refresh Request (LRR)
The Layer Refresh Request [I-D.ietf-avtext-lrr] allows a receiver to The Layer Refresh Request (LRR) [I-D.ietf-avtext-lrr] allows a
request a single layer of a spatially or temporally encoded stream to receiver to request a single layer of a spatially or temporally
be refreshed, without necessarily affecting the stream's other encoded stream to be refreshed, without necessarily affecting the
layers. stream's other layers.
+---------------+---------------+ +---------------+---------------+
|0|1|2|3|4|5|6|7|0|1|2|3|4|5|6|7| |0|1|2|3|4|5|6|7|0|1|2|3|4|5|6|7|
+---------------+---------+-----+ +---------------+---------+-----+
| RES | TID | RES | SID | | RES | TID | RES | SID |
+---------------+---------+-----+ +---------------+---------+-----+
Figure 5 Figure 5
Figure 5 shows the format of LRR's layer index fields for VP9 Figure 5 shows the format of LRR's layer index fields for VP9
streams. The two "RES" fields MUST be set to 0 on transmission and streams. The two "RES" fields MUST be set to 0 on transmission and
ingnored on reception. See Section 4.2 for details on the TID and ingnored on reception. See Section 4.2 for details on the TID and
SID fields. SID fields.
Identification of a layer refresh frame can be derived from the Identification of a layer refresh frame can be derived from the
reference IDs of each frame by backtracking the dependency chain reference IDs of each frame by backtracking the dependency chain
until reaching a point where only decodable frames are being until reaching a point where only decodable frames are being
referenced. Therefore it's recommended for both the flexible and the referenced. Therefore it's recommended for both the flexible and the
non-flexible mode that, when upgrade frames are being encoded in non-flexible mode that, when switching up points are being encoded in
response to a LRR, those packets should contain layer indices and the response to a LRR, those packets should contain layer indices and the
reference fields so that the decoder or an MCU can make this reference field(s) so that the decoder or a selective forwarding
derivation. middleboxes [RFC7667] can make this derivation.
Example: Example:
LRR {1,0}, {2,1} is sent by an MCU when it is currently relaying LRR {1,0}, {2,1} is sent by an MCU when it is currently relaying
{1,0} to a receiver and which wants to upgrade to {2,1}. In response {1,0} to a receiver and which wants to upgrade to {2,1}. In response
the encoder should encode the next frames in layers {1,1} and {2,1} the encoder should encode the next frames in layers {1,1} and {2,1}
by only referring to frames in {1,0}, or {0,0}. by only referring to frames in {1,0}, or {0,0}.
In the non-flexible mode, periodic upgrade frames can be defined by In the non-flexible mode, periodic upgrade frames can be defined by
the layer structure of the SS, thus periodic upgrade frames can be the layer structure of the SS, thus periodic upgrade frames can be
skipping to change at page 17, line 39 skipping to change at page 18, line 10
the VP9 profile the VP9 profile
corresponding to the corresponding to the
set of coding tools set of coding tools
supported. supported.
+=========+===========+=================+==========================+ +=========+===========+=================+==========================+
| Profile | Bit Depth | SRGB Colorspace | Chroma Subsampling | | Profile | Bit Depth | SRGB Colorspace | Chroma Subsampling |
+=========+===========+=================+==========================+ +=========+===========+=================+==========================+
| 0 | 8 | No | YUV 4:2:0 | | 0 | 8 | No | YUV 4:2:0 |
+---------+-----------+-----------------+--------------------------+ +---------+-----------+-----------------+--------------------------+
| 1 | 8 | Yes | YUV 4:2:0,4:4:0 or 4:4:4 | | 1 | 8 | Yes | YUV 4:2:2,4:4:0 or 4:4:4 |
+---------+-----------+-----------------+--------------------------+ +---------+-----------+-----------------+--------------------------+
| 2 | 10 or 12 | No | YUV 4:2:0 | | 2 | 10 or 12 | No | YUV 4:2:0 |
+---------+-----------+-----------------+--------------------------+ +---------+-----------+-----------------+--------------------------+
| 3 | 10 or 12 | Yes | YUV 4:2:0,4:4:0 or 4:4:4 | | 3 | 10 or 12 | Yes | YUV 4:2:2,4:4:0 or 4:4:4 |
+---------+-----------+-----------------+--------------------------+ +---------+-----------+-----------------+--------------------------+
Table 3: Table of profile capabilities. Table 3: Table of profile capabilities.
6.1. SDP Parameters 6.1. SDP Parameters
6.1.1. Mapping of Media Subtype Parameters to SDP 6.1.1. Mapping of Media Subtype Parameters to SDP
The media type video/VP9 string is mapped to fields in the Session The media type video/VP9 string is mapped to fields in the Session
Description Protocol (SDP) [RFC8866] as follows: Description Protocol (SDP) [RFC8866] as follows:
* The media name in the "m=" line of SDP MUST be video. * The media name in the "m=" line of SDP MUST be video.
* The encoding name in the "a=rtpmap" line of SDP MUST be VP9 (the * The encoding name in the "a=rtpmap" line of SDP MUST be VP9 (the
media subtype). media subtype).
skipping to change at page 19, line 19 skipping to change at page 19, line 31
values in the offer. values in the offer.
* To simplify the handling and matching of these configurations, the * To simplify the handling and matching of these configurations, the
same RTP payload type number used in the offer SHOULD also be used same RTP payload type number used in the offer SHOULD also be used
in the answer and in a subsequent offer, as specified in in the answer and in a subsequent offer, as specified in
[RFC3264]. An answer or subsequent offer MUST NOT contain the [RFC3264]. An answer or subsequent offer MUST NOT contain the
payload type number used in the offer unless the profile-id value payload type number used in the offer unless the profile-id value
is exactly the same as in the original offer. However, max-fr and is exactly the same as in the original offer. However, max-fr and
max-fs parameters MAY be changed in subsequent offers and answers, max-fs parameters MAY be changed in subsequent offers and answers,
with the same payload type number, if an endpoint wishes to change with the same payload type number, if an endpoint wishes to change
its declared receiver capabilties. its declared receiver capabilities.
7. Media Type Definition 7. Media Type Definition
This registration is done using the template defined in [RFC6838] and This registration is done using the template defined in [RFC6838] and
following [RFC4855]. following [RFC4855].
Type name: Type name:
video video
Subtype name: Subtype name:
skipping to change at page 21, line 11 skipping to change at page 21, line 11
Change controller: Change controller:
IETF AVTCore Working Group delegated from the IESG. IETF AVTCore Working Group delegated from the IESG.
8. Security Considerations 8. Security Considerations
RTP packets using the payload format defined in this specification RTP packets using the payload format defined in this specification
are subject to the security considerations discussed in the RTP are subject to the security considerations discussed in the RTP
specification [RFC3550], and in any applicable RTP profile such as specification [RFC3550], and in any applicable RTP profile such as
RTP/AVP [RFC3551], RTP/AVPF [RFC4585], RTP/SAVP [RFC3711], or RTP/ RTP/AVP [RFC3551], RTP/AVPF [RFC4585], RTP/SAVP [RFC3711], or RTP/
SAVPF [RFC5124]. SAVPF [RFC5124]. However, as "Securing the RTP SAVPF [RFC5124]. However, as "Securing the RTP Protocol Framework:
Protocol Framework: Why RTP Does Not Mandate a Single Media Security Why RTP Does Not Mandate a Single Media Security Solution" [RFC7202]
Solution" [RFC7202] discusses, it is not an RTP payload format's discusses, it is not an RTP payload format's responsibility to
responsibility to discuss or mandate what solutions are used to meet discuss or mandate what solutions are used to meet the basic security
the basic security goals like confidentiality, integrity and source goals like confidentiality, integrity and source authenticity for RTP
authenticity for RTP in general. This responsibility lays on anyone in general. This responsibility lays on anyone using RTP in an
using RTP in an application. They can find guidance on available application. They can find guidance on available security mechanisms
security mechanisms in Options for Securing RTP Sessions [RFC7201]. in Options for Securing RTP Sessions [RFC7201]. Applications SHOULD
Applications SHOULD use one or more appropriate strong security use one or more appropriate strong security mechanisms. The rest of
mechanisms. The rest of this security consideration section this security consideration section discusses the security impacting
discusses the security impacting properties of the payload format properties of the payload format itself.
itself.
Implementations of this RTP payload format need to take appropriate
security considerations into account. It is extremely important for
the decoder to be robust against malicious or malformed payloads and
ensure that they do not cause the decoder to overrun its allocated
memory or otherwise mis-behave. An overrun in allocated memory could
lead to arbitrary code execution by an attacker. The same applies to
the encoder, even though problems in encoders are typically rarer.
This RTP payload format and its media decoder do not exhibit any This RTP payload format and its media decoder do not exhibit any
significant non-uniformity in the receiver-side computational significant non-uniformity in the receiver-side computational
complexity for packet processing, and thus are unlikely to pose a complexity for packet processing, and thus are unlikely to pose a
denial-of-service threat due to the receipt of pathological data. denial-of-service threat due to the receipt of pathological data.
Nor does the RTP payload format contain any active content. Nor does the RTP payload format contain any active content.
9. Congestion Control 9. Congestion Control
Congestion control for RTP SHALL be used in accordance with RFC 3550 Congestion control for RTP SHALL be used in accordance with RFC 3550
 End of changes. 42 change blocks. 
100 lines changed or deleted 122 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/