Internet Engineering Task Force Internet Draft A. Klemets Document:
draft-ietf-avt-rtp-vc1-00.txtdraft-ietf-avt-rtp-vc1-01.txt Microsoft Expires: FebruaryApril 2006 AugustOctober 2005 RTP Payload Format for Video Codec 1 (VC-1) IPR Notice By submitting this Internet-Draft, each author represents that any applicable patent or other IPR claims of which he or she is aware have been or will be disclosed, and any of which he or she becomes aware will be disclosed, in accordance with Section 6 of BCP 79. Status of this Memo Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Distribution of this document is unlimited. Copyright Notice Copyright (C) The Internet Society (2005). Abstract This memo specifies an RTP payload format for encapsulating Video Codec 1 (VC-1) compressed bit streams, as defined by the proposed Society of Motion Picture and Television Engineers (SMPTE) standard, SMPTE 421M. SMPTE is the main standardizing body in the motion imaging industry and the proposed SMPTE 421M standard defines a compressed video bit stream format and decoding process for television. Table of Contents 1. Introduction...................................................2 1.1 Conventions used in this document..........................3 2. Definitions and abbreviations..................................3 3. Overview of VC-1...............................................4 3.1 VC-1 bit stream layering model.............................5 3.2 Bit-stream Data Units in Advanced profile..................5 3.3 Decoder initialization parameters..........................6 3.4 Ordering of frames.........................................7 4. Encapsulation of VC-1 format bit streams in RTP................8 4.1 Access Units...............................................8 4.2 Fragmentation of VC-1 frames...............................8 4.3 Time stamp considerations..................................9 4.4 Random Access Points......................................10 4.5 Removal of HRD parameters.................................10 4.6 Repeating the Sequence Layer header.......................11 4.7 Signaling of MIME format parameters.......................10parameters.......................11 4.8 MIME "mode=1" parameter...................................12 4.9 MIME "mode=3" parameter...................................12 5. RTP Payload Format syntax.....................................12syntax.....................................13 5.1 RTP header usage..........................................12usage..........................................13 5.2 AU header syntax..........................................12syntax..........................................13 5.3 AU Control field syntax...................................13syntax...................................14 6. RTP Payload format parameters.................................15parameters.................................16 6.1 Media Type Registration...................................15Registration...................................16 6.2 Mapping of MIME parameters to SDP.........................18SDP.........................19 7. Security Considerations.......................................19Considerations.......................................20 8. IANA Considerations...........................................20Considerations...........................................21 9. References....................................................20References....................................................21 9.1 Normative references......................................20references......................................21 9.2 Informative references....................................20references....................................21 1. Introduction The bit stream syntax for compressed video in Video Codec 1 (VC-1) format is defined by SMPTE 421M . SMPTE 421M also specifies constraints that must be met by VC-1 conformant bit streams, and it specifies the complete process required to decode the bit stream. However, it does not specify the VC-1 compression algorithm, thus allowing for different ways to implement a VC-1 encoder. The VC-1 bit stream syntax has three profiles. Each profile has specific bit stream syntax elements and algorithms associated with it. Depending on the application in which VC-1 is used, some profiles may be more suitable than others. For example, the Simple profile is designed for low bit rate Internet streaming and for playback on devices that can only handle low complexity decoding. The Advanced profile is designed for broadcast applications, such as digital TV, HD DVD or HDTV. The Advanced profile is the only VC-1 profile that supports interlaced video frames. Section 2 defines the abbreviations used in this document. Section 3 provides a more detailed overview of VC-1. Sections 4 and 5 define the RTP payload format for VC-1, and section 6 defines the MIME and SDP parameters for VC-1. See section 7 for security considerations. 1.1 Conventions used in this document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14, RFC 2119 . 2. Definitions and abbreviations This document uses the definitions in SMPTE 421M . For convenience, the following terms from SMPTE 421M are restated here: B-picture: A picture that is coded using motion compensated prediction from past and/or future reference fields or frames. A B- picture cannot be used for predicting any other picture. Bit-stream data unit (BDU): A unit of the compressed data which may be parsed (i.e., syntax decoded) independently of other information at the same hierarchical level. A BDU can be, for example, a sequence layer header, an entry-point segmentheader, a frame, or a slice. Encapsulated BDU (EBDU): A BDU which has been encapsulated using the encapsulation mechanism described in Annex E of SMPTE 421M , to prevent emulation of the start code prefix in the bit stream. Entry-point: A point in the bit stream that offers random access. frame: A frame contains lines of spatial information of a video signal. For progressive video, these lines contain samples starting from one time instant and continuing through successive lines to the bottom of the frame. For interlaced video, a frame consists of two fields, a top field and a bottom field. One of these fields will commence one field period later than the other. interlace: The property of frames where alternating lines of the frame represent different instances in time. In an interlaced frame, one of the fields is meant to be displayed first. I-picture: A picture coded using information only from itself. level: A defined set of constraints on the values which may be taken by the parameters (such as bit rate and buffer size) within a particular profile. A profile may contain one or more levels. P-picture: A picture that is coded using motion compensated prediction from past reference fields or frames. picture: For progressive video, a picture is identical to a frame, while for interlaced video, a picture may refer to a frame, or the top field or the bottom field of the frame depending on the context. profile: A defined subset of the syntax of VC-1, with a specific set of coding tools, algorithms, and syntax associated with it. There are three VC-1 profiles: Simple, Main and Advanced. progressive: The property of frames where all the samples of the frame represent the same instance in time. random access: A random access point in the bit stream is defined by the following guarantee: If decoding begins at this point, all frames needed for display after this point will have no decoding dependency on any data preceding this point, and are also present in the decoding sequence after this point. A random access point is also called an entry-point. sequence: A coded representation of a series of one or more pictures. In VC-1 Advanced profile, a sequence consists of a series of one or more entry-point segments, where each entry-point segment consists of a series of one or more pictures, and where the first picture in each entry-point segment provides random access. In VC-1 Simple and Main profiles, the first picture in each sequence is an I-picture. slice: A consecutive series of macroblock rows in a picture, which are encoded as a single unit. start codes (SC): 32-bit codes embedded in that coded bit stream that are unique, and identify the beginning of a BDU. Start codes consist of a unique three-byte Start Code Prefix (SCP), and a one-byte Start Code Suffix (SCS). 3. Overview of VC-1 The VC-1 bit stream syntax consists of three profiles: Simple, Main, and Advanced. The Simple and Main profiles are designed for relatively low bit rate applications. For example, the maximum bit rate supported by the Simple profile is 384 kbps. To help achieve high compression efficiency, certain features such as non-square pixels and support for interlaced pictures, are only included in the Advanced profile. The maximum bit rate supported by the Advanced profile is 135 Mbps, making it suitable for nearly lossless encoding of HDTV signals. Only the Advanced profile supports carrying user-data (meta-data) in- band with the compressed bit stream. The user-data can be used for closed captioning support, for example. Of the three profiles, only the Advanced profile allows codec configuration parameters, such as the picture aspect ratio, to be changed through in-band signaling in the compressed bit stream. For each of the profiles, a certain number of "levels" have been defined. Unlike a "profile", which implies a certain set of features or syntax elements, a "level" is a set of constraints on the values of parameters in a profile, such as the bit rate or buffer size. The VC-1 Simple profile has two levels, the Main profile has three, and the Advanced profile has five levels. See Annex D of SMPTE 421M  for a detailed list of the profiles and levels. 3.1 VC-1 bit stream layering model The VC-1 bit stream is defined as a hierarchy of layers. This is conceptually similar to the notion of a protocol stack of networking protocols. The outermost layer is called the sequence layer. The other layers are entry-point, picture, slice, macroblock and block. In the Simple and Main profiles, a sequence in the sequence layer consists of a series of one or more coded pictures. In the Advanced profile, a sequence consists of one or more entry-point segments, where each entry-point segment consists of a series of one or more pictures, and where the first picture in each entry-point segment provides random access. A picture is decomposed into macroblocks. A slice comprises one or more contiguous rows of macroblocks. The entry-point and slice layers are only present in the Advanced profile. In the Advanced profile, the start of each entry-point layer segment indicates a random access point. In the Simple and Main profiles each I-picture is a random access point. Each picture can be coded as an I-picture, P-picture, skipped picture, BI-picture, or as a B-picture. These terms are defined in section 2 of this document and in section 4.12 of SMPTE 421M . 3.2 Bit-stream Data Units in Advanced profile In the Advanced profile only, each picture and slice is byte-aligned and is considered a Bit-stream Data Unit (BDU). A BDU is defined as a unit that can be parsed (i.e., syntax decoded) independently of other information in the same layer. The beginning of a BDU is signaled by an identifier called Start Code (SC). Sequence layer headers and entry-point segmentheaders are also BDUs and thus can be easily identified by their Start Codes. See Annex E of SMPTE 421M  for a complete list of Start Codes. Note that blocks and macroblocks are not BDUs and thus do not have a Start Code and are not necessarily byte-aligned. The Start Code consists of four bytes. The first three bytes are 0x00, 0x00 and 0x01. The fourth byte is called the Start Code Suffix (SCS) and it is used to indicate the type of BDU that follows the Start Code. For example, the SCS of a sequence layer header (0x0F) is different from the SCS of an entry-point segmentheader (0x0E). The Start Code is always byte-aligned and is transmitted in network byte order. To prevent accidental emulation of the Start Code in the coded bit stream, SMPTE 421M defines an encapsulation mechanism that uses byte stuffing. A BDU which has been encapsulated by this mechanism is referred to as an Encapsulated BDU, or EBDU. 3.3 Decoder initialization parameters In the VC-1 Advanced profile, the sequence layer header contains parameters that are necessary to initialize the VC-1 decoder. These parameters apply to all entry-point segments until the next occurrence of a sequence layer header in the coded bit stream. The parameters in the sequence layer header include, among other things,include the Advanced profile level, the dimensions of the coded pictures, the aspect ratio, interlace information, the frame rate and up to 31 leaky bucket parameter sets for the Hypothetical Reference Decoder (HRD). Section 6.1 of SMPTE 421M  provides the formal specification of the sequence layer header. Each leaky bucket parameter set for the HRD specifies a peak transmission bit rate and a decoder buffer capacity. The coded bit stream is restricted by these parameters. The HRD model does not mandate buffering by the decoder. Its purpose is to limit the encoder's bit rate fluctuations according to a basic buffering model, so that the resources necessary to decode the bit stream are predictable. The HRD has a constant-delay mode and a variable-delay mode. The constant-delay mode is appropriate for broadcast and streaming applications, while the variable-delay mode is designed for video conferencing applications. Annex C of SMPTE 421M  specifies the usage of the hypothetical reference decoder for VC-1 bit streams. A general description of the theory of the HRD can be found in .. The concept of an entry-point layer applies only to the VC-1 Advanced profile. The presence of an entry-point segmentheader indicates a random access point within the bit stream. The entry-point segmentheader specifies current buffer fullness values for the leaky buckets in the HRD. The header also specifies coding control parameters that are in effect until the occurrence of the next entry-point segmentheader in the bit stream. See Section 6.2 of SMPTE 421M  for the formal specification of the entry-point segmentheader. Neither a sequence layer header nor an entry-point segmentheader is defined for the VC-1 Simple and Main profiles. For these profiles, decoder initialization parameters MUST be conveyed out-of-band from the coded bit stream. Section 4.44.7 of this document specifies how the parameters are conveyed by this RTP payload format. 3.4 Ordering of frames Frames are transmitted in the same order in which they are captured, except if the presence of B-pictures has been indicated in the decoder initialization parameters. In the latter case, the frames are reordered by the VC-1 encoder such that the frames that the B- pictures depend on are transmitted first. This is referred to as the coded order of the frames. When the presence of B-pictures has been indicated, the decoder is required to buffer one picture. When an I-picture or a P-picture is received, the picture is not displayed until the next I- or P-picture is received. However, B-pictures are displayed immediately. These rules are stated in section 5.4 in SMPTE 421M . Figure 1 illustrates the timing relationship between the capture of frames, their coded order, and the display order of the decoded frames. The figure shows that the display of frame P4 is delayed until frame P5 is received, while frames B2 and B3 are displayed immediately. Capture: |I0 P1 B2 B3 P4 ... | Coded order: | I0 P1 P4 B2 B3 P5 ... | Display order: | I0 P1 B2 B3 P4 ... | |+---+---+---+---+---+---+---+------> time 0 1 2 3 4 5 6 7 Figure 1. Frame reordering when B-pictures are indicated. 4. Encapsulation of VC-1 format bit streams in RTP 4.1 Access Units Each RTP packet contains an integral number of application data units (ADUs). For VC-1 format bit streams, an ADU is equivalent to one Access Unit (AU), as defined in this section. Figure 2 shows the layout of an RTP packet with multiple AUs. +-+-+-+-+-+-+-+-+-+-+-+-+-+- .. +-+-+-+-+ | RTP | AU(1) | AU(2) | | AU(n) | | Header | | | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+- .. +-+-+-+-+ Figure 2. RTP packet structure. Access Units MUST be byte-aligned. Each Access Unit MUST start with the AU header defined in section 5.2, and is followed by a variable length payload. The AU payload MUST contain data belonging to exactly one VC-1 frame. The following rules apply to the contents of each AU payload when the VC-1 Advanced profile is used: - The AU payload MUST contain VC-1 bit stream data in EBDU format (i.e., the bit stream must use the byte-stuffing encapsulation mode defined in Annex E of SMPTE 421M .) - The AU payload MAY contain multiple EBDUs, e.g., a sequence layer header, an entry-point segmentheader, a frame header and multiple slices and the associated user-data. (However, all slices and their corresponding macroblocks MUST belong to the same video frame.) - The AU payload MUST start at an EBDU boundary, except when the AU payload contains a fragmented frame, in which case the rules in section 4.2 apply. If the data in an AU (EBDUs in the case of Advanced profile and frame in the case of Simple and Main) does not end at an octet boundary, up to 7 zero-valued padding bits MUST be added to achieve octet- alignment. 4.2 Fragmentation of VC-1 frames Each AU payload SHOULD contain a complete VC-1 frame. However, if this would cause the RTP packet to exceed the MTU size, the frame SHOULD be fragmented into multiple AUs to avoid IP-level fragmentation. When an AU contains a fragmented frame, this MUST be indicated by setting the FRAG field in the AU header as defined in section 5.3. AU payloads that do not contain a fragmented frame, or that contain the first fragment of a frame, MUST start at an EBDU boundary if Advanced profile is used. In this case, for Simple and Main profiles, the AU payload MUST begin with the start of a frame. If Advanced profile is used, AU payloads that contain a fragment of a frame other than the first fragment, SHOULD start at an EBDU boundary, such as at the start of a slice. However, slices are only defined for the Advanced profile, and are not always used. Blocks and macroblocks are not BDUs (have no Start Code) and are not byte-aligned. Therefore, it may not always be possible to continue a fragmented frame at an EBDU boundary. If a RTP packet contains an AU with the last fragment of a frame, additional AUs SHOULD NOT be included in the RTP packet. If the PTS Delta field in the AU header is used, each fragment of a frame MUST have the same presentation time. If the DTS Delta field in the AU header is used, each fragment of a frame MUST have the same decode time. 4.3 Time stamp considerations Video frames MUST be transmitted in the coded order. Coded order implies that no frames are dependent on subsequent frames, as discussed in section 3.4. The RTP timestamp field MUST be set to the decodepresentation time of the video frame contained in the first AU in the RTP packet. The decodepresentation time is equivalent to the sampling instant of the frame, except when the codec initialization parameters indicate that the VC-1 bit stream contains B-pictures. When the presence of B- pictures has been indicated, the encoder may reorder frames, as explained in section 3.4 of this document and in section 5.4 of SMPTE 421M . The VC-1 bit stream does not carry any time stamps other than an optional Temporal Frame Reference Counter field, which, if it is present, can be used to calculate the decode time of aframe. However, the RTP senderEach AU header may have access to different externally provided time stamps depending on the method used to ingest the VC-1 bit stream. For example, if VC-1 is encapsulated in MPEG-2 Transport Stream, each frame is assigned a presentation time (PTS) andoptionally also a decode time (DTS). If a VC-1 bit stream is stored in an ASF file, onlyspecify the decode time of eachvideo frame is available.contained in the AU. If only presentationthe decode time informationis available,not specified, the RTP senderreceiver can approximate the decode time of a frameit by itsthe frame's presentation time, after taking frame reordering into account. Frame reordering can be handled by an algorithm similar to the one illustrated in Figure 1 in section 3.4. The algorithm requires buffering of only one frame. If only decode time information is available, determining the presentation time of a P-frame requires buffering, or looking ahead, to the first frame that does not depend onKnowing if the P-frame. Usingstream will contain B-pictures helps the coded order sequence in Figure 1decoder allocate resources more efficiently, as an example,the RTP sender cannot determine presentation timeencoder will not reorder any frames. In that case, the buffering of frame P4 until it has seen frame P5. This would be a more complicated and costly procedure than to estimate a decode time from the presentation time. Hence, this RTP payload format defines that the RTP timestamp field must represent the decode time of the frame. Knowing if the stream will contain B-pictures helps the decoder allocate resources more efficiently, as the encoder will not reorder any frames. In that case, the buffering of oneone frame as described in section 3.4 is not necessary. Avoiding this buffer reduces the end- to-end delay, which may be important for interactive applications. For Advanced profile, B-pictures are assumed to be present by default. If the coded bit stream never contains B-pictures, this can be indicated using the "bpic" MIME parameter defined in section 6.1. For Simple and Main profiles, the presence of B-pictures is indicated by a non-zero value for the MAXBFRAMES field in STRUCT_C decoder initialization parameter. STRUCT_C conveyed in the MIME "config" parameter, which is defined in section 6.1. 4.4 Signaling of MIME format parameters When this RTP payload formatRandom Access Points The entry-point header contains information that is used with SDP,needed by the decoder initialization parameters described in section 3.3 MUST be signaled in SDP using the MIME parameters specified in section 6.1. Section 6.2 specifies howto mapdecode the MIME parameters to SDP. Whenframes in that entry-point segment. This means that in the Advanced profile is used,event of lost RTP packets the decoder initialization parameters MAYmay be changed by inserting a new sequence layerunable to decode frames until the next entry-point header oris received. The first frame after an entry-point segmentheader inis a random access points into the coded bit stream. Note that the sequence layer header specifiesThe Simple and Main profiles do not have entry-point headers, so for those profiles each I-picture is a random access point. To allow the encoding level,RTP receiver to detect that a RTP packet which was lost contained a random access point, this RTP payload format defines a field called "RA Count". This field is present in every AU, and its value is incremented for every random access point. For additional details, see the maximum sizedefinition of the coded pictures and possibly"RA Count" in section 5.2. To make it easy to determine if a AU contains a random access point, this RTP payload format also defines a bit called the frame rate. Thus, if"RA" flag in the AU Control field. This bit is set to 1 only on those AU's that contain a random access point. The RA bit is defined in section 5.3. 4.5 Removal of HRD parameters The sequence layer header changes,of the newAdvanced profile may include up to 31 leaky bucket parameter sets for the Hypothetical Reference Decoder (HRD). Each leaky bucket parameter set specifies a possible peak transmission bit rate (HDR_RATE) and a decoder buffer capacity (HRD_BUFFER). (See section 3.3 for additional discussion about the HRD.) If the actual peak transmission rate is known by the RTP sender, the RTP sender MAY remove all leaky bucket parameter sets except for the one corresponding to the actual peak transmission rate. For each leaky bucket parameter set in the sequence layer header, there is also parameter in the entry-point header supersedesthat specifies the valuesinitial fullness (HRD_FULL) of the MIME parameters "level", "width", "height"leaky bucket. If the RTP sender has removed any leaky bucket parameter sets from the sequence layer header, then for any removed leaky bucket parameter set, it MUST also remove the corresponding HRD_FULL parameter in the entry-point header. Removing leaky bucket parameter sets, as described above, may significantly reduce the size of the sequence layer headers and "framerate".the entry-point headers. 4.6 Repeating the Sequence Layer header To improve robustness against loss of RTP packets, it is RECOMMENDED that if the sequence layer header changes, it should be repeated frequently in the bit stream. In this is case, it is RECOMMENDED that the number of leaky bucket parameters in the sequence layer header and the entry point headers be reduced to one, as described in section 4.5. This will help reduce the overhead caused by repeating the sequence layer header. Note that any data in the VC-1 bit stream, including repeated copies of the sequence header itself, must be accounted for when computing the leaky bucket parametersparameter for the HRD. (See section 3.3 for a discussion about the HRD.) The Seq Count field in the Access Unit header is used to track changes toNote that if the sequence layer header. Avalue of 0 is reserved for the case whenTFCNTRFLAG in the most recentsequence layer header of the bit streamis identical to1, each picture header contains a frame counter field (TFCNTR). Each time the sequence layer header is inserted in the MIME "config" parameter (defined in section 6.1.) Ifbit stream, the RTP sender cannot determinevalue of this counter MUST be reset. To allow the most recentRTP receiver to detect that a RTP packet which was lost contained a new sequence layer header, orthe AU Control field defines a bit called the "SL" flag. This bit is toggled when a sequence layer header is transmitted, but only if itthat header is different formfrom the most recently transmitted sequence layer headerheader. The SL bit is defined in thesection 5.3. 4.7 Signaling of MIME "config" parameter, a non-zero valueformat parameters When this RTP payload format is used with SDP, the decoder initialization parameters described in section 3.3 MUST be used forsignaled in SDP using the MIME parameters specified in section 6.1. Section 6.2 specifies how to map the Seq Count field.MIME parameters to SDP. When the RTP sender transmits an AU containingAdvanced profile is used, the decoder initialization parameters MAY be changed by inserting a new sequence layer header or an entry-point header in the coded bit stream. Note that is different fromthe previoussequence layer header,header specifies the value ofencoding level, the Seq Count field MUST be incremented. The Seq Count fieldmaximum size of all subsequent AU headers MUST be set to this new value untilthe coded pictures and possibly also the frame rate. Thus, if the sequence layer header changes again.changes, the new header supersedes the values of the MIME parameters "level", "width", "height" and "framerate". 4.8 MIME "mode=1" parameter In certain applications,applications using the Advanced profile, the sequence layer header never changes. This MAY be signaled with the MIME parameter "mode=1" or "mode=3", as appropriate. (See the definition of the"mode=1". (The "mode" parameter is defined in section 6.1.) The "mode=1" parameter serves as a "hint" to the RTP receiver that all sequence layer headers in the bit stream will be identical. If "mode=1" or "mode=3"is signaled and a sequence layer header is present in the coded bit stream, then it MUST be identical to the sequence layer header specified by the MIME "config" parameter. The entry-point segmentSince the sequence layer header contains information that is needed bynever changes in "mode=1", the decoder to decodeRTP sender MAY remove it from the frames inbit stream. Note, however, that segment. This meansif that inif the eventvalue of lost RTP packets the decoder may be unable to decode frames untilTFCNTRFLAG in the next entry-point segmentsequence layer header is received. Access Units that contain an entry-point segment1, each picture header MUST havecontains a frame counter field (TFCNTR). This field is reset each time the RA bit in AUsequence layer header setoccurs in the bit stream. If the RTP sender chooses to 1. (The RAremove the sequence layer header, then it MUST ensure that the resulting bit stream is defined in section 5.3.)still compliant with the VC-1 specification (e.g., by adjusting the TFCNTR field, if necessary.) 4.9 MIME "mode=3" parameter In certain applications,applications using the Advanced profile, both the sequence layer header and the entry-point segmentheader never changes.change. This MUSTMAY be signaled with the MIME parameter "mode=2" or "mode=3","mode=3". The same rules apply to "mode=3" as appropriate. In this case, anyfor "mode=1", described in section 4.8. Additionally, if "mode=3" is signaled, then the RTP sender MAY "compress" the coded bit stream by not including sequence layer headers and entry-point segmentheaders that are presentin the RTP packets. The RTP receiver MUST "decompress" the coded bit stream MAYby re- inserting the entry-point headers prior to delivering the coded bit stream to the VC-1 decoder. The sequence layer header does not need to be removeddecompressed by the RTP sender.receiver, since it never changes. If "mode=2" or"mode=3" is signaled and the RTP receiver receives a complete AU or the first fragment of an entry-point segment header is present inAU, and the codedRA bit stream, it MUST be identicalis set to 1 but the AU does not begin with an entry-point header, then this indicates that entry-point header has been "compressed". In that case, the RTP receiver MUST insert an entry-point segmentheader at the beginning of the AU. When inserting the entry-point header, the RTP receiver MUST use the one that was specified by the MIME "config" parameter. 5. RTP Payload Format syntax 5.1 RTP header usage The format of the RTP header is specified in RFC 3550  and is reprinted in Figure 3 for convenience. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|X| CC |M| PT | sequence number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | timestamp | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | synchronization source (SSRC) identifier | +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ | contributing source (CSRC) identifiers | | .... | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 3. RTP header according to RFC 3550 With the exception of the fields listed below, the RTP header fields are used as defined in RFC 3550 and by the RTP profile in use. Marker bit (M): 1 bit This bit is set to 1 if the RTP packet contains an Access Unit containing a complete VC-1 frame, or the last fragment of a VC-1 frame. Payload type (PT): 7 bits This document does not assign a RTP payload type for this RTP payload format. The assignment of a payload type has to be performed either through the RTP profile used or in a dynamic way. Timestamp: 32 bits The RTP timestamp is set to the decodepresentation time of the VC-1 frame in the first Access Unit. A 90 kHz clock rate MUST be used. 5.2 AU header syntax The Access Unit header consists of a one-byte AU Control field, the RA Count field and 43 optional fields. The structure of the AU header is illustrated in Figure 4. +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |AU | SeqRA | AUP | PTS | DTS | AUP ||Control| Count | DeltaLen | Delta | LenDelta | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 4. Structure of AU header. AU Control: 8 bits The usageusage of the AU Control field is defined in section 5.3. RA Count: 8 bits Random Access Point Counter. This field is a binary modulo 256 counter. The value of this field, MUST be incremented by 1, each time an AU is transmitted where the RA bit in the AU Control field is set to 1. The initial value of this field is undefined and MAY be chosen randomly. AUP Len: 16 bits Access Unit Payload Length. Specifies the size, in bytes, of the payload of the Access Unit. The field does not include the size of the AU Control field is defined in section 5.3. Seq Count: 8 bits Sequence Layer Counter. This field is a binary modulo 256 counter.header itself. The value of this field, if present,field MUST be incremented by 1,included in each time anAU containing a new sequence layerheader is transmitted. The value 0 is reserved for the case when thein an RTP sender knows that the current sequence layer header is identical topacket, except for the sequence layerlast AU header in the MIME "config" parameter (defined in section 6.1) and MUST NOT be used for any other purpose. If this field is not present, a value of 0 MUST be assumed.packet. PTS Delta: 32 bits Presentation time delta. Specifies the presentation time of the frame as a 2's complement offset (delta) from the timestamp in the RTP header of this RTP packet. The PTS Delta field MUST use the same clock rate as the timestamp field in the RTP header. This field SHOULD NOT be included in the first AU header in the RTP packet, because the RTP timestamp field specifies the presentation time of the frame in the first AU. DTS Delta: 32 bits Decode time delta. Specifies the decode time of the frame as a 2's complement offset (delta) from the timestamp in the RTP header of this RTP packet. The DTS Delta field MUST use the same clock rate as the timestamp field in the RTP header. This field SHOULD NOT be included in the first AU header in the RTP packet, because the RTP timestamp field specifies the decode time of the frame in the first AU. AUP Len: 16 bits Access Unit Payload Length. Specifies the size, in bytes, of the payload of the Access Unit. The field does not include the size of the AU header itself. The field MUST be included in each AU header in an RTP packet, except for the last AU header in the packet.5.3 AU Control field syntax The structure of the 8-bit AU Control field is shown in Figure 5. 0 1 2 3 4 5 6 7 +----+----+----+----+----+----+----+----+ | FRAG | RA | SCSL | LP | PT | DT | LP |R | +----+----+----+----+----+----+----+----+ Figure 5. Syntax of AU Control field. FRAG: 2 bits Fragmentation Information. This field indicates if the AU payload contains a complete frame or a fragment of a frame. It MUST be set as follows: 0: The AU payload contains a fragment of a frame other than the first or last fragment. 1: The AU payload contains the first fragment of a frame. 2: The AU payload contains the last fragment of a frame. 3: The AU payload contains a complete frame (not fragmented.) SC: 1 bit Sequence Layer Counter present. This bit MUST be set to 1 if the AU header includes the Seq Count field. The bit MUST be 0 for Simple and Main profile bit streams.AU payload contains a complete frame (not fragmented.) RA: 1 bit Random Access Point indicator. This bit MUST be set to 1 if the AU contains a frame that is a random access point. In the case of Simple and Main profiles, any I-picture is a random access point. In the case of Advanced profile, the first frame after an entry-point segmentheader is a random access point. Note that if entry-point segmentheaders are not transmitted at every random access point, this MUST be indicated using the MIME parameter "mode=2""mode=3". SL: 1 bit Sequence Layer Counter. This bit MUST be toggled, i.e., changed from 0 to 1 or "mode=3", as appropriate. PT:from 1 to 0, if the AU contains a sequence layer header and if it is different from the most recently transmitted sequence layer header. Otherwise, the value of this bit PTS Deltamust be identical to the value of the SL bit in the previous AU. The initial value of this bit is undefined and MAY be chosen randomly. The bit MUST be 0 for Simple and Main profile bit streams or if the sequence layer header never changes. LP: 1 bit Length Present. This bit MUST be set to 1 if the AU header includes the PTS DeltaAUP Len field. DT:PT: 1 bit DTSPTS Delta Present. This bit MUST be set to 1 if the AU header includes the DTSPTS Delta field. LP:DT: 1 bit LengthDTS Delta Present. This bit MUST be set to 1 if the AU header includes the AUP LenDTS Delta field. R: 1 bit Reserved. This bit MUST be set to 0 and MUST be ignored by receivers. 6. RTP Payload format parameters 6.1 Media Type Registration The media subtype for VC-1 is allocated from the standards tree. The top-level media type under which this payload format is registered is 'video'. The receiver MUST ignore any unrecognized parameter. Media type: video Media subtype: vc1 Required parameters: profile: The value is a decimal number indicating the VC-1 profile. The following values are defined: 0: Simple profile. 1: Main profile. 3: Advanced profile. config: The value is a hexadecimalbase16  (hexadecimal) representation of an octet string that expresses the decoder initialization parameters. Decoder initialization parameters are mapped onto the hexadecimalbase16 octet string in an MSB-first basis. The first bit of the decoder initialization parameters MUST be located at the MSB of the first octet. If the decoder initialization parameters are not multiple of 8 bits, in the last octet up to 7 zero-valued padding bits MUST be added to achieve octet alignment. For the Simple and Main profiles, the decoder initialization parameters are STRUCT_C, as defined in Annex J of SMPTE 421M . For the Advanced profile, the decoder initialization parameters are a sequence layer header directly followed by an entry-point segmentheader. The two headers MUST be in EBDU format, meaning that they must include their Start Codes and must use the encapsulation method defined in Annex E of SMPTE 421M . width: The value is a decimal number specifying the maximum horizontal size of the coded picture in pixels. Note: When Advanced profile is used, this parameter only applies while the sequence layer header specified in the config parameter is in use. height: The value is a decimal number specifying the maximum vertical size of the coded picture in pixels. Note: When Advanced profile is used, this parameter only applies while the sequence layer header specified in the config parameter is in use. bitrate: The value is a decimal number specifying the peak transmission rate of the coded bit stream. The number does not include RTP overhead. Note: When Advanced profile is used, this parameter only applies while the sequence layer header specified in the config parameter is in use. buffer: The value is a decimal number specifying the leaky bucket size, B, in milliseconds, required to contain a stream transmitted at the transmission rate specified by the bitrate parameter. This parameter is defined in the hypothetical reference decoder model for VC-1, in Annex C of SMPTE 421M . Note: When Advanced profile is used, this parameter only applies while the sequence layer header specified in the config parameter is in use. Optional parameters: level: The value is a decimal number specifying the level of the encoding profile. For Advanced profile, valid values are 0 to 4, which correspond to levels L0 to L4, respectively. For Simple and Main profiles, the following values are defined: 1: Low Level 2: Medium Level 3: High Level (only valid for Main profile) This parameter does not have a default value. Note: When Advanced profile is used, this parameter only applies while the sequence layer header specified in the config parameter is in use. framerate: The value is a decimal number specifying the number of frames per second, multiplied by 1000. For example, 29.97 frames per second is represented as 29970. This parameter does not have a default value. If the parameter is not specified, the frame rate can be determined from the level of the encoding profile, if it is known. Note: When Advanced profile is used, this parameter only applies while the sequence layer header specified in the config parameter is in use. bpic: This parameter signals if B-pictures may be present when the Advanced profile is used. If this parameter is present, and B-pictures may be present in the coded bit stream, this parameter MUST be equal to 1. If B-pictures will never be present in the coded bit stream, even if the sequence layer header changes, this parameter SHOULD be present and its value SHOULD be equal to 0. If this parameter is not specified, a value of 1 MUST be assumed. mode: The value is a decimal number specifying the use of the sequence layer header and the entry-point segmentheader. This parameter is only used for Advanced profile. The following values are defined: 0: Both the sequence layer header and the entry-point segmentheader may change, and changed headers will be included in the RTP packets. 1: The sequence layer header specified in the config parameter never changes. 2:3: The sequence layer header and the entry-point segmentheader specified in the config parameter never changes.change. Entry-point segmentheaders MAY not be included in the RTP packets.Access Units. Each Access Unit that has the RA bit set to 1 contains a random access point even if an entry-point segmentheader is not included in the Access Unit. If an entry-point header is not included at a random access point, then the RTP packet. 3: Modes 1 and 2 combined.receiver MUST insert the entry-point header into the VC-1 bit stream prior to delivering the bit stream to the VC-1 decoder. If the mode parameter is not specified, a value of 0 MUST be assumed. The mode parameter SHOULD be specified if any of themodes 1-31 or 3 apply to the VC-1 bit stream. Encoding considerations: This media type is framed and contains binary data. This media type depends on RTP framing, and hence is only defined for transfer via RTP .Security considerations: See Section 7 of this document. Interoperability considerations: None. Published specification: This payload format specification. Applications which use this media type: Multimedia streaming and conferencing tools. Additional Information: None. Person & email address to contact for further information: Anders Klemets <email@example.com> IETF AVT working group. Intended Usage: COMMON Restrictions on usage: This media type depends on RTP framing, and hence is only defined for transfer via RTP . Authors: Anders Klemets Change controller: IETF Audio/Video Transport Working Group delegated from the IESG. 6.2 Mapping of MIME parameters to SDP The information carried in the media type specification has a specific mapping to fields in the Session Description Protocol (SDP) . If SDP is used to specify sessions using this payload format, the mapping is done as follows: o The media name in the "m=" line of SDP MUST be video (the media type). o The encoding name in the "a=rtpmap" line of SDP MUST be vc1 (the media subtype). o The clock rate in the "a=rtpmap" line MUST be 90000. o The REQUIRED parameters "profile", "config", "width", "height", "bitrate" and "buffer" MUST be included in the "a=fmtp" line of SDP. These parameters are expressed as a MIME media type string, in the form of a semicolon separated list of parameter=value pairs. o The OPTIONAL parameters "level", "framerate", "bpic" and "mode", when present, MUST be included in the "a=fmtp" line of SDP. These parameters are expressed as a MIME media type string, in the form of a semicolon separated list of parameter=value pairs: a=fmtp:<dynamic payload type> <parameter name>=<value>[,<value>][; <parameter name>=<value>] o Any unknown parameters to the device that uses the SDP MUST be ignored. For example, parameters defined in later specifications MAY be copied into the SDP and MUST be ignored by receivers that do not understand them. An example of media representation in SDP is as follows (Simple profile, Medium level): m=video 49170 RTP/AVP 98 a=rtpmap:98 VC1/90000 a=fmtp:98 profile=0;level=2;width=352;height=288;framerate=15000; bitrate=384000;buffer=2000;config=4e291800 7. Security Considerations RTP packets using the payload format defined in this specification are subject to the security considerations discussed in the RTP specification , and in any appropriate RTP profile. This implies that confidentiality of the media streams is achieved by encryption; for example, through the application of SRTP .. A potential denial-of-service threat exists for data encodings using compression techniques that have non-uniform receiver-end computational load. The attacker can inject pathological RTP packets into the stream that are complex to decode and that cause the receiver to be overloaded. VC-1 is particularly vulnerable to such attacks, because it is possible for an attacker to generate RTP packets containing frames that affect the decoding process of many future frames. Therefore, the usage of data origin authentication and data integrity protection of at least the RTP packet is RECOMMENDED; for example, with SRTP .. Note that the appropriate mechanism to ensure confidentiality and integrity of RTP packets and their payloads is very dependent on the application and on the transport and signaling protocols employed. Thus, although SRTP is given as an example above, other possible choices exist. 8. IANA Considerations IANA is requested to register the media subtype name "vc1" for the media type "video" as specified in section 6.1 of this document. 9. References 9.1 Normative references  Proposed SMPTE 421M, "VC-1 Compressed Video Bitstream Format and Decoding Process", www.smpte.org.  Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.  Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, July 2003.  Handley, M. and V. Jacobson, "SDP: Session Description Protocol", RFC 2327, April 1998.  Josefsson, S., Ed., "The Base16, Base32, and Base64 Data Encodings", RFC 3548, July 2003. 9.2 Informative references  Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. Norrman, "The Secure Real-time Transport Protocol (SRTP)", RFC 3711, March 2004.  Ribas-Corbera, J., Chou, P.A., and S.L. Regunathan, "A generalized hypothetical reference decoder for H.264/AVC", IEEE Transactions on Circuits and Systems for Video Technology, August 2003. Author's Addresses Anders Klemets Microsoft Corp. 1 Microsoft Way Redmond, WA 98052 USA Email: firstname.lastname@example.org Acknowledgements Thanks to Shankar Regunathan for pointing out errors in the initial draft of this document. IPR Notices The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79. Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at email@example.com. Full Copyright Statement Copyright (C) The Internet Society (2005). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.