draft-ietf-avt-rtp-mpeg4-es-01.txt   draft-ietf-avt-rtp-mpeg4-es-02.txt 
Internet Engineering Task Force Yoshihiro Kikuchi - Toshiba Internet Engineering Task Force Yoshihiro Kikuchi - Toshiba
Internet Draft Toshiyuki Nomura - NEC Internet Draft Toshiyuki Nomura - NEC
Document: draft-ietf-avt-rtp-mpeg4-es-01.txt Shigeru Fukunaga - Oki Document: draft-ietf-avt-rtp-mpeg4-es-02.txt Shigeru Fukunaga - Oki
Yoshinori Matsui - Matsushita Yoshinori Matsui - Matsushita
Hideaki Kimata - NTT Hideaki Kimata - NTT
May 31, 2000 July 6, 2000
RTP payload format for MPEG-4 Audio/Visual streams RTP payload format for MPEG-4 Audio/Visual streams
Status of this Memo Status of this Memo
This document is an Internet-Draft and is in full conformance with all This document is an Internet-Draft and is in full conformance with all
provisions of Section 10 of RFC2026 [1]. provisions of Section 10 of RFC2026 [1].
Internet-Drafts are working documents of the Internet Engineering Task Internet-Drafts are working documents of the Internet Engineering Task
Force (IETF), its areas, and its working groups. Note that other groups Force (IETF), its areas, and its working groups. Note that other groups
skipping to change at page 1, line 31 skipping to change at page 1, line 31
replaced, or obsoleted by other documents at any time. It is replaced, or obsoleted by other documents at any time. It is
inappropriate to use Internet- Drafts as reference material or to cite inappropriate to use Internet- Drafts as reference material or to cite
them other than as "work in progress." them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
Abstract Abstract
This document describes RTP payload formats for the carriage of MPEG-4 This document describes RTP payload formats for carrying of MPEG-4 Audio
Audio and Visual streams[2][3], and an RTCP format for MPEG-4 upstream and Visual bitstreams[2][3]. For the purpose of directly mapping MPEG-4
messages functionalities[4]. In this specification, MPEG-4 Audio/Visual Audio/Visual bitstreams onto RTP packets, it provides specifications for
bitstreams are directly mapped into RTP packets. The RTP header fields the use of RTP header fields and also specifies fragmentation rules. It
usage and the fragmentation rule for MPEG-4 Visual and Audio bitstreams also provides specifications for MIME type registrations and the use of
are specified. It also specifies an RTCP packet usage to carry the MPEG-4 SDP.
upstream messages. In addition, MIME type registrations and SDP usages
for the MPEG-4 Audio and Visual streams are defined in this document.
1. Introduction 1. Introduction
1.1 Why MPEG-4 Audio/Visual RTP format needed?
The RTP payload formats described in this Internet-Draft specify a way of The RTP payload formats described in this Internet-Draft specify a way of
how MPEG-4 Audio and Visual streams are fragmented and mapped directly how MPEG-4 Audio and Visual streams are to be fragmented and mapped
onto RTP packets. directly onto RTP packets.
H.323 terminals could be an example where such RTP payload formats are These RTP payload formats enable to carry MPEG-4 Audio/Visual streams
used. MPEG-4 Audio/Visual streams are not managed by Object Descriptors without using the synchronization and stream management functionality of
of MPEG-4 Systems[6] but by H.245. The streams are directly mapped onto MPEG-4 Systems [6]. Such RTP payload format would be used within systems
RTP packets without using the synchronization functionality of MPEG-4 where their own stream management functionality is provided and thus such
Systems [6]. functionality in MPEG-4 Systems is not necessary. H.323 terminals are an
example of such systems. MPEG-4 Audio/Visual streams are not managed by
MPEG-4 Systems Object Descriptors but by H.245. The streams are directly
mapped onto RTP packets without using the synchronization functionality
of MPEG-4 Systems. Other examples are SIP and RTSP where attribute of the
video stream (e.g. media type, packetization format and configuration) is
specified in MIME and SDP parameters.
The semantics of RTP headers in such cases need to be clearly defined, The semantics of RTP headers in such cases need to be clearly defined,
including the association with the MPEG-4 Audio/Visual data elements. In including the association with MPEG-4 Audio/Visual data elements. In
addition, it would be beneficial to define the fragmentation rule of RTP addition, it would be beneficial to define the fragmentation rules of RTP
packets for MPEG-4 Video streams so as to enhance error resiliency by packets for MPEG-4 Video streams so as to enhance error resiliency by
utilizing the error resilience tools provided inside the MPEG-4 Video utilizing the error resilience tools provided inside the MPEG-4 Video
stream. However, these items are not covered by other RTP payload format stream. These issues, however, have yet to be addressed by other RTP
proposals. payload format specifications.
1.2 MPEG-4 Visual RTP payload format 1.1 MPEG-4 Visual RTP payload format
MPEG-4 Visual is a visual coding standard with many new functionalities: MPEG-4 Visual is a visual coding standard with many new features: high
high coding efficiency, high error resiliency, multiple arbitrary shaped coding efficiency; high error resiliency; multiple, arbitrary shape
object based coding, etc. [2]. It covers a wide range of bitrate from object-based coding; etc. [2]. It covers a wide range of bitrate from
several Kbps to many Mbps. It also covers a wide variety of networks scores of Kbps to several Mbps. It also covers a wide variety of
ranging from guarantied to be almost error-free to mobile networks with networks, ranging from those guaranteed to be almost error-free to mobile
high error rate due to its error resilience functionalities. networks with high error rates.
A fragmentation rule for an MPEG-4 visual bitstream into RTP packets is With respect to the fragmentation rules for an MPEG-4 visual bitstream
defined in this document. Since MPEG-4 Visual is used for a wide variety defined in this document, since MPEG-4 Visual is used for a wide variety
of networks, it is desirable not to apply too much restriction to the of networks, it is desirable not to apply too much restriction on
fragmentation. A fragmentation rule like "a single video packet shall fragmentation, and a fragmentation rule such as "a single video packet
always be mapped on a single RTP packet" may be inappropriate. On the shall always be mapped on a single RTP packet" may be inappropriate. On
other hand, a careless media unaware fragmentation may cause degradation the other hand, careless, media unaware fragmentation may cause
of the error resiliency and the bandwidth efficiency. The fragmentation degradation in error resiliency and bandwidth efficiency. The
rule described in this document is flexible but to define the minimum fragmentation rules described in this document are flexible but manage to
rules and guidelines for preventing the meaningless fragmentation and to define the minimum rules for preventing meaningless fragmentation and for
utilizing the error resilience functionality of MPEG-4 visual. utilizing the error resilience of MPEG-4 visual.
For video coding media such as H.261 or MPEG-1/2, the additional media While the additional media specific RTP header defined for such video
specific RTP header works effectively for recovering. e.g., of a picture coding tools as H.261 or MPEG-1/2 is effective in helping to recover
header corrupted by packet losses. However, there are error resilience picture headers corrupted by packet losses, in MPEG-4 Visual there are
functionalities inside MPEG-4 Visual to recover corrupt headers. These already error resilience functionalities for recovering corrupt headers,
functionalities can commonly be used on RTP/IP network as well as other and these can be used on RTP/IP networks, as well as on other networks.
networks. (H.223/mobile, MPEG-2/TS, etc.) Therefore, no extra RTP header
fields are defined in the MPEG-4 Visual RTP payload format.
1.3 Consideration on the MPEG-4 Audio RTP payload format (H.223/mobile, MPEG-2/TS, etc.) That is why no extra RTP header fields
are defined in the MPEG-4 Visual RTP payload format proposed here.
1.2 MPEG-4 Audio RTP payload format
MPEG-4 Audio is a new kind of audio standard that integrates many MPEG-4 Audio is a new kind of audio standard that integrates many
different types of audio coding tools. It also supports a mechanism different types of audio coding tools. It also supports a mechanism for
representing synthesized sounds. Low-overhead MPEG-4 Audio Transport representing synthesized sounds. Low-overhead MPEG-4 Audio Transport
Multiplex (LATM) manages the sequence of the compressed or the Multiplex (LATM) manages the sequences of audio data with relatively
represented audio data by MPEG-4 Audio tools with relatively small small overhead. In audio-only applications, then, it is desirable for
overhead. In audio-only applications, the LATM-based MPEG-4 Audio LATM-based MPEG-4 Audio bitstreams to be directly mapped onto the RTP
bitstreams, therefore, are desirable to be directly mapped into the RTP
packets without using MPEG-4 Systems. packets without using MPEG-4 Systems.
Furthermore, if the payload of a packet is a single audio frame, a packet For MPEG-4 Audio coding tools except synthesis tools, as is true for
loss does not impair the decodability of adjacent packets. Therefore, a other audio coders, if the payload of a packet is a single audio frame,
payload specific header for MPEG-4 Audio is not required as same as one packet loss will not impair the decodability of adjacent packets. On the
for the other audio coders. other hands, MPEG-4 Audio synthesis tools may be sensitive to error. For
example, an SA_access_unit in the payload may set a global value to a new
1.4 MPEG-4 Audio/Visual upstream messaging on RTCP packets value, which is then references throughout the audio content to make a
macro change in the performance. In this case, an error in the payload
Some particular tools of MPEG-4 Audio/Visual support upstream messaging influences all audio data produced after the error. In order to enhance
functionalities. These messages are extremely Audio/Visual specific, error resiliency, the element of SA_access_unit that makes the above
since coders directly use these messages for controlling coding macro change should be transmitted across several SA_access_unit
parameters. From the point of view of controlling parameters, these repeatedly. The number of repetition will be dependent on the network
messages should be transmitted without delay. Therefore, these messages condition. Therefore, the additional media specific header for recovering
are directly mapped onto some kind of low delay RTCP packets. The use of errors will not be required for MPEG-4 Audio.
this type of RTCP packets is limited to the case when the MPEG-4 upstream
functionalities in some particular profiles are used (e.g. MPEG-4 Visual
Advanced Real Time Simple Profile, NEWPRED tool).
2. Conventions used in this document 2. Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC-2119 [7]. document are to be interpreted as described in RFC-2119 [7].
3. RTP Packetization of MPEG-4 Visual bitstream 3. RTP Packetization of MPEG-4 Visual bitstream
This section specifies the RTP packetization rule for MPEG-4 Visual This section specifies RTP packetization rules for MPEG-4 Visual content.
content. An MPEG-4 Visual bitstream is mapped directly onto the RTP An MPEG-4 Visual bitstream is mapped directly onto the RTP payload
payload without any addition of extra header fields or removal of any without any addition of extra header fields or any removal of Visual
Visual syntax elements. The Combined Configuration/Elementary streams syntax elements. The Combined Configuration/Elementary stream mode is
mode is used so that the configuration information is carried in the same used so that configuration information will be carried to the same RTP
RTP port as the elementary stream. (see 6.2.1 "Start codes" of ISO/IEC port as the elementary stream. (see 6.2.1 "Start codes" of ISO/IEC 14496-
14496-2 [2][9][4]) 2 [2][9][4]) The configuration information MAY additionally be specified
When the short video header mode is used, RTP payload format for H.263 by some out-of-band means; in H.323 terminals, H.245 codepoint
specified in the relevant RFCs or other standards MAY be used. "decoderConfigurationInformation" MAY be used for this purpose; in
systems using MIME content type and SDP parameters, e.g. SIP and RTSP,
the optional parameter "config" MAY be used to specify the configuration
information. (see 5.1 and 5.2)
When the short video header mode is used, the RTP payload format used MAY
be that specified for H.263 in the relevant RFCs or in other relevant
standards.
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|X| CC |M| PT | sequence number | RTP |V=2|P|X| CC |M| PT | sequence number | RTP
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| timestamp | Header | timestamp | Header
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| synchronization source (SSRC) identifier | | synchronization source (SSRC) identifier |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
skipping to change at page 5, line 26 skipping to change at page 5, line 26
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| | RTP | | RTP
| MPEG-4 Visual stream (byte aligned) | Payload | MPEG-4 Visual stream (byte aligned) | Payload
| | | |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| :...OPTIONAL RTP padding | | :...OPTIONAL RTP padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 1 - An RTP packet for MPEG-4 Visual stream Figure 1 - An RTP packet for MPEG-4 Visual stream
3.1 RTP header fields usage for MPEG-4 Visual 3.1 Use of RTP header fields for MPEG-4 Visual
Payload Type (PT): Distinct payload type should be assigned to specify Payload Type (PT): Payload type is to be specifically assigned as the
MPEG-4 Visual RTP payload format. If the dynamic payload type assignment MPEG-4 Visual RTP payload format. If this assignment is to be carried out
is used, it is specified by some out-of-band means (e.g. H.245, SDP, dynamically, it can be performed by such out-of-band means as H.245, SDP,
etc.) that the MPEG-4 Visual payload format is used for the corresponding etc.
RTP packet.
Extension (X) bit: Defined by the RTP profile used. Extension (X) bit: Defined by the RTP profile used.
Sequence Number: Increment by one for each RTP data packet sent. It Sequence Number: Incremented by one for each RTP data packet sent,
starts with a random initial value for security reasons. starting, for security reasons, with a random initial value.
Marker (M) bit: The marker bit is set to one to indicate the last RTP Marker (M) bit: The marker bit is set to one to indicate the last RTP
packet (or only RTP packet) of a VOP. packet (or only RTP packet) of a VOP.
Timestamp: The timestamp indicates the composition time, or the Timestamp: The timestamp indicates the composition time, or the
presentation time in a no-compositor decoder by adding a constant random presentation time in a no-compositor decoder. A constant offset, which is
offset for security reasons. For a video object plane, it is defined by random, is added for security reasons. For a video object plane, it is
vop_time_increment (in units of 1/vop_time_increment_resolution seconds) defined as vop_time_increment (in units of
plus the cumulative number of whole seconds specified by module_time_base 1/vop_time_increment_resolution seconds) plus the cumulative number of
and time_code of Group_of_VideoObjectPlane() if present. In the case of whole seconds specified by module_time_base and, if present, time_code of
interlaced video, a VOP consists of lines from two fields and the Group_of_VideoObjectPlane() fields. In the case of interlaced video, a
timestamp indicates the composition time of the first field. If the RTP VOP will consist of lines from two fields, and the timestamp will
packet contains only configuration information and/or indicate the composition time of the first field. If the RTP packet
Group_of_VideoObjectPlane(), the composition time of the subsequent VOP contains only configuration information and/or
Group_of_VideoObjectPlane() fields, the composition time of the next VOP
in the coding order is used. If the RTP packet contains only in the coding order is used. If the RTP packet contains only
visual_object_sequence_end_code, the composition time of the immediately visual_object_sequence_end_code information, the composition time of the
preceding VOP in the coding order is used. immediately preceding VOP in the coding order is used.
Unless specified by an out-of-band means (e.g. SDP parameter or MIME The resolution of the timestamp is set to its default value of 90KHz,
parameter as defined in section 6), the resolution of the timestamp is unless specified by an out-of-band means (e.g. SDP parameter or MIME
set to its default (90KHz). parameter as defined in section 5).
SSRC, CC and CSRC fields are used as described in RFC 1889 [8]. SSRC, CC and CSRC fields are used as described in RFC 1889 [8].
3.2 Fragmentation of MPEG-4 Visual bitstream 3.2 Fragmentation of MPEG-4 Visual bitstream
A fragmented MPEG-4 Visual bitstream is mapped directly onto the RTP A fragmented MPEG-4 Visual bitstream is mapped directly onto the RTP
payload without any addition of extra header fields or removal of any payload without any addition of extra header fields or any removal of
Visual syntax elements. The Combined Configuration/Elementary streams Visual syntax elements. The Combined Configuration/Elementary streams
mode is used. The following rules apply for the fragmentation. mode is used. The following rules apply for the fragmentation.
(1) The configuration information and Group_of_VideoObjectPlane() SHALL (1) Configuration information and Group_of_VideoObjectPlane() fields
be placed at the beginning of the RTP payload (just after the RTP header) SHALL be placed at the beginning of the RTP payload (just after the RTP
or just after the header of the syntactically upper layer function. header) or just after the header of the syntactically upper layer
function.
(2) If one or more headers exist in the RTP payload, the RTP payload (2) If one or more headers exist in the RTP payload, the RTP payload
SHALL begin with the header of the syntactically highest function. SHALL begin with the header of the syntactically highest function.
Note: The visual_object_sequence_end_code is regarded as the lowest Note: The visual_object_sequence_end_code is regarded as the lowest
function. function.
(3) A header SHALL NOT be split into a plurality of RTP packets. (3) A header SHALL NOT be split into a plurality of RTP packets.
(4) Two or more VOPs SHALL be fragmented into different RTP packets so (4) Two or more VOPs SHALL be fragmented into different RTP packets so
that one RTP packet consists of the data bytes associated with an unique that one RTP packet consists of the data bytes associated with a unique
presentation time (that indicated to the timestamp field in the RTP presentation time (that is indicated in the timestamp field in the RTP
packet header). packet header).
(5) A single video packet SHOULD NOT be split into a plurality of RTP (5) A single video packet SHOULD NOT be split into a plurality of RTP
packets. The size of a video packet SHOULD be adjusted such that the packets. The size of a video packet SHOULD be adjusted in such a way that
resulting RTP packet is not larger than the path-MTU. A video packet MAY the resulting RTP packet is not larger than the path-MTU. A video packet
be split into a plurality of RTP packets when the size of the video MAY be split into a plurality of RTP packets when the size of the video
packet is large. packet is large.
(Rule (5) does not apply to the enhancement layer of the scalable streams
where the video packet is not supported.)
Here, header means: Here, header means:
- Configuration information (Visual Object Sequence Header, Visual Object - Configuration information (Visual Object Sequence Header, Visual Object
Header and Video Object Layer Header) Header and Video Object Layer Header)
- visual_object_sequence_end_code - visual_object_sequence_end_code
- The header of the entry point function for an elementary stream - The header of the entry point function for an elementary stream
(Group_of_VideoObjectPlane() or the header of VideoObjectPlane(), (Group_of_VideoObjectPlane() or the header of VideoObjectPlane(),
video_plane_with_short_header(), MeshObject() or FaceObject()) video_plane_with_short_header(), MeshObject() or FaceObject())
- The video packet header (video_packet_header() excluding - The video packet header (video_packet_header() excluding
next_resync_marker()) next_resync_marker())
- The header of gob_layer() - The header of gob_layer()
See 6.2.1 "Start codes" of ISO/IEC 14496-2[2][9][4] for the definition of See 6.2.1 "Start codes" of ISO/IEC 14496-2[2][9][4] for the definition of
the configuration information and the entry point functions. the configuration information and the entry point functions.
The video packet starts with the VOP header or the video packet header, The video packet starts with the VOP header or the video packet header,
followed by motion_shape_texture(), and ends with next_resync_marker() or followed by motion_shape_texture(), and ends with next_resync_marker() or
next_start_code). next_start_code().
3.3 Examples of packetized MPEG-4 Visual bitstream 3.3 Examples of packetized MPEG-4 Visual bitstream
Considering that MPEG-4 Visual is used on a wide variety of networks from Considering the fact that MPEG-4 Visual covers a wide variety of networks
several Kbps to many Mbps, from guaranteed networks which are almost ranging from scores of Kbps to several Mbps, and from those guaranteed to
error-free to mobile networks with high error rate, it is desirable not be almost error-free to mobile networks with high error rates, it is
to apply too much restriction to the fragmentation. On the other hand, a desirable not to apply too much restriction on fragmentation. On the
careless media unaware fragmentation will cause degradation of the error other hand, careless, media unaware fragmentation will cause degradation
resiliency and the bandwidth efficiency. The fragmentation criteria in error resiliency and bandwidth efficiency. The fragmentation criteria
described in 3.2 are flexible but to define the minimum rules to prevent described in 3.2 are flexible but to define the minimum rules to prevent
meaningless fragmentation. meaningless fragmentation.
For video coding media such as H.261 or MPEG-1/2, the additional media
specific RTP header works effectively for recovering, e.g., of a picture
header corrupted by packet losses. However, there is an error resilience
functionality inside MPEG-4 Visual to recover corrupt headers. This
functionality can commonly be used on RTP/IP network as well as other
networks. (H.223/mobile, MPEG-2/TS, etc.) Therefore, there is no strong
reason to define MPEG-4 Visual specific extra RTP header fields.
Figure 2 shows examples of RTP packets generated based on the criteria Figure 2 shows examples of RTP packets generated based on the criteria
described in 3.2 described in 3.2
(a) is an example of the first RTP packet or the random access point of (a) is an example of the first RTP packet or the random access point of
an MPEG-4 visual bitstream. This RTP packet contains the configuration an MPEG-4 visual bitstream containing the configuration information.
information. According to the criterion (1), the Visual Object Sequence According to criterion (1), the Visual Object Sequence Header(VS header)
Header(VS header) is placed at the beginning of the RTP payload, and the is placed at the beginning of the RTP payload, preceding the Visual
Visual Object Header and the Video Object Layer Header(VO header, VOL Object Header and the Video Object Layer Header(VO header, VOL header).
header) follow it. Since the fragmentation rule defined in 3.2 guarantees Since the fragmentation rule defined in 3.2 guarantees that the
that the configuration information, starting with configuration information, starting with
visual_object_sequence_start_code, is always placed at the beginning of visual_object_sequence_start_code, is always placed at the beginning of
the RTP payload, RTP receivers can detect the random access point by the RTP payload, RTP receivers can detect the random access point by
checking if the first 32-bit field of the RTP payload is checking if the first 32-bit field of the RTP payload is
visual_object_sequence_start_code. visual_object_sequence_start_code.
(b) is another example of the RTP packet containing the configuration (b) is another example of the RTP packet containing the configuration
information. The difference from the example (a) is that this RTP packet information. It differs from example (a) in that the RTP packet also
also contains a video packet in the VOP following the configuration contains a video packet in the VOP following the configuration
information. Since the length of the configuration information is information. Since the length of the configuration information is
relatively short (typically several ten bytes), an RTP packet containing relatively short (typically scores of bytes) and an RTP packet containing
only the configuration information may increase the overhead. Therefore, only the configuration information may thus increase the overhead, the
the configuration information and the immediately following GOV and/or (a configuration information and the immediately following GOV and/or (a
part of) VOP can be packetized into a single RTP packet like this part of) VOP can be effectively packetized into a single RTP packet as in
example. this example.
(c) is an example the RTP packet that contains (c) is an example of the RTP packet that contains
Group_of_VideoObjectPlane(GOV). Following the criterion (1), the GOV is Group_of_VideoObjectPlane(GOV). Following criterion (1), the GOV is
placed at the beginning of the RTP payload. It is a waste of RTP/IP placed at the beginning of the RTP payload. It would be a waste of RTP/IP
header overhead to generate a RTP packet containing only a GOV whose header overhead to generate an RTP packet containing only a GOV whose
length is 7 bytes. Therefore, (a part of) the following VOP can be placed length is 7 bytes. Therefore, (a part of) the following VOP can be placed
in the same RTP packet as shown in (c). in the same RTP packet as shown in (c).
(d) is an example of the case where one video packet is packetized into (d) is an example of the case where one video packet is packetized into
one RTP packet. When the packet-loss rate of the underlying network is one RTP packet. When the packet-loss rate of the underlying network is
high, this kind of packetization is recommended. It is strongly high, this kind of packetization is recommended. It is recommended to set
recommended to set resync_marker_disable to 0 in the VOL header to enable resync_marker_disable to 0 in the VOL header to enable the adjustment of
adjustment of the video packet size. Even when the RTP packet containing the video packet size. Even when the RTP packet containing the VOP header
the VOP header is discarded by a packet loss, the other RTP packets can is discarded by a packet loss, the other RTP packets can be decoded by
be decoded by using the HEC(Header Extension Code) information in the using the HEC(Header Extension Code) information in the video packet
video packet header. No extra RTP header field is necessary. header. No extra RTP header field is necessary.
(e) is an example of the case where more than one video packets are (e) is an example of the case where more than one video packets are
packetized into one RTP packet. This kind of packetization is effective packetized into one RTP packet. This kind of packetization is effective
to save the overhead of RTP/IP headers if the bit-rate of the underlying to save the overhead of RTP/IP headers when the bit-rate of the
network is low. However, it will decrease the packet-loss resiliency underlying network is low. However, it will decrease the packet-loss
because multiple video packets are discarded by a single RTP packet loss. resiliency because multiple video packets are discarded by a single RTP
The adequate number of video packets in a RTP packet and the RTP packet packet loss. The optimal number of video packets in an RTP packet and the
length depend the packet-loss rate and the bit-rate of the underlying length of the RTP packet can be determined considering the packet-loss
network. rate and the bit-rate of the underlying network.
Figure 3 shows examples of RTP packets prohibited by the criteria of 3.2. Figure 3 shows examples of RTP packets prohibited by the criteria of 3.2.
Fragmentation of a header into multiple RTP packets, like (a), will not Fragmentation of a header into multiple RTP packets, as in (a), will not
only increase the overhead of RTP/IP headers but also decrease the error only increase the overhead of RTP/IP headers but also decrease the error
resiliency. Therefore, it is prohibited by the criterion (3). resiliency. Therefore, it is prohibited by the criterion (3).
When concatenating more than one video packets into an RTP packet, VOP When concatenating more than one video packets into an RTP packet, VOP
header or video_packet_header() shall not be placed in the middle of the header or video_packet_header() shall not be placed in the middle of the
RTP payload. The packetization like (b) is not allowed by the criterion RTP payload. The packetization as in (b) is not allowed by criterion (2)
(2). This is because of the error resiliency. Comparing this example with due to the aspect of the error resiliency. Comparing this example with
Figure 2(c), two video packets are mapped onto two RTP packets in both Figure 2(d), although two video packets are mapped onto two RTP packets
cases. However, there is a difference between the packet-loss resiliency. in both cases, the packet-loss resiliency is not identical. Namely, if
When the second RTP packet is lost, both video packets 1 and 2 are lost the second RTP packet is lost, both video packets 1 and 2 are lost in the
in the case of Figure 3(b) whereas only video packet 2 is lost in the case of Figure 3(b) whereas only video packet 2 is lost in the case of
case of Figure 2(c). Figure 2(d).
An RTP packet containing more than one VOPs, like (c), is not allowed. An RTP packet containing more than one VOPs, as in (c), is not allowed.
+------+------+------+------+ +------+------+------+------+
(a) | RTP | VS | VO | VOL | (a) | RTP | VS | VO | VOL |
|header|header|header|header| |header|header|header|header|
+------+------+------+------+ +------+------+------+------+
+------+------+------+------+------------+ +------+------+------+------+------------+
(b) | RTP | VS | VO | VOL |Video Packet| (b) | RTP | VS | VO | VOL |Video Packet|
|header|header|header|header| | |header|header|header|header| |
+------+------+------+------+------------+ +------+------+------+------+------------+
skipping to change at page 10, line 7 skipping to change at page 10, line 7
+------+------+------------------+------+------------------+ +------+------+------------------+------+------------------+
(c) | RTP | VOP |Video Object Plane| VOP |Video Object Plane| (c) | RTP | VOP |Video Object Plane| VOP |Video Object Plane|
|header|header| (1) |header| (2) | |header|header| (1) |header| (2) |
+------+------+------------------+------+------------------+ +------+------+------------------+------+------------------+
Figure 3 - Examples of prohibited RTP packetization for MPEG-4 Visual Figure 3 - Examples of prohibited RTP packetization for MPEG-4 Visual
bitstream bitstream
4. RTP Packetization of MPEG-4 Audio bitstream 4. RTP Packetization of MPEG-4 Audio bitstream
When tools defined in MPEG-4 Systems are not used MPEG-4 Audio stream is This section specifies RTP packetization rules for MPEG-4 Audio
formatted by LATM (Low-overhead MPEG-4 Audio Transport Multiplex) bitstreams. MPEG-4 Audio streams are formatted by LATM (Low-overhead
format[5], and then mapped onto RTP packets as described the subsequent MPEG-4 Audio Transport Multiplex) tool[5], and the LATM-based streams are
section. then mapped onto RTP packets as described the three sections below.
4.1 RTP Packet Format 4.1 RTP Packet Format
The LATM consists of the sequence of audioMuxElements that include one or LATM-based streams consist of a sequence of audioMuxElements that include
more audio frames. A complete audioMuxElement or the part of one or more audio frames. A complete audioMuxElement or a part of one
audioMuxElements SHALL be mapped directly onto the RTP payload without SHALL be mapped directly onto an RTP payload without any removal of
removal of any audioMuxElement syntax elements as shown in Figure 4. The audioMuxElement syntax elements (see Figure 4). The first byte of each
first byte of each audioMuxElement SHALL be located at the first payload audioMuxElement SHALL be located at the first payload location in an RTP
location of an RTP packet. packet.
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|X| CC |M| PT | sequence number |RTP |V=2|P|X| CC |M| PT | sequence number |RTP
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| timestamp |Header | timestamp |Header
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| synchronization source (SSRC) identifier | | synchronization source (SSRC) identifier |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
skipping to change at page 10, line 41 skipping to change at page 10, line 41
| .... | | .... |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+ +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| |RTP | |RTP
: audioMuxElement (byte aligned) :Payload : audioMuxElement (byte aligned) :Payload
| | | |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| :...OPTIONAL RTP padding | | :...OPTIONAL RTP padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 4 - An RTP packet for MPEG-4 Audio Figure 4 - An RTP packet for MPEG-4 Audio
It is required for the audioMuxElement to indicate the following In order to decode the audioMuxElement, the following muxConfigPresent
muxConfigPresent information by an out-of-band means. information is required to be indicated by an out-of-band means.
muxConfigPresent: If this information is set to 1, the audioMuxElement muxConfigPresent: If this value is set to 1, the audioMuxElement SHALL
SHALL include an indication bit "useSameStreamMux" and MAY include the include an indication bit "useSameStreamMux" and MAY include the
configuration information for audio compression "StreamMuxConfig". The configuration information for audio compression "StreamMuxConfig". The
useSameStreamMux bit indicates whether the StreamMuxConfig element in the useSameStreamMux bit indicates whether the StreamMuxConfig element in the
previous frame is applied in the current frame. previous frame is applied in the current frame.
4.2 RTP Header Fields Usage 4.2 Use of RTP Header Fields for MPEG-4 Audio
Payload Type (PT): Distinct payload type should be assigned to specify Payload Type (PT): Payload type is to be specifically assigned as the
MPEG-4 Audio RTP payload format. If the dynamic payload type assignment MPEG-4 Audio RTP payload format. If this assignment is to be carried out
is used, it is specified by some out-of-band means (e.g. H.245, SDP, dynamically, it can be performed by such out-of-band means as H.245, SDP,
etc.) that the MPEG-4 Audio payload format is used for the corresponding etc.
RTP packet.
Marker (M) bit: The marker bit indicates audioMuxElement boundaries. This Marker (M) bit: The marker bit indicates audioMuxElement boundaries. It
bit is set to one to mark the RTP packet contains a complete is set to one to indicate that the RTP packet contains a complete
audioMuxElement or the last fragment of an audioMuxElement. audioMuxElement or the last fragment of an audioMuxElement.
Timestamp: The timestamp indicates the composition time, or the Timestamp: The timestamp indicates composition time, or presentation time
presentation time in a no-compositor decoder. Timestamps are recommended in a no-compositor decoder. Timestamps are recommended to start at a
to start at a random value for security reasons. random value for security reasons.
Unless specified by an out-of-band means, the resolution of the timestamp Unless specified by an out-of-band means, the resolution of the timestamp
is set to its default (90 kHz). is set to its default value of 90 kHz.
Sequence Number: Increment by one for each RTP packet sent. It starts Sequence Number: Incremented by one for each RTP packet sent, starting,
with a random value for security reasons. for security reasons, with a random value.
SSRC, CC and CSRC fields are used as described in RFC 1889 [8]. SSRC, CC and CSRC fields are used as described in RFC 1889 [8].
4.3 Fragmentation of MPEG-4 Audio bitstream 4.3 Fragmentation of MPEG-4 Audio bitstream
It is desirable to put one audioMuxElement per RTP packet. The size of an It is desirable to put one audioMuxElement in each RTP packet. If the
audioMuxElement is tried to be adjusted such that the resulting RTP size of an audioMuxElement can be kept small enough that the size of the
packet is not larger than the path-MTU. If this is not possible, the RTP packet containing it does not exceed the size of the path-MTU, this
audioMuxElement MAY be fragmented across several packets based on the will be no problem. If it cannot, the audioMuxElement MAY be fragmented
following rules. and spread across multiple packets, following the rules below:
(1) "payloadMux" which consists of payload elements MAY be fragmented (1) "payloadMux", which consists of payload elements, MAY be fragmented
into several RTP packets so that one RTP packet consists of one or more across several RTP packets, so that each of those RTP packets will
payload elements. A payload element SHOULD NOT be fragmented. contain one or more payload elements. Individual payload elements
themselves SHOULD NOT be fragmented.
(2) If the audioMuxElement includes StreamMuxConfig, StreamMuxConfig (2) If the audioMuxElement includes StreamMuxConfig, StreamMuxConfig
SHALL be included into the RTP packet containing the first payload SHALL be included in the RTP packet that contains the first payload
element. element.
5. RTCP Packetization of MPEG-4 upstream messages 5. MIME type registration for MPEG-4 Audio/Visual streams
This section specifies the usage of particular RTCP packets to carry the
upstream messages generated using the MPEG-4 Audio/Visual upstream
messaging functionalities. In the current specification, NEWPRED in the
MPEG-4 Visual Advance Real Time Simple (ARTS) Profile[4] is only the tool
which uses this RTCP payload specification. This particular RTCP packet
SHALL ONLY be used when it is indicated by some out of band means that
the corresponding MPEG-4 Visual codec is compliant with the ARTS profile
and it is indicated in the configuration information of the MPEG-4 visual
bitstream that the NEWPRED tool is enabled (newpred_enable is set to 1).
5.1. Abstract of NEWPRED in the ARTS profile
NEWPRED in the ARTS profile is an error resilience tool using the
upstream messages from the decoder to the encoder. As the inter-frame
coding is used in the MPEG-4 Visual standard, the image degradation by
packet loss will be propagated to the after several frames. In order to
prevent the temporal error propagation, the reference frames of the
inter-frame coding are switched according to the upstream messages in the
NEWPRED. As the correct frames are used as the reference frame, the
error propagation is refreshed.
As neither the re-transmission nor the intra refresh are used, the coding
efficiency can be kept high. And the NEWPRED can achieve the faster
error recovery than the intra refresh.
There are two types of upstream messages; acknowledged message (NP_ACK)
and non-acknowledged message (NP_NACK). NP_ACK and/or NP_NACK messages
are transmitted on the particular RTCP packets in the NEWPRED. The
selecting methods of reference frames are dependent on the kind of used
messages.
5.2. Particular RTCP packets keep low delay
The real-time Audio/Visual transmission is more sensitive to delay and
does not require full reliability. For Audio/Visual applications it is
more effective to send the MPEG-4 upstream message packets as soon as
possible, i.e. as soon as a loss is detected, without adding any random
delays.
5.3. Congestion control
In the cases of the demand type of intra refresh or the re-transmission,
the amount of bits during the congestion is larger than that in the error
free terms. Therefore they may cause some another congestion. While in
the NEWPRED, as the intra-frame coding is not used, the increased amount
of bits is much lower than that of the intra refresh or the re-
transmission even in the case of packet loss. Therefore NEWPRED causes
less additional burden for the congestion.
The amount of the upstream messages is dependent on the strategy of the
selecting methods of reference frames of the encoder and that of the
sending upstream messages of the decoder. In order to avoid congestion,
the amount of upstream message packets should be small. In the NEWPRED,
the decoder can control the amount of them by not sending some upstream
messages; For example, in the case that the NP_NACK messages are mainly
used to select the reference frames in the encoder, the decoder may not
send the NP_ACK messages even if it receives downstream data. On the
other hand, in the case that the NP_ACK messages are mainly used in the
encoder, the decoder may not send the NP_NACK messages. The amount of the
upstream messages is at most 5% (normally about 1%) of the visual
downstream data.
Especially the amount of NP_ACK messages is decreased in the case of
packet loss. Therefore the NP_ACK message has no additional burden for
the congestion. On the other hand, NP_NACK messages corresponding to the
lost packets are usually sent after the congestion, because the decoder
detects the packet loss after the next downstream packet reaches.
Therefore the NP_NACK message has less additional burden for the
congestion, too.
And to reduce the number of particular RTCP packets, multiple upstream
messages can be concatenated in the payload of one particular RTCP
packet. In this case, it is desirable to send these concatenated
messages as soon as possible.
The particular RTCP transmission interval is according to the interval of
the decoding the visual downstream data. Both the receiving interval of
the visual RTP packet and the decoding time for each packet data have
some jitter for themselves. Therefore the particular RTCP transmission
interval has some jitter for itself. It is effective for the congestion
control, and there is no need to add any random delays. This means that
the size of sending jitter is enough to avoid another congestion only in
case of the unicast.
5.4. Limiting to Unicast
The NEWPRED can work in multicast only in the case that the number of
decoders is small. However in order to avoid the additional congestion,
the NEWPRED over RTP/RTCP SHALL NOT be used in multicast.
5.5. Relations with SR and RR
The particular low delay RTCP packets for the MPEG-4 upstream messages
SHALL be treated as the completely different kind of packets from the
normal RTCP packets; such as SR, RR and so on.
For example, if the particular RTCP packets would be included in the
calculation of RTCP sending interval, the RR packets should be generated
in the timing of the particular low delay RTCP packets. In this case,
the interval of the RR packets would be smaller than 5 seconds, and the
number of the normal RTCP packets is much increased. It is bad for the
congestion.
Therefore all particular RTCP packets SHALL be ignored to analyze the
information in the sender and receiver reports (SR and RR), and only
normal RTCP packets are used.
Multiple particular RTCP packets can be concatenated without any
intervening separators to form a compound RTCP packet. The normal
compound RTCP packet SHOULD start with SR or RR packets. However in the
case of compound particular RTCP packet, other normal RTCP packets SHALL
NOT be included, and only particular RTCP packets SHALL be included in
one compound particular RTCP packet.
5.6. MPEG-4 Visual upstream message packets definition
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P| UMT | PT=RTCP_MP4U | length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| MPEG-4 Upstream Messages Payload (byte aligned) |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| : padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
version (V): 2 bits
Identifies the version of RTP, which is the same in RTCP packets as
in RTP data packets.
padding (P): 1 bit
If the padding bit is set, this RTCP packet contains some additional
padding octets at the end which are not part of the control
information. The last octet of the padding is a count of how many
padding octets should be ignored. In the case several upstream
messages are mapped onto one RTCP packet, padding should only be
required on the last individual message.
upstream message type (UMT): 5 bits
Identifies the type of the MPEG-4 upstream messages.
0: forbidden
1: MPEG-4 Visual NEWPRED in the ARTS Profile
2-63: reserved
In this internet-draft, only the NEWPRED in the ARTS profile is
assigned as the candidate of the UMT for the moment. Some other
MPEG-4 Audio/Visual applications using the upstream messages may be
assigned in the future.
packet type (PT): 8 bits
The value of the packet type (PT) identifier is the constant
RTCP_MP4U (TBD).
SSRC: 32 bits
SSRC is the synchronization source identifier for the sender of this
packet.
MPEG-4 Upstream Message Payload: variable
The syntax and semantics of the MPEG-4 upstream messages are defined
in the ISO/IEC 14496-2/3[4][5]. All messages are byte aligned.
Normally one message is mapped onto one RTCP packet, and several
messages with same UMT could be continuously mapped onto one RTCP
packet. One message SHALL NOT be fragmented into different RTCP
packets.
6. MIME type registration for MPEG-4 Audio/Visual streams
The following sections describe the MIME type registrations for the MPEG- The following sections describe the MIME type registrations for MPEG-4
4 Audio/Visual streams. MIME type registration and SDP usage for the Audio/Visual streams. MIME type registration and SDP usage for the MPEG-4
MPEG-4 Visual stream are described in sections 6.1 and 6.2, respectively. Visual stream are described in Sections 5.1 and 5.2, respectively, while
MIME type registration and SDP usage for the MPEG-4 Audio stream are MIME type registration and SDP usage for MPEG-4 Audio stream are
described in sections 6.3 and 6.4, respectively. described in Sections 5.3 and 5.4, respectively.
(In the following sections, the RFC number "XXXX" represents the RFC (In the following sections, the RFC number "XXXX" represents the RFC
number, which should be assigned for this Internet Draft.) number, which should be assigned for this Internet Draft.)
6.1 MIME type registration for MPEG-4 Visual 5.1 MIME type registration for MPEG-4 Visual
MIME media type name: video MIME media type name: video
MIME subtype name: MP4V MIME subtype name: MP4V
Required parameters: none Required parameters: none
Optional parameters: Optional parameters:
rate: This parameter is used only for RTP transport. It indicates the rate: This parameter is used only for RTP transport. It indicates the
resolution of the timestamp field in the RTP header. If this parameter resolution of the timestamp field in the RTP header. If this parameter
is not specified, the default value of 90000 (90KHz) is used. is not specified, its default value of 90000 (90KHz) is used.
profile-level-id: A decimal representation of MPEG-4 Visual Profile profile-level-id: A decimal representation of MPEG-4 Visual Profile
Level indication value (profile_and_level_indication) defined in Table Level indication value (profile_and_level_indication) defined in Table
G-1 of ISO/IEC 14496-2 [2][4]. G-1 of ISO/IEC 14496-2 [2][4].
mpeg4-newpred-upstream-message: A boolean number to indicate the config: A hexadecimal representation of an octet string that expresses
receiver capability of sending the upstream message of NEWPRED in the MPEG-4 Visual configuration information, as defined in subclause
MPEG-4 video. The upstream messages are delivered on the particular 6.2.1 Start codes of ISO/IEC14496-2[2][4][9]. The configuration
RTCP packets which are described in section 5. This optional exist information is mapped onto the octet string in an MSB-first basis. The
when and only when the "profile-level-id" is 145, 146, 147 or 148 first bit of the configuration information SHALL be located at the MSB
(Advance Real Time Simple Profile/Level 1, 2, 3 or 4). of the first octet. The configuration information indicated by this
parameter SHALL be the same as the configuration information in the
corresponding MPEG-4 Visual stream, except for
first_half_vbv_occupancy and latter_half_vbv_occupancy, if exist,
which may vary in the repeated configuration information inside an
MPEG-4 Visual stream (See 6.2.1 Start codes of ISO/IEC14496-2).
The parameter "profile-level-id" MAY be used in the capability
exchange procedure to indicate MPEG-4 Visual Profile and Level
combination of which the MPEG-4 Visual codec is capable. The parameter
"config" MAY be used to indicate the configuration of the
corresponding MPEG-4 visual bitstream, but SHALL NOT be used to
indicate the codec capability in the capability exchange procedure.
Example usages for these parameters are:
- MPEG-4 Visual Simple Profile/Level 1:
Content-type: video/mp4v; profile-level-id=1
Example usages for these parameters are show bellow:
- MPEG-4 Visual Core Profile/Level 2: - MPEG-4 Visual Core Profile/Level 2:
Content-type: video/mp4v; profile-level-id=34 Content-type: video/mp4v; profile-level-id=34
- MPEG-4 Visual Advanced Real Time Simple Profile/Level 1, upstream - MPEG-4 Visual Advanced Real Time Simple Profile/Level 1:
message is used: Content-type: video/mp4v; profile-level-id=145
Content-type: video/mp4v; profile-level-id=145; mpeg4-newpred-
upstream-message=1
Published specification: Published specification:
The specification of MPEG-4 Visual stream is presented in ISO/IEC The specifications for MPEG-4 Visual streams are presented in ISO/IEC
14469-2[2][4][9]. The RTP payload format is described in RFCXXXX. 14469-2[2][4][9]. The RTP payload format is described in RFCXXXX.
Encoding considerations: Encoding considerations:
A video bitstream must be generated according to the MPEG-4 Visual Video bitstreams must be generated according to MPEG-4 Visual
specification (ISO/IEC 14496-2). The video bitstream is binary data, specifications (ISO/IEC 14496-2). A video bitstream is binary data and
and must be encoded for non-binary transport; the Base64 encoding is must be encoded for non-binary transport (for Email, the Base64
suitable for Email. This type is also defined for transfer via RTP. encoding is sufficient). This type is also defined for transfer via
The RTP packets must be packetized according to the MPEG-4 Visual RTP RTP. The RTP packets MUST be packetized according to the MPEG-4 Visual
payload format defined in RFCXXXX. RTP payload format defined in RFCXXXX.
Security considerations: Security considerations:
See section 9 of RFCXXXX. See section 6 of RFCXXXX.
Interoperability considerations: Interoperability considerations:
MPEG-4 Visual provides a large and rich set of tools for the coding of MPEG-4 Visual provides a large and rich set of tools for the coding of
visual objects. In order to allow effective implementations of the visual objects. For effective implementation of the standard, subsets
standard, subsets of the MPEG-4 Visual tool sets have been identified, of the MPEG-4 Visual tool sets have been provided for use in specific
that can be used for specific applications. These subsets, called applications. These subsets, called 'Profiles', limit the size of the
'Profiles', limit the tool set a decoder has to implement. For each of tool set a decoder is required to implement. In order to restrict
these Profiles, one or more Levels have been set, restricting the computational complexity, one or more Levels are set for each Profiles.
computational complexity. A Profile@Level combination allows: A Profile@Level combination allows:
o a codec builder to implement only the subset of the standard he o a codec builder to implement only the subset of the standard he
needs, while maintaining interworking with other MPEG-4 devices built needs, while maintaining interworking with other MPEG-4 devices
to the same combination, and included in the same combination, and
o checking whether MPEG-4 devices comply with the standard o checking whether MPEG-4 devices comply with the standard
('conformance testing'). ('conformance testing').
The visual stream SHALL be compliant with the MPEG-4 Visual The visual stream SHALL be compliant with the MPEG-4 Visual
Profile@Level specified by the parameter "profile-level-id". The Profile@Level specified by the parameter "profile-level-id".
interoperability between a sender and a receiver may be achieved by Interoperability between a sender and a receiver may be achieved by
specifying the parameter "profile-level-id" in MIME content, or by specifying the parameter "profile-level-id" in MIME content, or by
exchanging this parameter in the capability exchange procedure. arranging in the capability exchange procedure to set this parameter
mutually to the same value.
Applications which use this media type: Applications which use this media type:
Audio and visual streaming and conferencing tools, Internet messaging Audio and visual streaming and conferencing tools, Internet messaging
and e-mail applications. and Email applications.
Additional information: none Additional information: none
Person & email address to contact for further information: Person & email address to contact for further information:
The authors of RFCXXXX. (See section 9) The authors of RFCXXXX. (See section 8)
Intended usage: COMMON Intended usage: COMMON
Author/Change controller: Author/Change controller:
The authors of RFCXXXX. (See section 9) The authors of RFCXXXX. (See section 8)
5.2 SDP usage of MPEG-4 Visual
6.2 SDP usage of MPEG-4 Visual
The MIME media type video/MP4V string is mapped to fields in the Session The MIME media type video/MP4V string is mapped to fields in the Session
Description Protocol (SDP), RFC 2327, as follows: Description Protocol (SDP), RFC 2327, as follows:
o The MIME type (video) goes in SDP "m=" as the media name. o The MIME type (video) goes in SDP "m=" as the media name.
o The MIME subtype (MP4V) goes in SDP "a=rtpmap" as the encoding name. o The MIME subtype (MP4V) goes in SDP "a=rtpmap" as the encoding name.
o The optional parameter "rate" goes in "a=rtpmap" as clock rate. o The optional parameter "rate" goes in "a=rtpmap" as the clock rate.
o The optional parameter "profile-level-id" MAY go in "a=fmtp" line. The o The optional parameter "profile-level-id" and "config" MAY go in the
optional parameter "mpeg4-newpred-upstream-message" MAY go in "a=fmtp" "a=fmtp" line to indicate the coder capability and configuration,
line, when and only when the "profile-level-id" is 145, 146, 147 or respectively. These parameters are expressed as a MIME media type string,
148(Advance Real Time Simple Profile/Level 1, 2, 3 or 4). The format and in the form of as a semicolon separated list of parameter=value pairs.
syntax of these parameters is the MIME media type string as a semicolon
separated list of parameter=value pairs.
The followings are some examples of the media representation in SDP: The following are some examples of media representation in SDP:
Simple Profile/Level 1, rate=90000(90KHz), "profile-level-id" is present Simple Profile/Level 1, rate=90000(90KHz), "profile-level-id" and
in "a=fmtp" line: "config" are present in "a=fmtp" line:
m=video 49170/2 RTP/AVP 98 m=video 49170/2 RTP/AVP 98
a=rtpmap:98 MP4V/90000 a=rtpmap:98 MP4V/90000
a=fmtp:98 profile-level-id=1 a=fmtp:98 profile-level-id=1;
config=000001B001000001B5090000010000000120008440FA282C2090A21F
Core Profile/Level 2, rate=90000(90KHz), "profile-level-id" is present in
"a=fmtp" line:
m=video 49170/2 RTP/AVP 98
a=rtpmap:98 MP4V/90000
a=fmtp:98 profile-level-id=34
Advance Real Time Simple Profile/Level 1, rate=25(25Hz), "profile-level- Advance Real Time Simple Profile/Level 1, rate=25(25Hz), "profile-level-
id" and " newpred- mpeg4- upstream-message" are present in "a=fmtp" line: id" is present in "a=fmtp" line:
m=video 49170/2 RTP/AVP 98 m=video 49170/2 RTP/AVP 98
a=rtpmap:98 MP4V/25 a=rtpmap:98 MP4V/25
a=fmtp:98 profile-level-id=145; mpeg4-newpred-upstream-message=1 a=fmtp:98 profile-level-id=145
6.3 MIME type registration of MPEG-4 Audio 5.3 MIME type registration of MPEG-4 Audio
MIME media type name: audio MIME media type name: audio
MIME subtype name: MP4A MIME subtype name: MP4A
Required parameters: Required parameters:
rate: the rate parameter indicates the RTP time stamp clock rate. The rate: the rate parameter indicates the RTP time stamp clock rate. The
default value is 90000. Other rates CAN be specified only if it would default value is 90000. Other rates CAN be specified only if they are
be set to the same value with the audio sampling rate (number of set to the same value as the audio sampling rate (number of samples
samples per second). per second).
Optional parameters: Optional parameters:
profile-level-id: a decimal representation of MPEG-4 Audio Profile profile-level-id: a decimal representation of MPEG-4 Audio Profile
Level indication value defined in ISO/IEC 14496-1 [11]. This parameter Level indication value defined in ISO/IEC 14496-1 [11]. This parameter
indicates the capability of subsets in MPEG-4 Audio tools. indicates which MPEG-4 Audio tool subsets the decoder is capable of
using.
object: a decimal representation of MPEG-4 Audio Object Type value object: a decimal representation of the MPEG-4 Audio Object Type value
defined in ISO/IEC 14496-3 [5]. This parameter specifies the tool to defined in ISO/IEC 14496-3 [5]. This parameter specifies the tool to
be used by the coder. It CAN be used to limit the capability within be used by the coder. It CAN be used to limit the capability within
the specified "profile-level-id". the specified "profile-level-id".
bitrate: the data rate for the audio bit stream. bitrate: the data rate for the audio bit stream.
cpresent: this parameter indicates whether audio payload configuration cpresent: this parameter indicates whether audio payload configuration
data is multiplexed into the RTP payload (See section 4.1 in this data has been multiplexed into an RTP payload (See section 4.1 in this
document). document).
config: a hexadecimal representation of octet string indicating the config: a hexadecimal representation of an octet string that expresses
audio payload configuration data "StreamMuxConfig" defined in ISO/IEC the audio payload configuration data "StreamMuxConfig", as defined in
14496-3 [5]. The configuration data is mapped into the octet string in ISO/IEC 14496-3 [5]. Configuration data is mapped onto the octet
an MSB-first basis. The first bit of the configuration data shall be string in an MSB-first basis. The first bit of the configuration data
located at the MSB of the first octet. In the last octet, zero-padding SHALL be located at the MSB of the first octet. In the last octet,
bits shall follow the configuration data, if necessary. zero-padding bits, if necessary, shall follow the configuration data.
If the size of the configuration data is quite large, such large
config data is RECOMMENDED to be indicated by in-band mode (cpresent
is set to 1).
ptime: RECOMMENDED duration of each packet in milliseconds. ptime: RECOMMENDED duration of each packet in milliseconds.
Published specification: Published specification:
The payload format specification is described in this document. The Payload format specifications are described in this document. Encoding
specification of encoding is provided in ISO/IEC 14496-3 [3][5]. specifications are provided in ISO/IEC 14496-3 [3][5].
Encoding considerations: Encoding considerations:
This type is only defined for transfer via RTP [RFC YYYY, draft-ietf- This type is only defined for transfer via RTP.
avt-rtp-new].
Security considerations: Security considerations:
See section 9 of RFCXXXX. See Section 6 of RFCXXXX.
Interoperability considerations: Interoperability considerations:
MPEG-4 Audio provides a large and rich set of tools for the coding of MPEG-4 Audio provides a large and rich set of tools for the coding of
visual objects. In order to allow effective implementations of the audio objects. For effective implementation of the standard, subsets of
standard, subsets of the MPEG-4 Audio tool sets have been identified the MPEG-4 Audio tool sets similar to those used in MPEG-4 Visual have
similar to MPEG-4 Audio (See section 6.1). been provided (see section 5.1).
The audio stream SHALL be compliant with the MPEG-4 Audio The audio stream SHALL be compliant with the MPEG-4 Audio
Profile@Level specified by the parameter "profile-level-id". The Profile@Level specified by the parameter "profile-level-id".
interoperability between a sender and a receiver may be achieved by Interoperability between a sender and a receiver may be achieved by
specifying the parameter "profile-level-id" in MIME content, or by specifying the parameter "profile-level-id" in MIME content, or by
exchanging this parameter in the capability exchange procedure. arranging in the capability exchange procedure to set this parameter
Furthermore, the "object" parameter can be used to limit the mutually to the same value. Furthermore, the "object" parameter can be
capability within the specified Profile@Level in capability exchange. used to limit the capability within the specified Profile@Level in
capability exchange.
Applications which use this media type: Applications which use this media type:
Audio and video streaming and conferencing tools. Audio and video streaming and conferencing tools.
Additional information: none Additional information: none
Personal & email address to contact for further information: Personal & email address to contact for further information:
See section 9 of RFCXXXX. See Section 8 of RFCXXXX.
Intended usage: COMMON Intended usage: COMMON
Author/Change controller: Author/Change controller:
See section 9 of RFCXXXX. See Section 8 of RFCXXXX.
6.4 SDP usage of MPEG-4 Audio 5.4 SDP usage of MPEG-4 Audio
The MIME media type audio/MP4A string is mapped to fields in the Session The MIME media type audio/MP4A string is mapped to fields in the Session
Description Protocol (SDP), RFC 2327, as follows: Description Protocol (SDP), RFC 2327, as follows:
o The MIME type (audio) goes in SDP "m=" as the media name. o The MIME type (audio) goes in SDP "m=" as the media name.
o The MIME subtype (MP4A) goes in SDP "a=rtpmap" as the encoding name. o The MIME subtype (MP4A) goes in SDP "a=rtpmap" as the encoding name.
o The required parameter "rate" goes in "a=rtpmap" as clock rate. o The required parameter "rate" goes in "a=rtpmap" as the clock rate.
o The optional parameter "ptime" goes in SDP "a=ptime" attribute. o The optional parameter "ptime" goes in SDP "a=ptime" attribute.
o The optional parameter "profile-level-id" goes in "a=fmtp" line to o The optional parameter "profile-level-id" goes in the "a=fmtp" line to
indicate the coder capability. The "object" parameter goes in "a=fmtp" indicate the coder capability. The "object" parameter goes in the
attribute. Any payload-format-specific parameters "bitrate", "cpresent" "a=fmtp" attribute. The payload-format-specific parameters "bitrate",
and "config" go in "a=fmtp" line. The format and syntax of these "cpresent" and "config" go in the "a=fmtp" line. If the string after
parameters is the MIME media type string as a semicolon separated list of "config=" is quite large, such large config data should not be
parameter=value pairs. transmitted by SDP but should be transmitted by in-band mode. These
parameters are expressed as a MIME media type string, in the form of as a
semicolon separated list of parameter=value pairs.
The followings are some examples of the media representation in SDP: The following are some examples of the media representation in SDP:
For 6 kb/s CELP bitstream (the audio sampling rate of 8 kHz), For 6 kb/s CELP bitstreams (with an audio sampling rate of 8 kHz),
m=audio 49230 RTP/AVP 96 m=audio 49230 RTP/AVP 96
a=rtpmap:96 MP4A/8000 a=rtpmap:96 MP4A/8000
a=fmtp:96 profile-level-id=9;object=8;cpresent=0;config=9128B1071070 a=fmtp:96 profile-level-id=9;object=8;cpresent=0;config=9128B1071070
a=ptime:20 a=ptime:20
For 64 kb/s AAC LC stereo bitstream (the audio sampling rate is 24 kHz), For 64 kb/s AAC LC stereo bitstreams (with an audio sampling rate of 24
kHz),
m=audio 49230 RTP/AVP 96 m=audio 49230 RTP/AVP 96
a=rtpmap:96 MP4A/24000 a=rtpmap:96 MP4A/24000
a=fmtp:96 profile-level-id=1; bitrate=64000; cpresent=0; a=fmtp:96 profile-level-id=1; bitrate=64000; cpresent=0;
config=9122620000 config=9122620000
In the above two examples, the audio configuration data is not In the above two examples, audio configuration data is not multiplexed
multiplexed into the RTP payload and is described only in SDP. into the RTP payload and is described only in SDP. Furthermore, the
Furthermore, the "clock rate" is set to the audio sampling rate. If it is "clock rate" is set to the audio sampling rate.
set to its default, the audio sampling rate can be obtained by parsing
the "config" parameter. If the clock rate has been set to its default value and it is necessary
to obtain the audio sampling rate, this can be done by parsing the
"config" parameter (see the following example).
m=audio 49230 RTP/AVP 96
a=rtpmap:96 MP4A/90000
a=fmtp:96 object=8; cpresent=0; config=9128B1071070
The following example shows that the audio configuration data appears in The following example shows that the audio configuration data appears in
the RTP payload. The value specified in "config" parameter is used as an the RTP payload.
initial value to setup coding parameters.
m=audio 49230 RTP/AVP 96 m=audio 49230 RTP/AVP 96
a=rtpmap:96 MP4A/90000 a=rtpmap:96 MP4A/90000
a=fmtp:96 cpresent=1; config=9128B1071070 a=fmtp:96 object=13; cpresent=1
7. Security Considerations 6. Security Considerations
RTP packets using the payload format defined in this specification are RTP packets using the payload format defined in this specification are
subject to the security considerations discussed in the RTP specification subject to the security considerations discussed in the RTP specification
[8]. This implies that confidentiality of the media streams is achieved [8]. This implies that confidentiality of the media streams is achieved
by encryption. Because the data compression used with this payload format by encryption. Because the data compression used with this payload format
is applied end-to-end, encryption may be performed on the compressed data is applied end-to-end, encryption may be performed on the compressed data
so there is no conflict between the two operations. so there is no conflict between the two operations.
This payload type does not exhibit any significant non-uniformity in the This payload type does not exhibit any significant non-uniformity in the
receiver side computational complexity for packet processing to cause a receiver side computational complexity for packet processing to cause a
potential denial-of-service threat. potential denial-of-service threat.
8. References 7. References
1 Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, 1 Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9,
RFC 2026, October 1996. RFC 2026, October 1996.
2 ISO/IEC 14496-2:1999, "Information technology - Coding of audio-visual 2 ISO/IEC 14496-2:1999, "Information technology - Coding of audio-visual
objects - Part2: Visual", December 1999. objects - Part2: Visual", December 1999.
3 ISO/IEC 14496-3:1999, "Information technology - Coding of audio-visual 3 ISO/IEC 14496-3:1999, "Information technology - Coding of audio-visual
objects - Part3: Audio", December 1999. objects - Part3: Audio", December 1999.
skipping to change at line 910 skipping to change at page 18, line 15
7 Bradner, S., "Key words for use in RFCs to Indicate Requirement 7 Bradner, S., "Key words for use in RFCs to Indicate Requirement
Levels", BCP 14, RFC 2119, March 1997 Levels", BCP 14, RFC 2119, March 1997
8 H. Schulzrinne, S. Casner, R. Frederick, V. Jacobson "RTP: A Transport 8 H. Schulzrinne, S. Casner, R. Frederick, V. Jacobson "RTP: A Transport
Protocol for Real Time Applications", RFC 1889, Internet Engineering Protocol for Real Time Applications", RFC 1889, Internet Engineering
Task Force, January 1996. Task Force, January 1996.
9 ISO/IEC 14496-2/COR1, "Information technology - Coding of audio-visual 9 ISO/IEC 14496-2/COR1, "Information technology - Coding of audio-visual
objects - Part2: Visual, Technical corrigendum 1", March 2000. objects - Part2: Visual, Technical corrigendum 1", March 2000.
9. Author's Addresses 8. Author's Addresses
Yoshihiro Kikuchi Yoshihiro Kikuchi
Toshiba corporation Toshiba corporation
1, Komukai Toshiba-cho, Saiwai-ku, Kawasaki, 212-8582, Japan 1, Komukai Toshiba-cho, Saiwai-ku, Kawasaki, 212-8582, Japan
Email: yoshihiro.kikuchi@toshiba.co.jp Email: yoshihiro.kikuchi@toshiba.co.jp
Yoshinori Matsui Yoshinori Matsui
Matsushita Electric Industrial Co., LTD. Matsushita Electric Industrial Co., LTD.
1006, Kadoma, Kadoma-shi, Osaka, Japan 1006, Kadoma, Kadoma-shi, Osaka, Japan
Email: matsui@drl.mei.co.jp Email: matsui@drl.mei.co.jp
 End of changes. 

This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/