draft-ietf-avt-mpeg4-simple-05.txt   draft-ietf-avt-mpeg4-simple-06.txt 
skipping to change at page 1, line 16 skipping to change at page 1, line 16
V. Swaminathan V. Swaminathan
Sun Microsystems Inc. Sun Microsystems Inc.
D. Singer D. Singer
Apple Computer Apple Computer
P. Gentric P. Gentric
Philips Electronics Philips Electronics
December 2002 December 2002
Expires June 2003 Expires June 2003
Document: draft-ietf-avt-mpeg4-simple-05.txt Document: draft-ietf-avt-mpeg4-simple-06.txt
Transport of MPEG-4 Elementary Streams Transport of MPEG-4 Elementary Streams
Status of this Memo Status of this Memo
This document is an Internet-Draft and is in full conformance with This document is an Internet-Draft and is in full conformance with
all provisions of section 10 of RFC2026. all provisions of section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
skipping to change at page 2, line 32 skipping to change at page 2, line 32
3. Payload format . . . . . . . . . . . . . . . . . . . . . . . 11 3. Payload format . . . . . . . . . . . . . . . . . . . . . . . 11
3.1. Usage of RTP header fields and RTCP . . . . . . . . . . . 11 3.1. Usage of RTP header fields and RTCP . . . . . . . . . . . 11
3.2. RTP payload structure . . . . . . . . . . . . . . . . . . 12 3.2. RTP payload structure . . . . . . . . . . . . . . . . . . 12
3.2.1. The AU Header Section . . . . . . . . . . . . . . . . . 12 3.2.1. The AU Header Section . . . . . . . . . . . . . . . . . 12
3.2.1.1. The AU-header . . . . . . . . . . . . . . . . . . . . 12 3.2.1.1. The AU-header . . . . . . . . . . . . . . . . . . . . 12
3.2.2. The Auxiliary Section . . . . . . . . . . . . . . . . . 14 3.2.2. The Auxiliary Section . . . . . . . . . . . . . . . . . 14
3.2.3. The Access Unit Data Section . . . . . . . . . . . . . . 15 3.2.3. The Access Unit Data Section . . . . . . . . . . . . . . 15
3.2.3.1. Fragmentation . . . . . . . . . . . . . . . . . . . . 16 3.2.3.1. Fragmentation . . . . . . . . . . . . . . . . . . . . 16
3.2.3.2. Interleaving . . . . . . . . . . . . . . . . . . . . . 16 3.2.3.2. Interleaving . . . . . . . . . . . . . . . . . . . . . 16
3.2.3.3. Constraints for interleaving . . . . . . . . . . . . . 17 3.2.3.3. Constraints for interleaving . . . . . . . . . . . . . 17
3.3. Usage of this specification . . . . . . . . . . . . . . . 21 3.2.3.4. Crucial and non-crucial AUs with MPEG-4 System data . 20
3.3.1. General . . . . . . . . . . . . . . . . . . . . . . . . 21 3.3. Usage of this specification . . . . . . . . . . . . . . . 22
3.3.2. The generic mode . . . . . . . . . . . . . . . . . . . . 21 3.3.1. General . . . . . . . . . . . . . . . . . . . . . . . . 22
3.3.3. Constant bit rate CELP . . . . . . . . . . . . . . . . . 22 3.3.2. The generic mode . . . . . . . . . . . . . . . . . . . . 22
3.3.4. Variable bit rate CELP . . . . . . . . . . . . . . . . . 22 3.3.3. Constant bit rate CELP . . . . . . . . . . . . . . . . . 23
3.3.5. Low bit rate AAC . . . . . . . . . . . . . . . . . . . . 23 3.3.4. Variable bit rate CELP . . . . . . . . . . . . . . . . . 23
3.3.6. High bit rate AAC . . . . . . . . . . . . . . . . . . . 24 3.3.5. Low bit rate AAC . . . . . . . . . . . . . . . . . . . . 24
3.3.7. Additional modes . . . . . . . . . . . . . . . . . . . . 25 3.3.6. High bit rate AAC . . . . . . . . . . . . . . . . . . . 25
4. IANA considerations . . . . . . . . . . . . . . . . . . . . 26 3.3.7. Additional modes . . . . . . . . . . . . . . . . . . . . 26
4.1. MIME type registration . . . . . . . . . . . . . . . . . . 26 4. IANA considerations . . . . . . . . . . . . . . . . . . . . 27
4.2. Registration of mode definitions with IANA . . . . . . . . 31 4.1. MIME type registration . . . . . . . . . . . . . . . . . . 27
4.3. Concatenation of parameters . . . . . . . . . . . . . . . 31 4.2. Registration of mode definitions with IANA . . . . . . . . 32
4.4. Usage of SDP . . . . . . . . . . . . . . . . . . . . . . . 32 4.3. Concatenation of parameters . . . . . . . . . . . . . . . 32
4.4.1. The a=fmtp keyword . . . . . . . . . . . . . . . . . . . 32 4.4. Usage of SDP . . . . . . . . . . . . . . . . . . . . . . . 33
5. Security considerations . . . . . . . . . . . . . . . . . . 32 4.4.1. The a=fmtp keyword . . . . . . . . . . . . . . . . . . . 33
6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 33 5. Security considerations . . . . . . . . . . . . . . . . . . 33
7. References . . . . . . . . . . . . . . . . . . . . . . . . . 33 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 34
8. Author addresses . . . . . . . . . . . . . . . . . . . . . . 34 7. References . . . . . . . . . . . . . . . . . . . . . . . . . 34
7.1 Normative references . . . . . . . . . . . . . . . . . . . . 34
APPENDIX: Usage of this payload format . . . . . . . . . . . 36 7.2 Informative references . . . . . . . . . . . . . . . . . . . 34
A. Examples of delay analysis with interleave . . . . . . . 36 8. Author addresses . . . . . . . . . . . . . . . . . . . . . . 35
A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . 36 APPENDIX: Usage of this payload format . . . . . . . . . . . 37
A.2 De-interleaving and error concealment . . . . . . . . . 36 A. Examples of delay analysis with interleave . . . . . . . 37
A.3 Simple Group interleave . . . . . . . . . . . . . . . . 36 A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . 37
A.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . 36 A.2 De-interleaving and error concealment . . . . . . . . . 37
A.3.2 Determining the de-interleave buffer size . . . . . . 37 A.3 Simple Group interleave . . . . . . . . . . . . . . . . 37
A.3.3 Determining the maximum displacement . . . . . . . . . 37 A.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . 37
A.4 More subtle group interleave . . . . . . . . . . . . . . 37 A.3.2 Determining the de-interleave buffer size . . . . . . 38
A.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . 37 A.3.3 Determining the maximum displacement . . . . . . . . . 38
A.4.2 Determining the de-interleave buffer size . . . . . . 38 A.4 More subtle group interleave . . . . . . . . . . . . . . 38
A.4.3 Determining the maximum displacement . . . . . . . . . 38 A.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . 38
A.5 Continuous interleave . . . . . . . . . . . . . . . . . 38 A.4.2 Determining the de-interleave buffer size . . . . . . 39
A.5.1 Introduction . . . . . . . . . . . . . . . . . . . . . 38 A.4.3 Determining the maximum displacement . . . . . . . . . 39
A.5.2 Determining the de-interleave buffer size . . . . . . 39 A.5 Continuous interleave . . . . . . . . . . . . . . . . . 39
A.5.3 Determining the maximum displacement . . . . . . . . . 39 A.5.1 Introduction . . . . . . . . . . . . . . . . . . . . . 39
A.5.2 Determining the de-interleave buffer size . . . . . . 40
A.5.3 Determining the maximum displacement . . . . . . . . . 41
1. Introduction 1. Introduction
The MPEG Committee is Working Group 11 (WG11) in ISO/IEC JTC1 SC29 The MPEG Committee is Working Group 11 (WG11) in ISO/IEC JTC1 SC29
that specified the MPEG-1, MPEG-2 and, more recently, the MPEG-4 that specified the MPEG-1, MPEG-2 and, more recently, the MPEG-4
standards [1]. The MPEG-4 standard specifies compression of standards [1]. The MPEG-4 standard specifies compression of
audio-visual data into for example an audio or video elementary audio-visual data into for example an audio or video elementary
stream. In the MPEG-4 standard, these streams take the form of stream. In the MPEG-4 standard, these streams take the form of
audio-visual objects that may be arranged into an audio-visual scene audio-visual objects that may be arranged into an audio-visual scene
by means of a scene description. Each MPEG-4 elementary stream by means of a scene description. Each MPEG-4 elementary stream
skipping to change at page 4, line 44 skipping to change at page 4, line 44
Applications transmitting streams that include crucial information, Applications transmitting streams that include crucial information,
such as OD commands, BIFS commands, or programmatic content such as such as OD commands, BIFS commands, or programmatic content such as
MPEG-J (Java) and ECMAScript, should include random access points MPEG-J (Java) and ECMAScript, should include random access points
sufficiently often, depending upon the probability of loss, to sufficiently often, depending upon the probability of loss, to
reduce stream corruption to an acceptable level. An example is the reduce stream corruption to an acceptable level. An example is the
carousel mechanism as defined by MPEG in ISO/IEC 14496-1. carousel mechanism as defined by MPEG in ISO/IEC 14496-1.
Such applications may also employ additional protocols or services Such applications may also employ additional protocols or services
to reduce the probability of loss. At the RTP layer, these measures to reduce the probability of loss. At the RTP layer, these measures
include payload formats and profiles for retransmission or forward include payload formats and profiles for retransmission or forward
error correction (such as in RFC 2733), which must be employed with error correction (such as in RFC 2733 [10]), which must be employed
due consideration to congestion control. Another solution that may with due consideration to congestion control. Another solution that
be appropriate for some applications is to carry RTP over TCP (such may be appropriate for some applications is to carry RTP over TCP
as in RFC 2326, section 10.12). At the network layer, resource (such as in RFC 2326 [8], section 10.12). At the network layer,
allocation or preferential service may be available to reduce the resource allocation or preferential service may be available to
probability of loss. For a general description of methods to repair reduce the probability of loss. For a general description of methods
streaming media see RFC 2354. to repair streaming media see RFC 2354 [9].
Though the RTP payload format defined in this document is capable Though the RTP payload format defined in this document is capable
of transporting any MPEG-4 stream, other, more specific, formats of transporting any MPEG-4 stream, other, more specific, formats
may exist, such as RFC 3016 for transport of MPEG-4 video (part 2). may exist, such as RFC 3016 [12] for transport of MPEG-4 video
(ISO/IEC 14496 [1] part 2).
Configuration of the payload is provided to accommodate transport Configuration of the payload is provided to accommodate transport
of any MPEG-4 stream at any possible bit rate. However, for a of any MPEG-4 stream at any possible bit rate. However, for a
specific MPEG-4 elementary stream typically only very few specific MPEG-4 elementary stream typically only very few
configurations are needed. So as to allow for the design of configurations are needed. So as to allow for the design of
simplified, but dedicated receivers, this specification requires simplified, but dedicated receivers, this specification requires
that specific modes are defined for transport of MPEG-4 streams. that specific modes are defined for transport of MPEG-4 streams.
This document defines modes for MPEG-4 CELP and AAC streams, as This document defines modes for MPEG-4 CELP and AAC streams, as
well as a generic mode that can be used to transport any MPEG-4 well as a generic mode that can be used to transport any MPEG-4
stream. In the future new RFCs are expected to specify additional stream. In the future new RFCs are expected to specify additional
skipping to change at page 5, line 32 skipping to change at page 5, line 33
information that may be contained in the MPEG-4 Sync Layer (SL) as information that may be contained in the MPEG-4 Sync Layer (SL) as
defined in MPEG-4 Systems [1]. This document does not prescribe how defined in MPEG-4 Systems [1]. This document does not prescribe how
to transcode or map information from the SL to fields defined in to transcode or map information from the SL to fields defined in
the RTP payload format. Such processing, if any, is left to the the RTP payload format. Such processing, if any, is left to the
discretion of the application. However, to anticipate the need for discretion of the application. However, to anticipate the need for
transport of any additional system-related information in future, transport of any additional system-related information in future,
an auxiliary field can be configured that may carry any such data. an auxiliary field can be configured that may carry any such data.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in
this document are to be interpreted as described in RFC 2119 [3]. this document are to be interpreted as described in RFC 2119 [4].
2. Carriage of MPEG-4 elementary streams over RTP 2. Carriage of MPEG-4 elementary streams over RTP
2.1 Introduction 2.1 Introduction
With this payload format a single MPEG-4 elementary stream can be With this payload format a single MPEG-4 elementary stream can be
transported. Information on the type of MPEG-4 stream carried in transported. Information on the type of MPEG-4 stream carried in
the payload is conveyed by MIME format parameters, for example in the payload is conveyed by MIME format parameters, for example in
an SDP [6] message or by other means (see section 4). These MIME an SDP [5] message or by other means (see section 4). These MIME
format parameters specify the configuration of the payload. To format parameters specify the configuration of the payload. To
allow for simplified and dedicated receivers, a MIME format allow for simplified and dedicated receivers, a MIME format
parameter is available to signal a specific mode of using this parameter is available to signal a specific mode of using this
payload. A mode definition MAY include the type of MPEG-4 payload. A mode definition MAY include the type of MPEG-4
elementary stream as well as the applied configuration, so as to elementary stream as well as the applied configuration, so as to
avoid the need in receivers to parse all MIME format parameters. avoid the need in receivers to parse all MIME format parameters.
The applied mode MUST be signaled. The applied mode MUST be signaled.
2.2 MPEG Access Units 2.2 MPEG Access Units
skipping to change at page 9, line 35 skipping to change at page 9, line 35
fields are defined for carriage in the RTP payload. However, their fields are defined for carriage in the RTP payload. However, their
use strongly depends on the type of MPEG-4 elementary stream that use strongly depends on the type of MPEG-4 elementary stream that
is carried. Sometimes a specific field is needed with a certain is carried. Sometimes a specific field is needed with a certain
length, while in other cases such field is not needed at all. To be length, while in other cases such field is not needed at all. To be
efficient in either case, the fields to support these features are efficient in either case, the fields to support these features are
configurable by means of MIME format parameters. In general, a MIME configurable by means of MIME format parameters. In general, a MIME
format parameter defines the presence and length of the associated format parameter defines the presence and length of the associated
field. A length of zero indicates absence of the field. As a field. A length of zero indicates absence of the field. As a
consequence, parsing of the payload requires knowledge of MIME consequence, parsing of the payload requires knowledge of MIME
format parameters. The MIME format parameters are conveyed to the format parameters. The MIME format parameters are conveyed to the
receiver via SDP [6] messages, as specified in section 4.4.1, or receiver via SDP [5] messages, as specified in section 4.4.1, or
through other means. through other means.
2.11 Global structure of payload format 2.11 Global structure of payload format
The RTP payload following the RTP header, contains three The RTP payload following the RTP header, contains three
octet-aligned data sections, of which the first two MAY be empty. octet-aligned data sections, of which the first two MAY be empty.
See figure 1. See figure 1.
+---------+-----------+-----------+---------------+ +---------+-----------+-----------+---------------+
| RTP | AU Header | Auxiliary | Access Unit | | RTP | AU Header | Auxiliary | Access Unit |
skipping to change at page 10, line 37 skipping to change at page 10, line 37
In this document several modes are defined for transport of MPEG-4 In this document several modes are defined for transport of MPEG-4
CELP and AAC streams, as well as a generic mode that can be used CELP and AAC streams, as well as a generic mode that can be used
for any MPEG-4 stream. In the future, new RFCs may specify other for any MPEG-4 stream. In the future, new RFCs may specify other
modes of using this specification. However, each mode MUST be in modes of using this specification. However, each mode MUST be in
full compliance with this specification (see section 3.3.7). full compliance with this specification (see section 3.3.7).
2.13 Alignment with RFC 3016 2.13 Alignment with RFC 3016
This payload can be configured to be nearly identical to the This payload can be configured to be nearly identical to the
payload format defined in RFC 3016 [5] for the MPEG-4 video payload format defined in RFC 3016 [12] for the MPEG-4 video
configurations recommended in RFC 3016. Hence, receivers that configurations recommended in RFC 3016. Hence, receivers that
comply with RFC 3016 can decode such RTP payload, providing that comply with RFC 3016 can decode such RTP payload, providing that
additional packets containing video decoder configuration (VO, additional packets containing video decoder configuration (VO,
VOL, VOSH) are inserted in the stream, as required by RFC 3016. VOL, VOSH) are inserted in the stream, as required by RFC 3016.
Conversely, receivers that comply with the specification in this Conversely, receivers that comply with the specification in this
document should be able to decode payloads, names and parameters document should be able to decode payloads, names and parameters
defined for MPEG-4 video in RFC 3016. In this respect it is defined for MPEG-4 video in RFC 3016. In this respect it is
strongly RECOMMENDED to implement the ability to ignore "in band" strongly RECOMMENDED to implement the ability to ignore "in band"
video decoder configuration packets in the RFC 3016 payload. video decoder configuration packets in the RFC 3016 payload.
skipping to change at page 12, line 49 skipping to change at page 12, line 46
zero-padding bits MUST be inserted at the end in order to achieve zero-padding bits MUST be inserted at the end in order to achieve
octet-alignment of the AU Header Section. octet-alignment of the AU Header Section.
3.2.1.1 The AU-header 3.2.1.1 The AU-header
Each AU-header may contain the fields given in figure 3. The length Each AU-header may contain the fields given in figure 3. The length
in bits of the above fields with the exception of the CTS-flag, the in bits of the above fields with the exception of the CTS-flag, the
DTS-flag and the RAP-flag fields is defined by MIME format DTS-flag and the RAP-flag fields is defined by MIME format
parameters; see section 4.1. If a MIME format parameter has the parameters; see section 4.1. If a MIME format parameter has the
default value of zero, then the associated field is not present. default value of zero, then the associated field is not present.
The number of bits for fields that are present and that represent
the value of a parameter MUST be chosen large enough to correctly
encode the largest value of that parameter during the session.
If present, the fields MUST occur in the mutual order given in If present, the fields MUST occur in the mutual order given in
figure 3. In the general case a receiver can only discover the size figure 3. In the general case a receiver can only discover the size
of an AU-header by parsing it since the presence of the CTS-delta of an AU-header by parsing it since the presence of the CTS-delta
and DTS-delta fields is signaled by the value of the CTS-flag and and DTS-delta fields is signaled by the value of the CTS-flag and
DTS-flag, respectively. DTS-flag, respectively.
+---------------------------------------+ +---------------------------------------+
| AU-size | | AU-size |
+---------------------------------------+ +---------------------------------------+
skipping to change at page 13, line 45 skipping to change at page 13, line 45
fragment, which is particularly useful after losing a packet fragment, which is particularly useful after losing a packet
carrying the last fragment of an AU. carrying the last fragment of an AU.
AU-Index: Indicates the serial number of the associated Access Unit AU-Index: Indicates the serial number of the associated Access Unit
(fragment). For each (in decoding order) consecutive AU or AU (fragment). For each (in decoding order) consecutive AU or AU
fragment, the serial number is incremented with 1. When fragment, the serial number is incremented with 1. When
present, the AU-Index field occurs in the first AU-header in present, the AU-Index field occurs in the first AU-header in
the AU Header Section, but MUST NOT occur in any subsequent the AU Header Section, but MUST NOT occur in any subsequent
(non-first) AU-header in that Section. To encode the serial (non-first) AU-header in that Section. To encode the serial
number in any such non-first AU-header, the AU-Index-delta number in any such non-first AU-header, the AU-Index-delta
field is used. If each AU-Index field is coded with the value field is used.
0, the serial number of the AU (fragment) is not specified,
and in that case receivers may ignore the AU-Index field.
AU-Index-delta: The AU-Index-delta field is an unsigned integer AU-Index-delta: The AU-Index-delta field is an unsigned integer
that specifies the serial number of the associated AU as the that specifies the serial number of the associated AU as the
difference with respect to the serial number of the previous difference with respect to the serial number of the previous
Access Unit. Hence, for the n-th (n>1) AU the serial number Access Unit. Hence, for the n-th (n>1) AU the serial number
is found from: is found from:
AU-Index(n) = AU-Index(n-1) + AU-Index-delta(n) + 1 AU-Index(n) = AU-Index(n-1) + AU-Index-delta(n) + 1
If the AU-Index field is present in the first AU-header in If the AU-Index field is present in the first AU-header in
the AU Header Section, then the AU-Index-delta field MUST be the AU Header Section, then the AU-Index-delta field MUST be
present in any subsequent (non-first) AU-header. When the present in any subsequent (non-first) AU-header. When the
skipping to change at page 16, line 46 skipping to change at page 16, line 46
A packet SHALL carry either one or more complete Access Units, or A packet SHALL carry either one or more complete Access Units, or
a single fragment of an Access Unit. Fragments of the same Access a single fragment of an Access Unit. Fragments of the same Access
Unit have the same time stamp but different RTP sequence numbers. Unit have the same time stamp but different RTP sequence numbers.
The marker bit in the RTP header is 1 on the last fragment of an The marker bit in the RTP header is 1 on the last fragment of an
Access Unit, and 0 on all other fragments. Access Unit, and 0 on all other fragments.
3.2.3.2 Interleaving 3.2.3.2 Interleaving
Access Units MAY be interleaved. Senders MAY perform interleaving. Access Units MAY be interleaved. Senders MAY perform interleaving.
Receivers MUST support interleaving, except if the receiver only Receivers MUST support interleaving, except if the receiver only
supports modes in which no interleaving is allowed. When supports modes in which no interleaving is allowed. When Access
interleaving of Access Units is used it SHALL be implemented using Units are interleaved, it SHALL be implemented using the AU-Index
the AU-Index and AU-Index-delta fields in the AU-header. and the AU-Index-delta fields in the AU-header.
Based on the RTP sequence number, the RTP time stamp, the AU-Index When a sender interleaves Access Units, then the transmitter needs
and the AU-Index-delta, a receiver can unambiguously reconstruct to provide sufficient information to enable a receiver to
the original order even in case of out-of-order packets, packet unambiguously reconstruct the original order, even in case of
loss or duplication. Note that for this purpose the AU-Index is out-of-order packets, packet loss or duplication. The information
redundant when the RTP time stamp and the AU-Index-delta values are that senders need to provide depends on whether or not the Access
sufficient for placing the AUs correctly in time. In such cases Units have a constant time duration. Access Units have a constant
receivers MAY ignore the AU-Index value and senders MAY code the time duration, if:
AU-Index field with the value 0, but only if they code each AU-Index
field with that value. If the AU-Index is not redundant, senders TS(i+1) TS(i) = constant, for any i, where
SHOULD use a length of the AU-Index field so that this field is not
coded with the value 0 in two subsequent RTP packets. i indicates the index of the AU in original order
TS(i) denotes the time stamp of AU(i)
If Access Units have a constant time duration then a receiver can
unambiguously reconstruct the original order based on the RTP
time stamp, the AU-Index and the AU-Index-delta. Note that for this
purpose the AU-Index is redundant, as the RTP time stamp and the
AU-Index-delta values are sufficient for placing the AUs correctly
in time. The RTP time stamp usually provides better robustness to
large bursts of packet losses, and is therefore to be preferred.
In order to unambiguously determine the index of each AU in the
most convenient way when the AUs have a constant time duration, the
value of the time duration SHOULD be signaled by the MIME format
parameter "constantDuration", see section 4.1.
If the "constantDuration" parameter is present, then the transmitter
MUST encode the AU-Index, if present, with the value 0 and the
receiver MUST use the RTP time stamp to determine the index of the
first AU in the RTP packet.
If the "constantDuration" parameter is not present, then Access
Units are assumed to have a variable duration. In this case, the
AU-Index is not redundant, and MUST provide the index information
required for re-ordering, and the receiver MUST use that value to
determine the index of the first AU in the RTP packet. The number
of bits of the AU-Index field MUST be chosen so that valid index
information is provided at the applied interleaving scheme, without
causing problems due to roll-over of the AU-Index field. For
variable duration AUs, index information is needed to reconstruct
the original order and to identify missing AUs, but to place the
AUs correctly in time, for each AU the time stamp is needed.
Therefore, if the "constantDuration" parameter is not present, then
the CTS-delta MUST be coded in the AU header for each non-first AU
in the RTP packet.
When interleaving is applied, a de-interleave buffer is needed in When interleaving is applied, a de-interleave buffer is needed in
receivers to put the Access Units in their correct logical receivers to put the Access Units in their correct logical
consecutive decoding order. This requires the computation of the consecutive decoding order. This requires the computation of the
time stamp for each Access Unit. In case of a fixed time duration time stamp for each Access Unit. In case of a constant time duration
per Access Unit, the time stamp of the i-th access unit in an RTP per Access Unit, the time stamp of the i-th access unit in an RTP
packet with RTP time stamp T is calculated as follows: packet with RTP time stamp T is calculated as follows:
Timestamp[0] = T Timestamp[0] = T
Timestamp[i, i > 0] = T +(Sum(for k=1 to i of (AU-Index-delta[k] Timestamp[i, i > 0] = T +(Sum(for k=1 to i of (AU-Index-delta[k]
+ 1))) * access-unit-duration + 1))) * access-unit-duration
When AU-Index-delta is always 0, this reduces to T + i * (access- When AU-Index-delta is always 0, this reduces to T + i * (access-
unit-duration). This is the non-interleaved case, where the frames unit-duration). This is the non-interleaved case, where the frames
are consecutive in decoding order. Note that the AU-Index field are consecutive in decoding order. Note that the AU-Index field
(present for the first Access Unit) is not needed in this (present for the first Access Unit) is indeed not needed in this
calculation. Hence in cases where the access-unit-duration has a calculation.
fixed and known value, the AU-Index does not need to provide index
information and can be coded with the value 0. See also the
semantics of the AU-Index field in 3.2.1.1.
If the Access Units are not fixed duration, the AU-Index is not
redundant, and MUST provide the index information required for
re-ordering. The number of bits of the AU-Index field MUST be chosen
so that valid index information is provided at the applied
interleaving scheme, without causing problems due to roll-over of
the AU-Index field. Note that the CTS-delta may be required to
compute the correct time stamp for each AU.
3.2.3.3 Constraints for interleaving 3.2.3.3 Constraints for interleaving
The size of the packets should be suitably chosen to be appropriate The size of the packets should be suitably chosen to be appropriate
to both the path MTU and the capacity of the receiver's to both the path MTU and the capacity of the receiver's
de-interleave buffer. The maximum packet size for a session SHOULD de-interleave buffer. The maximum packet size for a session SHOULD
be chosen not to exceed the path MTU. be chosen not to exceed the path MTU.
To allow receivers to allocate sufficient resources for To allow receivers to allocate sufficient resources for
de-interleaving, senders MUST provide the information to receivers de-interleaving, senders MUST provide the information to receivers
as specified in this section. as specified in this section.
AUs enter the decoder in decoding order. The de-interleave buffer AUs enter the decoder in decoding order. The de-interleave buffer
is used to re-order a stream of interleaved AUs back into decoding is used to re-order a stream of interleaved AUs back into decoding
order. When interleaving is applied, the decoding of "early" AUs order. When interleaving is applied, the decoding of "early" AUs
has to be postponed until all AUs that precede in decoding order has to be postponed until all AUs that precede in decoding order
have been received. Therefore these "early" AUs are stored in the are present. Therefore these "early" AUs are stored in the
de-interleave buffer. As an example in figure 6 the interleaving de-interleave buffer. As an example in figure 6 the interleaving
pattern from section 2.5 is considered. pattern from section 2.5 is considered.
+--+--+--+--+--+--+--+--+--+--+--+- +--+--+--+--+--+--+--+--+--+--+--+-
Interleaved AUs | 0| 3| 6| 1| 4| 7| 2| 5| 8| 9|12|.. Interleaved AUs | 0| 3| 6| 1| 4| 7| 2| 5| 8| 9|12|..
+--+--+--+--+--+--+--+--+--+--+--+- +--+--+--+--+--+--+--+--+--+--+--+-
Storage of "early" AUs 3 3 3 3 3 3 Storage of "early" AUs 3 3 3 3 3 3
6 6 6 6 6 6 6 6 6 6 6 6
4 4 4 4 4 4
7 7 7 7 7 7
12 12 12 12
Figure 6: Storage of "early" AUs in the de-interleave buffer per Figure 6: Storage of "early" AUs in the de-interleave buffer per
interleaved AU. interleaved AU.
AU(3) is to be delivered to the decoder after AU(0), AU(1)and AU(2); AU(3) is to be delivered to the decoder after AU(0), AU(1)and
of these AUs, AU(2) is most late and hence AU(3) needs to be stored AU(2); of these AUs, AU(2) is most late and hence AU(3) needs to be
until AU(2) is received. Similarly, AU(6) is to be stored until stored until AU(2) is present in the pattern. Similarly, AU(6) is
AU(5) is received, while AU(4) and AU(7) are to be stored until to be stored until AU(5) is present, while AU(4) and AU(7) are to
AU(2) and AU(5) are received, respectively. Note that the fullness be stored until AU(2) and AU(5) are present, respectively. Note
of the de-interleave buffer varies in time. In figure 6, the that the fullness of the de-interleave buffer varies in time. In
de-interleave buffer contains at most 4, but often less AUs. figure 6, the de-interleave buffer contains at most 4, but often
less AUs.
So as to give a rough indication of the resources needed in the So as to give a rough indication of the resources needed in the
receiver for de-interleaving, the maximum displacement in time of receiver for de-interleaving, the maximum displacement in time of
an AU is defined. The maximum displacement in time of an AU is the an AU is defined. For any AU in the pattern it can be verified
maximum difference between the time stamp of any received AU and which AUs are not yet present. The maximum displacement in time of
the time stamp of the earliest AU that is not yet received. In other an AU is the maximum difference between the time stamp of an AU in
words, when considering a sequence of interleaved AUs, then: the pattern and the time stamp of the earliest AU that is not yet
present. In other words, when considering a sequence of interleaved
AUs, then:
Maximum displacement = max{TS(i) - TS(j)}, for any i and any j>i, Maximum displacement = max{TS(i) - TS(j)}, for any i and any j>i,
where i and j indicate the index of the AU in the where i and j indicate the index of the AU in the
interleaving pattern and interleaving pattern and TS denotes the time stamp
TS denotes the time stamp of the AU of the AU
As an example in figure 7 the interleaving pattern from section 2.5 As an example in figure 7 the interleaving pattern from section 2.5
is considered. For each AU in the pattern the earliest not yet is considered. For each AU in the pattern the earliest not yet
received AU is indicated. A "-" indicates that all previous AUs present AU is indicated. A "-" indicates that all previous AUs
are received. If the AU period is constant, the maximum displacement are present. If the AU period is constant, the maximum displacement
equals 5 AU periods, as found for AU(6) and AU(7). equals 5 AU periods, as found for AU(6) and AU(7).
+--+--+--+--+--+--+--+--+--+--+--+- +--+--+--+--+--+--+--+--+--+--+--+-
Interleaved AUs | 0| 3| 6| 1| 4| 7| 2| 5| 8| 9|12|.. Interleaved AUs | 0| 3| 6| 1| 4| 7| 2| 5| 8| 9|12|..
+--+--+--+--+--+--+--+--+--+--+--+- +--+--+--+--+--+--+--+--+--+--+--+-
Earliest not yet received AU - 1 1 - 2 2 - - - - 10 Earliest not yet present AU - 1 1 - 2 2 - - - - 10
Figure 7: The earliest not yet received AU for each AU in the Figure 7: The earliest not yet present AU for each AU in the
interleaving pattern. interleaving pattern.
When interleaving, senders MUST signal the maximum displacement When interleaving, senders MUST signal the maximum displacement
in time during the session via the MIME format parameter in time during the session via the MIME format parameter
"maxDisplacement"; see section 4.1. "maxDisplacement"; see section 4.1.
An estimate of the size of the de-interleave buffer is found by An estimate of the size of the de-interleave buffer is found by
multiplying the maximum displacement by the maximum bit rate: multiplying the maximum displacement by the maximum bit rate:
size(de-interleave buffer) = {(maxDisplacement) * Rate(max)} / (RTP size(de-interleave buffer) = {(maxDisplacement) * Rate(max)} / (RTP
clock frequency), clock frequency),
where Rate(max) is the maximum bit-rate of the transported stream. where Rate(max) is the maximum bit-rate of the transported stream.
Note that receivers can derive Rate(max) from the MIME format Note that receivers can derive Rate(max) from the MIME format
parameters StreamType, Profile-level-id, and config. parameters StreamType, Profile-level-id, and config.
However, this calculation estimates the size of the de-interleave However, this calculation estimates the size of the de-interleave
buffer and its size may be larger than calculated. If this buffer and the really required size may differ from the calculated
calculation under-estimates the size of the de-interleave buffer, value. If this calculation under-estimates the size of the
then senders, when interleaving, MUST signal a size of the de-interleave buffer, then senders, when interleaving, MUST signal
de-interleave buffer that is large enough to contain all "early" a size of the de-interleave buffer via the MIME format parameter
AUs at any point in time during the session via the MIME format "de-interleaveBufferSize"; see section 4.1. If the calculation
parameter "de-interleaveBufferSize"; see section 4.1. over-estimates the size of the de-interleave buffer, then senders,
when interleaving, MAY signal a size of the de-interleave buffer
via the MIME format parameter "de-interleaveBufferSize".
The signaled size of the de-interleave buffer MUST be large enough
to contain all "early" AUs at any point in time during the session,
that is:
minimum de-interleave buffer size = max [sum {if TS(i) > TS(j) then
AU-size(i) else 0}] for any j
and any i<j, where
i and j indicate the index of an AU in the interleaving
pattern,
TS(i) denotes the time stamp of AU(i), and
AU-size(i) denotes the size of AU(i) in number of octets.
If the "de-interleaveBufferSize" parameter is present, then the If the "de-interleaveBufferSize" parameter is present, then the
applied buffer for de-interleaving in a receiver MUST have a size applied buffer for de-interleaving in a receiver MUST have a size
that is at least equal to the signaled size of the de-interleave that is at least equal to the signaled size of the de-interleave
buffer, else a size that is at least equal to the calculated size buffer, else a size that is at least equal to the calculated size
of the de-interleave buffer. of the de-interleave buffer.
No matter what interleaving scheme is used, the scheme must be No matter what interleaving scheme is used, the scheme must be
analyzed to calculate the applicable maxDisplacement value, as well analyzed to calculate the applicable maxDisplacement value, as well
as the required size of the de-interleave buffer. Senders SHOULD as the required size of the de-interleave buffer. Senders SHOULD
signal values that are not larger than the strictly required signal values that are not larger than the strictly required
values; if larger values are signalled, the receiver will buffer values; if larger values are signalled, the receiver will buffer
excessively. excessively.
Note that for low bit-rate material, the applied interleaving Note that for low bit-rate material, the applied interleaving
may make packets shorter than the MTU size. may make packets shorter than the MTU size.
3.2.3.5. Crucial and non-crucial AUs with MPEG-4 System data 3.2.3.4. Crucial and non-crucial AUs with MPEG-4 System data
Some Access Units with MPEG-4 system data, called "crucial" AUs, Some Access Units with MPEG-4 system data, called "crucial" AUs,
carry information whose loss cannot be tolerated, either in the carry information whose loss cannot be tolerated, either in the
presentation or in the decoder. At each crucial AU in an MPEG-4 presentation or in the decoder. At each crucial AU in an MPEG-4
system stream, the stream state changes. The stream-state MAY system stream, the stream state changes. The stream-state MAY
remain constant at non-crucial AUs. In ISO/IEC 14496-1, MPEG-4 remain constant at non-crucial AUs. In ISO/IEC 14496-1, MPEG-4
system streams use the AU_SequenceNumber to signal stream states. system streams use the AU_SequenceNumber to signal stream states.
Example: Given three AUs, AU1 = "Insertion of node X", AU2 = "Set Example: Given three AUs, AU1 = "Insertion of node X", AU2 = "Set
position of node X", AU3 = "Set position of node X". AU1 is crucial, position of node X", AU3 = "Set position of node X". AU1 is crucial,
skipping to change at page 21, line 17 skipping to change at page 22, line 17
3.3.1 General 3.3.1 General
Usage of this specification requires definition of a mode. A mode Usage of this specification requires definition of a mode. A mode
defines how to use this specification, as deemed appropriate. defines how to use this specification, as deemed appropriate.
Senders MUST signal the applied mode via the MIME format parameter Senders MUST signal the applied mode via the MIME format parameter
"Mode", as specified in section 4.1. This specification defines a "Mode", as specified in section 4.1. This specification defines a
generic mode that can be used for any MPEG-4 stream, as well as generic mode that can be used for any MPEG-4 stream, as well as
specific modes for transport of MPEG-4 CELP and MPEG-4 AAC streams, specific modes for transport of MPEG-4 CELP and MPEG-4 AAC streams,
defined in ISO/IEC 14496-3. defined in ISO/IEC 14496-3.
When use of this payload format is signaled using SDP [6], an When use of this payload format is signaled using SDP [5], an
"rtpmap" attribute is part of that signaling. The same requirements "rtpmap" attribute is part of that signaling. The same requirements
apply for the rtpmap attribute in any mode compliant to this apply for the rtpmap attribute in any mode compliant to this
specification. The general form of an rtpmap attribute is: specification. The general form of an rtpmap attribute is:
a=rtpmap:<payload type> <encoding name>/<clock rate>[/<encoding a=rtpmap:<payload type> <encoding name>/<clock rate>[/<encoding
parameters>] parameters>]
For audio streams, <encoding parameters> specifies the number of For audio streams, <encoding parameters> specifies the number of
audio channels: 2 for stereo material (see RFC 2327) and 1 for audio channels: 2 for stereo material (see RFC 2327 [5]) and 1 for
mono. Provided no additional parameters are needed, this parameter mono. Provided no additional parameters are needed, this parameter
may be omitted for mono material, hence its default value is 1. may be omitted for mono material, hence its default value is 1.
3.3.2 The generic mode 3.3.2 The generic mode
The generic mode can be used for any MPEG-4 stream. In this mode The generic mode can be used for any MPEG-4 stream. In this mode
no mode-specific constraints are applied; hence, in the generic no mode-specific constraints are applied; hence, in the generic
mode the full flexibility of this specification can be exploited. mode the full flexibility of this specification can be exploited.
The generic mode is signaled by mode=generic. The generic mode is signaled by mode=generic.
skipping to change at page 22, line 54 skipping to change at page 23, line 50
this mode. this mode.
In this mode the RTP payload consists of the AU Header Section, In this mode the RTP payload consists of the AU Header Section,
followed by one or more concatenated CELP frames. The Auxiliary followed by one or more concatenated CELP frames. The Auxiliary
Section MUST be empty. For each CELP frame contained in the payload Section MUST be empty. For each CELP frame contained in the payload
there MUST be a one octet AU-header in the AU Header Section to there MUST be a one octet AU-header in the AU Header Section to
provide: provide:
(a) the size of each CELP frame in the payload and (a) the size of each CELP frame in the payload and
(b) index information for computing the sequence (and hence timing) (b) index information for computing the sequence (and hence timing)
of each CELP frame. of each CELP frame.
Transport of CELP frames requires that the AU-size field is coded Transport of CELP frames requires that the AU-size field is coded
with 6 bits. In this mode therefore 6 bits are allocated to the with 6 bits. In this mode therefore 6 bits are allocated to the
AU-size field, and 2 bits to the AU-Index(-delta) field. Each AU-size field, and 2 bits to the AU-Index(-delta) field. Each
AU-Index field MUST be coded with the value 0. In the AU Header AU-Index field MUST be coded with the value 0. In the AU Header
Section, the concatenated AU-headers are preceded by the 16-bit Section, the concatenated AU-headers are preceded by the 16-bit
AU-headers-length field, as specified in section 3.2.1. AU-headers-length field, as specified in section 3.2.1.
In addition to the required MIME format parameters, the following In addition to the required MIME format parameters, the following
parameters MUST be present: SizeLength, IndexLength, and parameters MUST be present: SizeLength, IndexLength, and
IndexDeltaLength. IndexDeltaLength. CELP frames have fixed time duration per Access
When interleaving is applied (AU-Index-delta coded with a value Unit; when interleaving in this mode, the applicable duration MUST
larger than 0), the parameter InterleaveDelay MUST also be present. be signaled by the MIME format parameter constantDuration. In
addition, the parameter maxDisplacement MUST be present when
interleaving.
For example: For example:
m=audio 49230 RTP/AVP 96 m=audio 49230 RTP/AVP 96
a=rtpmap:96 mpeg4-generic/44100/2 a=rtpmap:96 mpeg4-generic/8000/1
a=fmtp:96 streamtype=5; profile-level-id=15; mode=CELP-vbr; config= a=fmtp:96 streamtype=5; profile-level-id=15; mode=CELP-vbr; config=
AudioSpecificConfig(); SizeLength=6; IndexLength=2; AudioSpecificConfig(); SizeLength=6; IndexLength=2;
IndexDeltaLength=2 IndexDeltaLength=2; constantDuration=xxx; maxDisplacement=yyy
Note: The a=fmtp line has been wrapped to fit the page, it comprises Note: The a=fmtp line has been wrapped to fit the page, it comprises
a single line in the SDP file. a single line in the SDP file.
AudioSpecificConfig() is the hexadecimal string as defined in AudioSpecificConfig() is the hexadecimal string as defined in
ISO/IEC 14496-3, AudioSpecificConfig() specifies that the audio ISO/IEC 14496-3, AudioSpecificConfig() specifies that the audio
stream type is CELP. For the description of MIME parameters see stream type is CELP. For the description of MIME parameters see
section 4.1. section 4.1.
3.3.5 Low bit-rate AAC 3.3.5 Low bit-rate AAC
This mode is signaled by mode=AAC-lbr. This mode supports transport This mode is signaled by mode=AAC-lbr. This mode supports transport
of one or more complete AAC frames of variable size. In this mode of one or more complete AAC frames of variable size. In this mode
the AAC frames are allowed to be interleaved and hence receivers the AAC frames are allowed to be interleaved and hence receivers
MUST support de-interleaving. The maximum size of an AAC frame in MUST support de-interleaving. The maximum size of an AAC frame in
this mode is 63 octets. CELP frames MUST not be fragmented when this mode is 63 octets. AAC frames MUST not be fragmented when
using this mode. using this mode.
The payload configuration in this mode is the same as in the The payload configuration in this mode is the same as in the
variable bit-rate CELP mode as defined in 3.3.4. The RTP payload variable bit-rate CELP mode as defined in 3.3.4. The RTP payload
consists of the AU Header Section, followed by concatenated AAC consists of the AU Header Section, followed by concatenated AAC
frames. The Auxiliary Section MUST be empty. For each AAC frame frames. The Auxiliary Section MUST be empty. For each AAC frame
contained in the payload the one octet AU-header MUST provide: contained in the payload the one octet AU-header MUST provide:
(a) the size of each AAC frame in the payload and (a) the size of each AAC frame in the payload and
(b) index information for computing the sequence (and hence timing) (b) index information for computing the sequence (and hence timing)
of each AAC frame. of each AAC frame.
skipping to change at page 23, line 51 skipping to change at page 25, line 4
variable bit-rate CELP mode as defined in 3.3.4. The RTP payload variable bit-rate CELP mode as defined in 3.3.4. The RTP payload
consists of the AU Header Section, followed by concatenated AAC consists of the AU Header Section, followed by concatenated AAC
frames. The Auxiliary Section MUST be empty. For each AAC frame frames. The Auxiliary Section MUST be empty. For each AAC frame
contained in the payload the one octet AU-header MUST provide: contained in the payload the one octet AU-header MUST provide:
(a) the size of each AAC frame in the payload and (a) the size of each AAC frame in the payload and
(b) index information for computing the sequence (and hence timing) (b) index information for computing the sequence (and hence timing)
of each AAC frame. of each AAC frame.
In the AU-header, the AU-size MUST be coded with 6 bits and the In the AU-header, the AU-size MUST be coded with 6 bits and the
AU-Index(-delta) with 2 bits; the AU-Index field MUST have the AU-Index(-delta) with 2 bits; the AU-Index field MUST have the
value 0 in each AU-header. value 0 in each AU-header.
In the AU-header Section, the concatenated AU-headers MUST be In the AU-header Section, the concatenated AU-headers MUST be
preceded by the 16-bit AU-headers-length field, as specified in preceded by the 16-bit AU-headers-length field, as specified in
section 3.2.1. section 3.2.1.
In addition to the required MIME format parameters, the following In addition to the required MIME format parameters, the following
parameters MUST be present: SizeLength, IndexLength, and parameters MUST be present: SizeLength, IndexLength, and
IndexDeltaLength. IndexDeltaLength. AAC frames have fixed time duration per Access
When interleaving is applied (AU-Index-delta coded with a value Unit; when interleaving in this mode, the applicable duration MUST
larger than 0), also the parameter InterleaveDelay MUST be present. be signaled by the MIME format parameter constantDuration. In
addition, the parameter maxDisplacement MUST be present when
interleaving.
For example: For example:
m=audio 49230 RTP/AVP 96 m=audio 49230 RTP/AVP 96
a=rtpmap:96 mpeg4-generic/44100/2 a=rtpmap:96 mpeg4-generic/44100/2
a=fmtp:96 streamtype=5; profile-level-id=15; mode=AAC-lbr; config= a=fmtp:96 streamtype=5; profile-level-id=15; mode=AAC-lbr; config=
AudioSpecificConfig(); SizeLength=6; IndexLength=2; AudioSpecificConfig(); SizeLength=6; IndexLength=2;
IndexDeltaLength=2 IndexDeltaLength=2; constantDuration=xxx; maxDisplacement=yyy
Note: The a=fmtp line has been wrapped to fit the page, it comprises Note: The a=fmtp line has been wrapped to fit the page, it comprises
a single line in the SDP file. a single line in the SDP file.
AudioSpecificConfig() is the hexadecimal string as defined in ISO/IEC AudioSpecificConfig() is the hexadecimal string as defined in ISO/IEC
14496-3. AudioSpecificConfig() specifies that the audio 14496-3. AudioSpecificConfig() specifies that the audio
stream type is AAC. For the description of MIME parameters see stream type is AAC. For the description of MIME parameters see
section 4.1. section 4.1.
3.3.6 High bit-rate AAC 3.3.6 High bit-rate AAC
skipping to change at page 24, line 50 skipping to change at page 25, line 56
AU-header in the AU Header Section to provide: AU-header in the AU Header Section to provide:
(a) the size of each AAC frame in the payload and (a) the size of each AAC frame in the payload and
(b) index information for computing the sequence (and hence timing) (b) index information for computing the sequence (and hence timing)
of each AAC frame. of each AAC frame.
To code the maximum size of an AAC frame requires 13 bits. Therefore To code the maximum size of an AAC frame requires 13 bits. Therefore
in this configuration 13 bits are allocated to the AU-size, and in this configuration 13 bits are allocated to the AU-size, and
3 bits to the AU-Index(-delta) field. Thus each AU-header has a size 3 bits to the AU-Index(-delta) field. Thus each AU-header has a size
of 2 octets. Each AU-Index field MUST be coded with the value 0. In of 2 octets. Each AU-Index field MUST be coded with the value 0. In
the AU Header Section, the concatenated AU-headers MUST be preceded the AU Header Section, the concatenated AU-headers MUST be preceded
by the 16-bit AU-headers-length field, as specified in section 3.2.1. by the 16-bit AU-headers-length field, as specified in
section 3.2.1.
In addition to the required MIME format parameters, the following In addition to the required MIME format parameters, the following
parameters MUST be present: SizeLength, IndexLength, and parameters MUST be present: SizeLength, IndexLength, and
IndexDeltaLength. IndexDeltaLength. AAC frames have fixed time duration per Access
When interleaving is applied (AU-Index-delta coded with a value Unit; when interleaving in this mode, the applicable duration MUST
larger than 0), also the parameter InterleaveDelay MUST be present. be signaled by the MIME format parameter constantDuration. In
addition, the parameter maxDisplacement MUST be present when
interleaving.
For example: For example:
m=audio 49230 RTP/AVP 96 m=audio 49230 RTP/AVP 96
a=rtpmap:96 mpeg4-generic/44100/2 a=rtpmap:96 mpeg4-generic/44100/2
a=fmtp:96 streamtype=5; profile-level-id=15; mode=AAC-hbr; a=fmtp:96 streamtype=5; profile-level-id=15; mode=AAC-hbr;
config=AudioSpecificConfig(); SizeLength=13; IndexLength=3; config=AudioSpecificConfig(); SizeLength=13; IndexLength=3;
IndexDeltaLength=3 IndexDeltaLength=3; constantDuration=xxx; maxDisplacement=yyy
Note: The a=fmtp line has been wrapped to fit the page, it comprises Note: The a=fmtp line has been wrapped to fit the page, it comprises
a single line in the SDP file. a single line in the SDP file.
AudioSpecificConfig() is the hexadecimal string as defined in AudioSpecificConfig() is the hexadecimal string as defined in
ISO/IEC 14496-3. AudioSpecificConfig() specifies that the audio ISO/IEC 14496-3. AudioSpecificConfig() specifies that the audio
stream type is AAC. For the description of MIME parameters see stream type is AAC. For the description of MIME parameters see
section 4.1. section 4.1.
3.3.7 Additional modes 3.3.7 Additional modes
skipping to change at page 26, line 9 skipping to change at page 27, line 9
they have the default value), even if its presence is redundant in they have the default value), even if its presence is redundant in
case the mode assigns a fixed value to a parameter. A mode may case the mode assigns a fixed value to a parameter. A mode may
define additionally that some MIME parameters are required instead define additionally that some MIME parameters are required instead
of optional, that some MIME parameters have fixed values (or of optional, that some MIME parameters have fixed values (or
ranges), and that there are rules restricting the usage. ranges), and that there are rules restricting the usage.
4. IANA considerations 4. IANA considerations
This section describes the MIME types and names associated with This section describes the MIME types and names associated with
this payload format. Section 4.1 registers the MIME types, as per this payload format. Section 4.1 registers the MIME types, as per
RFC 2048. RFC 2048 [3].
This format may require additional information about the mapping to This format may require additional information about the mapping to
be made available to the receiver. This is done using parameters be made available to the receiver. This is done using parameters
also described in the next section. also described in the next section.
4.1 MIME type registration 4.1 MIME type registration
MIME media type name: "video" or "audio" or "application" MIME media type name: "video" or "audio" or "application"
"video" MUST be used for MPEG-4 Visual streams (ISO/IEC 14496-2) "video" MUST be used for MPEG-4 Visual streams (ISO/IEC 14496-2)
skipping to change at page 29, line 5 skipping to change at page 30, line 5
defined in this document may not be suitable to carry a stream defined in this document may not be suitable to carry a stream
that is not defined by MPEG-4. ObjectType SHOULD NOT be set to that is not defined by MPEG-4. ObjectType SHOULD NOT be set to
a value that signals a stream that cannot be carried by this a value that signals a stream that cannot be carried by this
payload format. payload format.
ConstantSize: ConstantSize:
The constant size in octets of each Access Unit for this stream. The constant size in octets of each Access Unit for this stream.
The ConstantSize and the SizeLength parameters MUST NOT be The ConstantSize and the SizeLength parameters MUST NOT be
simultaneously present. simultaneously present.
ConstantDuration:
The constant duration of each Access Unit for this stream,
measured with the same units as the RTP time stamp.
maxDisplacement: maxDisplacement:
The decimal representation of the maximum displacement in time The decimal representation of the maximum displacement in time
of an interleaved AU, as defined in section 3.2.3.3, expressed of an interleaved AU, as defined in section 3.2.3.3, expressed
in units of the RTP time stamp clock. in units of the RTP time stamp clock.
This parameter MUST be present when interleaving is applied. This parameter MUST be present when interleaving is applied.
de-interleaveBufferSize: de-interleaveBufferSize:
The decimal representation in number of octets of the size of The decimal representation in number of octets of the size of
the de-interleave buffer, described in section 3.2.3.3. the de-interleave buffer, described in section 3.2.3.3.
When interleaving, this parameter MUST be present if the When interleaving, this parameter MUST be present if the
skipping to change at page 29, line 31 skipping to change at page 30, line 35
Optional configuration parameters: Optional configuration parameters:
SizeLength: SizeLength:
The number of bits on which the AU-size field is encoded in the The number of bits on which the AU-size field is encoded in the
AU-header. The SizeLength and the ConstantSize parameters MUST AU-header. The SizeLength and the ConstantSize parameters MUST
NOT be simultaneously present. NOT be simultaneously present.
IndexLength: IndexLength:
The number of bits on which the AU-Index is encoded in the first The number of bits on which the AU-Index is encoded in the first
AU-header. The default value of zero indicates the absence of AU-header. The default value of zero indicates the absence of
the AU-Index and AU-Index-delta fields in each AU-header. the AU-Index field in each first AU-header.
IndexDeltaLength: IndexDeltaLength:
The number of bits on which the AU-Index-delta field is encoded The number of bits on which the AU-Index-delta field is encoded
in any non-first AU-header. in any non-first AU-header. The default value of zero indicates
the absence of the AU-Index-delta field in each non-first
AU-header.
CTSDeltaLength: CTSDeltaLength:
The number of bits on which the CTS-delta field is encoded in The number of bits on which the CTS-delta field is encoded in
the AU-header. the AU-header.
DTSDeltaLength: DTSDeltaLength:
The number of bits on which the DTS-delta field is encoded in The number of bits on which the DTS-delta field is encoded in
the AU-header. the AU-header.
RandomAccessIndication: RandomAccessIndication:
skipping to change at page 30, line 14 skipping to change at page 31, line 20
AuxiliaryDataSizeLength: AuxiliaryDataSizeLength:
The number of bits that is used to encode the auxiliary-data-size The number of bits that is used to encode the auxiliary-data-size
field. field.
Applications MAY use more parameters, in addition to those defined Applications MAY use more parameters, in addition to those defined
above. Each additional parameter MUST be registered with IANA, to above. Each additional parameter MUST be registered with IANA, to
ensure that there is no clash of names. Each additional parameter ensure that there is no clash of names. Each additional parameter
MUST be accompanied by a specification in the form of an RFC, MPEG MUST be accompanied by a specification in the form of an RFC, MPEG
standard, or other permanent and readily available reference (the standard, or other permanent and readily available reference (the
"Specification Required" policy defined in RFC 2434). Receivers MUST "Specification Required" policy defined in RFC 2434 [6]). Receivers
tolerate the presence of such additional parameters, but these MUST tolerate the presence of such additional parameters, but these
parameters SHALL NOT impact the decoding of receivers that comply to parameters SHALL NOT impact the decoding of receivers that comply to
this specification. this specification.
Encoding considerations: Encoding considerations:
This MIME subtype is defined for RTP transport only. System This MIME subtype is defined for RTP transport only. System
bitstreams MUST be generated according to MPEG-4 Systems bitstreams MUST be generated according to MPEG-4 Systems
specifications (ISO/IEC 14496-1). Video bitstreams MUST be generated specifications (ISO/IEC 14496-1). Video bitstreams MUST be generated
according to MPEG-4 Visual specifications (ISO/IEC 14496-2). Audio according to MPEG-4 Visual specifications (ISO/IEC 14496-2). Audio
bitstreams MUST be generated according to MPEG-4 Audio bitstreams MUST be generated according to MPEG-4 Audio
specifications (ISO/IEC 14496-3). The RTP packets MUST be packetized specifications (ISO/IEC 14496-3). The RTP packets MUST be packetized
skipping to change at page 31, line 44 skipping to change at page 32, line 44
This specification can be used in a number of modes. The mode of This specification can be used in a number of modes. The mode of
operation is signaled using the "Mode" MIME parameter, with the operation is signaled using the "Mode" MIME parameter, with the
initial set of values specified in section 4.1. New modes may be initial set of values specified in section 4.1. New modes may be
defined at any time, as described in section 3.3.7. These modes defined at any time, as described in section 3.3.7. These modes
MUST be registered with IANA, to ensure that there is no clash MUST be registered with IANA, to ensure that there is no clash
of names. of names.
A new mode registration MUST be accompanied by a specification in A new mode registration MUST be accompanied by a specification in
the form of an RFC, MPEG standard, or other permanent and readily the form of an RFC, MPEG standard, or other permanent and readily
available reference (the "Specification Required" policy defined available reference (the "Specification Required" policy defined
in RFC 2434). in RFC 2434 [6]).
4.3 Concatenation of parameters 4.3 Concatenation of parameters
Multiple parameters SHOULD be expressed as a MIME media type string, Multiple parameters SHOULD be expressed as a MIME media type string,
in the form of a semicolon-separated list of parameter=value pairs in the form of a semicolon-separated list of parameter=value pairs
(for parameter usage examples see sections 3.3.2 up to 3.3.6). (for parameter usage examples see sections 3.3.2 up to 3.3.6).
4.4 Usage of SDP 4.4 Usage of SDP
4.4.1 The a=fmtp keyword 4.4.1 The a=fmtp keyword
It is assumed that one typical way to transport the above-described It is assumed that one typical way to transport the above-described
parameters associated with this payload format is via a SDP message parameters associated with this payload format is via a SDP message
[6] for example transported to the client in reply to a RTSP [5] for example transported to the client in reply to a RTSP
DESCRIBE [8] or via SAP [7]. In that case the (a=fmtp) keyword MUST DESCRIBE [8] or via SAP [11]. In that case the (a=fmtp) keyword
be used as described in RFC 2327 [6], section 6, the syntax being MUST be used as described in RFC 2327 [5], section 6, the syntax
then: being then:
a=fmtp:<format> <parameter name>=<value>[; <parameter name>=<value>] a=fmtp:<format> <parameter name>=<value>[; <parameter name>=<value>]
5. Security Considerations 5. Security Considerations
RTP packets using the payload format defined in this specification RTP packets using the payload format defined in this specification
are subject to the security considerations discussed in the RTP are subject to the security considerations discussed in the RTP
specification [2]. This implies that confidentiality of the media specification [2]. This implies that confidentiality of the media
streams is achieved by encryption. Because the data compression used streams is achieved by encryption. Because the data compression used
with this payload format is applied end-to-end, encryption may be with this payload format is applied end-to-end, encryption may be
skipping to change at page 33, line 32 skipping to change at page 34, line 27
contributions by people from the ISMA forum, from the IETF AVT contributions by people from the ISMA forum, from the IETF AVT
Working Group and from the 4-on-IP ad-hoc group within MPEG. The Working Group and from the 4-on-IP ad-hoc group within MPEG. The
authors wish to thank all involved people, and in particular Andrea authors wish to thank all involved people, and in particular Andrea
Basso, Stephen Casner, M. Reha Civanlar, Carsten Herpel, John Basso, Stephen Casner, M. Reha Civanlar, Carsten Herpel, John
Lazaro, Zvi Lifshitz, Young-kwon Lim, Alex MacAulay, Bill May, Lazaro, Zvi Lifshitz, Young-kwon Lim, Alex MacAulay, Bill May,
Colin Perkins, Dorairaj V and Stephan Wenger for their valuable Colin Perkins, Dorairaj V and Stephan Wenger for their valuable
comments and support. comments and support.
7. References 7. References
7.1 Normative references
[1] ISO/IEC International Standard 14496 (MPEG-4); "Information [1] ISO/IEC International Standard 14496 (MPEG-4); "Information
technology - Coding of audio-visual objects", January 2000 technology - Coding of audio-visual objects", January 2000
[2] H. Schulzrinne, S. Casner, R. Frederick, V. Jacobson RTP, "A [2] H. Schulzrinne, S. Casner, R. Frederick, V. Jacobson RTP, "A
Transport Protocol for Real Time Applications", RFC 1889, Internet Transport Protocol for Real Time Applications", RFC 1889, Internet
Engineering Task Force, January 1996. Engineering Task Force, January 1996.
[3] S. Bradner, "Key words for use in RFCs to Indicate Requirement [3] N. Freed, J. Klensin, J. Postel, " Multipurpose Internet Mail
Extensions (MIME) Part Four: Registration Procedures", RFC 2048,
Internet Engineering Task Force, November 1996.
[4] S. Bradner, "Key words for use in RFCs to Indicate Requirement
Levels", RFC 2119, March 1997. Levels", RFC 2119, March 1997.
[4] D. Hoffman, G. Fernando, V. Goyal, M. Civanlar, "RTP payload [5] M. Handley, V. Jacobson, "SDP: Session Description Protocol",
format for MPEG1/MPEG2 Video", RFC 2250, January 1998. RFC 2327, Internet Engineering Task Force, April 1998.
[5] Y. Kikuchi, T. Nomura, S. Fukunaga, Y. Matsui, H. Kimata, "RTP [6] T. Narten, H. Alvestrand, " Guidelines for Writing an IANA
payload format for MPEG-4 Audio/Visual streams", RFC 3016. Considerations Section in RFCs", RFC 2434, October 1998.
[6] M. Handley, V. Jacobson, "SDP: Session Description Protocol", 7.2 Informative references
RFC 2327, Internet Engineering Task Force, April 1998.
[7] M. Handley, C. Perkins, E. Whelan, "SAP: Session Announcement [7] D. Hoffman, G. Fernando, V. Goyal, M. Civanlar, "RTP payload
Protocol", RFC 2974, Internet Engineering Task Force, October 2000. format for MPEG1/MPEG2 Video", RFC 2250, January 1998.
[8] H. Schulzrinne, A. Rao, R. Lanphier, "RTSP: Real-Time Session [8] H. Schulzrinne, A. Rao, R. Lanphier, "RTSP: Real-Time Session
Protocol", RFC 2326, Internet Engineering Task Force, April 1998. Protocol", RFC 2326, Internet Engineering Task Force, April 1998.
[9] C. Perkins, O. Hudson, "Options for Repair of Streaming Media"
RFC 2354, Internet Engineering Task Force, June 1998.
[10] H. Schulzrinne, J. Rosenberg, "An RTP Payload Format for
Generic Forward Error Correction", RFC 2733, Internet Engineering
Task Force, December 1999.
[11] M. Handley, C. Perkins, E. Whelan, "SAP: Session Announcement
Protocol", RFC 2974, Internet Engineering Task Force, October 2000.
[12] Y. Kikuchi, T. Nomura, S. Fukunaga, Y. Matsui, H. Kimata, "RTP
payload format for MPEG-4 Audio/Visual streams", RFC 3016, Internet
Engineering Task Force, November 2000.
8. Author Addresses 8. Author Addresses
Jan van der Meer Jan van der Meer
Philips Digital Networks Philips Digital Networks
Cederlaan 4 Cederlaan 4
5600 JB Eindhoven 5600 JB Eindhoven
Netherlands Netherlands
Email : jan.vandermeer@philips.com Email : jan.vandermeer@philips.com
David Mackie David Mackie
skipping to change at page 36, line 37 skipping to change at page 37, line 37
When the decoding deadline is reached (i.e., the time when decoding When the decoding deadline is reached (i.e., the time when decoding
must begin in order to be completed by the time the AU is to be must begin in order to be completed by the time the AU is to be
presented), or if the decoder is some hardware that presents a presented), or if the decoder is some hardware that presents a
constant delay between initiation of decoding of an AU and constant delay between initiation of decoding of an AU and
presentation of that AU, then decoding must begin at that deadline presentation of that AU, then decoding must begin at that deadline
time. time.
If the next AU to be decoded is not present when the decoding If the next AU to be decoded is not present when the decoding
deadline is reached, then that AU is lost so the receiver must take deadline is reached, then that AU is lost so the receiver must take
whatever error concealment measures is deemed appropriate. The whatever error concealment measures is deemed appropriate. The
playout delay may need to be adjusted at that point (especially if play-out delay may need to be adjusted at that point (especially if
other AUs have also missed their deadline recently). Or, if it other AUs have also missed their deadline recently). Or, if it was
was a momentary delay, and maintaining the latency is important, a momentary delay, and maintaining the latency is important, then
then the receiver should minimize the glitch and continue processing the receiver should minimize the glitch and continue processing
with the next AU. with the next AU.
A.3 Simple Group interleave A.3 Simple Group interleave
A.3.1 Introduction A.3.1 Introduction
An example of regular interleave is when packets are formed into An example of regular interleave is when packets are formed into
groups. If the 'stride' of the interleave (the distance between groups. If the 'stride' of the interleave (the distance between
interleaved AUs) is N, packet 0 could contain AU(0), AU(N), AU(2N), interleaved AUs) is N, packet 0 could contain AU(0), AU(N), AU(2N),
and so on; packet 1 could contain AU(1), AU(1+N), AU(1+2N), and so and so on; packet 1 could contain AU(1), AU(1+N), AU(1+2N), and so
on. If there are M access units in a packet, then there are M*N on. If there are M access units in a packet, then there are M*N
access units in the group. access units in the group.
An example with N=M=3 follows; note that this is the same example An example with N=M=3 follows; note that this is the same example
as given in section 2.5: as given in section 2.5 and that a fixed time duration per Access
Unit is assumed:
Packet Time stamp Carried AUs AU-Index, AU-Index-delta Packet Time stamp Carried AUs AU-Index, AU-Index-delta
P(0) T[0] 0, 3, 6 0, 2, 2 P(0) T[0] 0, 3, 6 0, 2, 2
P(1) T[1] 1, 4, 7 0, 2, 2 P(1) T[1] 1, 4, 7 0, 2, 2
P(2) T[2] 2, 5, 8 0, 2, 2 P(2) T[2] 2, 5, 8 0, 2, 2
P(3) T[9] 9,12,15 0, 2, 2 P(3) T[9] 9,12,15 0, 2, 2
In the above example the AU-Index is coded with the value 0, as In this example the AU-Index is present in the first AU-header and
required for the modes defined in this document. The position of coded with the value 0, as required for fixed duration AUs. The
the first AU of each packet within the group is defined by the RTP position of the first AU of each packet within the group is defined
time stamp, while the AU-Index-delta field indicates the position by the RTP time stamp, while the AU-Index-delta field indicates the
of subsequent AUs relative to the first AU in the packet. All position of subsequent AUs relative to the first AU in the packet.
AU-Index-delta fields are coded with the value N-1, equal to 2 in All AU-Index-delta fields are coded with the value N-1, equal to 2
this example. Hence the RTP time stamp and the AU-Index-delta are in this example. Hence the RTP time stamp and the AU-Index-delta are
used to reconstruct the original order. See also section 3.2.3.2. used to reconstruct the original order. See also section 3.2.3.2.
A.3.2 Determining the de-interleave buffer size A.3.2 Determining the de-interleave buffer size
For the regular pattern as in this example, figure 6 in section For the regular pattern as in this example, figure 6 in section
3.2.3.3 shows that the de-interleave buffer size is equal to 4 AU 3.2.3.3 shows that the de-interleave buffer stores at most 4 AUs. A
sizes. de-interleaveBufferSize value may be signaled that is at least
equal to the total number of octets of any 4 "early" AUs that are
stored at the same time.
A.3.3 Determining the maximum displacement A.3.3 Determining the maximum displacement
For the regular pattern as in this example, figure 7 in section 3.3 For the regular pattern as in this example, figure 7 in section 3.3
shows that the value of the maxDisplacement equals 5 AU periods. shows that the maximum displacement in time equals 5 AU periods.
Hence the minimum maxDisplacement value that must be signaled is 5
AU periods. In case each AU has the same size, this maxDisplacement
value over-estimates the de-interleave buffer size with one AU.
However, note that in case of variable AU sizes the total size of
any 4 "early" AUs that must be stored at the same time may exceed
maxDisplacement times the maximum bitrate, in which case the
de-interleaveBufferSize must be signaled.
A.4 More subtle group interleave A.4 More subtle group interleave
A.4.1 Introduction A.4.1 Introduction
Another example of forming packets with group interleave is given Another example of forming packets with group interleave is given
below. In this example the packets are formed such that the loss of below. In this example the packets are formed such that the loss of
two subsequent RTP packets does not cause the loss of two subsequent two subsequent RTP packets does not cause the loss of two subsequent
AUs. Note that in this example the RTP time stamps of packet 3 and AUs. Note that in this example the RTP time stamps of packet 3 and
packet 4 are earlier than the RTP time stamps of packets 1 and 2, packet 4 are earlier than the RTP time stamps of packets 1 and 2,
respectively. respectively; a fixed time duration per Access Unit is assumed.
Packet Time stamp Carried AUs AU-Index, AU-Index-delta Packet Time stamp Carried AUs AU-Index, AU-Index-delta
0 T[0] 0, 5 0, 5 0 T[0] 0, 5 0, 4
1 T[2] 2, 7 0, 5 1 T[2] 2, 7 0, 4
2 T[4] 4, 9 0, 5 2 T[4] 4, 9 0, 4
3 T[1] 1, 6 0, 5 3 T[1] 1, 6 0, 4
4 T[3] 3, 8 0, 5 4 T[3] 3, 8 0, 4
5 T[10] 10, 15 0, 4
5 T[10] 10, 15 0, 5
and so on .. and so on ..
In this example the AU-Index is coded with the value 0, as required In this example the AU-Index is present in the first AU-header and
for the modes defined in this document. To reconstruct the original coded with the value 0, as required for AUs with a fixed duration.
order, the RTP time stamp and the AU-Index-delta (coded with the To reconstruct the original order, the RTP time stamp and the
value 5) are used. See also section 3.2.3.2. AU-Index-delta (coded with the value 4) are used. See also
section 3.2.3.2.
A.4.2 Determining the de-interleave buffer size A.4.2 Determining the de-interleave buffer size
From figure 8 it can be to determined that at most 5 "early" AUs From figure 8 it can be to determined that at most 5 "early" AUs
are to be stored. If the AUs are of constant size, then this value are to be stored. If the AUs are of constant size, then this value
equals 5 times the AU size. equals 5 times the AU size. The minimum size of the de-interleave
buffer equals the maximum total number of octets of the "early" AUs
that are to be stored at the same time. This gives the minimum
value of the de-interleaveBufferSize that may be signaled.
+--+--+--+--+--+--+--+--+--+--+ +--+--+--+--+--+--+--+--+--+--+
Interleaved AUs | 0| 5| 2| 7| 4| 9| 1| 6| 3| 8| Interleaved AUs | 0| 5| 2| 7| 4| 9| 1| 6| 3| 8|
+--+--+--+--+--+--+--+--+--+--+ +--+--+--+--+--+--+--+--+--+--+
- - 5 - 5 - 2 7 4 9 - - 5 - 5 - 2 7 4 9
7 4 9 5 7 4 9 5
Received "early" AUs 5 6 "Early" AUs 5 6
7 7 7 7
9 9 9 9
Figure 8: Storage of "early" AUs in the de-interleave buffer per Figure 8: Storage of "early" AUs in the de-interleave buffer per
interleaved AU. interleaved AU.
A.4.2 Determining the maximum displacement A.4.2 Determining the maximum displacement
From figure 9 it can be seen that max-interleaveDisplacement has From figure 9 it can be seen that the maximum displacement in time
a value of 8 AU periods. equals 8 AU periods. Hence the minimum maxDisplacement value to be
signaled is 8 AU periods.
+--+--+--+--+--+--+--+--+--+--+ +--+--+--+--+--+--+--+--+--+--+
Interleaved AUs | 0| 5| 2| 7| 4| 9| 1| 6| 3| 8| Interleaved AUs | 0| 5| 2| 7| 4| 9| 1| 6| 3| 8|
+--+--+--+--+--+--+--+--+--+--+ +--+--+--+--+--+--+--+--+--+--+
Earliest not yet received AU - 1 1 1 1 1 - 3 - - Earliest not yet present AU - 1 1 1 1 1 - 3 - -
Figure 9: The earliest not yet received AU for each AU in the Figure 9: The earliest not yet present AU for each AU in the
interleaving pattern. interleaving pattern.
In case each AU has the same size, the found maxDisplacement value
over-estimates the de-interleave buffer size with three AUs.
However, in case of variable AU sizes the total size of any 5
"early" AUs stored at the same time may exceed maxDisplacement
times the maximum bitrate, in which case de-interleaveBufferSize
must be signaled.
A.5 Continuous interleave A.5 Continuous interleave
A.5.1 Introduction A.5.1 Introduction
In continuous interleave, once the scheme is 'primed', the number In continuous interleave, once the scheme is 'primed', the number
of AUs in a packet exceeds the 'stride' (the distance between of AUs in a packet exceeds the 'stride' (the distance between
them). This shortens the buffering needed, smooths the data-flow, them). This shortens the buffering needed, smooths the data-flow,
and gives slightly larger packets -- and thus lower overhead -- for and gives slightly larger packets -- and thus lower overhead -- for
the same interleave. For example, here is a continuous interleave the same interleave. For example, here is a continuous interleave
also over a stride of 3 AUs, but with 4 AUs per packet, for a run also over a stride of 3 AUs, but with 4 AUs per packet, for a run
of 20 AUs. This shows both how the scheme 'starts up' and how it of 20 AUs. This shows both how the scheme 'starts up' and how it
finishes. finishes. Once again, the example assumes fixed time duration per
Access Unit.
Packet Time-stamp Carried AUs AU-Index, AU-Index-delta Packet Time-stamp Carried AUs AU-Index, AU-Index-delta
0 T[0] 0 0 0 T[0] 0 0
1 T[1] 1 4 0 2 1 T[1] 1 4 0 2
2 T[2] 2 5 8 0 2 2 2 T[2] 2 5 8 0 2 2
3 T[3] 3 6 9 12 0 2 2 2 3 T[3] 3 6 9 12 0 2 2 2
4 T[7] 7 10 13 16 0 2 2 2 4 T[7] 7 10 13 16 0 2 2 2
5 T[11] 11 14 17 20 0 2 2 2 5 T[11] 11 14 17 20 0 2 2 2
6 T[15] 15 18 0 2 6 T[15] 15 18 0 2
7 T[19] 19 0 7 T[19] 19 0
Also in this example the AU-Index is coded with the value 0, as In this example the AU-Index is present in the first AU-header and
required for the modes defined in this document. To reconstruct the coded with the value 0, as required for AUs with a fixed duration.
original order, the RTP time stamp and the AU-Index-delta (coded To reconstruct the original order, the RTP time stamp and the
with the value 2) are used. See also 3.2.3.2. Note that this AU-Index-delta (coded with the value 2) are used. See also 3.2.3.2.
example has RTP time-stamps in increasing order. Note that this example has RTP time-stamps in increasing order.
A.5.2 Determining the de-interleave buffer size A.5.2 Determining the de-interleave buffer size
For this example the de-interleave buffer size can be derived from For this example the de-interleave buffer size can be derived from
figure 10. The maximum number of "early" AUs is three. If the AUs figure 10. The maximum number of "early" AUs is three. If the AUs
are of constant size, then this value equals 3 times the AU size. are of constant size, then this value equals 3 times the AU size.
Compared to the example in A.2, for constant size AUs the Compared to the example in A.2, for constant size AUs the
de-interleave buffer size is reduced from 4 to 3 times the AU size, de-interleave buffer size is reduced from 4 to 3 times the AU size,
while maintaining the same 'stride'. while maintaining the same 'stride'.
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+- +--+--+--+--+--+--+--+--+--+--+--+--+--+--+-
Interleaved AUs | 0| 1| 4| 2| 5| 8| 3| 6| 9|12| 7|10|13|16| Interleaved AUs | 0| 1| 4| 2| 5| 8| 3| 6| 9|12| 7|10|13|16|
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+- +--+--+--+--+--+--+--+--+--+--+--+--+--+--+-
- - - 4 - - 4 8 - - 8 12 - - - - - 4 - - 4 8 - - 8 12 - -
5 9 5 9
Received "early" AUs 8 12 "Early" AUs 8 12
Figure 10: Storage of "early" AUs in the de-interleave buffer per Figure 10: Storage of "early" AUs in the de-interleave buffer per
interleaved AU. interleaved AU.
A.5.3 Determining the maximum displacement A.5.3 Determining the maximum displacement
For this example the maxDisplacement has a value of 5 AU periods. For this example the maximum displacement has a value of 5 AU
See figure 11. periods. See figure 11. Compared to the example in A.2, the maximum
displacement does not decrease, though in fact less de-interleave
buffering is required.
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+- +--+--+--+--+--+--+--+--+--+--+--+--+--+--+-
Interleaved AUs | 0| 1| 4| 2| 5| 8| 3| 6| 9|12| 7|10|13|16| Interleaved AUs | 0| 1| 4| 2| 5| 8| 3| 6| 9|12| 7|10|13|16|
+--+--+--+--+--+--+--+--+--+--+--+--+--+--+- +--+--+--+--+--+--+--+--+--+--+--+--+--+--+-
Earliest not yet Earliest not yet
received AU - - 2 - 3 3 - - 7 7 - - 11 11 present AU - - 2 - 3 3 - - 7 7 - - 11 11
Figure 11: The earliest not yet received AU for each AU in the Figure 11: The earliest not yet present AU for each AU in the
interleaving pattern. interleaving pattern.
 End of changes. 

This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/