draft-ietf-avt-rtcp-feedback-01.txt   draft-ietf-avt-rtcp-feedback-02.txt 
INTERNET-DRAFT Jųrg Ott/UniversitĄt Bremen TZI INTERNET-DRAFT Joerg Ott/Uni Bremen TZI
draft-ietf-avt-rtcp-feedback-01.txt Stephan Wenger/TU Berlin draft-ietf-avt-rtcp-feedback-02.txt Stephan Wenger/TU Berlin
Shigeru Fukunaga/Oki Shigeru Fukunaga/Oki
Noriyuki Sato/Oki Noriyuki Sato/Oki
Koichi Yano/Fast Forward Networks Koichi Yano/Fast Forward Networks
Akihiro Miyazaki/Matsushita Akihiro Miyazaki/Matsushita
Koichi Hata/Matsushita Koichi Hata/Matsushita
Rolf Hakenberg/Matsushita Rolf Hakenberg/Matsushita
Carsten Burmeister/Matsushita Carsten Burmeister/Matsushita
21 November, 2001 1 March 2002
Expires May 2002 Expires September 2002
Extended RTP Profile for RTCP-based Feedback (RTP/AVPF) Extended RTP Profile for RTCP-based Feedback (RTP/AVPF)
Status of this Memo Status of this Memo
This document is an Internet-Draft and is in full conformance with all This document is an Internet-Draft and is in full conformance with all
provisions of Section 10 of RFC 2026. Internet-Drafts are working provisions of Section 10 of RFC 2026. Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas, and documents of the Internet Engineering Task Force (IETF), its areas,
its working groups. Note that other groups may also distribute working and its working groups. Note that other groups may also distribute
documents as Internet-Drafts. working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet- Drafts as reference material time. It is inappropriate to use Internet- Drafts as reference
or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
Abstract Abstract
Real-time media streams are not resilient against packet losses. RTP Real-time media streams that use RTP are not resilient against
[1] provides all the necessary mechanisms to restore ordering and packet losses. RTP [1] provides all the necessary mechanisms to
timing to properly reproduce a media stream at the recipient. RTP restore ordering and timing to properly reproduce a media stream at
also provides continuous feedback about the overall reception quality the recipient. RTP also provides continuous feedback about the
from all receivers -- thereby allowing the sender(s) in the mid-term overall reception quality from all receivers -- thereby allowing
(in the order of several seconds to minutes) to adapt their coding the sender(s) in the mid-term (in the order of several seconds to
scheme and transmission behavior to the observed network QoS. minutes) to adapt their coding scheme and transmission behavior to
However, except for a few payload specific mechanisms [10], RTP makes the observed network QoS. However, except for a few payload
no provision for timely feedback that would allow a sender to repair specific mechanisms [10], RTP makes no provision for timely
the media stream immediately: through retransmissions, retro-active feedback that would allow a sender to repair the media stream
FEC, or media-specific mechanisms such as reference picture immediately: through retransmissions, retro-active FEC, or media-
selection. specific mechanisms such as reference picture selection for some
video codecs.
Ott et al. Expires January 2002 [Page 1] Generally, real-time transport of media streams across IP
Generally, real-time transport of media streams across IP networks networks follows RTP[1] in conjunction with the RTP Profile for
follows RTP[1] in conjunction with the RTP Profile for Audio and Audio and Video Conferences with Minimal Control [2]. This
Video Conferences with Minimal Control [2]. This document modifies document modifies the profile defined in [2] in two ways:
the profile defined in [2] in two ways:
. by providing additional RTCP messages that enable a receiver to o by providing additional RTCP messages that enable a receiver to
convey more precise feedback to a sender and convey more precise feedback to a sender and
. by adapting the timing algorithm for scheduling RTCP packets in o by adapting the timing algorithm for scheduling RTCP packets in
order to allow for occasional timely feedback about events order to allow for occasional timely feedback about events
observed by a receiver (such as lost packets). observed by a receiver (such as lost packets).
The result is an RTP Profile for Audio and Video Conferences with The result is an RTP Profile for Audio and Video Conferences with
Minimal Control that allows for more explicit and more immediate Minimal Control that allows for more explicit and more immediate
receiver feedback but shares all other properties (including all receiver feedback but shares all other properties (including all
other message types and formats, all code points for codecs, payload other message types and formats, all code points for codecs,
formats, scaling capabilities, etc. of [2]). Therefore, this payload formats, scaling capabilities, etc. of [2]). Therefore,
document only specifies the additions and modifications to [2] rather this document only specifies the additions and modifications to [2]
than the repeating the entire specification. rather than the repeating the entire specification.
1. Introduction 1. Introduction
Real-time media streams are not resilient against packet losses. RTP Real-time media streams that use RTP are not resilient against
[1] provides all the necessary mechanisms to restore ordering and packet losses. RTP [1] provides all the necessary mechanisms to
timing present at the sender to properly reproduce a media stream at restore ordering and timing present at the sender to properly
a recipient. RTP also provides continuous feedback about the overall reproduce a media stream at a recipient. RTP also provides
reception quality from all receivers -- thereby allowing the continuous feedback about the overall reception quality from all
sender(s) in the mid-term (in the order of several seconds to receivers -- thereby allowing the sender(s) in the mid-term (in the
minutes) to adapt their coding scheme and transmission behavior to order of several seconds to minutes) to adapt their coding scheme
the observed network QoS. However, except for a few payload specific and transmission behavior to the observed network QoS. However,
mechanisms [10], RTP makes no provision for timely feedback that except for a few payload specific mechanisms [10], RTP makes no
would allow a sender to repair the media stream immediately: through provision for timely feedback that would allow a sender to repair
retransmissions, retro-active FEC, or media-specific mechanisms such the media stream immediately: through retransmissions, retro-active
as reference picture selection. FEC, or media-specific mechanisms such as reference picture
selection for some video codecs.
Current mechanisms available with RTP to improve error resilience Current mechanisms available with RTP to improve error resilience
include audio redundancy coding [7], video redundancy coding [11], include audio redundancy coding [7], video redundancy coding [11],
RTP-level FEC [5], and general considerations on more robust media RTP-level FEC [5], and general considerations on more robust media
streams transmission [6]. These mechanisms may be applied pro- streams transmission [6]. These mechanisms may be applied pro-
actively (thereby increasing the bandwidth of a given media stream). actively (thereby increasing the bandwidth of a given media
Alternatively, in sufficiently small groups with short RTTs, the stream). Alternatively, in sufficiently small groups with small
senders may perform repair on-demand, using the above mechanisms RTTs, the senders may perform repair on-demand, using the above
and/or media-encoding-specific approaches. Note that "small group" mechanisms and/or media-encoding-specific approaches. Note that
and "sufficiently short RTT" are both highly application dependent. "small group" and "sufficiently small RTT" are both highly
application dependent.
This document specifies a modified RTP Profile for Audio and Video This document specifies a modified RTP Profile for audio and video
conferences with minimal control based upon [1] and [2] by means of conferences with minimal control based upon [1] and [2] by means of
two modifications/additions: To achieve timely feedback the concepts two modifications/additions: To achieve timely feedback, the
of Immediate Feedback messages and Early RTCP messages as well as concepts of Immediate Feedback messages and Early RTCP messages as
algorithms allowing for low delay feedback in small multicast groups well as algorithms allowing for low delay feedback in small
(and preventing feedback implosion in large ones) are introduced. multicast groups (and preventing feedback implosion in large ones)
Special consideration is given to point-to-point scenarios. And a are introduced. Special consideration is given to point-to-point
small number general-purpose feedback messages as well as a format scenarios. And a small number of general-purpose feedback messages
as well as a format for codec and application-specific feedback
Ott et al. Expires May 2002 [Page 2] information are defined as specific RTCP payloads.
for codec and application-specific feedback information is defined as
specific RTCP payloads.
1.1 Definitions 1.1 Definitions
The definitions from [1] and [2] apply. In addition, the following The definitions from [1] and [2] apply. In addition, the following
definitions are used in this document: definitions are used in this document:
Early RTCP mode: Early RTCP mode:
The mode of operation in which a receiver of a media stream The mode of operation in which a receiver of a media stream
is, statistically, often (but not always) capable of is, statistically, often (but not always) capable of
reporting events of interest back to the sender close to reporting events of interest back to the sender close to
their occurrence. In Early RTCP mode, RTCP feedback messages their occurrence. In Early RTCP mode, RTCP feedback
are transmitted according to the timing rules defined in this messages are transmitted according to the timing rules
document. defined in this document.
Early RTCP packet: Early RTCP packet:
An Early RTCP packet is a packet which is transmitted earlier An Early RTCP packet is a packet which is transmitted
than would be allowed following the scheduling algorithm of earlier than would be allowed following the scheduling
[1], the reason being that an event observed by a receiver. algorithm of [1], the reason being an "event" observed by a
Early RTCP packets may be sent in Immediate feedback and in receiver. Early RTCP packets may be sent in Immediate
Early RTCP mode. feedback and in Early RTCP mode.
Event: Event:
An observation made by the receiver of a media stream that is An observation made by the receiver of a media stream that
(potentially) of interest to the sender -- such as a packet is (potentially) of interest to the sender -- such as a
loss or packet reception, frame loss, etc. -- and thus to be packet loss or packet reception, frame loss, etc. -- and
reported back to the sender by means of a Feedback message. thus useful to be reported back to the sender by means of a
Feedback message.
Feedback (FB) message: Feedback (FB) message:
An RTCP message as defined in this document used to convey An RTCP message as defined in this document is used to
events observed at a receiver -- in addition to long term convey information about events observed at a receiver --
receiver status information which is carried in RTCP RRs Ż in addition to long term receiver status information which
back to the sender of the media stream. is carried in RTCP RRs -- back to the sender of the media
stream.
Feedback (FB) threshold: Feedback (FB) threshold:
The FB threshold indicates the "borderline" between Immediate The FB threshold indicates the transition between Immediate
Feedback and Early RTCP mode. For a multicast scenario, the Feedback and Early RTCP mode. For a multicast scenario,
FB threshold indicates the maximum group size at which, on the FB threshold indicates the maximum group size at which,
average, each receiver is able to report each event back to on average, each receiver is able to report each event back
the sender(s) immediately, i.e. without having to wait for to the sender(s) immediately, i.e. by means of an Early
its regularly scheduled RTCP interval. This threshold is RTCP packet without having to wait for its regularly
highly dependent on network QoS (e.g. packet loss probability scheduled RTCP interval. This threshold is highly
and distribution), codec and packetization in use, and dependent on the type of feedback to be provided, network
application requirements. Hence, no formal definition is QoS (e.g. packet loss probability and distribution), codec
presented in this document. and packetization in use, the session bandwidth, and
application requirements. Note that the algorithms do
not depend on all senders and receivers agreeing on the
same value for this threshold. It is merely intended to
provide conceptual guidance to application designers and is
not used in any calculations.
Immediate Feedback mode: Immediate Feedback mode:
Mode of operation in which each receiver of a media is, A mode of operation in which each receiver of a media
statistically, capable of reporting each event of interest stream is, statistically, capable of reporting each event
immediately back to the media stream sender. In Immediate of interest immediately back to the media stream sender.
Feedback mode, RTCP feedback messages are transmitted In Immediate Feedback mode, RTCP feedback messages are
according to the timing rules defined in this document. transmitted according to the timing rules defined in this
document.
Ott et al. Expires May 2002 [Page 3]
Regular RTCP mode: Regular RTCP mode:
Mode of operation in which no preferred transmission of Mode of operation in which no preferred transmission of
feedback messages is allowed. Instead, RTCP messages are feedback messages is allowed. Instead, RTCP messages are
sent following the rules of [1] and may contain feedback sent following the rules of [1]. Such RTCP messages may
messages information as defined in this document. contain feedback information as defined in this document.
Regularly Scheduled RTCP packet: Regularly Scheduled RTCP packet:
An RTCP packet that is not sent as an Early RTCP packet. An RTCP packet that is not sent as an Early RTCP packet.
1.2 Terminology 1.2 Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL"
document are to be interpreted as described in RFC 2119 [8] in this document are to be interpreted as described in RFC 2119
[8]
2. RTP and RTCP Packet Formats and Protocol Behavior 2. RTP and RTCP Packet Formats and Protocol Behavior
The rules defined in [2] also apply to this profile except for those The rules defined in [2] also apply to this profile except for
rules mentioned in the following: those rules mentioned in the following:
RTCP packet types: RTCP packet types:
Three additional RTCP packet types to convey feedback Three additional RTCP packet types to convey feedback
information are defined in section 4. information are defined in section 4.
RTCP report intervals: RTCP report intervals:
This memo describes three modes of operation which influence This memo describes three modes of operation which
the RTCP report intervals (see section 3.2). In regular influence the RTCP report intervals (see section 3.2). In
RTCP mode, all rules from [1] apply. In both Immediate regular RTCP mode, all rules from [1] apply. In both
Feedback and Early RTCP modes the minimal interval of 5 Immediate Feedback and Early RTCP modes the minimal
seconds between 2 RTCP reports is dropped and the rules interval of 5 seconds between 2 RTCP reports is dropped and
specified in section 3 apply if RTCP packets containing the rules specified in section 3 apply if RTCP packets
feedback messages (defined in section 4) are to be containing feedback messages (defined in section 4) are to
transmitted. be transmitted.
The rules set forth in [1] may be overridden by session The rules set forth in [1] may be overridden by session
descriptions specifying different parameters (e.g. for the descriptions specifying different parameters (e.g. for the
bandwidth share assigned to RTCP for senders and receivers, bandwidth share assigned to RTCP for senders and receivers,
respectively. For sessions defined using the Session respectively). For sessions defined using the Session
Description Protocol (SDP) [3], the rules of [4] apply. Description Protocol (SDP) [3], the rules of [4] apply.
Congestion control: Congestion control:
The same basic rules as detailed in [2] apply. Beyond this, The same basic rules as detailed in [2] apply. Beyond
in section 5, further consideration is given to the impact of this, in section 5, further consideration is given to the
feedback and a sender's reaction to feedback messages. impact of feedback and a sender's reaction to feedback
messages.
3. Rules for RTCP Feedback 3. Rules for RTCP Feedback
3.1 Compound RTCP Feedback Packets 3.1 Compound RTCP Feedback Packets
Ott et al. Expires May 2002 [Page 4]
Two components constitute RTCP-based feedback as described in this Two components constitute RTCP-based feedback as described in this
memo: memo:
. Status reports are contained in SR/RR messages and are transmitted o Status reports are contained in SR/RR messages and are
at regular intervals as part of compound RTCP packets (which also transmitted at regular intervals as part of compound RTCP
include SDES and possibly other messages); these status reports packets (which also include SDES and possibly other messages);
provide an overall indication for the recent reception quality of these status reports provide an overall indication for the
a media stream. recent reception quality of a media stream.
. Feedback messages as defined in this document that indicate loss o Feedback messages as defined in this document that indicate loss
or reception of particular pieces of a media stream (or provide or reception of particular pieces of a media stream (or provide
some other form of rather immediate feedback on the data some other form of rather immediate feedback on the data
received). Rules for the transmission of feedback messages are received). Rules for the transmission of feedback messages are
newly introduced in this memo. newly introduced in this memo.
RTCP Feedback (FB) messages are just another RTCP packet type (see RTCP Feedback (FB) messages are just another RTCP packet type (see
section 4). Therefore, multiple FB messages MAY be combined in a section 4). Therefore, multiple FB messages MAY be combined in a
single compound RTCP packet and they MAY also be sent combined with single compound RTCP packet and they MAY also be sent combined with
other RTCP packets. other RTCP packets.
RTCP packets containing Feedback packets as defined in this document RTCP packets containing Feedback packets as defined in this
MUST contain RTCP packets in the order as defined in [1]: document MUST contain RTCP packets in the order as defined in [1]:
. OPTIONAL encryption prefix that MUST be present if the RTCP o OPTIONAL encryption prefix that MUST be present if the RTCP
message is to be encrypted. message is to be encrypted.
. MANDATORY SR or RR. o MANDATORY SR or RR.
. MANDATORY SDES which MUST contain the CNAME item; all other SDES o MANDATORY SDES which MUST contain the CNAME item; all other SDES
items are OPTIONAL. items are OPTIONAL.
. One or more FB messages. o One or more FB messages.
The FB MUST be placed in the compound packet after RR and SDES RTCP The FB message(s) MUST be placed in the compound packet after RR
packets defined in [1]. The ordering with respect to other RTCP and SDES RTCP packets defined in [1]. The ordering with respect to
extensions is not defined. other RTCP extensions is not defined.
Two types of compound RTCP packets carrying feedback packets are used Two types of compound RTCP packets carrying feedback packets are
in this document: used in this document:
a) Minimal compound RTCP feedback packet a) Minimal compound RTCP feedback packet
A minimal compound RTCP feedback packet MUST contain only the A minimal compound RTCP feedback packet MUST contain only the
mandatory information as listed above: encryption prefix if mandatory information as listed above: encryption prefix if
necessary, exactly one RR or SR, exactly one SDES with only the necessary, exactly one RR or SR, exactly one SDES with only the
CNAME item present, and the feedback message(s). This is to CNAME item present, and the feedback message(s). This is to
minimize the size of the RTCP packet transmitted to convey minimize the size of the RTCP packet transmitted to convey
feedback and thus to maximize the frequency at which feedback can feedback and thus to maximize the frequency at which feedback
be provided while still adhering to the RTCP bandwidth can be provided while still adhering to the RTCP bandwidth
limitations. limitations.
This packet format SHOULD be used whenever an RTCP feedback This packet format SHOULD be used whenever an RTCP feedback
message is sent as part of an Early RTCP packet. message is sent as part of an Early RTCP packet.
b) (Full) compound RTCP feedback packet b) (Full) compound RTCP feedback packet
Ott et al. Expires May 2002 [Page 5] A (full) compound RTCP feedback packet MAY contain any
A (full) compound RTCP feedback packet MAY contain any additional additional number of RTCP packets (additional RRs, further SDES
number of RTCP packets (additional RRs, further SDES items, items, etc.). The above ordering rules MUST be adhered to.
etc.).
This packet format MUST be used whenever an RTCP feedback message This packet format MUST be used whenever an RTCP feedback
is sent as part of a regularly scheduled RTCP packet or in message is sent as part of a regularly scheduled RTCP packet or
Regular RTCP mode. This packet format MAY also be used to send in Regular RTCP mode. It MAY also be used to send RTCP
RTCP feedback messages in Immediate Feedback or Early RTCP mode. feedback messages in Immediate Feedback or Early RTCP mode.
RTCP packets that do not contain FB messages are referred to as non- RTCP packets that do not contain FB messages are referred to as
FB RTCP packets. non-FB RTCP packets. Such packets MUST follow the format rules in
[1].
3.2 Algorithm Outline 3.2 Algorithm Outline
FB messages are part of the RTCP control streams and are thus subject FB messages are part of the RTCP control streams and are thus
to the same bandwidth constraints as other RTCP traffic. This means subject to the same bandwidth constraints as other RTCP traffic.
in particular that it may not be possible to report an event observed This means in particular that it may not be possible to report an
at a receiver immediately back to the sender. However, the value of event observed at a receiver immediately back to the sender.
feedback given to a sender typically decreases over time -- in terms
of the media quality as perceived by the user at the receiving end However, the value of feedback given to a sender typically
and/or the cost required to achieve media stream repair. decreases over time -- in terms of the media quality as perceived
by the user at the receiving end and/or the cost required to
achieve media stream repair.
RTP [1] and the commonly used RTP profile [2] specify rules when RTP [1] and the commonly used RTP profile [2] specify rules when
compound RTCP packets should be sent. This document modifies those compound RTCP packets should be sent. This document modifies those
rules in order to allow applications to timely report media loss or rules in order to allow applications to timely report events (e.g.
reception events to accommodate algorithms that use FB messages and loss or reception of media packets) to accommodate algorithms that
are sensitive to the feedback timing. use FB messages and are sensitive to the feedback timing.
The modified algorithm can be outlined as follows: Normally, when no The modified RTCP transmission algorithm can be outlined as
FB messages have to be conveyed, compound RTCP packets are sent follows: Normally, when no FB messages have to be conveyed,
following the rules of RTP [1] -- except that the 5s minimum interval compound RTCP packets are sent following the rules of RTP [1] --
between RTCP reports is not enforced. If a receiver detects the need except that the 5s minimum interval between RTCP reports is not
for an FB message, the receiver waits for a short, random dithering enforced and the interval between RTCP reports is only derived from
the average RTCP packet size and the RTCP bandwidth share available
to the RTP/RTCP entity. If a receiver detects the need to send an
FB message, the receiver waits for a short, random dithering
interval (in case of multicast) and then checks whether it has interval (in case of multicast) and then checks whether it has
already seen a corresponding FB message from any other receiver already seen a corresponding FB message from any other receiver
(which it can do with all FB messages that are transmitted via (which it can do with all FB messages that are transmitted via
multicast; for unicast sessions, there is no such delay). If this is multicast; for unicast sessions, there is no such delay). If this
the case then the receiver refrains from sending the FB message and is the case then the receiver refrains from sending the FB message
continues to follow the regular RTCP sending schedule. If the and continues to follow the regular RTCP sending schedule. If the
receiver has not yet seen a similar FB message from any other receiver has not yet seen a similar FB message from any other
receiver, it checks whether it has recently exceeded its RTCP bit receiver, it checks whether it has recently exceeded its RTCP bit
rate budget to transmit another FB message (without waiting for its rate budget to transmit another FB message (without waiting for its
regularly scheduled RTCP transmission time). Only if this is not the regularly scheduled RTCP transmission time). Only if this is not
case, it sends the FB message as part of a (minimal) compound RTCP the case, it sends the FB message as part of a (minimal) compound
packet. RTCP packet.
FB messages may also be sent as part of full compound RTCP packets FB messages may also be sent as part of full compound RTCP packets
which are interspersed as per [1] in regular intervals. which are interspersed as per [1] (except for the five second lower
bound) in regular intervals.
3.3 Modes of Operation 3.3 Modes of Operation
RTCP-based feedback may operate in one of three modes (figure 1): RTCP-based feedback may operate in one of three modes (figure 1) as
described below. The mode is a hint whether or not a receiver
should send early feedback at all and, if so, whether,
statistically, all events observed at the receiver can be reported
back to the sender in a timely fashion. The current mode of
operation is continuously derived independently at each receiver
and the receivers do not have to agree on a common mode.
Ott et al. Expires May 2002 [Page 6] a) Immediate feedback mode: the group size is below the FB
a) Immediate feedback mode: the group size is below the FB threshold threshold which gives each receiving party sufficient bandwidth
which gives each receiving party sufficient bandwidth to transmit to transmit the RTCP feedback packets for the intended purpose.
the feedback traffic for the intended purpose. This means, for
each receiver there is enough bandwidth to report each event it is
supposed/expected to by means of a virtually "immediate" RTCP
feedback packet.
The group size threshold is a function of a number of parameters This means that, for each receiver, there is enough bandwidth
including (but not necessarily limited to) the type of feedback to report each event it is supposed/expected to by means of a
used (e.g. ACK vs. NACK), bandwidth, packet rate, packet loss virtually "immediate" RTCP feedback packet.
probability and distribution, media type, codec, and -- again
depending on the type of FB used -- the (worst case or observed) The group size threshold is a function of a number of
frequency of events to report (e.g. frame received, packet lost). parameters including (but not necessarily limited to) the type
of feedback used (e.g. ACK vs. NACK), bandwidth, packet rate,
packet loss probability and distribution, media type, codec,
and -- again depending on the type of FB used -- the (worst
case or observed) frequency of events to report (e.g. frame
received, packet lost).
A special case of this is the ACK mode (where positive A special case of this is the ACK mode (where positive
acknowledgements are used to confirm reception of data) which is acknowledgements are used to confirm reception of data) which
restricted to point-to-point communications. is restricted to point-to-point communications.
b) Early RTCP mode: In this mode, the group size and other parameters As a rough estimate, let N be the average number of events to
no longer allow each receiver to react to each event that would be be reported per interval T by a receiver, B the RTCP bandwidth
worth (or needed) to report. But feedback can still be given fraction for this particular receiver and R the average RTCP
sufficiently often so that it allows the sender to adapt the media packet size, then the receiver operates in Immediate Feedback
stream transmission accordingly and thereby increase the overall mode as long as N<=B*T/R.
reproduced media quality.
b) Early RTCP mode: In this mode, the group size and other
parameters no longer allow each receiver to react to each event
that would be worth (or needed) to report. But feedback can
still be given sufficiently often so that it allows the sender
to adapt the media stream transmission accordingly and thereby
increase the overall reproduced media quality.
Using the above notation, Early RTCP mode can be roughly
characterized by N > B*T/R as "lower bound". An estimate for
an upper bound is more difficult. Setting N=1, we obtain for a
given R and B the interval T = R/B as average interval between
events to be reported. This information can be used as a hint
to determine whether or not early transmission of RTCP packets
is useful.
c) From some group size upwards, it is no longer useful to provide c) From some group size upwards, it is no longer useful to provide
feedback from individual receivers at all -- because of the time feedback from individual receivers at all -- because of the
scale in which the feedback could be provided and/or because in time scale in which the feedback could be provided and/or
large groups the sender(s) have no chance to react to individual because in large groups the sender(s) have no chance to react
feedback anymore. to individual feedback anymore.
No threshold can be specified when this occurs.
As the feedback algorithm described in this memo scales smoothly, As the feedback algorithm described in this memo scales smoothly,
there is no need for an agreement among the participants on the there is no need for an agreement among the participants on the
precise values of the respective "thresholds" within the group. precise values of the respective "thresholds" within the group.
Hence the borders between all these modes are allowed to be fluent. Hence the borders between all these modes are allowed to be
fluent.
ACK ACK
feedback feedback
V V
:<- - - - NACK feedback - - - ->// :<- - - - NACK feedback - - - ->//
: :
: Immediate || : Immediate ||
: Feedback mode ||Early RTCP mode Regular RTCP mode : Feedback mode ||Early RTCP mode Regular RTCP mode
:<=============>||<=============>//<=================> :<=============>||<=============>//<=================>
: || : ||
-+---------------||---------------//------------------> group size -+---------------||---------------//------------------> group size
2 || 2 ||
Application-specific FB Threshold Application-specific FB Threshold
= f(data rate, packet loss, codec, ...) = f(data rate, packet loss, codec, ...)
Figure 1: Modes of operation Figure 1: Modes of operation
Ott et al. Expires May 2002 [Page 7] As stated before, the respective thresholds depend on a number of
The respective thresholds depend on a number of technical parameters technical parameters (of the codec, the transport, the type of
(of the codec, the transport, the feedback used, etc.) but also on feedback used, etc.) but also on the respective application
the respective application scenarios. Section 3.5 provides some scenarios. Section 3.6 provides some useful hints (but no precise
useful hints (but no complete precise calculations) on estimating calculations) on estimating these thresholds.
these thresholds.
3.4 Definitions 3.4 Definitions
The following pieces of state information need to be maintained per The following pieces of state information need to be maintained per
receiver (largely taken from [1]). Note that all variables (except receiver (largely taken from [1]). Note that all variables (except
for h) are calculated independently at each receiver and so their for g) are calculated independently at each receiver and so their
local values may differ at a given point in time. local values may differ at a given point in time.
a) Let senders be the number of active senders in the RTP session. a) Let senders be the number of active senders in the RTP session.
b) Let members be the current estimate of the number of receivers b) Let members be the current estimate of the number of receivers
in the RTP session. in the RTP session.
c) Let T_rtt be the maximum round trip time as measured by RTCP c) Let tn and tp be the time for the next (last) scheduled
(if available to the receiver). Note that this may be asymmetric.
d) Let tn and tp be the time for the next (last) scheduled
RTCP RR transmission calculated prior to reconsideration. RTCP RR transmission calculated prior to reconsideration.
e) Let T_rr be the interval after which, having just sent a regularly d) Let T_rr be the interval after which, having just sent a
scheduled RTCP packet, a receiver would schedule the transmission regularly scheduled RTCP packet, a receiver would schedule the
of its next RTCP packet following the rules of [1]: T_rr = tn - transmission of its next RTCP packet following the rules of [1]:
tp. Note that the 5s minimum interval between two report as T_rr = tn - tp. Note that the 5s minimum interval between two
defined in [1] SHOULD NOT be enforced. report as defined in [1] SHOULD NOT be enforced.
f) Let t0 be the time at which an event that is to be reported is e) Let t0 be the time at which an event that is to be reported is
detected by a receiver. detected by a receiver.
g) Let T_dither_max be the maximum interval for which an RTCP f) Let T_dither_max be the maximum interval for which an RTCP
feedback packet may be additionally delayed (to prevent feedback packet may be additionally delayed (to prevent
implosions). implosions).
h) Let T_max_fb_delay be the upper bound within which feedback to g) Let T_max_fb_delay be the upper bound within which feedback to
an event needs to be reported back to the sender to be useful at an event needs to be reported back to the sender to be useful at
all. Note that this value is application-specific. all. Note that this value is application-specific.
i) Let te be the time for which a feedback packet is scheduled. h) Let te be the time for which a feedback packet is scheduled.
j) Let T_fd be the actual (randomized) delay for the transmission of i) Let T_fd be the actual (randomized) delay for the transmission
feedback message in response to an event that a certain packet P of feedback message in response to an event that a certain
caused. packet P caused.
k) Let allow_early be a Boolean variable that indicates whether the j) Let allow_early be a Boolean variable that indicates whether the
receiver currently may transmit feedback messages prior to its receiver currently may transmit feedback messages prior to its
next regularly scheduled RTCP interval tn. This variable is used next regularly scheduled RTCP interval tn. This variable is
to throttle the feedback sent by a single receiver. allow_early used to throttle the feedback sent by a single receiver.
is adjusted (set to FALSE) after early feedback transmission and allow_early is adjusted (set to FALSE) after early feedback
is reset to TRUE as soon as the next regular RTCP transmission is transmission and is reset to TRUE as soon as the next regular
scheduled. RTCP transmission is scheduled.
Ott et al. Expires May 2002 [Page 8] k) Let avg_rtcp_size be the moving average on the RTCP packet size
l) Let avg_rtcp_size be the moving average on the RTCP packet size as as defined in [1].
defined in [1].
The feedback situation for an event to report at a receiver is The feedback situation for an event to report at a receiver is
depicted in figure 2 below. At time t0, such an event (e.g. a packet depicted in figure 2 below. At time t0, such an event (e.g. a
loss) is detected at the receiver. The receiver decides -- based packet loss) is detected at the receiver. The receiver decides --
upon current T_rtt, group size, and other (application-specific) based upon current bandwidth, group size, and other (application-
parameters -- that a feedback message needs to be sent back to the specific) parameters -- that a feedback message needs to be sent
sender. back to the sender.
To avoid an implosion of immediate feedback packets, the receiver To avoid an implosion of immediate feedback packets, the receiver
MUST delay the transmission of the compound feedback packet by a MUST delay the transmission of the RTCP feedback packet by a random
random amount T_fd (with the random number evenly distributed in the amount T_fd (with the random number evenly distributed in the
interval [0, T_dither_max]. Transmission of the compound RTCP packet interval [0, T_dither_max]). Transmission of the compound RTCP
is then scheduled for te = t0 + T_fd. packet MUST then be scheduled for te = t0 + T_fd.
The T_dither_max parameter is chosen based upon the round-trip time The T_dither_max parameter is derived from the regular RTCP
or, if the round-trip time is not available, based upon the group interval (which, in turn, is based upon the group size).
size.
Based upon the parameters influencing T_dither_max and a number of For a certain application scenario, a receiver may determine an
other parameters (such as the type of feedback to be provided) the upper bound for the acceptable local delay of feedback messages:
receiver may determine T_max_fb_delay (as static value or dynamically T_max_fb_delay. If an a priori estimation or the actual
adjusted) as the upper bound for the feedback information to be calculation of T_dither_max indicates that this upper bound MAY be
useful when it reaches the sender. violated (e.g. because T_dither_max > T_max_fb_delay), the receiver
MAY decide not to send any feedback at all because the achievable
gain is considered insufficient.
If a compound RTCP feedback packet is scheduled, the time slot for If an RTCP feedback packet is scheduled, the time slot for the next
the next scheduled compound RTCP packet is updated accordingly to a scheduled (full) compound RTCP packet MUST be updated accordingly
new tn. to a new tn (which will then be in the order of tn=tp+2*T_rr).
This is to ensure that the short term average bandwidth used for
RTCP with feedback does not exceed the bandwidth limit that would
be used without feedback.
event to event to
report report
detected detected
| |
| RTCP feedback range | RTCP feedback range
| (T_max_fb_delay) | (T_max_fb_delay)
vXXXXXXXXXXXXXXXXXXXXXXXXXXX ) ) vXXXXXXXXXXXXXXXXXXXXXXXXXXX ) )
|---+--------+-------------+-----+------------| |--------+---------> |---+--------+-------------+-----+------------| |--------+--->
| | | | ( ( | | | | | ( ( |
| t0 te | | t0 te |
tp tn tp tn
\_______ ________/ \_______ ________/
\/ \/
T_dither_max T_dither_max
Figure 2: Event report and parameters for Early RTCP scheduling Figure 2: Event report and parameters for Early RTCP scheduling
3.5 Early RTCP Algorithm 3.5 Early RTCP Algorithm
Ott et al. Expires May 2002 [Page 9]
Assume an active sender S0 (out of S senders) and a number N of Assume an active sender S0 (out of S senders) and a number N of
receivers with R being one of these receivers. receivers with R being one of these receivers.
Assume further that R has verified that using feedback mechanisms is Assume further that R has verified that using feedback mechanisms
reasonable at the current constellation (which is highly application is reasonable at the current constellation (which is highly
specific and hence not specified in this memo). application specific and hence not specified in this memo).
Then, receiver R MUST use the following rules for transmitting one or Then, receiver R MUST use the following rules for transmitting one
more Feedback messages as minimal or full compound RTCP packet: or more Feedback messages as minimal or full compound RTCP packet:
Initially, R MUST set allow_early := TRUE. Initially, R MUST set allow_early = TRUE.
R has transmitted the last RTCP RR packet at tp and has scheduled the Assume that R has transmitted the last RTCP RR packet at tp and has
next transmission (prior to reconsideration) for tn. scheduled the next transmission (prior to reconsideration) for tn.
At time t0, R detects the need to transmit one or more feedback At time t0, R detects the need to transmit one or more RTCP
messages (e.g. because media "units" needs to be ACKed or NACKed) and feedback messages (e.g. because media "units" needs to be ACKed or
finds that sending the feedback information is useful for the sender. NACKed) and finds that sending the feedback information is useful
for the sender.
R first checks whether there is still a compound RTCP feedback packet R first checks whether there is still a compound RTCP feedback
waiting for transmission (scheduled as early or regular RTCP packet). packet waiting for transmission (scheduled as early or regular RTCP
If so, the new feedback message MUST be appended to the packet; the packet). If so, the new feedback message MUST be appended to the
schedule for the waiting RTCP feedback packet MUST remain unchanged. packet; the schedule for the waiting RTCP feedback packet MUST
When appending, the feedback information of several RTCP feedback remain unchanged. When appending, the feedback information of
packets SHOULD be merged as few packets as possible. several RTCP feedback packets SHOULD be merged to produce as few
packets as possible.
If no RTCP feedback message is already awaiting transmission, a new If no RTCP feedback message is already awaiting transmission, a new
(minimal) compound RTCP feedback packet MUST be created and the (minimal or full) compound RTCP feedback packet MUST be created and
minimal interval for T_dither_max MUST be chosen as follows: the minimal interval for T_dither_max MUST be chosen as follows:
i) If the session is a unicast session (group size = 2) then i) If the session is a unicast session (group size = 2) then
T_dither_max := 0. T_dither_max = 0.
ii) If the receiver has an RTT estimate to the originator of the
media unit to provide feedback about, then
T_dither_max := k * T_rtt/2 * members
with k=1.
iii) If the receiver does not have an RTT estimate to the originator, ii) If the session is a multicast session with potentially more
then than two group members then
T_dither_max := l * T_rr T_dither_max = l * T_rr
with l=0.5. with l=0.5.
The values given above for T_dither_max are minimal values. The values given above for T_dither_max are minimal values.
Application-specific feedback considerations may make it worthwhile Application-specific feedback considerations may make it worthwhile
to increase T_dither_max beyond this value. This is up to the to increase T_dither_max beyond this value. This is up to the
discretion of the implementer. discretion of the implementer.
Then, R MUST check whether its next regularly scheduled RTCP packet Then, R MUST check whether its next regularly scheduled RTCP packet
is within the time bounds for the RTCP FB (t0 + T_dither_max > tn). is within the time bounds for the RTCP FB (t0 + T_dither_max > tn).
Ott et al. Expires May 2002 [Page 10]
If so, an Early RTCP packet MUST NOT be scheduled; instead the FB If so, an Early RTCP packet MUST NOT be scheduled; instead the FB
message(s) MUST be stored to be appended to the regular RTCP packet message(s) MUST be stored to be appended to the regular RTCP packet
scheduled for tn. scheduled for tn.
Otherwise, R MUST check whether it is allowed to transmit an Early Otherwise, R MUST check whether it is allowed to transmit an Early
RTCP packet (allow_early == TRUE). RTCP packet (allow_early == TRUE).
If so, R MUST schedule an Early RTCP packet for te := t0 + RND * If so, R MUST schedule an Early RTCP packet for te = t0 + RND *
T_dither_max with the RND function evenly distributed between 0 T_dither_max with RND being a pseudo random function evenly
and 1. distributed between 0 and 1.
If, while waiting for te, R receives RTCP feedback packets If, while waiting for te, R receives RTCP feedback packets
contained in one or more (minimal) compound RTCP packets, R MUST contained in one or more (minimal) compound RTCP packets, R MUST
act as follows for each of the RTCP feedback packets in the one or act as follows for each of the RTCP feedback packets in the one
more compound RTCP packets received: or more compound RTCP packets received:
1. If R understands the received feedback message's semantics and 1. If R understands the received feedback message's semantics
the message contents is a superset of the feedback R wanted to and the message contents is a superset of the feedback R
send then R MUST discard its own feedback message and MUST re- wanted to send then R MUST discard its own feedback message
schedule the next regular RTCP message transmission for tn (as and MUST re-schedule the next regular RTCP message
calculated before). transmission for tn (as calculated before).
2. If R understands the received feedback message's semantics and 2. If R understands the received feedback message's semantics
the message contents is not a superset of the feedback R and the message contents is not a superset of the feedback R
wanted to send then R SHOULD transmit its own feedback message wanted to send then R SHOULD transmit its own feedback
as scheduled. If there is an overlap between the feedback message as scheduled. If there is an overlap between the
information to send and the feedback information to receive, feedback information to send and the feedback information
the amount of feedback transmitted is up to R: R MAY send its received, the amount of feedback transmitted is up to R: R
feedback information unchanged, R MAY as well eliminate any MAY send its feedback information unchanged, R MAY as well
redundancy between its own feedback and the feedback received eliminate any redundancy between its own feedback and the
so far. feedback received so far.
3. If R does not understand the received feedback message's 3. If R does not understand the received feedback message's
semantics, R checks whether the compound RTCP packet contains semantics, R MAY send its own feedback message as Early RTCP
a Generic INFO message. If a Generic INFO message is present packet, or R MAY re-schedule the next regular RTCP message
R performs the comparison based upon this information and transmission for tn (as calculated before) and MAY append
proceeds with alternative 1. or 2. above depending on the the feedback message to the now regularly scheduled RTCP
outcome of the comparison. If no Generic INFO message is message.
present, then R MAY send its own feedback message as or Early
RTCP packet. Alternatively, R MAY re-schedule the next
regular RTCP message transmission for tn (as calculated
before) and MAY append the feedback message to the now
regularly scheduled RTCP message.
Refer to section 4 on the comparison of feedback messages and for Note: With rule #3, receiving unknown feedback packets may
which feedback messages MUST be understood by a receiver. not lead to feedback suppression at a particular receiver.
As a consequence, a given event may cause M different types
of feedback packets (which are all appropriate but not the
same and mutually not understood) to be scheduled, and a
"large" receiver group may be partitioned into at most M
groups. Among members of each of these M groups, feedback
suppression will occur following the rules #1 and #2 but no
suppression will happen across groups. As a result, O(M)
RTCP feedback messages may be received by the sender. Given
that these M groups consist of receivers for the same
application using the same (set of) codecs in the same RTP
session, M is assumed to be small in the general case.
Given further that the O(M) feedback packets are randomly
distributed over a time interval of T_dither_max, the
resulting limited number of extra feedback packets (a) is
assumed not to overwhelm the sender and (b) should be
conveyed as all contain complementary pieces of information.
Refer to section 4 on the comparison of feedback messages and
for which feedback messages MUST be understood by a receiver.
Otherwise, when te is reached, R MUST transmit the RTCP packet Otherwise, when te is reached, R MUST transmit the RTCP packet
containing the FB message. R then MUST set allow_early := FALSE containing the FB message. R then MUST set allow_early = FALSE
and MUST recalculate tn := tp + 2*T_rr. As soon as R sends its and MUST recalculate tn = tp + 2*T_rr. The value from the last
next regularly scheduled RTCP RR (at the new tn), it MUST set calculation of T_rr SHOULD be used. As soon as R sends its next
allow_early := TRUE again. regularly scheduled RTCP RR (at the new tn), it MUST set
allow_early = TRUE again.
Ott et al. Expires May 2002 [Page 11]
If allow_early == FALSE then R MUST check the time for the next If allow_early == FALSE then R MUST check the time for the next
scheduled RR: scheduled RR:
1. If tn Ż t0 < T_max_fb_delay (i.e. if, despite late reception, the 1. If tn - t0 < T_max_fb_delay (i.e. if, despite late reception,
feedback could still be useful for the sender) then R MAY create the feedback could still be useful for the sender) then R MAY
an RTCP FB message for transmission along with the RTCP packet at create an RTCP FB message for transmission along with the RTCP
tn. packet at tn.
2. Otherwise, R MUST discard the RTCP feedback message. 2. Otherwise, R MUST discard the RTCP feedback message.
In regular RTCP intervals as specified by [1] (except for the five In regular RTCP intervals as specified by [1] (except for the five
second minimum), a full compound RTCP packet is sent (which may also second minimum), a full compound RTCP packet MUST be sent (which
contain a feedback message if one has been created according to the MAY also contain a feedback message if one has been created
above rules and scheduled for transmission along the full compound according to the above rules and scheduled for transmission along
RTCP message). the full compound RTCP message).
Whenever an RTCP packet is sent or received -- minimal or full Whenever an RTCP packet is sent or received -- minimal or full
compound, early or regularly scheduled -- the avg_rtcp_size variable compound, early or regularly scheduled -- the avg_rtcp_size
is updated accordingly (see [1]) and the tn is calculated using the variable MUST be updated accordingly (see [1]) and the tn MUST be
new avg_rtcp_size. calculated using the new avg_rtcp_size.
3.6 Considerations on the Group Size 3.6 Considerations on the Group Size
This section provides guidelines to the group sizes at which the This section provides some guidelines to the group sizes at which
various feedback modes may be used. the various feedback modes may be used.
3.6.1 ACK mode 3.6.1 ACK mode
The group size MUST be exactly two participants, i.e. point-to-point The group size MUST be exactly two participants, i.e. point-to-
communications. Unicast addresses SHOULD be used in the session point communications. Unicast addresses MUST be used in the
description. session description.
For unidirectional as well as bi-directional communication between For unidirectional as well as bi-directional communication between
two parties, 2.5% of the RTP session bandwidth are available for RTCP two parties, 2.5% of the RTP session bandwidth are available for
traffic from the receivers including feedback. Assuming that out of RTCP traffic from the receivers including feedback. For a 64
ten RTCP packets, nine are sent as minimal compound RTCP packets and kbit/s stream this yields 1600 bit/s for RTCP. As every other RTCP
one as full compound RTCP packet, at 64kbit/s unidirectional packet needs to be a full compound packet, we assume an average of
communication scenario, a receiver can report 1.5 events per second 96 bytes (=768 bits) per RTCP packet so that a receiver can report
back to the sender, at 256kbit/s 6 events and so forth. 2 events per second back to the sender. If acknowledgments for 10
events are collected in each feedback message then 20 events can be
acknowledged per second. At 256 kbit/s 8 events could be reported
per second; thus the ACKs may be sent in a finer granularity (e.g.
only combining only three ACKs per RTCP feedback message).
From 1 Mbit/s upwards, a receiver would be able to acknowledge each From 1 Mbit/s upwards, a receiver would be able to acknowledge each
individual frame (not packet!) in a 25 fps video stream. individual frame (not packet!) in a 30 fps video stream.
ACK strategies MUST be defined accordingly to work properly with ACK strategies MUST be defined accordingly to work properly with
these bandwidth limitations. An indication whether or not ACKs are these bandwidth limitations. An indication whether or not ACKs are
allowed for a session and, if so, which ACK strategy should be used, allowed for a session and, if so, which ACK strategy should be
MAY be conveyed by out-of-band mechanisms, e.g. media-specific used, MAY be conveyed by out-of-band mechanisms, e.g. media-
attributes in a session description using SDP. specific attributes in a session description using SDP.
Ott et al. Expires May 2002 [Page 12]
3.6.2 NACK mode 3.6.2 NACK mode
Negative acknowledgements (or similar types of feedback) MUST be Negative acknowledgements (or similar types of feedback) MUST be
used for all groups larger than two. Of course, NACKs MAY be used used for all groups larger than two. Of course, NACKs MAY be used
for point-to-point communications as well. for point-to-point communications as well.
Whether or not the use of Immediate or Early RTCP packets should be Whether or not the use of Immediate or Early RTCP packets should be
considered depends upon a number of parameters including session considered depends upon a number of parameters including session
bandwidth, codec, special type of feedback, number of senders and bandwidth, codec, special type of feedback, number of senders and
receivers, among many others. receivers, among many others.
The crucial parameters -- to which virtually all of the above can be The crucial parameters -- to which virtually all of the above can
reduced -- is the allowed minimal interval between two RTCP reports be reduced -- is the allowed minimal interval between two RTCP
and the (average) number of events that presumably need reporting per reports and the (average) number of events that presumably need
time interval (plus their distribution over time, of course). The reporting per time interval (plus their distribution over time, of
minimum interval is derived from the available RTCP bandwidth and the course). The minimum interval can be derived from the available
expected average size of an RTCP packet. The number of events to RTCP bandwidth and the expected average size of an RTCP packet.
report e.g. per second may be derived from the packet loss rate and The number of events to report e.g. per second may be derived from
sender's rate of transmitting packets. From these two values, the the packet loss rate and sender's rate of transmitting packets.
allowable group size for the Immediate feedback mode can be From these two values, the allowable group size for the Immediate
calculated. feedback mode can be calculated.
Let N be the average number of events to be reported per
interval T by a receiver, B the RTCP bandwidth fraction for
this particular receiver and R the average RTCP packet size,
then the receiver operates in Immediate Feedback mode is used
as long as N<=B*T/R.
The upper bound for the Early RTCP mode then solely depends on the The upper bound for the Early RTCP mode then solely depends on the
acceptable quality degradation, i.e. how many events per time acceptable quality degradation, i.e. how many events per time
interval may go unreported. interval may go unreported.
Using the above notation, Early RTCP mode can be roughly
characterized by N > B*T/R as "lower bound". An estimate for
an upper bound is more difficult. Setting N=1, we obtain for a
given R and B the interval T = R/B as average interval between
events to be reported. This information can be used as a hint
to determine whether or not early transmission of RTCP packets
is useful.
Example: If a 256kbit/s video with 30 fps is transmitted through a Example: If a 256kbit/s video with 30 fps is transmitted through a
network with an MTU size of some 1500 bytes, then, in most cases, network with an MTU size of some 1500 bytes, then, in most cases,
each frame would fit in its own packet leading to a packet rate of 30 each frame would fit in its own packet leading to a packet rate of
packets per second. If 5% packet loss occurs in the network (equally 30 packets per second. If 5% packet loss occurs in the network
distributed, no inter-dependence between receivers), then each (equally distributed, no inter-dependence between receivers), then
receiver will have to report 3 packets lost each two seconds. each receiver will have to report 3 packets lost each two seconds.
Assuming a single sender and more than three receivers, this yields Assuming a single sender and more than three receivers, this yields
3.75% of the RTCP bandwidth allocated to the receivers and thus 3.75% of the RTCP bandwidth allocated to the receivers and thus
9.6kbit/s. Assuming further a size of 120 bytes for the average 9.6kbit/s. Assuming further a size of 120 bytes for the average
compound RTCP packet allows 10 RTCP packets to be sent per second or compound RTCP packet allows 10 RTCP packets to be sent per second
20 in two seconds. If every receiver needs to report three packets, or 20 in two seconds. If every receiver needs to report three
this yields a maximum group size of 6-7 receivers if all loss events packets, this yields a maximum group size of 6-7 receivers if all
shall be reported. The rules for transmission of immediate RTCP loss events shall be reported. The rules for transmission of
packets should provide sufficient flexibility for most of this immediate RTCP packets should provide sufficient flexibility for
reporting to occur in a timely fashion. most of this reporting to occur in a timely fashion.
Extending this example to determine the upper bound for Early RTCP Extending this example to determine the upper bound for Early RTCP
mode leads to the following considerations: assume that the mode could lead to the following considerations: assume that the
underlying coding scheme and the application (as well as the tolerant underlying coding scheme and the application (as well as the
users) allow on the order of one loss without repair per two seconds. tolerant users) allow on the order of one loss without repair per
Thus the number of packets to be reported by each receiver decreases two seconds. Thus the number of packets to be reported by each
to two per two seconds second and increases the group size to 10. receiver decreases to two per two seconds second and increases the
Assuming further that some number of packet losses are correlated, group size to 10. Assuming further that some number of packet
feedback traffic is further reduced and group sizes of some 12 to 16 losses are correlated, feedback traffic is further reduced and
(maybe even 20) can be reasonably well supported using Early RTCP group sizes of some 12 to 16 (maybe even 20) can be reasonably well
mode. supported using Early RTCP mode. Note, of course, that all those
considerations are based upon statistics and will fail to hold in
some cases.
Ott et al. Expires May 2002 [Page 13]
3.7 Summary of decision steps 3.7 Summary of decision steps
3.7.1 General Hints 3.7.1 General Hints
Before even considering whether or not to send RTCP feedback Before even considering whether or not to send RTCP feedback
information an application has to determine whether this mechanism is information an application has to determine whether this mechanism
applicable: is applicable:
1) An application has to decide whether -- for the current ratio of 1) An application has to decide whether -- for the current ratio of
packet rate with the associated (application-specific) maximum packet rate with the associated (application-specific) maximum
feedback delay and the currently observed round-trip time (if feedback delay and the currently observed round-trip time (if
available) -- feedback mechanisms can be applied at all. available) -- feedback mechanisms can be applied at all.
This decision may obviously be based upon (and dynamically revised This decision may obviously be based upon (and dynamically
following) regular RTCP reception statistics. revised following) regular RTCP reception statistics as well as
out-of-band mechanisms.
2) The application has to decide whether -- for a certain observed 2) The application has to decide -- for a certain observed error
error rate, assigned bandwidth, frame rate, and group size -- (and rate, assigned bandwidth, frame/packet rate, and group size --
which) feedback mechanisms can be applied. whether (and which) feedback mechanisms can be applied.
Regular RTCP provides valuable input to this step, too. Regular RTCP provides valuable input to this step, too.
3) If these tests pass, the application has to follow the rules for 3) If these tests pass, the application has to follow the rules for
transmitting Early RTCP packets or regularly scheduled RTCP transmitting Early RTCP packets or regularly scheduled RTCP
packets with piggybacked feedback. packets with piggybacked feedback.
3.7.2 Media Session Attributes 3.7.2 Media Session Attributes
Media sessions are typically described using out-of-band mechanisms Media sessions are typically described using out-of-band mechanisms
to convey transport addresses, codec information, etc. between to convey transport addresses, codec information, etc. between
sender(s) and receiver(s). Such a mechanisms is composed of a format sender(s) and receiver(s). Such a mechanisms consists of a format
used to describe a media session and another mechanism for used to describe a media session and another mechanism for
transporting this description. transporting this description.
In the IETF, the Session Description Protocol (SDP) is currently used In the IETF, the Session Description Protocol (SDP) is currently
to describe media sessions while protocols such as SIP, SAP, RTSP, used to describe media sessions while protocols such as SIP, SAP,
and HTTP are used to convey the description. RTSP, and HTTP (among others) are used to convey the descriptions.
A present media session description format MAY include parameters to A media session description format MAY include parameters to
indicate that RTCP feedback mechanisms are supported in this session indicate that RTCP feedback mechanisms MAY be used (=are supported)
and which of the feedback mechanisms may be applied. in this session and which of the feedback mechanisms MAY be
applied.
To do so, the profile "AVPF" MUST be indicated instead of "AVP". To do so, the profile "AVPF" MUST be indicated instead of "AVP".
Further attributes may be defined to show which type(s) of feedback Further attributes may be defined to show which type(s) of feedback
are supported. are supported.
Section 4 contains the syntax specification to support RTCP feedback Section 4 contains the syntax specification to support RTCP
with SDP. Similar specifications for other media session description feedback with SDP. Similar specifications for other media session
formats are outside the scope of this specification. description formats are outside the scope of this document.
4. SDP Definitions 4. SDP Definitions
Ott et al. Expires May 2002 [Page 14]
This section defines a number of additional SDP parameters that are This section defines a number of additional SDP parameters that are
used to describe a session. All of these are defined as media level used to describe a session. All of these are defined as media
attributes. level attributes.
4.1 Profile identification 4.1 Profile identification
The AV profile defined in [4] is referred to as "AVP" in the context The AV profile defined in [4] is referred to as "AVP" in the
of e.g. the Session Description Protocol (SDP) [3]. The profile context of e.g. the Session Description Protocol (SDP) [3]. The
specified in this document is referred to as "AVPF". profile specified in this document is referred to as "AVPF".
Feedback information following the modified timing rules as specified Feedback information following the modified timing rules as
in this document MUST NOT be sent for a particular media session specified in this document MUST NOT be sent for a particular media
unless the profile for this session indicates the use of the "AVPF" session unless the profile for this session indicates the use of
profile. the "AVPF" profile (exclusively or jointly with other AV profiles).
4.2 RTCP Feedback Capability Attribute 4.2 RTCP Feedback Capability Attribute
A new payload format-specific SDP attribute (for use with "a=fmtp:") A new payload format-specific SDP attribute (for use with
is defined to indicate the capability of using RTCP feedback as "a=fmtp:") is defined to indicate the capability of using RTCP
specified in this document: "rtcp-fb". The "rtcp-fb" attribute MAY feedback as specified in this document: "rtcp-fb". The "rtcp-fb"
only be used as an SDP media attribute and MUST NOT be provided at attribute MUST only be used as an SDP media attribute and MUST NOT
the session level. The rtcp-fb attribute MUST only be used in media be provided at the session level. The "rtcp-fb" attribute MUST
sessions for which the "AVPF" is specified. only be used in media sessions for which the "AVPF" is specified.
The rtcp-fb attribute is used to indicate which RTCP feedback The "rtcp-fb" attribute SHOULD be used to indicate which RTCP
messages MAY be used in this media session for the indicated payload feedback messages MAY be used in this media session for the
type. If several types of feedback are supported, several a=rtcp-fb: indicated payload type. If several types of feedback are
lines MUST be used. supported, several "a=rtcp-fb:" lines MUST be used.
If no rtcp-fb attribute is specified the RTP receivers SHOULD assume If no "rtcp-fb" attribute is specified the RTP receivers SHOULD
that the RTP senders only support generic NACKs. In addition, the assume that the RTP senders only support generic NACKs. In
RTP receivers MAY send feedback using other suitable RTCP feedback addition, the RTP receivers MAY send feedback using other suitable
packets as defined for the respective media type. The RTP receivers RTCP feedback packets as defined for the respective media type.
MUST NOT rely on the RTP senders reacting to any of the feedback The RTP receivers MUST NOT rely on the RTP senders reacting to any
messages. of the feedback messages.
If one or more rtcp-fb attributes are present in a media session If one or more "rtcp-fb" attributes are present in a media session
description, the RTP receivers for the media session(s) containing description, the RTCP receivers for the media session(s) containing
the "rtcp-fb" the "rtcp-fb"
. MUST ignore all rtcp-fb attributes of which they do not fully o MUST ignore all "rtcp-fb" attributes of which they do not fully
understand the semantics (i.e. understand the meaning of all understand the semantics (i.e. where they do not understand the
values in the a=fmtp:rtcp-fb line); meaning of all values in the a=fmtp:rtcp-fb line);
. SHOULD provide feedback information as specified in this document o SHOULD provide feedback information as specified in this
using any of the RTCP feedback packets as specified in one of the document using any of the RTCP feedback packets as specified in
rtcp-fb attributes for this media session; and one of the "rtcp-fb" attributes for this media session; and
. MUST NOT use other feedback messages than those listed in one of o MUST NOT use other feedback messages than those listed in one of
the rtcp-fb attribute lines. the "rtcp-fb" attribute lines.
Ott et al. Expires May 2002 [Page 15]
RTP senders MUST be prepared to receive any kind of RTCP feedback RTP senders MUST be prepared to receive any kind of RTCP feedback
messages and MUST silently discard all those RTCP feedback messages messages and MUST silently discard all those RTCP feedback messages
that they do not understand. that they do not understand.
The syntax of the rtcp-fb attribute is as follows (the feedback types The syntax of the "rtcp-fb" attribute is as follows (the feedback
and optional parameters are all case sensitive): types and optional parameters are all case sensitive):
rtcp-fb-syntax = "a=fmtp:" <format> WS "rtcp-fb" WS rtcp-fb-value rtcp-fb-syntax = "a=fmtp:" <format> WS "rtcp-fb" WS rtcp-fb-val
rtcp-fb-value = "ack" rtcp-fb-param rtcp-fb-val = "ack" rtcp-fb-ack-param
| "nack" rtcp-fb-nack-param | "nack" rtcp-fb-nack-param
| rtcp-fb-id rtcp-fb-param | rtcp-fb-id rtcp-fb-param
rtcp-fb-id = 1*(alpha-numeric | "-" | "_") rtcp-fb-id = 1*(alpha-numeric | "-" | "_")
rtcp-fb-param = "app" rtcp-fb-param = "app"
| byte-string | byte-string
| ; empty | ; empty
rtcp-fb-ack-param = "rpsi"
| "app"
| byte-string
| ; empty
rtcp-fb-nack-param = "pli" rtcp-fb-nack-param = "pli"
| "sli" | "sli"
| "rpsi" | "rpsi"
| "app" | "app"
| byte-string | byte-string
| ; empty | ; empty
The literals of the above grammar have the following semantics: The literals of the above grammar have the following semantics:
Feedback type "ack": Feedback type "ack":
This feedback type indicates that positive acknowledgements for This feedback type indicates that positive acknowledgements
feedback are supported. for feedback are supported.
The feedback type "ack" MUST only be used if the media session The feedback type "ack" MUST only be used if the media session
is allowed to operate in ACK mode as defined in 3.6.1.2. is allowed to operate in ACK mode as defined in 3.6.1.2.
Parameters may be provided to further distinguish different Parameters MAY be provided to further distinguish different
types of positive acknowledgement feedback. If no parameters types of positive acknowledgement feedback. If no parameters
are present, the Generic ACK as specified in section 4.1.2 is are present, the Generic ACK as specified in section 6.2.2 is
implied. implied.
The parameter "rpsi" indicates the use of Reference Picture
Selection Indication feedback as defined in section 6.3.3.
If the parameter "app" is specified, this indicates the use of If the parameter "app" is specified, this indicates the use of
application layer feedback. In this case, additional parameters application layer feedback. In this case, additional
following "app" MAY be used to further differentiate various parameters following "app" MAY be used to further
types of application layer feedback. This document does not differentiate various types of application layer feedback.
define any parameters specific to "app". This document does not define any parameters specific to
"app".
Further parameters for "ack" MAY be defined in other documents. Further parameters for "ack" MAY be defined in other
documents.
Feedback type "nack": Feedback type "nack":
This feedback type indicates that negative acknowledgements for This feedback type indicates that negative acknowledgements
feedback are supported. for feedback are supported.
Ott et al. Expires May 2002 [Page 16]
The feedback type "nack", without parameters, indicates use of The feedback type "nack", without parameters, indicates use of
the General NACK feedback format as defined in section 4.2.1. the General NACK feedback format as defined in section 6.2.1.
The following three parameters are defined in this document for The following three parameters are defined in this document
use with "nack" in conjunction with the media type "video": for use with "nack" in conjunction with the media type
"video":
. "pli" indicates the use of Picture Loss Indication feedback o "pli" indicates the use of Picture Loss Indication feedback
as defined in section 4.3.1. as defined in section 6.3.1.
. "sli" indicates the use of Slice Loss Indication feedback as o "sli" indicates the use of Slice Loss Indication feedback
defined in section 4.3.2. as defined in section 6.3.2.
. "rpsi" indicates the use of Reference Picture Selection o "rpsi" indicates the use of Reference Picture Selection
Indication feedback as defined in section 4.3.3. Indication feedback as defined in section 6.3.3.
. "app" indicates the use of application layer feedback.
"app" indicates the use of application layer feedback.
Additional parameters after "app" MAY be provided to Additional parameters after "app" MAY be provided to
differentiate different types of application layer feedback. differentiate different types of application layer feedback.
No parameters specific to "app" are defined in this document. No parameters specific to "app" are defined in this document.
Further parameters for "nack" MAY be defined in other documents. Further parameters for "nack" MAY be defined in other
documents.
Other feedback types <rtcp-fb-id>: Other feedback types <rtcp-fb-id>:
Other documents MAY define additional types of feedback; to keep Other documents MAY define additional types of feedback; to
the grammar extensible for those cases, the rtcp-fb-id is keep the grammar extensible for those cases, the rtcp-fb-id is
introduced as a placeholder. A new feedback scheme name needs introduced as a placeholder. A new feedback scheme name MUST
to be unique (and thus has to be registered with IANA). Along to be unique (and thus MUST be registered with IANA). Along
with a new name, its semantics, packet formats (if necessary), with a new name, its semantics, packet formats (if necessary),
and rules for its operation need to be specified. and rules for its operation MUST be specified.
Note that it is assumed that more specific information about Note that it is assumed that more specific information about
application layer feedback (as defined in section 4.2.3) will be application layer feedback (as defined in section 6.4) will be
conveyed as feedback types and parameters defined elsewhere. Hence, conveyed as feedback types and parameters defined elsewhere.
no further provision for any types and parameters is made in this Hence, no further provision for any types and parameters is made in
document. this document.
Further types of feedback as well as further parameters may be Further types of feedback as well as further parameters may be
defined in other documents. defined in other documents.
It is up to the recipients whether or not they send feedback It is up to the recipients whether or not they send feedback
information and up to the sender(s) to make use of feedback provided. information and up to the sender(s) to make use of feedback
provided.
4.3 Unicasting 4.3 Unicasting vs. Multicasting
If an m= line in the SDP describing a session indicates unicast If a media session description indicates unicast addresses for a
addresses for a particular media type (and does not operate in multi- particular media type (and does not operate in multi-unicast mode
unicast mode with all recipients listed explicitly but still with all recipients listed explicitly but still addressed via
addressed via unicast), the RTCP feedback MAY operate in ACK feedback unicast), the RTCP feedback MAY operate in ACK feedback mode.
mode.
4.4 RTCP Bandwidth Modifiers If a media session description indicates multicast addresses for a
particular media type or a multi-unicast session, ACK feedback mode
MUST NOT be used.
The standard RTCP bandwidth assignments as defined in [1] and [2] may 4.4 RTCP Bandwidth Modifiers
be overridden by bandwidth modifiers as specified in [4]: b=RS:<bw>
Ott et al. Expires May 2002 [Page 17] The standard RTCP bandwidth assignments as defined in [1] and [2]
and b=RR:<bw> MAY be used to assign a different bandwidth (measured may be overridden by bandwidth modifiers that explicitly define the
in bits per second) to RTP senders and receivers, respectively. The maximum RTCP bandwidth. For use with SDP, such modifiers are
precedence rules of [4] apply to determine the actual bandwidth to be specified in [4]: "b=RS:<bw>" and "b=RR:<bw>" MAY be used to assign
used by senders and receivers. a different bandwidth (measured in bits per second) to RTP senders
and receivers, respectively. The precedence rules of [4] apply to
determine the actual bandwidth to be used by senders and receivers.
Applications operating knowingly over highly asymmetric links (such Applications operating knowingly over highly asymmetric links (such
as satellite links) SHOULD use this mechanism to reduce the feedback as satellite links) SHOULD use this mechanism to reduce the
rate for high bandwidth streams to prevent deterministic congestion feedback rate for high bandwidth streams to prevent deterministic
of the feedback path(s). congestion of the feedback path(s).
4.5 Examples 4.5 Examples
Example 1: The following session description indicates a session made Example 1: The following session description indicates a session
up from an audio and a DTMF for point-to-point communication in which made up from an audio and a DTMF for point-to-point communication
the DTMF stream uses Generic ACKs. This session description could be in which the DTMF stream uses Generic ACKs. This session
contained in a SIP INVITE, 200 OK, or ACK message to indicate that description could be contained in a SIP INVITE, 200 OK, or ACK
its sender is capable of and willing to receive feedback for the DTMF message to indicate that its sender is capable of and willing to
stream it transmits. receive feedback for the DTMF stream it transmits.
v=0 v=0
o=alice 3203093520 3203093520 IN IP4 host.example.com o=alice 3203093520 3203093520 IN IP4 host.example.com
s=Media with feedback s=Media with feedback
t=0 0 t=0 0
c=IN IP4 host.example.com c=IN IP4 host.example.com
m=audio 49170 RTP/AVPF 0 96 m=audio 49170 RTP/AVPF 0 96
a=rtpmap:0 PCMU/8000 a=rtpmap:0 PCMU/8000
a=rtpmap:96 telephone-event/8000 a=rtpmap:96 telephone-event/8000
a=fmtp:96 0-16 a=fmtp:96 0-16
a=fmtp:96 rtcp-fb ack a=fmtp:96 rtcp-fb ack
Example 2: The following session description indicates a multicast Example 2: The following session description indicates a multicast
video-only session (using H.263+) with the video source accepting video-only session (using H.263+) with the video source accepting
Generic NACKs and Reference Picture Selection. Such a description Generic NACKs and Reference Picture Selection. Such a description
may have been conveyed using the Session Announcement Protocol (SAP). may have been conveyed using the Session Announcement Protocol
(SAP).
v=0 v=0
o=alice 3203093520 3203093520 IN IP4 host.example.com o=alice 3203093520 3203093520 IN IP4 host.example.com
s=Multicast video with feedback s=Multicast video with feedback
t=3203130148 3203137348 t=3203130148 3203137348
m=audio 49170 RTP/AVP 0 m=audio 49170 RTP/AVP 0
c=IN IP4 224.2.1.183 c=IN IP4 224.2.1.183
a=rtpmap:0 PCMU/8000 a=rtpmap:0 PCMU/8000
m=video 51372 RTP/AVPF 98 m=video 51372 RTP/AVPF 98
c=IN IP4 224.2.1.184 c=IN IP4 224.2.1.184
a=rtpmap:98 H263-1998/90000 a=rtpmap:98 H263-1998/90000
a=fmtp:98 rtcp-fb nack a=fmtp:98 rtcp-fb nack
a=fmtp:98 rtcp-fb nack rpsi a=fmtp:98 rtcp-fb nack rpsi
5. Interworking and Co-Existence of AVP and AVPF Entities Example 3: The following session description defines the same media
session as example 2 but allows for mixed mode operation of AVP and
AVPF RTP entities (see also next section). Note that both media
descriptions use the same addresses; however, two m= lines are
needed to convey information about both applicable RTP profiles.
The AVPF profile defined in this document is an extension of the AVP v=0
profile as defined in [2]. Both profiles follow the same basic rules o=alice 3203093520 3203093520 IN IP4 host.example.com
s=Multicast video with feedback
t=3203130148 3203137348
m=audio 49170 RTP/AVP 0
c=IN IP4 224.2.1.183
a=rtpmap:0 PCMU/8000
m=video 51372 RTP/AVP 98
c=IN IP4 224.2.1.184
a=rtpmap:98 H263-1998/90000
m=video 51372 RTP/AVPF 98
c=IN IP4 224.2.1.184
a=rtpmap:98 H263-1998/90000
a=fmtp:98 rtcp-fb nack
a=fmtp:98 rtcp-fb nack rpsi
Ott et al. Expires May 2002 [Page 18] Note that these two m= lines SHOULD be grouped by some appropriate
(including the upper bandwidth limit for RTCP and the bandwidth mechanisms to indicate that both are alternatives actually
assignments to senders and receivers. Therefore, senders and conveying the same contents. A sample mechanism by which this can
receivers of using either of the two profiles can be mixed in a be achieved is defined in [14].
single session.
5. Interworking and Co-Existence of AVP and AVPF Entities
The AVPF profile defined in this document is an extension of the
AVP profile as defined in [2]. Both profiles follow the same basic
rules (including the upper bandwidth limit for RTCP and the
bandwidth assignments to senders and receivers). Therefore,
senders and receivers of using either of the two profiles can be
mixed in a single session (see e.g. example 3 in section 4.5).
AVP and AVPF are defined in a way that, from a robustness point of AVP and AVPF are defined in a way that, from a robustness point of
view,, the RTP entities do not need to be aware of entities of the view, the RTP entities do not need to be aware of entities of the
respective other profile: they will not disturb each other's respective other profile: they will not disturb each other's
functioning. However, the quality of the media presented may suffer. functioning. However, the quality of the media presented may
suffer.
The following considerations apply to senders and receivers when used The following considerations apply to senders and receivers when
in a combined session. used in a combined session.
. AVP entities (senders and receivers) o AVP entities (senders and receivers)
AVP senders will receive RTCP feedback packets from AVPF receivers AVP senders will receive RTCP feedback packets from AVPF
and ignore these packets. They will see occasional closer spacing receivers and ignore these packets. They will see occasional
of RTCP messages (e.g. violating the 5s rule) by AVPF entities. closer spacing of RTCP messages (e.g. violating the 5s rule) by
As the overall bandwidth constraints are adhered to by both types AVPF entities. As the overall bandwidth constraints are adhered
of entities, they will still get their share of the RTCP to by both types of entities, they will still get their share of
bandwidth. However, while AVP entities are bound by the 5s rule, the RTCP bandwidth. However, while AVP entities are bound by
depending on the group size and session bandwidth, AVPF entities the 5s rule, depending on the group size and session bandwidth,
may provide more frequent RTCP reports than AVP ones will. Also, AVPF entities may provide more frequent RTCP reports than AVP
the overall reporting may decrease slightly as AVPF entities are ones will. Also, the overall reporting may decrease slightly as
may to send bigger RTCP packets (due to the extra fields). AVPF entities may send bigger compound RTCP packets (due to the
extra RTCP packets).
. AVPF senders o AVPF senders
AVPF senders will receive feedback information only from AVPF AVPF senders will receive feedback information only from AVPF
receivers. If they rely on feedback to provide the target media receivers. If they rely on feedback to provide the target media
quality, the quality achieved for AVP receivers may be sub- quality, the quality achieved for AVP receivers may be sub-
optimal. optimal.
. AVPF receivers o AVPF receivers
AVPF receivers SHOULD send immediate or early RTCP feedback AVPF receivers SHOULD send immediate or early RTCP feedback
packets only if all (sending) entities in the media session packets only if all (sending) entities in the media session
support AVPF. AVPF receivers MAY send feedback information as support AVPF. AVPF receivers MAY send feedback information as
part of regularly scheduled compound RTCP packets following the part of regularly scheduled compound RTCP packets following the
timing rules of [1] and [2] also in media sessions operating in timing rules of [1] and [2] also in media sessions operating in
mixed mode. In this case, however, the receiver providing mixed mode. However, the receiver providing feedback MUST NOT
feedback MUST NOT rely on the sender reacting to the feedback at rely on the sender reacting to the feedback at all.
all.
6. Format of RTCP Feedback Messages 6. Format of RTCP Feedback Messages
This section defines the format of the low delay RTCP feedback This section defines the format of the low delay RTCP feedback
messages. These messages classified into three categories as messages. These messages classified into three categories as
follows: follows:
- Transport layer feedback messages - Transport layer feedback messages
- Payload-specific feedback messages - Payload-specific feedback messages
- Application layer feedback messages - Application layer feedback messages
Ott et al. Expires May 2002 [Page 19]
Transport layer feedback messages are intended to transmit general Transport layer feedback messages are intended to transmit general
purpose feedback information, i.e. information independent of the purpose feedback information, i.e. information independent of the
particular codec or the application in use. The information is particular codec or the application in use. The information is
expected to be generated and processed at the transport/RTP layer. expected to be generated and processed at the transport/RTP layer.
Currently, only a general positive acknowledgement (ACK) and negative Currently, only a general positive acknowledgement (ACK) and
acknowledgement (NACK) message are defined. negative acknowledgement (NACK) message are defined.
Payload-specific feedback messages transport information that is Payload-specific feedback messages transport information that is
specific to a certain payload and will be generated and acted upon at specific to a certain payload type and will be generated and acted
the codec "layer". This document defines a common header to be used upon at the codec "layer". This document defines a common header
in conjunction with all payload-specific feedback messages. The to be used in conjunction with all payload-specific feedback
definition of specific messages is left to either RTP Payload Format messages. The definition of specific messages is left to either
specifications or to additional feedback format documents. RTP Payload Format specifications or to additional feedback format
documents.
Application layer feedback messages provide a means to transparently Application layer feedback messages provide a means to
convey feedback from the receiver's to the sender's application. The transparently convey feedback from the receiver's to the sender's
information contained in such a message is not expected to be acted application. The information contained in such a message is not
upon at the transport/RTP or the codec layer. The data to be expected to be acted upon at the transport/RTP or the codec layer.
exchanged between two application instances is usually defined in the The data to be exchanged between two application instances is
application protocol's specification and thus can be identified by usually defined in the application protocol's specification and
the application so that there is no need for additional external thus can be identified by the application so that there is no need
information. Hence, this document defines only a common header to be for additional external information. Hence, this document defines
used along with all application layer feedback messages. From a only a common header to be used along with all application layer
protocol point of view, an application layer feedback message is feedback messages. From a protocol point of view, an application
treated as a special case of a payload-specific feedback message. layer feedback message is treated as a special case of a payload-
specific feedback message.
This document defines two transport layer feedback and three (video) This document defines two transport layer feedback and three
payload-specific feedback messages as well as a container for (video) payload-specific feedback messages as well as a single
application layer feedback messages. Additional transport layer and container for application layer feedback messages. Additional
payload specific feedback messages may be defined in other documents transport layer and payload specific feedback messages MAY be
and are registered through IANA (see section IANA considerations). defined in other documents and MUST be registered through IANA (see
section IANA considerations).
The general syntax and semantics for the above RTCP feedback message The general syntax and semantics for the above RTCP feedback
types is described in the following subsections. message types is described in the following subsections.
6.1 Common Packet Format for Feedback Message 6.1 Common Packet Format for Feedback Message
All feedback message share a common packet format that is depicted in All feedback message MUST use a common packet format that is
figure 3: depicted in figure 3:
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|0| FMT | PT | length | |V=2|P|0| FMT | PT | length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC of packet sender | | SSRC of packet sender |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC of media source | | SSRC of media source |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: Feedback Control Information (FCI) : : Feedback Control Information (FCI) :
: : : :
Figure 3: Common Packet Format for Feedback Messages Figure 3: Common Packet Format for Feedback Messages
Ott et al. Expires May 2002 [Page 20]
The various fields V, P, SSRC and length are defined in the RTP The various fields V, P, SSRC and length are defined in the RTP
specification [2], the respective meaning being summarized below: specification [2], the respective meaning being summarized below:
version (V): 2 bits version (V): 2 bits
This field identifies the RTP version. The current version is 2. This field identifies the RTP version. The current version is
2.
padding (P): 1 bit padding (P): 1 bit
If set, the padding bit indicates that the packet contains If set, the padding bit indicates that the packet contains
additional padding octets at the end which are not part of the additional padding octets at the end which are not part of the
control information but are included in the length field. control information but are included in the length field.
Feedback message type (FMT): 4 bits Feedback message type (FMT): 4 bits
This field identifies the type of the feedback message and is This field identifies the type of the feedback message and is
interpreted relative to the RTCP message type (transport, interpreted relative to the RTCP message type (transport,
payload-specific, or application feedback). The values for each payload-specific, or application layer feedback). The values
of the three feedback types are defined in the respective for each of the three feedback types are defined in the
sections below. respective sections below.
Payload type (PT): 8 bits Payload type (PT): 8 bits
This is the RTCP packet type which identifies the packet as being This is the RTCP packet type which identifies the packet as
an RTCP Feedback Message. Two values are defined (TBA. By IANA): being an RTCP Feedback Message. Two values are defined (TBA.
by IANA):
Name | Value | Brief Description Name | Value | Brief Description
----------+-------+-------------------------------------- ----------+-------+------------------------------------
RTPFB | 2xx | Transport layer feedback message RTPFB | 205 | Transport layer feedback message
PSFB | 2xy | Payload-specific feedback message PSFB | 206 | Payload-specific feedback message
Length: 16 bits Length: 16 bits
The length of this packet in 32-bit words minus one, including The length of this packet in 32-bit words minus one, including
the header and any padding. This is in line with the definition the header and any padding. This is in line with the
of the length field used in RTCP sender and receiver reports [3]. definition of the length field used in RTCP sender and receiver
reports [3].
SSRC of packet sender: 32 bits SSRC of packet sender: 32 bits
The synchronization source identifier for the originator of this The synchronization source identifier for the originator of
packet. this packet.
SSRC of media source: 32 bits SSRC of media source: 32 bits
The synchronization source identifier of the media source that The synchronization source identifier of the media source that
this piece of feedback information is related to. this piece of feedback information is related to.
Feedback Control Information (FCI): variable length Feedback Control Information (FCI): variable length
The following three sections define which additional information The following three sections define which additional
is included in the feedback message for each type of feedback. information MAY be included in the feedback message for each
Each RTCP feedback packet MUST contain exactly one FCI field of type of feedback (further FCI contents MAY be specified in
the types defined in sections 6.2 and 6.3. If multiple FCI further documents). Each RTCP feedback packet MUST contain
fields (even of the same type) need to be conveyed, then several exactly one FCI field of the types defined in sections 6.2 and
RTCP feedback packets MUST be generated and concatenated in the 6.3. If multiple FCI fields (even of the same type) need to be
same compound RTCP packet. conveyed, then several RTCP feedback packets MUST be generated
and concatenated in the same compound RTCP packet.
6.2 Transport Layer Feedback Messages 6.2 Transport Layer Feedback Messages
Transport Layer Feedback messages are identified by the value RTPFB Transport Layer Feedback messages are identified by the value RTPFB
as RTCP message type. as RTCP message type.
Ott et al. Expires May 2002 [Page 21] Two general purpose transport layer feedback messages are defined
Two general purpose transport layer feedback messages are defined so so far: General ACK and General NACK. They are identified by means
far: General ACK and General NACK. They are identified by means of of the FMT parameter as follows:
the FMT parameter as follows:
0: forbidden 0: forbidden
1: Generic NACK 1: Generic NACK
2: Generic ACK 2: Generic ACK
3: Generic INFO 3-15: reserved
4-15: reserved
The following two subsections define the packet formats for these The following two subsections define the packet formats for these
messages. messages.
6.2.1 Generic NACK 6.2.1 Generic NACK
The Generic NACK message is identified by PT=RTPFB and FMT=1. The Generic NACK message is identified by PT=RTPFB and FMT=1.
The Generic NACK packet is used to indicate the loss of one or more The Generic NACK packet is used to indicate the loss of one or more
RTP packets. The lost packet(s) are identified by the means of a RTP packets. The lost packet(s) are identified by the means of a
skipping to change at line 1161 skipping to change at page 27, line 16
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PID | BLP | | PID | BLP |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 4: Syntax for the Generic NACK message Figure 4: Syntax for the Generic NACK message
Packet ID (PID): 16 bits Packet ID (PID): 16 bits
The PID field is used to specify a lost packet. Typically, the The PID field is used to specify a lost packet. Typically, the
RTP sequence number is used for PID as the default format, but RTP sequence number is used for PID as the default format, but
RTP Payload Formats may decide to identify a packet differently. RTP Payload Formats may decide to identify a packet
differently.
bitmask of following lost packets (BLP): 16 bits bitmask of following lost packets (BLP): 16 bits
The BLP allows for reporting losses of any of the 16 RTP packets The BLP allows for reporting losses of any of the 16 RTP
immediately following the RTP packet indicated by the PID. The packets immediately following the RTP packet indicated by the
BLP's definition is identical to that given in [10]. Denoting PID. The BLP's definition is identical to that given in [10].
the BLP's least significant bit as bit 1, and its most Denoting the BLP's least significant bit as bit 1, and its most
significant bit as bit 16, then bit i of the bit mask is set to 1 significant bit as bit 16, then bit i of the bit mask is set to
if the sender has not received RTP packet number PID+i (modulo 1 if the sender has not received RTP packet number PID+i
2^16) and the receiver decides this packet is lost; bit i is set (modulo 2^16) and the receiver decides this packet is lost; bit
to 0 otherwise. Note that the sender MUST NOT assume that a i is set to 0 otherwise. Note that the sender MUST NOT assume
receiver has received a packet because its bit mask was set to 0. that a receiver has received a packet because its bit mask was
For example, the least significant bit of the BLP would be set to set to 0. For example, the least significant bit of the BLP
1 if the packet corresponding to the PID and the following packet would be set to 1 if the packet corresponding to the PID and
have been lost. However, the sender cannot infer that packets the following packet have been lost. However, the sender
PID+2 through PID+16 have been received simply because bits 2 cannot infer that packets PID+2 through PID+16 have been
received simply because bits 2 through 15 of the BLP are 0; all
Ott et al. Expires May 2002 [Page 22] the sender knows is that the receiver has not reported them as
through 15 of the BLP are 0; all the sender knows is that the lost at this time.
receiver has not reported them as lost at this time.
6.2.2 Generic ACK 6.2.2 Generic ACK
The Generic ACK message is identified by PT=RTPFB and FMT=2. The Generic ACK message is identified by PT=RTPFB and FMT=2.
The Generic ACK packet is used to indicate that one or several RTP The Generic ACK packet is used to indicate that one or several RTP
packets were received correctly. The received packet(s) are packets were received correctly. The received packet(s) are
identified by the means of a packet identifier and a bit mask. identified by the means of a packet identifier and a bit mask.
ACKing of a range of consecutive packets is also possible. ACKing of a range of consecutive packets is also possible.
skipping to change at line 1204 skipping to change at page 28, line 15
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PID |R| BLP/#packets | | PID |R| BLP/#packets |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 5: Syntax for the Generic ACK message Figure 5: Syntax for the Generic ACK message
Packet ID (1st PID): 16 bits Packet ID (1st PID): 16 bits
This PID field is used to specify a correctly received packet. This PID field is used to specify a correctly received packet.
Typically, the RTP sequence number is used for PID as the default Typically, the RTP sequence number is used for PID as the
format, but RTP Payload Formats may decide to identify a packet default format, but RTP Payload Formats may decide to identify
differently. a packet differently.
Range of ACKs (R): 1 bit Range of ACKs (R): 1 bit
The R-bit indicates that a range of consecutive packets are The R-bit indicates that a range of consecutive packets are
received correctly. If R=1 then the PID field specifies the received correctly. If R=1 then the PID field specifies the
first packet of that range and the next field (BLP/#packets) will first packet of that range and the next field (BLP/#packets)
carry the number of packets being acknowledged. If R=0 then PID will carry the number of packets being acknowledged. If R=0
specifies the first packet to be acknowledged and BLP/#packets then PID specifies the first packet to be acknowledged and
provides a bit mask to selectively indicate individual packets BLP/#packets provides a bit mask to selectively indicate
that are acknowledged. individual packets that are acknowledged.
Bit mask of lost packets (BLP)/#packets (PID): 15 bits Bit mask of lost packets (BLP)/#packets (PID): 15 bits
The semantics of this field depends on the value of the R-bit. The semantics of this field depends on the value of the R-bit.
If R=1, this field is used to identify the number of additional If R=1, this field is used to identify the number of additional
packets of to be acknowledged: packets of to be acknowledged:
#packets = <highest seq# to be ACKed> - <PID> #packets = <highest seq# to be ACKed> - <PID>
That is, #packets MUST indicate the number of packet to be ACKed That is, #packets MUST indicate the number of packet to be
minus one. In particular, if only a single packet is to be ACKed ACKed minus one. In particular, if only a single packet is to
and R=1 then #packets MUST be set to 0x0000. be ACKed and R=1 then #packets MUST be set to 0x0000.
Example: If all packets between and including PIDx=380 and PIDy =
422 have been received, the Generic ACK would contain PID = PIDx
= 380 and #packets = PIDy Ż PID = 42. In case the PID wraps
Ott et al. Expires May 2002 [Page 23] Example: If all packets between and including PIDx = 380 and
around, modulo arithmetic is used to calculate the number of PIDy = 422 have been received, the Generic ACK would contain
packets. PID = PIDx = 380 and #packets = PIDy - PID = 42. In case the
PID wraps around, modulo arithmetic is used to calculate the
number of packets.
If R=0, this field carries a bit mask. The BLP allows for If R=0, this field carries a bit mask. The BLP allows for
reporting reception of any of the 15 RTP packets immediately reporting reception of any of the 15 RTP packets immediately
following the RTP packet indicated by the PID. The BLP's following the RTP packet indicated by the PID. The BLP's
definition is identical to that given in [10] except that, here, definition is identical to that given in [10] except that,
BLP is only 15 bits wide. Denoting the BLP's least significant here, BLP is only 15 bits wide. Denoting the BLP's least
bit as bit 1, and its most significant bit as bit 15, then bit i significant bit as bit 1, and its most significant bit as bit
of the bitmask is set to 1 if the sender has received RTP packet 15, then bit i of the bitmask is set to 1 if the sender has
number PID+i (modulo 2^16) and the receiver decides to ACK this received RTP packet number PID+i (modulo 2^16) and the receiver
packet; bit i is set to 0 otherwise. If only the packet decides to ACK this packet; bit i is set to 0 otherwise. If
indicated by PID is to be ACKed and R=0 then BLP MUST be set to only the packet indicated by PID is to be ACKed and R=0 then
0x0000. BLP MUST be set to 0x0000.
6.2.3 Generic INFO
The Generic INFO message is identified by PT=RTPFB and FMT=3.
The Generic INFO packet MUST only be used in conjunction with an
application-specific feedback message. The Generic INFO message
indicates which RTP packets the payload-specific message is about.
The packet(s) in question are identified by the means of a packet
identifier and a bit mask.
The sole purpose of the Generic INFO packet is to avoid unnecessary
feedback suppression when payload-specific feedback messages are
mixed with generic ones.
The packet format is the same as for the Generic NACK message defined
in section 6.2.3.
6.3 Payload Specific Feedback Messages 6.3 Payload Specific Feedback Messages
Payload-Specific Feedback Messages are identified by the value PSFB Payload-Specific Feedback Messages are identified by the value
as RTCP message type. PT=PSFB as RTCP message type.
Three payload-specific feedback messages are defined so far. They Three payload-specific feedback messages are defined so far plus an
are identified by means of the FMT parameter as follows: application layer feedback message. They are identified by means
of the FMT parameter as follows:
0: forbidden 0: forbidden
1: Picture Loss Indication (PLI) 1: Picture Loss Indication (PLI)
2: Slice Lost Indication (SLI) 2: Slice Lost Indication (SLI)
3: Reference Picture Selection Indication (RPSI) 3: Reference Picture Selection Indication (RPSI)
4-14: reserved 4-14: reserved
15: Application layer feedback message 15: Application layer feedback message
The following subsections define the packet formats for these The following subsections define the packet formats for the
messages. payload-specific messages, section 6.4 defines the application
layer feedback message.
AVPF entities MUST include Generic INFO messages along with any
payload-specific ones in compound RTCP packets (early as well as
regularly scheduled ones). The INFO message(s) MUST cover all the
Ott et al. Expires May 2002 [Page 24]
RTP packets to which the payload-specific message(s) apply. This is
to avoid that AVPF entities that do not understand the payload-
specific messages unnecessarily suppress their feedback messages.
6.3.1 Picture Loss Indication (PLI) 6.3.1 Picture Loss Indication (PLI)
The PLI feedback message is identified by PT=PSFB and FMT=1. The PLI feedback message is identified by PT=PSFB and FMT=1.
6.3.1.1 Semantics 6.3.1.1 Semantics
With the Picture Loss Indication message a decoder informs the With the Picture Loss Indication message, a decoder informs the
encoder about the loss of one or more full pictures. encoder about the loss of an undefined amount of coded video data
belonging to one or more pictures. When used in conjunction with
any video coding scheme that is based on inter-picture prediction,
an encoder that receives a PLI becomes aware that the prediction
chain may be broken. The sender MAY react to a PLI by transmitting
an intra-picture to achieve resynchronization (making effectively
similar to the FIR as defined in [10]); however, the sender MUST
consider congestion control as outlined in section 7 which MAY
restrict its ability to send an intra frame.
Other RTP payload specifications such as RFC 2032 [10] already
define a feedback mechanism for some for certain codecs. An
application supporting both schemes MUST use the feedback mechanism
defined in this specification when sending feedback. For backward
compatibility reasons, such an application SHOULD also be capable
to receive and react to the feedback scheme defined in the
respective RTP payload format, if this is required by that payload
format.
6.3.1.2 Message Format 6.3.1.2 Message Format
PLI does not require parameters. Therefore, the length field MUST be PLI does not require parameters. Therefore, the length field MUST
2, and there MUST NOT be any Feedback Control Information. be 2, and there MUST NOT be any Feedback Control Information.
6.3.1.3 Timing Rules 6.3.1.3 Timing Rules
The timing follows the rules outlined in section 3. In systems that The timing follows the rules outlined in section 3. In systems
employ both PLI and other types of feedback it may be advisable to that employ both PLI and other types of feedback it may be
follow the regular RTCP RR timing rules for PLI, since PLI is not as advisable to follow the regular RTCP RR timing rules for PLI, since
delay critical as other FB types. PLI is not as delay critical as other FB types.
6.3.1.4 Remarks 6.3.1.4 Remarks
PLI messages typically trigger the sending of full Intra pictures. PLI messages typically trigger the sending of full intra pictures.
Intra Pictures are several times larger then predicted (Inter) Intra pictures are several times larger then predicted (inter)
pictures. Their size is independent of the time they are generated. pictures. Their size is independent of the time they are
In most environments, especially when employing bandwidth-limited generated. In most environments, especially when employing
links, the use of an Intra picture implies an allowed delay that is a bandwidth-limited links, the use of an intra picture implies an
significant multitude of the typical frame duration. An example: If allowed delay that is a significant multitude of the typical frame
the sending frame rate is 10 fps, and an Intra picture is assumed to duration. An example: If the sending frame rate is 10 fps, and an
be 10 times as big as an Inter picture (not an unrealistic intra picture is assumed to be 10 times as big as an inter picture,
assumption, see [14] for details), then a full second of latency has then a full second of latency has to be accepted. In such an
to be accepted. In such an environment there is no need for a environment there is no need for a particular short delay in
particular short delay in sending the feedback message. Hence sending the feedback message. Hence waiting for the next possible
waiting for the next possible time slot allowed by RTCP timing rules time slot allowed by RTCP timing rules as per [2] does not have a
as per [2] does not have a negative impact on the system performance. negative impact on the system performance.
6.3.2 Slice Lost Indication (SLI) 6.3.2 Slice Lost Indication (SLI)
The SLI feedback message is identified by PT=PSFB and FMT=2. The SLI feedback message is identified by PT=PSFB and FMT=2.
Ott et al. Expires May 2002 [Page 25]
6.3.2.1 Semantics 6.3.2.1 Semantics
With the Slice Lost Indication a decoder can inform an encoder that With the Slice Lost Indication a decoder can inform an encoder that
it was unable to decode one, or several consecutive, macroblocks. it has detected the loss or corruption of one or several
The encoder can take appropriate action in order to re-synchronize consecutive macroblock(s) in scan order (see below). This feedback
encoder and decoder by means of its choice, typically by sending the message MUST NOT be used for video codecs with non-uniform,
lost macroblocks in Intra mode. This feedback message SHALL NOT be dynamically changeable macroblock sizes such as H.263 with enabled
used for video codecs with non-uniform, dynamically changeable Annex Q. In such a case, an encoder cannot always identify the
macroblock sizes such as H.263 with enabled Annex Q. In such a case, corrupted spatial region.
an encoder cannot always identify the corrupted spatial region.
6.3.2.2 Format 6.3.2.2 Format
When FBT indicates a Slice Lost Indication, then there is one The Slice Lost Indication uses one additional PCI field the
additional PCI field the content of which is depicted in figure 6. content of which is depicted in figure 6. The length of the
The length of the feedback message MUST be set to 3. feedback message MUST be set to 3.
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| First | Number | TR | | First | Number | PictureID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 6: Syntax of the Slice Lost Indication (SLI) Figure 6: Syntax of the Slice Lost Indication (SLI)
First: 13 bits First: 13 bits
The macroblock (MB) address of the first lost macroblock. The MB The macroblock (MB) address of the first lost macroblock. The
numbering is done such that the macroblock in the upper left MB numbering is done such that the macroblock in the upper left
corner of the picture is considered macroblock number 1 and the corner of the picture is considered macroblock number 1 and the
number for each macroblock increases from left to right and then number for each macroblock increases from left to right and
from top to bottom in raster-scan order (such that if there is a then from top to bottom in raster-scan order (such that if
total of N macroblocks in a picture, the bottom right macroblock there is a total of N macroblocks in a picture, the bottom
is considered macroblock number N). right macroblock is considered macroblock number N).
Number: 13 bits Number: 13 bits
The number of lost macroblocks, in scan order as discussed above. The number of lost macroblocks, in scan order as discussed
above.
TR: 6 bits PictureID: 6 bits
The six least significant bits of the Temporal Reference of the The six least significant bits of the a codec-specific
picture. identifier that is used to reference the picture in which the
loss of the macroblock (s) has occurred. For many video
codecs, the PictureID is identical to the Temporal Reference..
6.3.2.3 Timing Rules 6.3.2.3 Timing Rules
The efficiency of algorithms using the Slice Lost Indication is The efficiency of algorithms using the Slice Lost Indication is
reduced greatly when the Indication is not transmitted in a timely reduced greatly when the Indication is not transmitted in a timely
fashion. Motion compensation propagates corrupted pixels that are fashion. Motion compensation propagates corrupted pixels that are
not reported as being corrupted. Therefore, the use of the algorithm not reported as being corrupted. Therefore, the use of the
discussed in section 3 is highly recommended. algorithm discussed in section 3 is highly recommended.
Ott et al. Expires May 2002 [Page 26]
6.3.2.4 Remarks 6.3.2.4 Remarks
The First field of the UCI defines the first macroblock of a picture The term Slice is defined and used here in the sense of MPEG-1 -- a
as 1 and not, as one could suspect, as 0. This was done to align consecutive number of macroblocks in scan order. More recent video
this specification with the comparable mechanism available in H.245. coding standards sometimes have a different understanding of the
The maximum number of macroblocks in a picture (2**13 or 8192) term Slice. In H.263 (1998), for example, a concept known as
corresponds to the maximum picture sizes of the ITU-T and ISO/IEC "rectangular Slice" exist. The loss of one Rectangular Slice may
video codecs. If future video codecs offer larger picture sizes lead to the necessity of sending more than one SLI in order to
and/or smaller macroblock sizes, then an additional feedback message precisely identify the region of lost/damaged MBs.
has to be defined. The six least significant bits of the Temporal
Reference field are deemed to be sufficient to indicate the picture
in which the loss occurred.
Algorithms were reported that keep track of the regions effected by The first field of the FCI defines the first macroblock of a
picture as 1 and not, as one could suspect, as 0. This was done to
align this specification with the comparable mechanism available in
H.245. The maximum number of macroblocks in a picture (2**13 or
8192) corresponds to the maximum picture sizes of most of the ITU-T
and ISO/IEC video codecs. If future video codecs offer larger
picture sizes and/or smaller macroblock sizes, then an additional
feedback message has to be defined. The six least significant bits
of the Temporal Reference field are deemed to be sufficient to
indicate the picture in which the loss occurred.
The reaction to a SLI is not part of this specification. One
typical way of reacting to a SLI is to use intra refresh for the
affected spatial region.
Algorithms were reported that keep track of the regions affected by
motion compensation, in order to allow for a transmission of Intra motion compensation, in order to allow for a transmission of Intra
macroblocks to all those areas, regardless of the timing of the FB macroblocks to all those areas, regardless of the timing of the FB
(see H.263 (2000) Appendix I [13]] and [15]. While, when those (see H.263 (2000) Appendix I [13] and [15]). While, when those
algorithms are used, the timing of the FB is less critical then algorithms are used, the timing of the FB is less critical then
without, it has to be observed that those algorithms correct large without, it has to be observed that those algorithms correct large
parts of the picture and, therefore, have to transmit many for bits parts of the picture and, therefore, have to transmit much higher
in case of delayed FBs. data volume in case of delayed FBs.
6.3.3 Reference Picture Selection Indication (RPSI) 6.3.3 Reference Picture Selection Indication (RPSI)
The RPSI feedback message is identified by PT=PSFB and FMT=3. The RPSI feedback message is identified by PT=PSFB and FMT=3.
6.3.3.1 Semantics 6.3.3.1 Semantics
Modern video coding standards such as MPEG-4 visual version 2 [12] or Modern video coding standards such as MPEG-4 visual version 2 [12]
H.263 version 2 [13] allow the use of older reference pictures then or H.263 version 2 [13] allow to use older reference pictures than
the most recent one. Typically, a first-in-first-out queue of the most recent one for predictive coding. Typically, a first-in-
reference pictures is maintained. If an encoder has learned about a first-out queue of reference pictures is maintained. If an encoder
loss of encoder-decoder synchronicity, a known-as-correct reference has learned about a loss of encoder-decoder synchronicity, a known-
picture can be used. As this reference picture is temporally further as-correct reference picture can be used. As this reference picture
away then usual, the resulting predictively coded picture will use is temporally further away then usual, the resulting predictively
more bits. coded picture will use more bits.
Both MPEG-4 and H.263 define a binary format for the ŰpayloadŲ of an Both MPEG-4 and H.263 define a binary format for the "payload" of
RPSI message that includes information such as the temporal ID of the an RPSI message that includes information such as the temporal ID
damaged picture and the size of the damaged region. This bit string of the damaged picture and the size of the damaged region. This
is typically small Ż- a couple of dozen bits -Ż, of variable length, bit string is typically small -- a couple of dozen bits --, of
and self-contained, i.e. contains all information that is necessary variable length, and self-contained, i.e. contains all information
to perform reference picture selection. that is necessary to perform reference picture selection.
Note that both MPEG-4 and H.263 allow the use of RPSI with positive Note that both MPEG-4 and H.263 allow the use of RPSI with positive
feedback information as well. That is, all corrected pictures are feedback information as well. That is, pictures (or Slices) are
reported. Any form of positive feedback MUST NOT be used when in a reported that were decoded without error. Note that any form of
multicast environment (reporting positive feedback about individual positive feedback MUST NOT be used when in a multicast environment
reference pictures at RTCP intervals is not expected to be of much (reporting positive feedback about individual reference pictures at
use anyway). For point-to-point communication, positive feedback MAY RTCP intervals is not expected to be of much use anyway).
be used but, again, the bit rate budget of RTCP feedback will prevent
the use in most scenarios anyway.
Ott et al. Expires May 2002 [Page 27]
6.3.3.2 Format 6.3.3.2 Format
When FB indicates an RPSI, then the length field is set to the number The FCI for the RPSI message follows the format depicted in figure
of bits of the following bit string that contains the RPS 7:
information. This bit string follows byte aligned in the UCI field.
Bit padding is used to achieve 32-bit word alignment of the UCI 0 1 2 3
message (and the whole packet). 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PB | Native RPSI bit string defined per codec |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | Padding (0)|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 7: Syntax of the Reference Picture Selection Indication
(RPSI)
PB: 8 bits
The number of unused bits required to pad the length of the
RPSI message to a multiple of 32 bits.
Native RPSI bit string: variable length
The RPSI information as natively defined by the video codec.
Padding: #PB bits
A number of bits set to zero to fill up the contents of the
RPSI message to the next 32 bit boundary. The number of
padding bits MUST be indicated by the PB field.
6.3.3.3 Timing Rules 6.3.3.3 Timing Rules
RPS is even more critical to delay then algorithms using SLI. This RPS is even more critical to delay then algorithms using SLI. This
is due to the fact that the older the RPS message is, the more bits is due to the fact that the older the RPS message is, the more bits
the encoder has to spend to achieve encoder-decoder synchronicity. the encoder has to spend to re-establish encoder-decoder
See [14] and [15] for some information about the overhead of RPS for synchronicity. See [15] for some information about the overhead of
certain bit rate/frame rate/loss rate scenarios. RPS for certain bit rate/frame rate/loss rate scenarios.
Therefore, RPS messages should typically be sent as soon as possible, Therefore, RPS messages should typically be sent as soon as
employing the algorithm of section 3. possible, employing the algorithm of section 3.
6.4 Application Layer Feedback Messages 6.4 Application Layer Feedback Messages
Payload-Specific Feedback Messages are a special case of payload- Payload-Specific Feedback Messages are a special case of payload-
specific messages and identified by PT=PSFB and FMT=15. specific messages and identified by PT=PSFB and FMT=15.
These messages are used to transport application defined data These messages are used to transport application defined data
directly from the receiver's to the sender's application. The data directly from the receiver's to the sender's application. The data
that is transported is not identified by the feedback message. that is transported is not identified by the feedback message.
Therefore the application must be able to identify the messages Therefore, the application MUST be able to identify the messages
payload. payload.
Usually applications define their own set of messages, e.g. NEWPRED Usually, applications define their own set of messages, e.g.
messages in MPEG-4 or feedback messages in H.263/Annex N,U. These NEWPRED messages in MPEG-4 or feedback messages in H.263/Annex N,
messages do not need any additional information from the RTCP U. These messages do not need any additional information from the
message. Thus the application message is simply placed into the FCI RTCP message. Thus the application message is simply placed into
field as follows and the length field is set accordingly. the FCI field as follows and the length field is set accordingly.
Application Message (FCI): variable length Application Message (FCI): variable length
This field contains the original application message that should This field contains the original application message that
be transported from the receiver to the source. The format is should be transported from the receiver to the source. The
application dependent. The length of this field is variable. If format is application dependent. The length of this field is
the application data is not four-byte aligned, padding must be variable. If the application data is not byte aligned, padding
added. bits must be added. Identification of padding bits is up to
the application layer and not defined in this specification.
7. Early Feedback and Congestion Control 7. Early Feedback and Congestion Control
In the previous sections, the feedback messages were defined as well In the previous sections, the feedback messages were defined as
as the timing rules according to which to send these messages. The well as the timing rules according to which to send these messages.
way to react to the feedback received depends on the application The way to react to the feedback received depends on the
using the feedback mechanisms and hence is beyond the scope of this application using the feedback mechanisms and hence is beyond the
document. scope of this document.
Ott et al. Expires May 2002 [Page 28]
However, across all applications, there is a common requirement for However, across all applications, there is a common requirement for
(TCP-friendly) congestion control on the media stream as defined in (TCP-friendly) congestion control on the media stream as defined in
[1] and [2] when operating in a best-effort network environment. [1] and [2] when operating in a best-effort network environment.
Low delay feedback supports the use of congestion control algorithms Low delay feedback supports the use of congestion control
in two ways: algorithms in two ways:
. The potentially more frequent RTCP messages allow the sender to o The potentially more frequent RTCP messages allow the sender to
monitor the network state more closely than with regular RTCP monitor the network state more closely than with regular RTCP
and therefore enable reacting to upcoming congestion in a more and therefore enable reacting to upcoming congestion in a more
timely fashion. timely fashion.
. The feedback messages themselves may convey additional o The feedback messages themselves may convey additional
information as input to congestion control algorithms and thus information as input to congestion control algorithms and thus
improve reaction over conventional RTCP. (For example, ACK-based improve reaction over conventional RTCP. (For example, ACK-based
feedback may even allow to construct closed loop algorithms and feedback may even allow to construct closed loop algorithms and
NACK-based systems may provide further information on the packet NACK-based systems may provide further information on the packet
loss distribution.) loss distribution.)
A congestion control algorithm that shares the available bandwidth A congestion control algorithm that shares the available bandwidth
fair with competing TCP connections, e.g. TFRC [16], SHOULD be used fair with competing TCP connections, e.g. TFRC [16], SHOULD be used
to determine the data rate for the media stream (if the low delay RTP to determine the data rate for the media stream (if the low delay
session is transmitted in a best effort environment). RTP session is transmitted in a best effort environment).
RTCP feedback messages or RTCP SR/RR packets that indicate recent RTCP feedback messages or RTCP SR/RR packets that indicate recent
packet loss MUST NOT lead to a (mid-term) increase in the packet loss MUST NOT lead to a (mid-term) increase in the
transmission data rate and SHOULD lead to a (short-term) decrease of transmission data rate and SHOULD lead to a (short-term) decrease
the transmission data rate. Such messages SHOULD cause the sender to of the transmission data rate. Such messages SHOULD cause the
adjust the transmission data rate to the order of the throughput TCP sender to adjust the transmission data rate to the order of the
would achieve under similar conditions (e.g. using TFRC). throughput TCP would achieve under similar conditions (e.g. using
TFRC).
RTCP feedback messages or RTCP SR/RR packets that indicate no recent RTCP feedback messages or RTCP SR/RR packets that indicate no
packet loss MAY cause the sender to increase the transmission data recent packet loss MAY cause the sender to increase the
rate to roughly the throughput TCP would achieve under similar transmission data rate to roughly the throughput TCP would achieve
conditions (e.g. using TFRC). under similar conditions (e.g. using TFRC).
8. Security Considerations 8. Security Considerations
RTP packets transporting information with the proposed payload for RTP packets transporting information with the proposed payload
mat are subject to the security considerations discussed in the RTP format are subject to the security considerations discussed in the
specification [1] and in the RTP/AVP profile specification [2]. RTP specification [1] and in the RTP/AVP profile specification [2].
This profile does not specify any different security services. This profile does not specify any additional security services.
This profile modifies the timing behavior of RTCP and eliminates the
minimum RTCP interval of 5 seconds and allows for earlier feedback to
be provided by receivers. This approach does not increase the
potential for denial-of-service attacks beyond those discussed in [1]
and [2].
Feedback information is suppressed if unknown RTCP feedback packets This profile modifies the timing behavior of RTCP and eliminates
are received. This introduces the risk of a malicious group member the minimum RTCP interval of 5 seconds and allows for earlier
eliminating all early feedback by simply transmitting payload- feedback to be provided by receivers. Group members of the
specific RTCP feedback packets with random contents that are neither associated RTP session (possibly pretending to represent a large
number of entities) may disturb the operation of RTCP by sending
large numbers of RTCP packets thereby reducing the RTCP bandwidth
available for regular RTCP reporting as well as for early feedback
messages. (Note that an entity need not be member of a multicast
group to cause these effects.)
Ott et al. Expires May 2002 [Page 29] Feedback information may be suppressed if unknown RTCP feedback
recognized by any receiver (so they will suppress feedback) nor by packets are received. This introduces the risk of a malicious
the sender (so no repair actions will be taken). group member reducing early feedback by simply transmitting
payload-specific RTCP feedback packets with random contents that
are neither recognized by any receiver (so they will suppress
feedback) nor by the sender (so no repair actions will be taken).
A malicious group member can also report arbitrary high loss rates in A malicious group member can also report arbitrary high loss rates
the feedback information to make the sender throttle the data in the feedback information to make the sender throttle the data
transmission and increase the amount of redundancy information or transmission and increase the amount of redundancy information or
take other action to deal with the pretended packet loss. This may take other action to deal with the pretended packet loss (e.g. send
result in a degradation of the quality of the reproduced media fewer frames or decrease audio/video quality). This may result in
stream. a degradation of the quality of the reproduced media stream.
Finally, a malicious group member can act as a large number of group
members and thereby obtain an artificially large share of the early Finally, a malicious group member can act as a large number of
feedback bandwidth and reduce the reactivity of the other group group members and thereby obtain an artificially large share of the
members -- possibly even causing them to no longer operate in early feedback bandwidth and reduce the reactivity of the other
group members -- possibly even causing them to no longer operate in
immediate or early feedback mode and thus undermining the whole immediate or early feedback mode and thus undermining the whole
purpose of this profile. purpose of this profile.
Senders as well as receivers SHOULD behave conservative when
observing strange reporting behavior. For excessive failure
reporting from one or a few receivers, the sender MAY decide to no
longer consider this feedback when adapting its transmission
behavior for the media stream. In any case, senders and receivers
SHOULD still adhere to the maximum RTCP bandwidth but make sure
that they are capable of transmitting at least regularly scheduled
RTCP packets. Senders SHOULD carefully consider how to adjust
their transmission bandwidth when encountering strange reporting
behavior; they MUST NOT increase their transmission bandwidth even
if ignoring suspicious feedback.
Attacks using false RTCP packets (regular as well as early ones)
can be avoided by authenticating all RTCP messages. This can be
achieved by using the AVPF profile together with the Secure RTP
profile as defined in [17].
9. IANA Considerations 9. IANA Considerations
The feedback profile as an extension to the profile for audio-visual The feedback profile as an extension to the profile for audio-
conferences with minimal control needs to be registered: "RTP/AVPF". visual conferences with minimal control needs to be registered:
"RTP/AVPF".
For the Session Description Protocol, the following "fmtp:" attribute For the Session Description Protocol, the following "fmtp:"
needs to be registered: "rtcp-fb". attribute needs to be registered: "rtcp-fb".
Along with "rtcp-fb", the feedback types "ack" and "nack" need to be Along with "rtcp-fb", the feedback types "ack" and "nack" need to
registered. be registered.
Along with "nack", the feedback type parameters "sli", "pli", and Along with "nack", the feedback type parameters "sli" and "pli"
"rpsi" need to be registered. need to be registered.
Along with "ack" and "nack", the feedback type parameters "rpsi"
and "app" need to be registered.
Two RTCP Control Packet Types: for the class of transport layer Two RTCP Control Packet Types: for the class of transport layer
feedback messages ("RTPFB") and for the class of payload-specific feedback messages ("RTPFB") and for the class of payload-specific
feedback messages ("PSFB"). feedback messages ("PSFB"). Section 6 suggests RTPFB=205 and
PSFB=206 to be added to the RTCP registry.
Within the RTPFB range, three format (FMT) values need to be Within the RTPFB range, three format (FMT) values need to be
registered: registered:
0: forbidden 0: forbidden
1: General NACK 1: General NACK
2: General ACK 2: General ACK
Within the PSFB range, five format (FMT) values need to be Within the PSFB range, five format (FMT) values need to be
registered: registered:
0: forbidden 0: forbidden
1: Picture Loss Indication (PLI) 1: Picture Loss Indication (PLI)
2: Slice Loss Indication (SLI) 2: Slice Loss Indication (SLI)
3: Reference Picture Selection Indication (SLI) 3: Reference Picture Selection Indication (SLI)
15: Application layer feedback (AFB) 15: Application layer feedback (AFB)
Ott et al. Expires May 2002 [Page 30]
10. Acknowledgements 10. Acknowledgements
This document is a product of the Audio-Visual Transport (AVT) This document is a product of the Audio-Visual Transport (AVT)
Working Group of the IETF. The authors would like to thank Steve Working Group of the IETF. The authors would like to thank Steve
Casner and Colin Perkins for their comments and suggestions as well Casner and Colin Perkins for their comments and suggestions as well
as for their responsiveness to numerous questions. as for their responsiveness to numerous questions.
11. Full Copyright Statement 11. Full Copyright Statement
Copyright (C) The Internet Society (2001). All Rights Reserved. Copyright (C) The Internet Society (2001). All Rights Reserved.
skipping to change at line 1607 skipping to change at page 37, line 36
10. Acknowledgements 10. Acknowledgements
This document is a product of the Audio-Visual Transport (AVT) This document is a product of the Audio-Visual Transport (AVT)
Working Group of the IETF. The authors would like to thank Steve Working Group of the IETF. The authors would like to thank Steve
Casner and Colin Perkins for their comments and suggestions as well Casner and Colin Perkins for their comments and suggestions as well
as for their responsiveness to numerous questions. as for their responsiveness to numerous questions.
11. Full Copyright Statement 11. Full Copyright Statement
Copyright (C) The Internet Society (2001). All Rights Reserved. Copyright (C) The Internet Society (2001). All Rights Reserved.
This document and translations of it may be copied and furnished to This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it others, and derivative works that comment on or otherwise explain
or assist in its implementation may be prepared, copied, published it or assist in its implementation may be prepared, copied,
and distributed, in whole or in part, without restriction of any published and distributed, in whole or in part, without restriction
kind, provided that the above copyright notice and this paragraph are of any kind, provided that the above copyright notice and this
included on all such copies and derivative works. paragraph are included on all such copies and derivative works.
However, this document itself may not be modified in any way, such as However, this document itself may not be modified in any way, such
by removing the copyright notice or references to the Internet Soci- as by removing the copyright notice or references to the Internet
ety or other Internet organizations, except as needed for the purpose Society or other Internet organizations, except as needed for the
of developing Internet standards in which case the procedures for purpose of developing Internet standards in which case the
copyrights defined in the Internet Standards process must be fol- procedures for copyrights defined in the Internet Standards process
lowed, or as required to translate it into languages other than must be followed, or as required to translate it into languages
English. other than English.
The limited permissions granted above are perpetual and will not be The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns. revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an This document and the information contained herein is provided on
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MER- THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
CHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE." WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE."
12. Authors' Addresses 12. Authors' Addresses
Jųrg Ott {sip,mailto}:jo@tzi.org Joerg Ott {sip,mailto}:jo@tzi.org
UniversitĄt Bremen TZI Uni Bremen TZI
MZH 5180 MZH 5180
Bibliothekstr. 1 Bibliothekstr. 1
D-28359 Bremen D-28359 Bremen
Germany Germany
Stephan Wenger stewe@cs.tu-berlin.de Stephan Wenger stewe@cs.tu-berlin.de
TU Berlin TU Berlin
Sekr. FR 6-3 Sekr. FR 6-3
Franklinstr. 28-29 Franklinstr. 28-29
D-10587 Berlin D-10587 Berlin
Germany Germany
Ott et al. Expires May 2002 [Page 31]
Shigeru Fukunaga Shigeru Fukunaga
Oki Electric Industry Co., Ltd. Oki Electric Industry Co., Ltd.
1-2-27 Shiromi, Chuo-ku, Osaka 540-6025 Japan 1-2-27 Shiromi, Chuo-ku, Osaka 540-6025 Japan
Tel. +81 6 6949 5101 Tel. +81 6 6949 5101
Fax. +81 6 6949 5108 Fax. +81 6 6949 5108
Mail fukunaga444@oki.com Mail fukunaga444@oki.com
Noriyuki Sato Noriyuki Sato
Oki Electric Industry Co., Ltd. Oki Electric Industry Co., Ltd.
1-2-27 Shiromi, Chuo-ku, Osaka 540-6025 Japan 1-2-27 Shiromi, Chuo-ku, Osaka 540-6025 Japan
skipping to change at line 1700 skipping to change at page 39, line 34
Carsten Burmeister Carsten Burmeister
Panasonic European Laboratories GmbH Panasonic European Laboratories GmbH
Monzastr. 4c, 63225 Langen, Germany Monzastr. 4c, 63225 Langen, Germany
Tel. +49-(0)6103-766-263 Tel. +49-(0)6103-766-263
Fax. +49-(0)6103-766-166 Fax. +49-(0)6103-766-166
Mail burmeister@panasonic.de Mail burmeister@panasonic.de
11. Bibliography 11. Bibliography
[1] H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, "RTP - [1] H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, "RTP
A Transport Protocol for Real-time Applications," Internet - A Transport Protocol for Real-time Applications," Internet
Draft, draft-ietf-avt-rtp-new-10.txt, Work in Progress, July Draft, draft-ietf-avt-rtp-new-11.txt, Work in Progress,
2001. November 2001.
Ott et al. Expires May 2002 [Page 32]
[2] H. Schulzrinne and S. Casner, "RTP Profile for Audio and Video [2] H. Schulzrinne and S. Casner, "RTP Profile for Audio and Video
Conferences with Minimal Control," Internet Draft draft-ietf- Conferences with Minimal Control," Internet Draft draft-ietf-
avt-profile-new-11.txt, July 2001. avt-profile-new-12.txt, November 2001.
[3] M. Handley and V. Jacobson, "SDP: Session Description Protocol", [3] M. Handley and V. Jacobson, "SDP: Session Description
RFC 2327, April 1998. Protocol", RFC 2327, April 1998.
[4] S. Casner, "SDP Bandwidth Modifiers for RTCP Bandwidth", [4] S. Casner, "SDP Bandwidth Modifiers for RTCP Bandwidth",
Internet Draft draft-ietf-avt-rtcp-bw-03.txt, July 2001. Internet Draft draft-ietf-avt-rtcp-bw-05.txt, November 2001.
[5] C. Perkins and O. Hodson, "2354 Options for Repair of Streaming [5] C. Perkins and O. Hodson, "2354 Options for Repair of
Media," RFC 2354, June 1998. Streaming Media," RFC 2354, June 1998.
[6] J. Rosenberg and H. Schulzrinne, "An RTP Payload Format for [6] J. Rosenberg and H. Schulzrinne, "An RTP Payload Format for
Generic Forward Error Correction,", RFC 2733, December 1999. Generic Forward Error Correction,", RFC 2733, December 1999.
[7] C. Perkins, I. Kouvelas, O. Hodson, V. Hardman, M. Handley, J.C. [7] C. Perkins, I. Kouvelas, O. Hodson, V. Hardman, M. Handley,
Bolot, A. Vega-Garcia, and S. Fosse-Parisis, "RTP Payload for J.C. Bolot, A. Vega-Garcia, and S. Fosse-Parisis, "RTP Payload
Redundant Audio Data," RFC 2198, September 1997. for Redundant Audio Data," RFC 2198, September 1997.
[8] S. Bradner, "Key words for use in RFCs to Indicate Requirement [8] S. Bradner, "Key words for use in RFCs to Indicate Requirement
Levels," RFC 2119, March 1997. Levels," RFC 2119, March 1997.
[9] H. Schulzrinne and S. Petrack, "RTP Payload for DTMF Digits, [9] H. Schulzrinne and S. Petrack, "RTP Payload for DTMF Digits,
Telephony Tones and Telephony Signals," RFC 2833, May 2000. Telephony Tones and Telephony Signals," RFC 2833, May 2000.
[10] T. Turletti and C. Huitema, "RTP Payload Format for H.261 Video [10] T. Turletti and C. Huitema, "RTP Payload Format for H.261
Streams, RFC 2032, October 1996. Video Streams, RFC 2032, October 1996.
[11] C. Bormann, L. Cline, G. Deisher, T. Gardos, C. Maciocco, D. [11] C. Bormann, L. Cline, G. Deisher, T. Gardos, C. Maciocco, D.
Newell, J. Ott, G. Sullivan, S. Wenger, and C. Zhu, "RTP Payload Newell, J. Ott, G. Sullivan, S. Wenger, and C. Zhu, "RTP
Format for the 1998 Version of ITU-T Rec. H.263 Video (H.263+)," Payload Format for the 1998 Version of ITU-T Rec. H.263 Video
RFC 2429, October 1998. (H.263+)," RFC 2429, October 1998.
[12] ISO/IEC 14496-2:1999/Amd.1:2000, "Information technology - [12] ISO/IEC 14496-2:1999/Amd.1:2000, "Information technology -
Coding of audio-visual objects - Part2: Visual", July 2000. Coding of audio-visual objects - Part2: Visual", July 2000.
[13] ITU-T Recommendation H.263, "Video Coding for Low Bit Rate [13] ITU-T Recommendation H.263, "Video Coding for Low Bit Rate
Communication," November 2000. Communication," November 2000.
[14] S. Wenger, "Media-aware Protocols -- transport aware Media [14] G. Camarillo, J. Holler, G. Eriksson, H. Schulzrinne,
Coding," Habilitation thesis, in preparation, 2001. "Grouping of media lines in SDP," Internet Draft, draft-ietf-
mmusic-fid-05.txt, Work in Progress, September 2001.
[15] B. Girod, N. Faerber, "Feedback-based error control for mobile [15] B. Girod, N. Faerber, "Feedback-based error control for mobile
video transmission," Proceedings IEEE, Vol. 87, No. 10, pp. 1707 video transmission," Proceedings IEEE, Vol. 87, No. 10, pp.
Ż 1723, October, 1999. 1707 - 1723, October, 1999.
[16] M. Handley, J. Padhye, S. Floyd, J. Widmer, "TCP friendly Rate [16] M. Handley, J. Padhye, S. Floyd, J. Widmer, "TCP friendly Rate
Control (TFRC): Protocol Specification," Internet Draft, draft- Control (TFRC): Protocol Specification," Internet Draft,
ietf-tsvwg-02.txt, Work in Progress, May 2001. draft-ietf-tsvwg-tfrc-03.txt, Work in Progress, July 2001.
Ott et al. Expires May 2002 [Page 33]
Appendix A. Some Background and Motivation (Informative)
A.1 Example: Predictive Video Coding
A.1.1 Video Encoder-decoder synchronicity
Most current video coding schemes for compressed video, such as the
ITU-T H.261 and H.263 and ISO/IEC MPEG[124] employ a mechanism known
as Inter Picture Prediction. Each picture is divided into
macroblocks of uniform size. For each macroblock, one or more
motion vectors may be identified and transmitted. The residual
signal after motion compensation is DCT-transformed, quantized,
entropy coded, and transmitted as well. The encoder reconstructs,
based on this information, a so-called reference picture, which is
used to perform the motion compensation and residual signal coding
steps for the subsequent picture. Since the reference picture is
generated using only such information that is also available at the
decoder, the reference picture is identical to the reconstructed
picture at the decoder. Having identical reference pictures at the
encoder and decoder is referred to as encoder-decoder-synchronicity.
Whenever data is damaged or lost on the way between the encoder and
the decoder, the reconstructed picture at the decoder is no more
identical with the encoder's reference picture -- the encoder-decoder
synchronicity is lost.
Any loss of the encoder-decoder synchronicity results in annoying
artifacts at the decoder. Because the prediction of subsequent
pictures in the decoder is based on a damaged reference picture, the
annoying artifacts are present not only in the picture in which the
loss occurred; they propagate to all subsequent pictures, until,
through source coding based mechanisms, the encoder-decoder
synchronicity is restored. Therefore, the goal of systems employing
predictive video coding in a lossy environment must be to keep the
encoder-decoder synchronicity, or, if this is not possible, to regain
that synchronicity as quickly as possible.
A.1.2. Non-feedback based mechanisms
Avoiding the loss of the encoder-decoder synchronicity corresponds to
avoiding the loss of coded picture data. Such a task can be
performed on the transport layer. In RTP environments, the use of
packet-based FEC is a good example for such a technique. (The use of
TCP or reliable multicast as the transport for media streams would be
an even better one but is inappropriate for low-delay (interactive)
real-time systems.) FEC schemes, interleaving, and other means for
repairing real-time media streams may also add additional delay and
significant bit rate overhead without being able to guarantee
compensation of virtually all packet losses.
Once the encoder-decoder synchronicity is lost, only source coding
oriented mechanisms can help to regain it. One common way is to send
a non-predictively coded picture (known as Intra picture). Intra
pictures have the disadvantage of being several times bigger than
Ott et al. Expires May 2002 [Page 34]
predictively coded pictures (Inter pictures). Therefore, sending
Intra pictures has negative implications both on the bandwidth and
(in bandwidth limited environments) delay. Another way is to use
Intra macroblock refresh. Here, certain parts of the picture (those
affected by a packet loss) are coded non-predictively in order to
resynchronize the encoder and decoder over time. Intra macroblock
refresh has better delay characteristics then full Intra pictures
because the picture size can be kept constant, but is less efficient
in terms of bit rate/distortion than full Intra pictures. More
sophisticated means such as Reference Picture Selection (RPS) are
also available in modern video coding standards.
Systems not employing feedback channels may use any combination of
the mechanisms described above to add error resilience -- at the cost
of added bit rate and, sometimes, added delay. The number of
additional bits spent for error resilience can be adapted using the
long-term packet loss rate information in the RTCP receiver reports.
But, even when using such adaptive means, it is still likely that
systems spend many more bits then theoretically necessary to achieve
error resilience in order to be on the safe side. Plus, as regular
RTCP feedback is aimed at longer terms, reactivity to sudden losses
is limited. In all practical applications today this means that
fewer bits are available for non redundant picture data, and hence
the overall picture quality suffers.
A.1.3 Feedback based systems
Feedback-based systems try to avoid spending too many bits for
redundant information by informing the encoder about a loss situation
at the decoder(s). The encoder can then react accordingly and spend
redundant bits only when needed possibly only for the part of the
picture that was effected by the loss -- thereby reducing the number
of redundant bits and leaving more bits for useful information. As a
result, a higher reproduced picture quality can generally be expected
when feedback channels are available.
Similar to the observations of section 2.1.2, transport and source
coding based mechanisms can be distinguished that react on loss
situations reported by feedback.
Transport based systems employing feedback react media unaware, by
re-transmitting lost packets. TCP is a good example for a protocol
following such a scheme. Transport-based feedback in real-time
and/or multicast environments is a complex matter and subject of a
lot of engineering and research in and outside of the IETF. This
specification is not concerned with pure transport-based feedback.
Source coding based mechanisms may react upon the arrival of a
feedback message indicating a loss situation by adding bits that
restore, or at least make an effort to restore, the encoder-decoder
synchronicity. This process has to be performed by a real-time
encoder. However, schemes were reported, that allow the use of
feedback also for non-real-time encoders by storing multiple
Ott et al. Expires May 2002 [Page 35]
representations of the same data (e.g. Inter and Intra coded), and
dynamically switching between those representations.
Several types of feedback messages, called Feedback Messages or FB
messages, can be defined for such a case. An FB message can be as
simple as a Boolean condition, indicating for example the loss of a
full picture (and, therefore, the need of a full Intra picture
transmission). Other feedback messages may contain more complex
information such as information about the damage of a spatial region
of the picture. A special form consists of a message the format and
semantics of which are not known at the transport level, because they
are defined in the video codec standards.
A.2 Feedback Messages
Most FB messages contain negative acknowledge information, indicating
an erroneous situation at the decoder. In others, the nature of the
acknowledge (positive, negative, or both) is part of the feedback
message itself. When used in multicast environments, positive
acknowledge must not be used.
This document assumes that feedback messages are transmitted using
RTCP packets. RTCP messages from the receivers to the sender cannot
be sent at any possible time, in order to prevent traffic explosion
in case of large multicast groups. Instead, the bit rate for all
RTCP messages of all receivers together has to obey a maximum
fraction of the total RTP session bit rate, yielding a very limited
bit rate budget for a single receiver when having a large multicast
group. This, in turn, leads to an increased average delay when the
size of the receiving multicast group grows. (see section 6 of [1]
for details)
This specification defines an algorithm that adheres to the bit rate
limitations for the feedback channel on the long term, but allows
short-term overdrafting for any receiver (but not all of them
simultaneously). Thus, the algorithm allows for better real-time
performance then the one specified in [1]. Traffic explosion in such
cases in which many receivers identify a picture damage
simultaneously is prevented by dithering.
As this specification assumes a sender that has full control over its
transmission bit rate (e.g. a real-time encoder), there is no scaling
problem on the forward channel. Any reaction to negative feedback
generates additional bits, which have to be conveyed but this is
taken from the sender∆s total bit rate budget. The encoder can take
this into account by, for example, changing the encoding mode, packet
size, and so forth. The sender is also free to simply ignore
feedback messages. Adjusting the tradeoff between the reproduced
media quality of all receivers of a multicast group and the amount of
additional repair traffic is a media-dependent, very complex task and
is not covered in this specification.
Ott et al. Expires May 2002 [Page 36]
Finally, frequent RTCP-based feedback messages may provide additional
input to the sender(s)'s congestion control algorithms and thus
improve its reactivity towards network congestion.
Feedback messages as well as sender and receiver behavior are to be
specified in separate documents (such as [7]). Such specifications
need to consider that, frequently, packet loss is an indication of
network congestion and thus define mechanisms for media-specific
congestion control in the presence of feedback as defined in this
memo.
A.3. Applications and Relationships to other Standards
This specification is based on RTCP, which implies its use in an RTP
environment. RTP itself is used in a variety of systems such as in
SIP- or H.323-based multimedia conferencing/telephony, SAP-announced
Mbone conferences, and RTSP-based media streaming.
As for the video codecs, there is currently a small set of standards
that are, for the purpose of this discussion, roughly comparable.
Many mechanisms for regaining encoder-decoder synchronicity are
applicable to all video codecs. Others require certain tools (such
as Reference Picture Selection, aka NEWPRED) that are available only
in certain versions of the standards, and/or optional tools whose use
must be negotiated prior to being used.
A few RTP payload specifications such as RFC 2032 [10] already define
a feedback mechanism for some of the coding algorithms considered in
this specification. An application capable of performing both
schemes MUST use the feedback mechanism defined in this
specification, although, for backward compatibility reasons, it MUST
also be capable to conform to the feedback scheme defined in the
respective RTP payload format, if this is required by that payload
format.
Also, audio, DTMF, and text streams could benefit from more immediate
feedback even though the redundancy payload formats work well for
these media.
All kinds of non-interactive media streams (such as RTSP-controlled
media streaming applications) could benefit significantly as without
interactivity there is more time available for media repair.
A.4 Remarks on the size of the multicast group
This specification prevents traffic explosion on the feedback channel
in a very similar way as RTP does, with the exception of allowing
individual receivers to overdraft their bit rate budget from time to
time. This is necessary in order to allow for low delay, which is
needed by the algorithms reacting to Feedback messages.
This scaling, however, limits the usefulness of this mechanism in
multicast groups from a certain size upwards (where the size
Ott et al. Expires May 2002 [Page 37]
threshold depends on a number of parameters including loss rate,
frame rate, number of packets per frame, and session bandwidth). The
maximum size of the multicast group is soft and also depends on
application requirements and is therefore not specified here.
Considerations on the multicast group sizes are presented in section
3.5.
Ott et al. Expires May 2002 [Page 38] [17] M. Baugher, R. Blom, E. Carrarra, D. McGrew, M. Naslund, K.
Norrman, D. Oran, "The Secure Real-Time Transport Protocol,"
Internet Draft, draft-ietf-avt-srtp-02.txt, Work in Progress,
November 2001.
 End of changes. 

This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/