draft-ietf-avt-avpf-ccm-03.txt   draft-ietf-avt-avpf-ccm-04.txt 
Network Working Group Stephan Wenger Network Working Group Stephan Wenger
INTERNET-DRAFT Umesh Chandra INTERNET-DRAFT Umesh Chandra
Expires: May 2007 Nokia Expires: May 2007 Nokia
Magnus Westerlund Magnus Westerlund
Bo Burman Bo Burman
Ericsson Ericsson
November 30, 2006 March 5, 2007
Codec Control Messages in the Codec Control Messages in the
RTP Audio-Visual Profile with Feedback (AVPF) RTP Audio-Visual Profile with Feedback (AVPF)
draft-ietf-avt-avpf-ccm-03.txt> draft-ietf-avt-avpf-ccm-04.txt>
Status of this Memo Status of this Memo
By submitting this Internet-Draft, each author represents that any By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of BCP 79. aware will be disclosed, in accordance with Section 6 of BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
skipping to change at page 1, line 40 skipping to change at page 1, line 40
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt. http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
Copyright Notice Copyright Notice
Copyright (C) The Internet Society (2006). Copyright (C) The IETF Trust (2007).
Abstract Abstract
This document specifies a few extensions to the messages defined in This document specifies a few extensions to the messages defined in
the Audio-Visual Profile with Feedback (AVPF). They are helpful the Audio-Visual Profile with Feedback (AVPF). They are helpful
primarily in conversational multimedia scenarios where centralized primarily in conversational multimedia scenarios where centralized
multipoint functionalities are in use. However some are also usable multipoint functionalities are in use. However some are also usable
in smaller multicast environments and point-to-point calls. The in smaller multicast environments and point-to-point calls. The
extensions discussed are H.271 video back channel, Full Intra extensions discussed are messages related to the ITU-T H.271 Video
Request, Temporary Maximum Media Bit-rate and Temporal Spatial Trade- Back Channel, Full Intra Request, Temporary Maximum Media Stream Bit-
off. rate and Temporal Spatial Trade-off.
TABLE OF CONTENTS TABLE OF CONTENTS
1. Introduction....................................................5 1. Introduction....................................................5
2. Definitions.....................................................7 2. Definitions.....................................................7
2.1. Glossary...................................................7 2.1. Glossary...................................................7
2.2. Terminology................................................8 2.2. Terminology................................................8
2.3. Topologies.................................................9 2.3. Topologies.................................................9
3. Motivation (Informative)........................................9 3. Motivation (Informative).......................................10
3.1. Use Cases.................................................10 3.1. Use Cases.................................................10
3.2. Using the Media Path......................................12 3.2. Using the Media Path......................................12
3.3. Using AVPF................................................12 3.3. Using AVPF................................................13
3.3.1. Reliability..........................................12 3.3.1. Reliability..........................................13
3.4. Multicast.................................................13 3.4. Multicast.................................................13
3.5. Feedback Messages.........................................13 3.5. Feedback Messages.........................................13
3.5.1. Full Intra Request Command...........................13 3.5.1. Full Intra Request Command...........................13
3.5.1.1. Reliability.....................................14 3.5.1.1. Reliability.....................................14
3.5.2. Temporal Spatial Trade-off Request and Announcement..15 3.5.2. Temporal Spatial Trade-off Request and Announcement..15
3.5.2.1. Point-to-point..................................15 3.5.2.1. Point-to-point..................................16
3.5.2.2. Point-to-Multipoint using Multicast or Translators16 3.5.2.2. Point-to-Multipoint using Multicast or Translators16
3.5.2.3. Point-to-Multipoint using RTP Mixer.............16 3.5.2.3. Point-to-Multipoint using RTP Mixer.............17
3.5.2.4. Reliability.....................................16 3.5.2.4. Reliability.....................................17
3.5.3. H.271 Video Back Channel Message conforming to ITU-T Rec. 3.5.3. H.271 Video Back Channel Message conforming to ITU-T Rec.
H.271.......................................................17 H.271.......................................................17
3.5.3.1. Reliability.....................................20 3.5.3.1. Reliability.....................................20
3.5.4. Temporary Maximum Media Bit-rate Request.............20 3.5.4. Temporary Maximum Media Bit-rate Request.............20
3.5.4.1. MCU based Multi-point operation.................21 3.5.4.1. MCU based Multi-point operation.................25
3.5.4.2. Point-to-Multipoint using Multicast or Translators22 3.5.4.2. Point-to-Multipoint using Multicast or Translators27
3.5.4.3. Point-to-point operation........................22 3.5.4.3. Point-to-point operation........................27
3.5.4.4. Reliability.....................................23 3.5.4.4. Reliability.....................................28
4. RTCP Receiver Report Extensions................................24 4. RTCP Receiver Report Extensions................................29
4.1. Design Principles of the Extension Mechanism..............24 4.1. Design Principles of the Extension Mechanism..............29
4.2. Transport Layer Feedback Messages.........................25 4.2. Transport Layer Feedback Messages.........................30
4.2.1. Temporary Maximum Media Bit-rate Request (TMMBR).....25 4.2.1. Temporary Maximum Media Bit-rate Request (TMMBR).....30
4.2.1.1. Semantics.......................................25 4.2.1.1. Semantics.......................................31
4.2.1.2. Message Format..................................27 4.2.1.2. Message Format..................................33
4.2.1.3. Timing Rules....................................28 4.2.1.3. Timing Rules....................................34
4.2.2. Temporary Maximum Media Bit-rate Notification (TMMBN) 28 4.2.2. Temporary Maximum Media Bit-rate Notification (TMMBN) 35
4.2.2.1. Semantics.......................................28 4.2.2.1. Semantics.......................................35
4.2.2.2. Message Format..................................29 4.2.2.2. Message Format..................................36
4.2.2.3. Timing Rules....................................30 4.2.2.3. Timing Rules....................................36
4.3. Payload Specific Feedback Messages........................30 4.3. Payload Specific Feedback Messages........................37
4.3.1. Full Intra Request (FIR) command.....................30 4.3.1. Full Intra Request (FIR) command.....................37
4.3.1.1. Semantics.......................................30 4.3.1.1. Semantics.......................................37
4.3.1.2. Message Format..................................32 4.3.1.2. Message Format..................................39
4.3.1.3. Timing Rules....................................33 4.3.1.3. Timing Rules....................................40
4.3.1.4. Remarks.........................................33 4.3.1.4. Remarks.........................................40
4.3.2. Temporal-Spatial Trade-off Request (TSTR)............34 4.3.2. Temporal-Spatial Trade-off Request (TSTR)............41
4.3.2.1. Semantics.......................................34 4.3.2.1. Semantics.......................................41
4.3.2.2. Message Format..................................34 4.3.2.2. Message Format..................................41
4.3.2.3. Timing Rules....................................35 4.3.2.3. Timing Rules....................................42
4.3.2.4. Remarks.........................................35 4.3.2.4. Remarks.........................................42
4.3.3. Temporal-Spatial Trade-off Announcement (TSTA).......36 4.3.3. Temporal-Spatial Trade-off Announcement (TSTA).......43
4.3.3.1. Semantics.......................................36 4.3.3.1. Semantics.......................................43
4.3.3.2. Message Format..................................36 4.3.3.2. Message Format..................................44
4.3.3.3. Timing Rules....................................37 4.3.3.3. Timing Rules....................................44
4.3.3.4. Remarks.........................................37 4.3.3.4. Remarks.........................................45
4.3.4. H.271 VideoBackChannelMessage (VBCM).................37 4.3.4. H.271 VideoBackChannelMessage (VBCM).................45
5. Congestion Control.............................................40 5. Congestion Control.............................................48
6. Security Considerations........................................41 6. Security Considerations........................................48
7. SDP Definitions................................................41 7. SDP Definitions................................................49
7.1. Extension of rtcp-fb attribute............................42 7.1. Extension of rtcp-fb attribute............................49
7.2. Offer-Answer..............................................43 7.2. Offer-Answer..............................................51
7.3. Examples..................................................43 7.3. Examples..................................................51
8. IANA Considerations............................................46 8. IANA Considerations............................................54
9. Acknowledgements...............................................47 9. Acknowledgements...............................................54
10. References....................................................48 10. References....................................................56
10.1. Normative references.....................................48 10.1. Normative references.....................................56
10.2. Informative references...................................48 10.2. Informative references...................................56
11. Authors' Addresses............................................49 11. Authors' Addresses............................................57
12. List of Changes relative to previous drafts...................49 12. List of Changes relative to previous draftsError! Bookmark not defined.
1. Introduction 1. Introduction
When the Audio-Visual Profile with Feedback (AVPF) [RFC4585] was When the Audio-Visual Profile with Feedback (AVPF) [RFC4585] was
developed, the main emphasis lied in the efficient support of point- developed, the main emphasis lay in the efficient support of point-
to-point and small multipoint scenarios without centralized to-point and small multipoint scenarios without centralized
multipoint control. However, in practice, many small multipoint multipoint control. However, in practice, many small multipoint
conferences operate utilizing devices known as Multipoint Control conferences operate utilizing devices known as Multipoint Control
Units (MCUs). Long standing experience of the conversational video Units (MCUs). Long standing experience of the conversational video
conferencing industry suggests that there is a need for a few conferencing industry suggests that there is a need for a few
additional feedback messages, to efficiently support MCU-based additional feedback messages, to efficiently support centralized
multipoint conferencing. Some of the messages have applications multipoint conferencing. Some of the messages have applications
beyond centralized multipoint, and this is indicated in the beyond centralized multipoint, and this is indicated in the
description of the message. This is especially true for the message description of the message. This is especially true for the message
intended to carry ITU-T Rec. H.271 [H.271] bitstrings for video back intended to carry ITU-T Rec. H.271 [H.271] bitstrings for Video Back
channel messages. Channel messages.
In RTP [RFC3550] terminology, MCUs comprise mixers and translators. In RTP [RFC3550] terminology, MCUs comprise mixers and translators.
Most MCUs also include signalling support. During the development of Most MCUs also include signaling support. During the development of
this memo, it was noticed that there is considerable confusion in the this memo, it was noticed that there is considerable confusion in the
community related to the use of terms such as mixer, translator, and community related to the use of terms such as mixer, translator, and
MCU. In response to these concerns, a number of topologies have been MCU. In response to these concerns, a number of topologies have been
identified that are of practical relevance to the industry, but were identified that are of practical relevance to the industry, but not
not envisioned (or at least not documented in sufficient detail) in documented in sufficient detail in RTP. These topologies are
RTP. These topologies are documented in [Topologies], and documented in [Topologies], and understanding this memo requires
understanding this memo requires previous or parallel study of previous or parallel study of [Topologies].
[Topologies].
Some of the messages defined here are forward only, in that they do Some of the messages defined here are forward only, in that they do
not require an explicit acknowledgement. Other messages require not require an explicit notification to the message emitter
acknowledgement, leading to a two way communication model that could indicating their reception and/or the message receiver's actions.
suggest to some to be useful for control purposes. It is not the Other messages require notification, leading to a two way
intention of this memo to open up RTCP to a generalized control communication model that could suggest to some to be useful for
protocol. All mentioned messages have relatively strict real-time control purposes. It is not the intention of this memo to open up
constraints -- in the sense that their value diminishes with RTCP to a generalized control protocol. All mentioned messages have
increased delay. This makes the use of more traditional control relatively strict real-time constraints -- in the sense that their
protocol means, such as SIP re-invites, undesirable. Furthermore, value diminishes with increased delay. This makes the use of more
all messages are of a very simple format that can be easily processed traditional control protocol means, such as SIP re-invites [RFC3261],
by an RTP/RTCP sender/receiver. Finally, all messages infer only to undesirable. Furthermore, all messages are of a very simple format
the RTP stream they are related to, and not to any other property of that can be easily processed by an RTP/RTCP sender/receiver.
a communication system. Finally, all messages infer only to the RTP stream they are related
to, and not to any other property of a communication system.
The Full Intra Request (FIR) Command requires the receiver of the The Full Intra Request (FIR) requires the receiver of the message
message (and sender of the stream) to immediately insert a decoder (and sender of the stream) to immediately insert a decoder refresh
refresh point. In video coding, one commonly used form of a decoder point. In video coding, one commonly used form of a decoder refresh
refresh point is an IDR or Intra picture. Other codecs may have point is an IDR or Intra picture, depending on the video compression
other forms of decoder refresh points. In order to fulfil congestion technology in use. Other codecs may have other forms of decoder
control constraints, sending a decoder refresh point may imply a refresh points. In order to fulfill congestion control constraints,
significant drop in frame rate, as they are commonly much larger than sending a decoder refresh point may imply a significant drop in frame
regular predicted content. The use of this message is restricted to rate, as they are commonly much larger than regular predicted
cases where no other means of decoder refresh can be employed, e.g. content. The use of this message is restricted to cases where no
during the join-phase of a new participant in a multipoint other means of decoder refresh can be employed, e.g. during the join-
conference. It is explicitly disallowed to use the FIR command for phase of a new participant in a multipoint conference. It is
error resilience purposes, and instead it is referred to AVPF's explicitly disallowed to use the FIR command for error resilience
[RFC4585] PLI message, which reports lost pictures and has been purposes, and instead it is referred to AVPF's [RFC4585] PLI message,
included in AVPF for precisely that purpose. The message does not which reports lost pictures and has been included in AVPF for
require an acknowledgement, as the presence of a decoder refresh precisely that purpose. The message does not require a reception
point can be easily derived from the media bit stream. Today, the notification, as the presence of a decoder refresh point can be
FIR message appears to be useful primarily with video streams, but in easily derived from the media bit stream. Today, the FIR message
the future it may become helpful also in conjunction with other media appears to be useful primarily with video streams, but in the future
codecs that support prediction across RTP packets. it may also prove helpful in conjunction with other media codecs that
support prediction across RTP packets.
The Temporary Maximum Media Bandwidth Request (TMMBR) Message allows The Temporary Maximum Media Stream Bitrate Request (TMMBR) allows to
to signal, from media receiver to media sender, the current maximum signal, from media receiver to media sender, the current maximum
supported media bit-rate for a given media stream. Once a bandwidth media stream bit-rate for a given media stream. The maximum media
limitation is established by the media sender, that sender notifies stream bit-rate is defined as a tuple. The first value is the bit-
the initiator of the request, and all other session participants, by rate available for the packet stream at the layer reported on. The
sending a TMMBN notification message. One usage scenarios can be second value is the measured header sizes between the start of the
seen as limiting media senders in multiparty conferencing to the header for the layer reported on and the beginning of the RTP
slowest receiver's maximum media bandwidth reception/handling payload. Once, the media sender has received the TMMBR request on
capability. Such a use is helpful, for example, because the the bitrate limitation, it notifies the initiator of the request, and
receiver's situation may have changed due to computational load, or all other session participants, by sending a Temporal Maximum Media
because the receiver has just joined the conference and it is helpful Stream Bitrate Notification (TMMBN). The TMMBN contains a list of
to inform media sender(s) about its constraints, without waiting for the current applicable restrictions to help the participants to
congestion induced bandwidth reduction. Another application involves suppress TMMBR requests that wouldn't result in further restrictions
graceful bandwidth adaptation in scenarios where the upper limit for the sender. One usage scenario can be seen as limiting media
connection bandwidth to a receiver changes, but is known in the senders in multiparty conferencing to the slowest receiver's Maximum
interval between these dynamic changes. The TMMBR message is useful Media Stream bitrate reception/handling capability. Such a use is
for all media types that are not inherently of constant bit rate. helpful, for example, because the receiver's situation may have
changed due to computational load, or because the receiver has just
joined the conference, and considers it helpful to inform media
sender(s) about its constraints, without waiting for congestion
induced bitrate reduction. Another application involves graceful
bitrate adaptation in scenarios where the upper limit connection
bitrate to a receiver changes, but is known in the interval between
these dynamic changes. The TMMBR/TMMBN messages are useful for all
media types that are not inherently of constant bit rate. However,
TMMBR is not a congestion control mechanism and can't replace the
need to implement one.
The Video back channel message (VBCM) allows conveying bit streams The Video Back Channel Message (VBCM) allows conveying bit streams
conforming to ITU-T Rec. H.271 [H.271], from a video receiver to conforming to ITU-T Rec. H.271 [H.271], from a video receiver to
video sender. This ITU-T Recommendation defines codepoints for a video sender. This ITU-T Recommendation defines codepoints for a
number of video-specific feedback messages. Examples include number of video-specific feedback messages. Examples include
messages to signal: messages to signal:
- the corruption of reference pictures or parts thereof, - the corruption of reference pictures or parts thereof,
- the corruption of decoder state information, e.g. parameter sets, - the corruption of decoder state information, e.g. parameter sets,
- the suggestion of using a reference picture other than the one - the suggestion of using a reference picture other than the one
typically used, e.g. to support the NEWPRED algorithm [NEWPRED]. typically used, e.g. to support the NEWPRED algorithm [NEWPRED].
The ITU-T plans to add codepoints to H.271 every time a need arises, The ITU-T has the authority to add codepoints to H.271 every time a
e.g. with the introduction of new video codecs or new tools into need arises, e.g. with the introduction of new video codecs or new
existing video codecs. tools into existing video codecs.
There exists some overlap between H.271 messages and "native" There exists some overlap between VBCM messages and native messages
messages specified in this memo and in AVPF. Examples include the specified in this memo and in AVPF. Examples include the PLI message
PLI message of [RFC4585] and the FIR message specified herein. As a of [RFC4585] and the FIR message specified herein. As a general
general rule, the "native" messages should be prefered over the rule, the native messages should be preferred over the sending of
sending of VBCM messages when all senders and receivers implement VBCM messages when all senders and receivers implement this memo.
this memo. However, if gateways are in the picture, it may be more However, if gateways are in the picture, it may be more advisable to
advisable to utilize VBCM. Similarly, for feedback message types utilize VBCM. Similarly, for feedback message types that exist in
that exist in H.271 but do not exist in this memo or AVPF, there is H.271 but do not exist in this memo or AVPF, there is no other choice
no other choice but using VBCM. but using VBCM.
Video feedback channel messages according to H.271 do not require
acknowledgements on a protocol level, because the appropriate
reaction of the video encoder and sender can be derived from the
forward video bit stream.
Finally, the Temporal-Spatial Trade-off Request (TSTR) Message Video Back Channel Messages according to H.271 do not require a
enables a video receiver to signal to the video sender its preference notification on a protocol level, because the appropriate reaction of
for spatial quality or high temporal resolution (frame rate). The the video encoder and sender can be derived from the forward video
receiver of the video stream generates this signal typically based on bit stream.
input from its user interface, so to react to explicit requests of
the user. However, some implicit use forms are also known. For Finally, the Temporal-Spatial Trade-off Request (TSTR) enables a
example, the trade-offs commonly used for live video and document video receiver to signal to the video sender its preference for
camera content are different. Obviously, this indication is relevant spatial quality or high temporal resolution (frame rate). Typically,
only with respect to video transmission. The message is acknowledged the receiver of the video stream generates this signal based on input
by an announcement message indicating the newly chosen tradeoff, so from its user interface, in reaction to explicit requests of the
to allow immediate user feedback. user. However, some implicit use forms are also known. For example,
the trade-offs commonly used for live video and document camera
content are different. Obviously, this indication is relevant only
with respect to video transmission. The message is acknowledged by a
notification message indicating the newly chosen tradeoff, so to
allow immediate user feedback.
2. Definitions 2. Definitions
2.1. Glossary 2.1. Glossary
AMID - Additive Increase Multiplicative Decrease
ASM - Asynchronous Multicast ASM - Asynchronous Multicast
AVPF - The Extended RTP Profile for RTCP-based Feedback AVPF - The Extended RTP Profile for RTCP-based Feedback
FEC - Forward Error Correction FEC - Forward Error Correction
FIR - Full Intra Request FIR - Full Intra Request
MCU - Multipoint Control Unit MCU - Multipoint Control Unit
MPEG - Moving Picture Experts Group MPEG - Moving Picture Experts Group
PtM - Point to Multipoint PtM - Point to Multipoint
PtP - Point to Point PtP - Point to Point
TMMBN - Temporary Maximum Media Bit-rate Notification TMMBN - Temporary Maximum Media Stream Bitrate Notification
TMMBR - Temporary Maximum Media Bit-rate Request TMMBR - Temporary Maximum Media Stream Bitrate Request
PLI - Picture Loss Indication PLI - Picture Loss Indication
TSTA - Temporal Spatial Trade-off Announcement TSTN - Temporal Spatial Trade-off Notification
TSTR - Temporal Spatial Trade-off Request TSTR - Temporal Spatial Trade-off Request
VBCM - Video Back Channel Message indication. VBCM - Video Back Channel Message indication.
2.2. Terminology 2.2. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119]. document are to be interpreted as described in RFC 2119 [RFC2119].
Message: Message:
Codepoint defined by this specification, of one of the Codepoint defined by this specification, of one of the
following types: following types:
Request: Request:
Message that requires Acknowledgement Message that requires Acknowledgement
Acknowledgment:
Message that answers a Request
Command: Command:
Message that forces the receiver to an action Message that forces the receiver to an action
Indication: Indication:
Message that reports a situation Message that reports a situation
Notification: Notification:
See Indication. See Indication.
Note that, with the exception of "Notification", this terminology Note that, with the exception of ''Notification'', this
is in alignment with ITU-T Rec. H.245. terminology is in alignment with ITU-T Rec. H.245.
Decoder Refresh Point: Decoder Refresh Point:
A bit string, packetised in one or more RTP packets, which A bit string, packetised in one or more RTP packets, which
completely resets the decoder to a known state. Typical completely resets the decoder to a known state. Typical
examples of Decoder Refresh Points are H.261 Intra pictures examples of Decoder Refresh Points are H.261 Intra pictures
and H.264 IDR pictures. However, there are also much more and H.264 IDR pictures. However, there are also much more
complex decoder refresh points. complex decoder refresh points, as discussed below.
Typical examples for "hard" decoder refresh points are Intra Examples for "hard" decoder refresh points are Intra pictures
pictures in H.261, H.263, MPEG 1, MPEG 2, and MPEG-4 part 2, in H.261, H.263, MPEG 1, MPEG 2, and MPEG-4 part 2, and IDR
and IDR pictures in H.264. "Gradual" decoder refresh points pictures in H.264. "Gradual" decoder refresh points may also
may also be used; see for example [AVC]. While both "hard" be used; see for example [AVC]. While both "hard" and
and "gradual" decoder refresh points are acceptable in the "gradual" decoder refresh points are acceptable in the scope
scope of this specification, in most cases the user of this specification, in most cases the user experience will
experience will benefit from using a "hard" decoder refresh benefit from using a "hard" decoder refresh point.
point.
A decoder refresh point also contains all header information A decoder refresh point also contains all header information
above the picture layer (or equivalent, depending on the above the picture layer (or equivalent, depending on the
video compression standard) that is conveyed in-band. In video compression standard) that is conveyed in-band. In
H.264, for example, a decoder refresh point contains H.264, for example, a decoder refresh point contains
parameter set NAL units that generate parameter sets parameter set NAL units that generate parameter sets
necessary for the decoding of the following slice/data necessary for the decoding of the following slice/data
partition NAL units (and that are not conveyed out of band). partition NAL units (and that are not conveyed out of band).
To the best of the author's knowledge, the term "Decoder
Refresh Point" has been formally defined only in H.264; hence
we are referring here to this video compression standard.
Decoding: Decoding:
The operation of reconstructing the media stream. The operation of reconstructing the media stream.
Rendering: Rendering:
The operation of presenting (parts of) the reconstructed The operation of presenting (parts of) the reconstructed
media stream to the user. media stream to the user.
Stream thinning: Stream thinning:
The operation of removing some of the packets from a media The operation of removing some of the packets from a media
stream. Stream thinning, preferably, is performed media stream. Stream thinning, preferably, is media-aware,
aware, implying that media packets are removed in the order implying that media packets are removed in the order of their
of their relevance to the reproductive quality. However even relevance to the reproductive quality. However even when
when employing media-aware stream thinning, most media employing media-aware stream thinning, most media streams
streams quickly lose quality when subject to increasing quickly lose quality when subject to increasing levels of
levels of thinning. Media-unaware stream thinning leads to thinning. Media-unaware stream thinning leads to even worse
even worse quality degradation. quality degradation. In contrast to transcoding, stream
thinning is typically seen as a computationally lightweight
operation
2.3. Topologies Media: Often used (sometimes in conjunction with terms like
bitrate, stream, sender, ...) to identify the content of the
forward RTP packet stream carrying the codec data to which
the codec control message applies to.
Media Stream: The stream of packets carrying the media (and in some
case also repair information such as retransmission or
Forward Error Correction (FEC) information). We further
include within this specification the RTP packetization and
the usage of additional protocol headers on these packets to
carry them from sender to receiver.
2.3. Topologies
Please refer to [Topologies] for an in depth discussion. the Please refer to [Topologies] for an in depth discussion. the
topologies referred to throughout this memo are labeled (consistent topologies referred to throughout this memo are labeled (consistent
with [Topologies] as follows: with [Topologies] as follows:
Topo-Point-to-Point . . . . . point-to-point communication Topo-Point-to-Point . . . . . point-to-point communication
Topo-Multicast . . . . . . . multicast communication as in RFC 3550 Topo-Multicast . . . . . . . multicast communication as in RFC 3550
Topo-Translator . . . . . . . translator based as in RFC 3550 Topo-Translator . . . . . . . translator based as in RFC 3550
Topo-Mixer . . . . . . . . . mixer based as in RFC 3550 Topo-Mixer . . . . . . . . . mixer based as in RFC 3550
Topo-Video-switch-MCU . . . . video switching MCU, Topo-Video-switch-MCU . . . . video switching MCU,
Topo-RTCP-terminating-MCU . . mixer but terminating RTCP Topo-RTCP-terminating-MCU . . mixer but terminating RTCP
skipping to change at page 10, line 35 skipping to change at page 11, line 9
using unrelated content as reference data for inter picture using unrelated content as reference data for inter picture
prediction. After requesting the decoder refresh point, the video prediction. After requesting the decoder refresh point, the video
mixer stops the delivery of the current RTP stream and monitors mixer stops the delivery of the current RTP stream and monitors
the RTP stream from the new source until it detects data belonging the RTP stream from the new source until it detects data belonging
to the decoder refresh point. At that time, the RTP mixer starts to the decoder refresh point. At that time, the RTP mixer starts
forwarding the newly selected stream to the receiver(s). forwarding the newly selected stream to the receiver(s).
3. An application needs to signal to the remote encoder a request of 3. An application needs to signal to the remote encoder a request of
change of the desired trade-off in temporal/spatial resolution. change of the desired trade-off in temporal/spatial resolution.
For example, one user may prefer a higher frame rate and a lower For example, one user may prefer a higher frame rate and a lower
spatial quality, and another use may prefer the opposite. This spatial quality, and another user may prefer the opposite. This
choice is also highly content dependent. Many current video choice is also highly content dependent. Many current video
conferencing systems offer in the user interface a mechanism to conferencing systems offer in the user interface a mechanism to
make this selection, usually in the form of a slider. The make this selection, usually in the form of a slider. The
mechanism is helpful in point-to-point, centralized multipoint and mechanism is helpful in point-to-point, centralized multipoint and
non-centralized multipoint uses. non-centralized multipoint uses.
4. Use case 4 of the Basso draft applies only to AVPF's PLI [RFC4585] 4. Use case 4 of the Basso draft applies only to AVPF's PLI [RFC4585]
and is not reproduced here. and is not reproduced here.
5. Use case 5 of the Basso draft relates to a mechanism known as 5. Use case 5 of the Basso draft relates to a mechanism known as
skipping to change at page 11, line 12 skipping to change at page 11, line 32
problematic. Therefore, no freeze picture request has been problematic. Therefore, no freeze picture request has been
included in this memo, and the use case discussion is not included in this memo, and the use case discussion is not
reproduced here. reproduced here.
6. A video mixer dynamically selects one of the received video 6. A video mixer dynamically selects one of the received video
streams to be sent out to participants and tries to provide the streams to be sent out to participants and tries to provide the
highest bit rate possible to all participants, while minimizing highest bit rate possible to all participants, while minimizing
stream transrating. One way of achieving this is to setup sessions stream transrating. One way of achieving this is to setup sessions
with endpoints using the maximum bit rate accepted by that with endpoints using the maximum bit rate accepted by that
endpoint, and by the call admission method used by the mixer. By endpoint, and by the call admission method used by the mixer. By
means of commands that allow reducing the maximum media bitrate means of commands that allow reducing the Maximum Media Stream
beyond what has been negotiated during session setup, the mixer bitrate beyond what has been negotiated during session setup, the
can then reduce the maximum bit rate sent by endpoints to the mixer can then reduce the maximum bit rate sent by endpoints to
lowest common denominator of all received streams. As the lowest the lowest common denominator of all received streams. As the
common denominator changes due to endpoints joining, leaving, or lowest common denominator changes due to endpoints joining,
network congestion, the mixer can adjust the limits to which leaving, or network congestion, the mixer can adjust the limits to
endpoints can send their streams to match the new limit. The mixer which endpoints can send their streams to match the new limit. The
then would request a new maximum bit rate, which is equal or less mixer then would request a new maximum bit rate, which is equal or
than the maximum bit-rate negotiated at session setup, for a less than the maximum bit-rate negotiated at session setup, for a
specific media stream, and the remote endpoint can respond with specific media stream, and the remote endpoint can respond with
the actual bit-rate that it can support. the actual bit-rate that it can support.
The picture Basso, et al draws up covers most applications we The picture Basso, et al draws up covers most applications we
foresee. However we would like to extend the list with two additional foresee. However we would like to extend the list with two additional
use cases: use cases:
7. The used congestion control algorithms (AMID and TFRC) probe for 7. The used congestion control algorithms (AMID and TFRC [RFC3448])
more bandwidth as long as there is something to send. With probe for more available capacity as long as there is something to
congestion control using packet-loss as the indication for send. With congestion control using packet-loss as the indication
congestion, this probing does generally result in reduced media for congestion, this probing does generally result in reduced
quality (often to a point where the distortion is large enough to media quality (often to a point where the distortion is large
make the media unusable), due to packet loss and increased delay. enough to make the media unusable), due to packet loss and
In a number of deployment scenarios, especially cellular ones, the increased delay. In a number of deployment scenarios, especially
bottleneck link is often the last hop link. That cellular link cellular ones, the bottleneck link is often the last hop link.
also commonly has some type of QoS negotiation enabling the That cellular link also commonly has some type of QoS negotiation
cellular device to learn the maximal bit-rate available over this enabling the cellular device to learn the maximal bit-rate
last hop. Thus indicating the maximum available bit-rate to the available over this last hop. Thus, indicating the maximum
transmitting part can be beneficial to prevent it from even trying available bit-rate to the transmitting part can be beneficial to
to exceed the known hard limit that exists. For cellular or other prevent it from even trying to exceed the known hard limit that
mobile devices the available known bit-rate can also quickly exists. For cellular or other mobile devices the available known
change due to handover to another transmission technology, QoS bit-rate can also quickly change due to handover to another
renegotiation due to congestion, etc. To enable minimal disruption transmission technology, QoS renegotiation due to congestion, etc.
of service a possibility for quick convergence, especially in To enable minimal disruption of service quick convergence is
cases of reduced bandwidth, a media path signalling method is necessary, and therefore media path signaling is desirable.
desired.
8. The use of reference picture selection as an error resilience tool 8. The use of reference picture selection (RPS) as an error
has been introduced in 1997 as NEWPRED [NEWPRED], and is now resilience tool has been introduced in 1997 as NEWPRED [NEWPRED],
widely deployed. It operates the receiver sending a feedback and is now widely deployed. When RPS is in use, simplisticly put,
message to the sender, indicating a reference picture that should the receiver can send a feedback message to the sender, indicating
be used for future prediction. AVPF contains a mechanism for a reference picture that should be used for future prediction.
conveying such a message, but did not specify for which codec and ([NEWPRED] mentions other forms of feedback as well.) AVPF
according to which syntax the message conforms to. Recently, the contains a mechanism for conveying such a message, but did not
ITU-T finalized Rec. H.271 which (among other message types) also specify for which codec and according to which syntax the message
includes a feedback message. It is expected that this feedback conforms to. Recently, the ITU-T finalized Rec. H.271 which
message will enjoy wide support and fairly quickly. Therefore, a (among other message types) also includes a feedback message. It
mechanism to convey feedback messages according to H.271 appears is expected that this feedback message will enjoy wide support and
to be desirable. fairly quickly. Therefore, a mechanism to convey feedback
messages according to H.271 appears to be desirable.
3.2. Using the Media Path 3.2. Using the Media Path
There are multiple reasons why we propose to use the media path for There are multiple reasons why we use the media path for the codec
the codec control messages. First, systems employing MCUs are often control messages.
separating the control and media processing parts. As these messages
are intended or generated by the media part rather than the
signalling part of the MCU, having them on the media path avoids
interfaces and unnecessary control traffic between signalling and
processing. If the MCU is physically decomposite, the use of the
media path avoids the need for media control protocol extensions
(e.g. in MEGACO [RFC3525]).
Secondly, the signalling path quite commonly contains several First, systems employing MCUs are often separating the control and
signalling entities, e.g. SIP-proxies and application servers. media processing parts. As these messages are intended or generated
Avoiding signalling entities avoids delay for several reasons. by the media part rather than the signaling part of the MCU, having
Proxies have less stringent delay requirements than media processing them on the media path avoids interfaces and unnecessary control
and due to their complex and more generic nature may result in traffic between signaling and processing. If the MCU is physically
significant processing delay. The topological locations of the decomposite, the use of the media path avoids the need for media
signalling entities are also commonly not optimized for minimal control protocol extensions (e.g. in MEGACO [RFC3525]).
delay, rather other architectural goals. Thus the signalling path can
be significantly longer in both geographical and delay sense. Secondly, the signaling path quite commonly contains several
signaling entities, e.g. SIP-proxies and application servers.
Avoiding going through signaling entities avoids delay for several
reasons. Proxies have less stringent delay requirements than media
processing and due to their complex and more generic nature may
result in significant processing delay. The topological locations of
the signaling entities are also commonly not optimized for minimal
delay, but rather towards other architectural goals. Thus the
signaling path can be significantly longer in both geographical and
delay sense.
3.3. Using AVPF 3.3. Using AVPF
The AVPF feedback message framework [RFC4585] provides a simple way The AVPF feedback message framework [RFC4585] provides a simple way
of implementing the new messages. Furthermore, AVPF implements rules of implementing the new messages. Furthermore, AVPF implements rules
controlling the timing of feedback messages so to avoid congestion controlling the timing of feedback messages so to avoid congestion
through network flooding. We re-use these rules by referencing to through network flooding by RTCP traffic. We re-use these rules by
AVPF. referencing AVPF.
The signalling setup for AVPF allows each individual type of function The signaling setup for AVPF allows each individual type of function
to be configured or negotiated on a RTP session basis. to be configured or negotiated on a RTP session basis.
3.3.1. Reliability 3.3.1. Reliability
The use of RTCP messages implies that each message transfer is The use of RTCP messages implies that each message transfer is
unreliable, unless the lower layer transport provides reliability. unreliable, unless the lower layer transport provides reliability.
The different messages proposed in this specification have different The different messages proposed in this specification have different
requirements in terms of reliability. However, in all cases, the requirements in terms of reliability. However, in all cases, the
reaction to an (occasional) loss of a feedback message is specified. reaction to an (occasional) loss of a feedback message is specified.
3.4. Multicast 3.4. Multicast
The media related requests might be used with multicast. The RTCP The codec control messages might be used with multicast. The RTCP
timing rules specified in [RFC3550] and [RFC4585] ensure that the timing rules specified in [RFC3550] and [RFC4585] ensure that the
messages do not cause overload of the RTCP connection. The use of messages do not cause overload of the RTCP connection. The use of
multicast may result in the reception of messages with inconsistent multicast may result in the reception of messages with inconsistent
semantics. The reaction to inconsistencies depends on the message semantics. The reaction to inconsistencies depends on the message
type, and is discussed for each message type separately. type, and is discussed for each message type separately.
3.5. Feedback Messages 3.5. Feedback Messages
This section describes the semantics of the different feedback This section describes the semantics of the different feedback
messages and how they apply to the different use cases. messages and how they apply to the different use cases.
3.5.1. Full Intra Request Command 3.5.1. Full Intra Request Command
A Full Intra Request (FIR) command, when received by the designated A Full Intra Request (FIR) Command, when received by the designated
media sender, requires that the media sender sends a "decoder refresh media sender, requires that the media sender sends a Decoder Refresh
point" (see 2.2) at the earliest opportunity. The evaluation of such Point (see 2.2) at the earliest opportunity. The evaluation of such
opportunity includes the current encoder coding strategy and the opportunity includes the current encoder coding strategy and the
current available network resources. current available network resources.
FIR is also known as an "instantaneous decoder refresh request" or FIR is also known as an ''instantaneous decoder refresh request''
"video fast update request". or ''video fast update request''.
Using a decoder refresh point implies refraining from using any Using a decoder refresh point implies refraining from using any
picture sent prior to that point as a reference for the encoding picture sent prior to that point as a reference for the encoding
process of any subsequent picture sent in the stream. For predictive process of any subsequent picture sent in the stream. For predictive
media types that are not video, the analogue applies. For example, media types that are not video, the analogue applies. For example,
if in MPEG-4 systems scene updates are used, the decoder refresh if in MPEG-4 systems scene updates are used, the decoder refresh
point consists of the full representation of the scene and is not point consists of the full representation of the scene and is not
delta-coded relative to previous updates. delta-coded relative to previous updates.
Decoder Refresh points, especially Intra or IDR pictures, are in Decoder Refresh Points, especially Intra or IDR pictures, are in
general several times larger in size than predicted pictures. Thus, general several times larger in size than predicted pictures. Thus,
in scenarios in which the available bandwidth is small, the use of a in scenarios in which the available bit-rate is small, the use of a
decoder refresh point implies a delay that is significantly longer Decoder Refresh Point implies a delay that is significantly longer
than the typical picture duration. than the typical picture duration.
Usage in multicast is possible; however aggregation of the commands Usage in multicast is possible; however aggregation of the commands
is recommended. A receiver that receives a request closely (within 2 is recommended. A receiver that receives a request closely (within 2
times the longest Round Trip Time (RTT) known) after sending a times the longest Round Trip Time (RTT) known) after sending a
decoder refresh point should await a second request message to ensure Decoder Refresh Point should await a second request message to ensure
that the media receiver has not been served by the previously that the media receiver has not been served by the previously
delivered decoder refresh point. The reason for delaying 2 times the delivered Decoder Refresh Point. The reason for delaying 2 times the
longest known RTT is to avoid sending unnecessary decoder refresh longest known RTT is to avoid sending unnecessary Decoder Refresh
points. A session participant may have sent its own request while Points. A session participant may have sent its own request while
another participants request was in-flight to them. Thus suppressing another participant's request was in-flight to them. Suppressing
those requests that may have been sent without knowledge about the those requests that may have been sent without knowledge about the
other request avoids this issue. other request avoids this issue.
Full Intra Request is applicable in use-case 1, 2, and 5. Full Intra Request is applicable in use-case 1, 2, and 5.
3.5.1.1. Reliability 3.5.1.1. Reliability
The FIR message results in the delivery of a decoder refresh point, The FIR message results in the delivery of a Decoder Refresh Point,
unless the message is lost. Decoder refresh points are easily unless the message is lost. Decoder Refresh Points are easily
identifiable from the bit stream. Therefore, there is no need for identifiable from the bit stream. Therefore, there is no need for
protocol-level acknowledgement, and a simple command repetition protocol-level notification, and a simple command repetition
mechanism is sufficient for ensuring the level of reliability mechanism is sufficient for ensuring the level of reliability
required. However, the potential use of repetition does require a required. However, the potential use of repetition does require a
mechanism to prevent the recipient from responding to messages mechanism to prevent the recipient from responding to messages
already received and responded to. already received and responded to.
To ensure the best possible reliability, a sender of FIR may repeat To ensure the best possible reliability, a sender of FIR may repeat
the FIR request until a response has been received. The repetition the FIR request until a response has been received. The repetition
interval is determined by the RTCP timing rules the session operates interval is determined by the RTCP timing rules applicable to the
under. Upon reception of a complete decoder refresh point or the session. Upon reception of a complete Decoder Refresh Point or the
detection of an attempt to send a decoder refresh point (which got detection of an attempt to send a Decoder Refresh Point (which got
damaged due to a packet loss) the repetition of the FIR must stop. If damaged due to a packet loss), the repetition of the FIR must stop.
another FIR is necessary, the request sequence number must be If another FIR is necessary, the request sequence number must be
increased. To combat loss of the decoder refresh points sent, the increased. To combat loss of the Decoder Refresh Points sent, the
sender that receives repetitions of the FIR 2*RTT after the sender that receives repetitions of the FIR 2*RTT after the
transmission of the decoder refresh point shall send a new decoder transmission of the Decoder Refresh Point shall send a new Decoder
refresh point. Two round trip times allow time for the request to Refresh Point. Two round trip times allow time for the request to
arrive at the media sender and the decoder refresh point to arrive arrive at the media sender and the Decoder Refresh Point to arrive
back to the requestor. A FIR sender shall not have more than one FIR back to the requestor. A FIR sender shall not have more than one FIR
request (different request sequence number) outstanding at any time request (different request sequence number) outstanding at any time
per media sender in the session. per media sender in the session.
An RTP Mixer that receives an FIR from a media receiver is An RTP Mixer that receives an FIR from a media receiver is
responsible to ensure that a decoder refresh point is delivered to responsible to ensure that a Decoder Refresh Point is delivered to
the requesting receiver. It may be necessary to generate FIR commands the requesting receiver. It may be necessary for the mixer to
by the MCU. The two legs (FIR-requesting endpoint to MCU, and MCU to generate FIR commands. The two legs (FIR-requesting endpoint to
decoder refresh point generating MCU) are handled independently from mixer, and mixer to Decoder Refresh Point generating endpoint) are
each other from a reliability perspective. handled independently from each other from a reliability perspective.
3.5.2. Temporal Spatial Trade-off Request and Announcement 3.5.2. Temporal Spatial Trade-off Request and Notification
The Temporal Spatial Trade-off Request (TSTR) instructs the video The Temporal Spatial Trade-off Request (TSTR) instructs the video
encoder to change its trade-off between temporal and spatial encoder to change its trade-off between temporal and spatial
resolution. Index values from 0 to 31 indicate monotonically a resolution. Index values from 0 to 31 indicate monotonically a
desire for higher frame rate. In general the encoder reaction time desire for higher frame rate. That is, a requester asking for an
may be significantly longer than the typical picture duration. See index of 0 prefers a high quality and is willing to accept a low
use case 3 for an example. The encoder decides if the request frame rate, whereas a requester asking for 31 wishes a high frame
results in a change of the trade off. An acknowledgement process has rate, potentially at the cost of low spatial quality.
been defined to provide feedback of the trade-off that is used
henceforth.
Informative note: TSTR and TSTA have been introduced primarily In general the encoder reaction time may be significantly longer than
the typical picture duration. See use case 3 for an example. The
encoder decides if the request results in a change of the trade off.
The Temporal Spatial Trade-Off Notification message (TSTN) has been
defined to provide feedback of the trade-off that is used henceforth.
Informative note: TSTR and TSTN have been introduced primarily
because it is believed that control protocol mechanisms, e.g. a SIP because it is believed that control protocol mechanisms, e.g. a SIP
re-invite, are too heavyweight, and too slow to allow for a re-invite, are too heavyweight, and too slow to allow for a
reasonable user experience. Consider, for example, a user reasonable user experience. Consider, for example, a user
interface where the remote user selects the temporal/spatial trade- interface where the remote user selects the temporal/spatial trade-
off with a slider (as it is common in state-of-the-art video off with a slider (as it is common in state-of-the-art video
conferencing systems). An immediate feedback to any slider conferencing systems). An immediate feedback to any slider
movement is required for a reasonable user experience. A SIP re- movement is required for a reasonable user experience. A SIP re-
invite would require at least 2 round-trips more (compared to the invite [RFC3261] would require at least 2 round-trips more
TSTR/TSTA mechanism) and may involve proxies and other complex (compared to the TSTR/TSTN mechanism) and may involve proxies and
mechanisms. Even in a well-designed system, it may take a second other complex mechanisms. Even in a well-designed system, it may
or so until finally the new trade-off is selected. take a second or so until finally the new trade-off is selected.
Furthermore the use of RTCP solves very efficiently the multicast Furthermore the use of RTCP solves very efficiently the multicast
use case. use case.
The use of TSTR and TSTA in multipoint scenarios is a non-trivial The use of TSTR and TSTN in multipoint scenarios is a non-trivial
subject, and can be solved in many implementation specific ways. subject, and can be solved in many implementation-specific ways.
Problems are stemming from the fact that TSTRs will typically arrive Problems are stemming from the fact that TSTRs will typically arrive
unsynchronized, and may request different trade-off values for the unsynchronized, and may request different trade-off values for the
same stream and/or endpoint encoder. This memo does not specify a same stream and/or endpoint encoder. This memo does not specify a
MCU's or endpoint's reaction to the reception of a suggested trade- translator, mixer or endpoint's reaction to the reception of a
off as conveyed in the TSTR -- we only require the receiver of a TSTR suggested trade-off as conveyed in the TSTR -- we only require the
message to reply to it by sending a TSTA, carrying the new trade-off receiver of a TSTR message to reply to it by sending a TSTN, carrying
chosen by its own criteria (which may or may not be based on the the new trade-off chosen by its own criteria (which may or may not be
trade-off conveyed by TSTR). In other words, the trade-off sent in based on the trade-off conveyed by TSTR). In other words, the trade-
TSTR is a non-binding recommendation; nothing more. off sent in TSTR is a non-binding recommendation; nothing more.
With respect to TSTR/TSTA, four scenarios based on the topologies With respect to TSTR/TSTN, four scenarios based on the topologies
described in [Topologies] need to be distinguished. The scenarios are described in [Topologies] need to be distinguished. The scenarios are
described in the following sub-clauses. described in the following sub-clauses.
3.5.2.1. Point-to-point 3.5.2.1. Point-to-point
In this most trivial case (Topo-Point-to-Point), the media sender In this most trivial case (Topo-Point-to-Point), the media sender
typically adjusts its temporal/spatial trade-off based on the typically adjusts its temporal/spatial trade-off based on the
requested value in TSTR, and within its capabilities. The TSTA requested value in TSTR, and within its capabilities. The TSTN
message conveys back the new trade-off value (which may be identical message conveys back the new trade-off value (which may be identical
to the old one if, for example, the sender is not capable to adjust to the old one if, for example, the sender is not capable of
its trade-off). adjusting its trade-off).
3.5.2.2. Point-to-Multipoint using Multicast or Translators 3.5.2.2. Point-to-Multipoint using Multicast or Translators
RTCP Multicast is used either with media multicast according to Topo- RTCP Multicast is used either with media multicast according to Topo-
Multicast, or following RFC 3550's translator model according to Multicast, or following RFC 3550's translator model according to
Topo-Translator. In these cases, TSTR messages from different Topo-Translator. In these cases, TSTR messages from different
receivers may be received unsynchronized, and possibly with different receivers may be received unsynchronized, and possibly with different
requested trade-offs (because of different user preferences). This requested trade-offs (because of different user preferences). This
memo does not specify how the media sender tunes its trade-off. memo does not specify how the media sender tunes its trade-off.
Possible strategies include selecting the mean, or median, of all Possible strategies include selecting the mean, or median, of all
trade-off requests received, prioritize certain participants, or trade-off requests received, prioritize certain participants, or
continue using the previously selected trade-off (e.g. when the continue using the previously selected trade-off (e.g. when the
sender is not capable of adjusting it). Again, all TSTR messages sender is not capable of adjusting it). Again, all TSTR messages
need to be acknowledged by TSTA, and the value conveyed back has to need to be acknowledged by TSTN, and the value conveyed back has to
reflect the decision made. reflect the decision made.
3.5.2.3. Point-to-Multipoint using RTP Mixer 3.5.2.3. Point-to-Multipoint using RTP Mixer
In this scenario (Topo-Mixer) the RTP Mixer receives all TSTR In this scenario (Topo-Mixer) the RTP Mixer receives all TSTR
messages, and has the opportunity to act on them based on its own messages, and has the opportunity to act on them based on its own
criteria. In most cases, the MCU should form a "consensus" of criteria. In most cases, the Mixer should form a ''consensus'' of
potentially conflicting TSTR messages arriving from different potentially conflicting TSTR messages arriving from different
participants, and initiate its own TSTR message(s) to the media participants, and initiate its own TSTR message(s) to the media
sender(s). The strategy of forming this "consensus" is open for the sender(s). The strategy of forming this ''consensus'' is open for
implementation, and can, for example, encompass averaging the the implementation, and can, for example, encompass averaging the
participant's request values, prioritizing certain participants, or participants request values, prioritizing certain participants, or
use session default values. If the Mixer changes its trade-off, it use session default values. If the Mixer changes its trade-off, it
needs to request from the media sender(s) the use of the new value, needs to request from the media sender(s) the use of the new value,
by creating a TSTR of its own. Upon reaching a decision on the used by creating a TSTR of its own. Upon reaching a decision on the used
trade-off it includes that value in the acknowledgement. trade-off it includes that value in the acknowledgement.
Even if a Mixer or Translator performs transcoding, it is very Even if a Mixer or Translator performs transcoding, it is very
difficult to deliver media with the requested trade-off, unless the difficult to deliver media with the requested trade-off, unless the
content the MCU receives is already close to that trade-off. Only in content the Mixer or Translator receives is already close to that
cases where the original source has substantially higher quality (and trade-off. Only in cases where the original source has substantially
bit-rate), it is likely that transcoding can result in the requested higher quality (and bit-rate), it is likely that transcoding can
trade-off. result in the requested trade-off.
3.5.2.4. Reliability 3.5.2.4. Reliability
A request and reception acknowledgement mechanism is specified. The A request and reception acknowledgement mechanism is specified. The
Temporal Spatial Trade-off Announcement (TSTA) message informs the Temporal Spatial Trade-off Notification (TSTN) message informs the
request-sender that its request has been received, and what trade-off request-sender that its request has been received, and what trade-off
is used henceforth. This acknowledgment mechanism is desirable for at is used henceforth. This acknowledgment mechanism is desirable for at
least the following reasons: least the following reasons:
o A change in the trade-off cannot be directly identified from the o A change in the trade-off cannot be directly identified from the
media bit stream, media bit stream,
o User feedback cannot be implemented without information of the o User feedback cannot be implemented without information of the
chosen trade-off value, according to the media sender's chosen trade-off value, according to the media sender's
constraints, constraints,
o Repetitive sending of messages requesting an unimplementable trade- o Repetitive sending of messages requesting an unimplementable trade-
off can be avoided. off can be avoided.
3.5.3. H.271 Video Back Channel Message 3.5.3. H.271 Video Back Channel Message
ITU-T Rec. H.271 defines syntax, semantics, and suggested encoder ITU-T Rec. H.271 defines syntax, semantics, and suggested encoder
reaction to a video back channel message. The codepoint defined in reaction to a video back channel message. The codepoint defined in
this memo is used to transparently convey such a message from media this memo is used to transparently convey such a message from media
receiver to media sender. receiver to media sender. In this memo, we refrain from an in-depth
discussion of the available codepoints within H.271 and refer to the
In this memo, we refrain from an in-depth discussion of the available specification text instead [H.271].
codepoints within H.271 for a number of reasons. The perhaps most
important reason is that we expect backward-compatible additions of
codepoints to H.271 outside the update/maturity cycle of this memo.
Another reason lies in the complexity of the H.271 specification: it
is a dense document with currently 16 pages of content. It does not
make any sense to try to summarize its content in a few sentences of
IETF lingo -- oversimplification and misguidance would be inevitable.
Finally, please note that H.271 contains many statements of
applicability and interpretation of its various messages in
conjunction with specific video compression standards. This type of
discussion would overload the present memo.
In so far, this memo follows the guidance of a decade of RTP payload
format specification work -- the details of the media format carried
is normally not described in any significant detail.
However, we note that some H.271 messages bear similarities with However, we note that some H.271 messages bear similarities with
native messages of AVPF and this memo. Furthermore, we note that native messages of AVPF and this memo. Furthermore, we note that
some H.271 message are known to require caution in multicast some H.271 message are known to require caution in multicast
environments -- or are plainly not usable in multicast or multipoint environments -- or are plainly not usable in multicast or multipoint
scenarios. Table 1 provides a brief, oversimplifying overview of the scenarios. Table 1 provides a brief, oversimplifying overview of the
messages currenty defined in H.271, their similar AVPF or CCM messages currently defined in H.271, their similar AVPF or CCM
messages (the latter as specified in this memo), and an indication of messages (the latter as specified in this memo), and an indication of
our current knowledge of their multicast safety. our current knowledge of their multicast safety.
H.271 msg type AVPF/CCM msg type multicast-safe H.271 msg type AVPF/CCM msg type multicast-safe
---------------------------------------------------------------------
0 (when used for 0 (when used for
reference picture reference picture
selection) AVPF RPSI No (positive ACK of pictures) selection) AVPF RPSI No (positive ACK of pictures)
1 AVPF PLI Yes 1 AVPF PLI Yes
2 AVPF SLI Yes 2 AVPF SLI Yes
3 N/A Yes (no required sender action) 3 N/A Yes (no required sender action)
4 N/A Yes (no required sender action) 4 N/A Yes (no required sender action)
Table 1: H.271 messages and their AVPF/CCM equivalents Table 1: H.271 messages and their AVPF/CCM equivalents
skipping to change at page 18, line 32 skipping to change at page 18, line 42
for the same purpose as AVPF's RPSI -- although other use for the same purpose as AVPF's RPSI -- although other use
forms are also possible. forms are also possible.
In response to the opaqueness of the H.271 messages especially with In response to the opaqueness of the H.271 messages especially with
respect to the multicast safety, the following guidelines MUST be respect to the multicast safety, the following guidelines MUST be
followed when an implementation wishes to employ the H.271 video back followed when an implementation wishes to employ the H.271 video back
channel message: channel message:
1. Implementations utilizing the H.271 feedback message MUST stay in 1. Implementations utilizing the H.271 feedback message MUST stay in
compliance with congestion control principles, as outlined in compliance with congestion control principles, as outlined in
section 5. section 5 ..
2. An implementation SHOULD utilize the native messages as defined in 2. An implementation SHOULD utilize the native messages as defined in
[RFC4585] and in this memo instead of similar messages defined in [RFC4585] and in this memo instead of similar messages defined in
[H.271]. Our current understanding of similar messages is [H.271]. Our current understanding of similar messages is
documented in Table 1 above. One good reason to divert from the documented in Table 1 above. One good reason to divert from the
SHOULD statement above would be if it is clearly understood that, SHOULD statement above would be if it is clearly understood that,
for a given application and video compression standard, the for a given application and video compression standard, the
aforementioned "similarity" is not given, in contrast to what aforementioned ''similarity'' is not given, in contrast to what
the table indicates. the table indicates.
3. It has been observed that some of the H.271 codepoints currently 3. It has been observed that some of the H.271 codepoints currently
in existence are not multicast-safe. Therefore, the sensible in existence are not multicast-safe. Therefore, the sensible
thing to do is not to use the H.271 feedback message type in thing to do is not to use the H.271 feedback message type in
multicast environments. It MAY be used only when all the issues multicast environments. It MAY be used only when all the issues
mentioned later are fully understood by the implementer, and mentioned later are fully understood by the implementer, and
properly taken into account by all endpoints. In all other cases, properly taken into account by all endpoints. In all other cases,
the H.271 message type MUST NOT be used in conjunction with the H.271 message type MUST NOT be used in conjunction with
multicast. multicast.
4. It has been observed that even in centralized multipoint 4. It has been observed that even in centralized multipoint
environments, where the mixer should theoretically be able to environments, where the mixer should theoretically be able to
skipping to change at page 19, line 38 skipping to change at page 19, line 50
sender is a non-trivial task. As reasons, we note that these sender is a non-trivial task. As reasons, we note that these
messages may be contradicting each other, and that their transport messages may be contradicting each other, and that their transport
is unreliable (there may well be other reasons). In case of many is unreliable (there may well be other reasons). In case of many
H.271 messages (i.e. types 0, 2, 3, and 4), the algorithm for H.271 messages (i.e. types 0, 2, 3, and 4), the algorithm for
combining must be both aware of the network/protocol environment combining must be both aware of the network/protocol environment
(i.e. with respect to congestion) and of the media codec employed, (i.e. with respect to congestion) and of the media codec employed,
as H.271 messages of a given type can have different semantics for as H.271 messages of a given type can have different semantics for
different media codecs. different media codecs.
3. The suppression of requests may need to go beyond the basic 3. The suppression of requests may need to go beyond the basic
mechanism described in AVPF (which are driven exclusively by mechanism described in AVPF (which are driven exclusively by
timing/bandwidth considerations on the protocol level). For timing and transport considerations on the protocol level). For
example, a receiver is often required to refrain from (or delay) example, a receiver is often required to refrain from (or delay)
generating requests, based on information it receives from the generating requests, based on information it receives from the
media stream. For instance, it makes no sense for a receiver to media stream. For instance, it makes no sense for a receiver to
issue a FIR when a transmission of an Intra/IDR picture is issue a FIR when a transmission of an Intra/IDR picture is
ongoing. ongoing.
4. When using the not multicast save messages (e.g. H.271 type 0 4. When using the non-multicast-safe messages (e.g. H.271 type 0
positive ACK of received pictures/slices) in larger multicast positive ACK of received pictures/slices) in larger multicast
groups, the media receiver will likely be forced to delay or even groups, the media receiver will likely be forced to delay or even
omit sending these messages. For the media sender this looks like omit sending these messages. For the media sender this looks like
data has not been properly received (although it was received data has not been properly received (although it was received
properly), and a naively implemented media sender reacts to these properly), and a naively implemented media sender reacts to these
perceived problems where it shouldn't. perceived problems where it shouldn't.
3.5.3.1. Reliability 3.5.3.1. Reliability
H.271 video back channel messages do not require reliable H.271 Video Back Channel messages do not require reliable
transmission, and the reception of a message can be derived from the transmission, and the reception of a message can be derived from the
forward video bit stream. Therefore, no specific reception forward video bit stream. Therefore, no specific reception
acknowledgement is specified. acknowledgement is specified.
With respect to re-sending rules, clause 3.5.1.1. applies. With respect to re-sending rules, clause 3.5.1.1. applies.
3.5.4. Temporary Maximum Media Bit-rate Request 3.5.4. Temporary Maximum Media Stream Bit-rate Request and Notification
A receiver, translator or mixer uses the Temporary Maximum Media Bit- A receiver, translator or mixer uses the Temporary Maximum Media
rate Request (TMMBR, "timber") to request a sender to limit the Stream Bit-rate Request (TMMBR, "timber") to request a sender to
maximum bit-rate for a media stream to, or below, the provided value. limit the maximum bit-rate for a media stream to, or below, the
The primary usage for this is a scenario with MCU (use case 6), provided value. The Temporary Maximum Media Stream Bit-rate
corresponding to Topo-Translator or Topo-Mixer, but also Topo-Point- Notification (TMMBN) advises the media receiver(s) of the changed
to-Point. bitrate it is not going to exceed henceforth. The primary usage for
this is a scenario with a MCU or Mixer (use case 6), corresponding to
Topo-Translator or Topo-Mixer, but also Topo-Point-to-Point.
The temporary maximum media bit-rate messages are generic messages The temporary limitation on the media stream is expressed as a tuple;
that can be applied to any media. one value limiting the bit-rate at the layer for which the overhead
is calculated to. A second value provides the per packet header
overhead between the layer for which bit-rate is reported and the
start of the RTP payload. By having both values the media stream
sender can determine the effect of changing the packet rate for the
media stream in an environment which contains translators or mixers
that affect the amount of per packet overhead. For example a gateway
that convert between IPv4 and IPv6 would affect the per packet
overhead commonly with 20 bytes. There exist also other mechanisms,
like tunnels, that change the amount of headers that are present at a
particular bottleneck for which the TMMBR sending entity has
knowledge about. The problem with varying overhead is also discussed
in [RFC3890].
The above way of measuring allows for one to provide bit-rate and
overhead values for different protocol layers, for example on IP
level, out part of a tunnel protocol, or the link layer. The level a
peer report on, is fully dependent on the level of integration the
peer has, as it needs to be able to extract the information from that
level. It is expected that peers will be able to report values at
least for the IP layer, but in certain implementations link layer may
be available to allow for more precise information.
The temporary maximum media stream bit-rate messages are generic
messages that can be applied to any RTP packet stream. This
separates it a bit from the other codec control messages defined in
this specification that applies only to specific media types or
payload formats. The TMMBR functionality applies to the transport and
the requirements it places on the media encoding.
The reasoning below assumes that the participants have negotiated a The reasoning below assumes that the participants have negotiated a
session maximum bit-rate, using the signalling protocol. This value session maximum bit-rate, using a signaling protocol. This value can
can be global, for example in case of point-to-point, multicast, or be global, for example in case of point-to-point, multicast, or
translators. It may also be local between the participant and the translators. It may also be local between the participant and the
peer or mixer. In both cases, the bit-rate negotiated in signalling peer or mixer. In both cases, the bit-rate negotiated in signaling is
is the one that the participant guarantees to be able to handle the one that the participant guarantees to be able to handle (encode
(encode and decode). In practice, the connectivity of the and decode). In practice, the connectivity of the participant also
participant also bears an influence to the negotiated value -- it bears an influence to the negotiated value -- it does not necessarily
does not necessarily make much sense to negotiate a media bit rate make much sense to negotiate a media bit rate that one's network
that one's network interface does not support. interface does not support.
An already established temporary bit-rate value may be changed at any It is also beneficial to have negotiated a maximum packet rate for
time (subject to the timing rules of the feedback message sending), the session or sender. RFC 3890 provides such a SDP [RFC4566]
and to any value between zero and the session maximum, as negotiated attribute, however that is not usable in RTP sessions established
during signalling. Even if a sender has received a TMMBR message using offer/answer [RFC3264]. Therefore a max packet rate signaling
increasing the bit-rate, all increases must be governed by a parameter is specified.
congestion control algorithm. TMMBR only indicates known limitations,
usually in the local environment, and does not provide any
guarantees.
If it is likely that the new bit-rate indicated by TMMBR will be An already established temporary limit may be changed at any time
valid for the remainder of the session, the TMMBR sender can perform (subject to the timing rules of the feedback message sending), and to
a renegotiation of the session upper limit using the session any values between zero and the session maximum, as negotiated during
signalling protocol. session establishment signaling. Even if a sender has received a
TMMBR message allowing an increase in the bit-rate, all increases
must be governed by a congestion control mechanism. TMMBR only
indicates known limitations, usually in the local environment, and
does not provide any guarantees about the full path.
3.5.4.1. MCU based Multi-point operation If it is likely that the new value indicated by TMMBR will be valid
for the remainder of the session, the TMMBR sender can perform a
renegotiation of the session upper limit using the session signaling
protocol.
3.5.4.1. Behavior for media receivers using TMMBR
In multipart scenarios, different receivers likely have different
limits for receiving bitrate. Therefore, an algorithm to identify
the most restrictive TMMBR requests is specified in section 4 ..2.2.1.
The general behavior is explaind in this section and the gist of the
algorithm to determine the most restrictive values are explained
informally in the next section.
Immediately after session setup, the bitrate limit is set to the
session limit as established by the session setup signaling (or
equivalent). The overhead value is set to 0. When the session setup
signaling does not specify a limit, then unlimited bitrate is
assumed. Note that many codecs specify their own limits, e.g.
through H.264's level concept.
At any given time, a media receiver can send a TMMBR with a limit
that is lower than the current limit. The media receiver use the
algorithm outlined in the below Section 3.5.4.2 to determine if its
limit is stricter than already existing ones. The media sender upon
receiving the TMMBR request will also excersie the algorithm to
determine the set of most restrictive limitations and then send a
TMMBN containg that set. Once the media sender has sent the TMMBN
message, the receivers indicated in that message becomes ''owners''
of the limitations. Most likely, the owner is the original sender of
the TMMBR -- for the handling of corner-cases (i.e. concurrent TMMBRs
from different receivers, lost TMMBRs and sender side optimisations)
please see the formal specification. ''Owners'' and limits are
usually known session wide, as both TMMBR and TMMBN are forwarded to
all in the session unless a Mixer or Translator separate the session
from RTCP handling point of view.
Only a ''owner'' is allowed to raise the bitrate limit to a value
higher than the session has been notified of, but not higher than the
session limit negotiated by the session setup signaling (see above).
A ''owner'' does not need to take into account TMMBR messages sent by
anyone else (although that may well be a desirable optimization). If
a ''owner'' sets a new session limit that is too high for someone
else's liking, other media receivers can react to the situation by
emmitting their own TMMBR message (and, in the process, become a
''owner''). Limitations belonging to ''owners'' timing out from the
session are removed by the media sender who notifies the session
about the event by sending a TMMBN.
Obviously, when there is only one media receiver, this receiver
becomes ''owner'' once it receives the first TMMBN in response to its
own TMMBR, and stays ''owner'' for the rest of the session.
Therefore, when it is known that there will always be only a single
media receiver, the above algorithm is not required. Media receivers
that are aware they are the only ones in a session can send TMMBR
messages with bitrate limits both higher and lower than the
previously notified limit at any time (subject to AVPF's RTCP RR send
timing rules). However, it may be difficult for a session
participant to determine if it is the only receiver in the session.
Due to that any one implementing TMMBR are required to implement this
algorithm.
3.5.4.2. Algorithm for exstablishing current limitations
First it is important to consider the implications of using a tuple
for limiting the media sender's behavior. The bit-rate and the
overhead value results in a 2-dimensional solution space for possible
media streams. Fortunately the two variables are linked. The bit-rate
available for RTP payloads will be equal to the TMMBR reported bit-
rate minus the packet rate used times the TMMBR reported overhead.
This has the result in a session with two different participants
having set limitations, the used packet rate will determine which of
the two that applies.
Example:
Receiver A: TMMBR_BR = 35 kbps, TMMBR_OH = 40
Receiver B: TMMBR_BR = 40 kbps, TMMBR_OH = 60
For a given packet rate (PR) the bit-rate available for media
payloads in RTP will be:
Max_media_BR_A = TMMBR_BR_A - PR * TMMBR_OH_A * 8
Max_media_BR_B = TMMBR_BR_B - PR * TMMBR_OH_B * 8
For a PR = 20 these calculations will yield a Max_media_BR_A = 28600
bps and Max_media_BR_B = 30400 bps, which shows that receiver A is
the limiting one for this packet rate. However there will be a PR
when the difference in bit-rate restriction will be equal to the
difference in packet overheads. This can be found by setting
Max_media_BR_A equal to Max_media_BR_B and breaking out PR:
TMMBR_BR_A - TMMBR_BR_B
PR = ---------------------------
8*(TMMBR_OH_A - TMMBR_OH_B)
Which, for the numbers above yields 31.25 as the intersection point
between the two limits. The implications of this have to be
considered by application implementors that are going to control
media encoding and its packetization. Because, as exemplified above,
there might be multiple TMMBR limits that applies to the trade-off
between media bit-rate and packet rate. Which limitation that applies
depends on the packet rate considered to be used.
This also has implications for how the TMMBR mechanism needs to work.
First, there is the possibility that multiple TMMBR tuples are
providing limitations on the media sender. Secondly there is a need
for any session participant (meda sender and receivers) to be able to
determine if a given tuple will become a limitation upon the media
sender, or if the set of already given limitations are stricter than
the given values. Otherwise the suppression of TMMBR requests would
not work.
Thus any session participant needs to be able from a given set X of
tuples determine which is the minimal set need to express the
limitations for all packet rates from 0 to highest possible. Where
the highest possible either is application limited and indicated
trough session setup signaling or as a result of the given
limitations when the available bit-rate is fully consumed by headers.
First determine what the highest possible bit-rate given all the
limitations is. If there is provided a session maximum packet rate
(SMAXPR) then this can be used. In addition one needs to calculate
for each tuple in the set what its maximum is by calculating bit-rate
(BR) divided by overhead (OH) per packet converted to bits.
MaxPR = SMAXPR
For i=1 to size(X) {
tmp_pr = X(i).BR / 8*X(i).OH;
If (tmp_pr < MaxPR) then MaxPR = tmp_pr
}
For a zero packet rate the TMMBR signaled bit-rate will be the only
limiting factor, thus the tuple with the smallest available bit-rate
is a limitation at this point of the range and function as a start
value in the algorithm.
Start by finding the element X(l) in X with the lowest bit-rate value
and the highest overhead if there are multiple on the same bit-rate.
The set Y that is the minimal set of tuples that provide restrictions
initially contain only X(l). Then for each other tuple X(i) calculate
if there exist an intersection between the currently selected tuple
X(s) (initially s=l) and which of the tuples within the set that has
this intersection at the lowest packet rate. Having found the lowest
packet rate, compare it with the sessions maximum packet rate. If
lower than that limit this tuple provide a session limit and the
tuple is added to Y. Update the value of s to the found tuple and
repeat search for the tuple that has the intersection at the lowest
packet rate, but still higher than the previous intersection.
Algorithm has finished when it can't find any new tuple with an
intersection at a packet rate lower than the session maximum.
// Find the element with the lowest bit-rate in X
l=0;
for (i=1:size(X)){
if (X(i).BR <= X(l).BR) & (X(i).OH > X(l).OH) then
l=I;
}
tuple_index = l; // The lowest bit-rate tuple
Y = X(l); // Initilize Y to X(l)
start_pr = 0; // Start from zero bit-rate
do {
current_low = MaxPr; //Reset packet-rate
current_index = tuple_index; // To allow for no intersection
For i=each element in X
pr = (X(i).BR - X(tuple_index).BR) /
(X(i).OH - X(tuple_index).OH)
// Calculate packet rate compared to element i
If (pr < current_low && pr > start_pr) then {
// Update lowest intersection packet rate
current_low = pr;
current_index = i;
}
}
If (current_index != tuple_index) {
// A tuple intersecting below maxpacket rate
Y(size(Y)+1) = X(current_index) // Add to Y
tuple_index = current_index; // Update which to compare with
start_pr = current_low; // Update packet rate to seek from.
}
} while (current_low < MaxPr)
The above algorithm yields the set of applicable restriction Y.
3.5.4.3. Use of TMMBR in a Mixer based Multi-point operation
Assume a small mixer-based multiparty conference is ongoing, as Assume a small mixer-based multiparty conference is ongoing, as
depicted in Topo-Mixer of [Topologies]. All participants (A-D) have depicted in Topo-Mixer of [Topologies]. All participants have
negotiated a common maximum bit-rate that this session can use. The negotiated a common maximum bit-rate that this session can use. The
conference operates over a number of unicast links between the conference operates over a number of unicast paths between the
participants and the MCU. The congestion situation on each of these participants and the mixer. The congestion situation on each of
links can easily be monitored by the participant in question and by these paths can be monitored by the participant in question and by
the MCU, utilizing, for example, RTCP Receiver Reports. However, any the mixer, utilizing, for example, RTCP Receiver Reports or the
given participant has no knowledge of the congestion situation of the transport protocol, e.g. DCCP [RFC4340]. However, any given
participant has no knowledge of the congestion situation of the
connections to the other participants. Worse, without mechanisms connections to the other participants. Worse, without mechanisms
similar to the ones discussed in this draft, the MCU (who is aware of similar to the ones discussed in this draft, the mixer (who is aware
the congestion situation on all connections it manages) has no of the congestion situation on all connections it manages) has no
standardized means to inform participants to slow down, short of standardized means to inform media senders to slow down, short of
forging its own receiver reports (which is undesirable). In forging its own receiver reports (which is undesirable). In
principle, an MCU confronted with such a situation is obliged to thin principle, a mixer confronted with such a situation is obliged to
or transcode streams intended for connections that detected thin or transcode streams intended for connections that detected
congestion. congestion.
In practice, stream thinning - if performed media aware - is In practice, media-aware stream thinning is unfortunately a very
unfortunately a very difficult and cumbersome operation and adds difficult and cumbersome operation and adds undesirable delay. If
undesirable delay. If done media unaware, it leads very quickly to media-unaware, it leads very quickly to unacceptable reproduced media
unacceptable reproduced media quality. Hence, means to slow down quality. Hence, means to slow down senders even in the absence of
senders even in the absence of congestion on their connections to the congestion on their connections to the mixer are desirable.
MCU are desirable.
To allow the MCU to perform congestion control on the individual To allow the mixer to perform congestion control on the individual
links, without performing transcoding, there is a need for a links, without performing transcoding, there is a need for a
mechanism that enables the MCU to request the participant's media mechanism that enables the mixer to request the participant's media
encoders to limit their maximum media bit-rate currently used. The encoders to limit their Maximum Media Stream bit-rate currently used.
MCU handles the detection of a congestion state between itself and a The mixer handles the detection of a congestion state between itself
participant as follows: and a participant as follows:
1. Start thinning the media traffic to the supported bit-rate. 1. Start thinning the media traffic to the supported bit-rate.
2. Use the TMMBR to request the media sender(s) to reduce the media 2. Use the TMMBR to request the media sender(s) to reduce the media
bit-rate sent by them to the MCU, to a value that is in compliance bit-rate sent by them to the mixer, to a value that is in
with congestion control principles for the slowest link. Slow compliance with congestion control principles for the slowest
refers here to the available bandwidth and packet rate after link. Slow refers here to the available
congestion control. bandwidth/bitrate/capacity and packet rate after congestion
control.
3. As soon as the bit-rate has been reduced by the sending part, the 3. As soon as the bit-rate has been reduced by the sending part, the
MCU stops stream thinning implicitly, because there is no need for mixer stops stream thinning implicitly, because there is no need
it any more as the stream is in compliance with congestion for it any more as the stream is in compliance with congestion
control. control.
Above algorithms may suggest to some that there is no need for the Above algorithms may suggest to some that there is no need for the
TMMBR - it should be sufficient to solely rely on stream thinning. TMMBR - it should be sufficient to solely rely on stream thinning.
As much as this is desirable from a network protocol designer's As much as this is desirable from a network protocol designer's
viewpoint, it has the disadvantage that it doesn't work very viewpoint, it has the disadvantage that it doesn't work very
well - the reproduced media quality quickly becomes unusable. well - the reproduced media quality quickly becomes unusable.
It appears to be a reasonable compromise to rely on stream thinning It appears to be a reasonable compromise to rely on stream thinning
as an immediate reaction tool to combat congestions, and have a quick as an immediate reaction tool to combat congestions, and have a quick
control mechanism that instructs the original sender to reduce its control mechanism that instructs the original sender to reduce its
bitrate. bitrate.
Note also that the standard RTCP receiver report cannot serve for the Note also that the standard RTCP receiver report cannot serve for the
purpose mentioned. In an environment with RTP Mixers, the RTCP RR is purpose mentioned. In an environment with RTP mixers, the RTCP RR is
being sent between the RTP receiver in the endpoint and the RTP being sent between the RTP receiver in the endpoint and the RTP
sender in the Mixer only - as there is no multicast transmission. sender in the mixer only - as there is no multicast transmission.
The stream that needs to be bandwidth-reduced, however, is the one The stream that needs to be bitrate-reduced, however, is the one
between the original sending endpoint and the Mixer. This endpoint between the original sending endpoint and the mixer. This endpoint
doesn't see the aforementioned RTCP RRs, and hence needs explicitly doesn't see the aforementioned RTCP RRs, and hence needs to be
informed about desired bandwidth adjustments. explicitly informed about desired bitrate adjustments.
In this topology it is the Mixer's responsibility to collect, and In this topology it is the mixer's responsibility to collect, and
consider jointly, the different bit-rates which the different links consider jointly, the different bit-rates which the different links
may support, into the bit rate requested. This aggregation may also may support, into the bit rate requested. This aggregation may also
take into account that the Mixer may contain certain transcoding take into account that the mixer may contain certain transcoding
capabilities (as discussed in under Topo-Mixer in [Topologies]), capabilities (as discussed in under Topo-Mixer in [Topologies]),
which can be employed for those few of the session participants that which can be employed for those few of the session participants that
have the lowest available bit-rates. have the lowest available bit-rates.
3.5.4.2. Point-to-Multipoint using Multicast or Translators 3.5.4.4. Use of TMMBR in Point-to-Multipoint using Multicast or
Translators
In these topologies, corresponding to Topo-Multicast or Topo- In these topologies, corresponding to Topo-Multicast or Topo-
Translator RTCP RRs are transmitted globally which allows for the Translator RTCP RRs are transmitted globally which allows for the
detection of transmission problems such as congestion, on a medium detection of transmission problems such as congestion, on a medium
timescale. As all media senders are aware of the congestion timescale. As all media senders are aware of the congestion
situation of all media receivers, the rationale of the use of TMMBR situation of all media receivers, the rationale of the use of TMMBR
of section 3.5.4.1 does not apply. However, even in this case the of section 3.5.4.3 does not apply. However, even in this case the
congestion control response can be improved when the unicast links congestion control response can be improved when the unicast links
are employing congestion controlled transport protocols (such as TCP are employing congestion controlled transport protocols (such as TCP
or DCCP). A peer may also report local limitation to the media or DCCP). A peer may also report local limitation to the media
sender. sender.
3.5.4.3. Point-to-point operation 3.5.4.5. Use of TMMBR in Point-to-point operation
In use case 7 it is possible to use TMMBR to improve the performance In use case 7 it is possible to use TMMBR to improve the performance
at times of changes in the known upper limit of the bit-rate. In at times of changes in the known upper limit of the bit-rate. In
this use case the signalling protocol has established an upper limit this use case the signaling protocol has established an upper limit
for the session and media bit-rates. However at the time of for the session and media bit-rates. However, at the time of
transport link bit-rate reduction, a receiver could avoid serious transport link bit-rate reduction, a receiver could avoid serious
congestion by sending a TMMBR to the sending side. congestion by sending a TMMBR to the sending side. Thus TMMBR is
useful for putting restrictions on the application and thus placing
the congestion control mechanism in the right ballpark. However TMMBR
is usually unable to have continuously quick feedback loop required
for real congestion control. Its semantics is also not a match for
congestion control due to its different purpose. Because of these
reasons TMMBR SHALL NOT be used for congestion control.
3.5.4.4. Reliability 3.5.4.6. Reliability
The reaction of a media sender to the reception of a TMMBR message is The reaction of a media sender to the reception of a TMMBR message is
not immediately identifiable through inspection of the media stream. not immediately identifiable through inspection of the media stream.
Therefore a more explicit mechanism is needed to avoid unnecessary Therefore, a more explicit mechanism is needed to avoid unnecessary
re-sending of TMMBR messages. Using a statistically based re-sending of TMMBR messages. Using a statistically based
retransmission scheme would only provide statistical guarantees of retransmission scheme would only provide statistical guarantees of
the request being received. It would also not avoid the the request being received. It would also not avoid the
retransmission of already received messages. In addition it does not retransmission of already received messages. In addition, it does not
allow for easy suppression of other participants requests. For the allow for easy suppression of other participants requests. For the
reasons mentioned, a mechanism based on explicit notification is reasons mentioned, a mechanism based on explicit notification is
used. used, as discussed already in section 3.5.4.1.
Upon the reception of a request a media sender sends a notification Upon the reception of a request a media sender sends a notification
containing the current applicable limitation of the bit-rate, and containing the current applicable limitation of the bit-rate, and
which session participants that own that limit. That allows all other which session participants that own that limit. In multicast
participants to suppress any request they may have, with limitation scenarios, that allows all other participants to suppress any request
value equal or higher to the current one. The identity of the owner they may have, with limitation values less strict than the current
allows for small message sizes and media sender states. A media ones. The identity of the owners allows for small message sizes and
sender only keeps state for the SSRC of the current owner of the media sender states. A media sender only keeps state for the SSRCs of
limitation; all other requests and their sources are not saved. Only the current owners of the limitations; all other requests and their
the participant with the lowest value is allowed to remove or change sources are not saved. Only the owners are allowed to remove or
its limitation. Otherwise anyone that ever set a limitation would change its limitation. Otherwise, anyone that ever set a limitation
need to remove it to allow the maximum bit-rate to be raised beyond would need to remove it to allow the maximum bit-rate to be raised
that value. beyond that value.
4. RTCP Receiver Report Extensions 4. RTCP Receiver Report Extensions
This memo specifies six new feedback messages. The Full Intra Request This memo specifies six new feedback messages. The Full Intra Request
(FIR), Temporal-Spatial Trade-off Request (TSTR), Temporal-Spatial (FIR), Temporal-Spatial Trade-off Request (TSTR), Temporal-Spatial
Trade-off Announcement (TSTA), and Video Back Channel Message (VBCM) Trade-off Notification (TSTN), and Video Back Channel Message (VBCM)
are "Payload Specific Feedback Messages" in the sense of section 6.3 are "Payload Specific Feedback Messages" as defined in Section 6.3 of
of AVPF [RFC4585]. The Temporary Maximum Media Bit-rate Request AVPF [RFC4585]. The Temporary Maximum Media Stream Bit-rate Request
(TMMBR) and Temporary Maximum Media Bit-rate Notification (TMMBN) are (TMMBR) and Temporary Maximum Media Stream Bit-rate Notification
"Transport Layer Feedback Messages" in the sense of section 6.2 of (TMMBN) are "Transport Layer Feedback Messages" as defined in Section
AVPF. 6.2 of AVPF.
In the following subsections, the new feedback messages are defined, In the following subsections, the new feedback messages are defined,
following a similar structure as in the AVPF specification's sections following a similar structure as in the AVPF specification's sections
6.2 and 6.3, respectively. 6.2 and 6.3, respectively.
4.1. Design Principles of the Extension Mechanism 4.1. Design Principles of the Extension Mechanism
RTCP was originally introduced as a channel to convey presence, RTCP was originally introduced as a channel to convey presence,
reception quality statistics and hints on the desired media coding. reception quality statistics and hints on the desired media coding.
A limited set of media control mechanisms have been introduced in A limited set of media control mechanisms have been introduced in
early RTP payload formats for video formats, for example in RFC 2032 early RTP payload formats for video formats, for example in RFC 2032
[RFC2032]. However, this specification, for the first time, suggests [RFC2032]. However, this specification, for the first time, suggests
a two-way handshake for one of its messages. There is danger that a two-way handshake for some of its messages. There is danger that
this introduction could be misunderstood as the precedence for the this introduction could be misunderstood as the precedence for the
use of RTCP as an RTP session control protocol. In order to prevent use of RTCP as an RTP session control protocol. In order to prevent
these misunderstandings, this subsection attempts to clarify the these misunderstandings, this subsection attempts to clarify the
scope of the extensions specified in this memo, and strongly suggests scope of the extensions specified in this memo, and strongly suggests
that future extensions follow the rationale spelled out here, or that future extensions follow the rationale spelled out here, or
compellingly explain why they divert from the rationale. compellingly explain why they divert from the rationale.
In this memo, and in AVPF [RFC4585], only such messages have been In this memo, and in AVPF [RFC4585], only such messages have been
included which included which
a) have comparatively strict real-time constraints, which prevent the a) have comparatively strict real-time constraints, which prevent the
use of mechanisms such as a SIP re-invite in most application use of mechanisms such as a SIP re-invite in most application
scenarios. The real-time constraints are explained separately for scenarios. The real-time constraints are explained separately for
each message where necessary each message where necessary
b) are multicast-safe in that the reaction to potentially b) are multicast-safe in that the reaction to potentially
contradicting feedback messages is specified, as necessary for contradicting feedback messages is specified, as necessary for
each message each message
c) are directly related to activities of a certain media codec, class c) are directly related to activities of a certain media codec, class
of media codecs (e.g. video codecs), or the given media stream. of media codecs (e.g. video codecs), or a given RTP packet stream.
In this memo, a two-way handshake is only introduced for such In this memo, a two-way handshake is only introduced for such
messages that messages that
a) require a notification or acknowledgement due to their nature, a) require a notification or acknowledgement due to their nature,
which is motivated separately for each message which is motivated separately for each message
b) the notification or acknowledgement cannot be easily derived from b) the notification or acknowledgement cannot be easily derived from
the media bit stream. the media bit stream.
All messages in AVPF [RFC4585] and in this memo follow a number of All messages in AVPF [RFC4585] and in this memo implement their
common design principles. In particular: codepoints in a simple, fixed binary format. The reason behind this
design principle lies in that media receivers do not always implement
a) Media receivers are not always implementing higher control higher control protocol functionalities (SDP, XML parsers and such)
protocol functionalities (SDP, XML parsers and such) in their in their media path. Therefore, simple binary representations are
media path. Therefore, simple binary representations are used in used in the feedback messages and not an (otherwise desirable)
the feedback messages and not an (otherwise desirable) flexible flexible format such as, for example, XML.
format such as, for example, XML.
4.2. Transport Layer Feedback Messages 4.2. Transport Layer Feedback Messages
Transport Layer FB messages are identified by the value RTPFB (205) Transport Layer FB messages are identified by the value RTPFB (205)
as RTCP packet type. as RTCP packet type (see section 6.1 of RFC 4585 [RFC4585].
In AVPF, one message of this category had been defined. This memo In AVPF, one message of this category had been defined. This memo
specifies two more messages for a total of three messages of this specifies two more messages, for a total of three messages of this
type. They are identified by means of the FMT parameter as follows: type. They are identified by means of the FMT parameter as follows:
0: unassigned 0: unassigned
1: Generic NACK (as per AVPF) 1: Generic NACK (as per AVPF)
2: Temporary Maximum Media Bit-rate Request 2: reserved (see note below)
3: Temporary Maximum Media Bit-rate Notification 3: Temporary Maximum Media Stream Bit-rate Request (TMMBR)
4-30: unassigned 4: Temporary Maximum Media Stream Bit-rate Notification (TMMBN)
5-30: unassigned
31: reserved for future expansion of the identifier number space 31: reserved for future expansion of the identifier number space
Note: early drafts of AVPF [RFC4585] reserved FMT=2 for a
codepoint that has later been removed. It has been pointed
out that there may be implementations in the field using this
value for according to the expired draft. As there is
sufficient numbering space available, we mark FMT=2 as
reserved so to avoid possible interoperability problems with
implementations that are standard-incompliant with respect to
RFC 4585 in this very point.
The following subsection defines the formats of the FCI field for The following subsection defines the formats of the FCI field for
this type of FB message. this type of FB message.
4.2.1. Temporary Maximum Media Bit-rate Request (TMMBR) 4.2.1. Temporary Maximum Media Stream Bit-rate Request and Notification
The FCI field of a TMMBR Feedback message SHALL contain one or more The FCI field of a Temporary Maximum Media Stream Bit-Rate Request
FCI entries. (TMMBR) message SHALL contain one or more FCI entries.
4.2.1.1. Semantics 4.2.1.1. Semantics
The TMMBR is used to indicate the highest bit-rate per sender of a TMMBR is used to indicate the transport related limitation in the
media, which the receiver currently supports in this RTP session. form of a tuple. The first value is the highest bit-rate per sender
The media sender MAY use any lower bit-rate, as it may need to of a media, which the receiver currently supports in this RTP session
address a congestion situation or other limiting factors. See observed at a particular protocol layer. The second value is the
section 5 (congestion control) for more discussion. measured header overhead in bytes on the packets received for the
stream. Counting from the start of the header on the protocol layer
for which the bit-rate is reported until the RTP payload's start.
The measurement of the overhead is a running averaging that is
updated for each packet received for this particular media source
(SSRC). For each packet received the overhead is calculated (pckt_OH)
and then added to the average overhead (avg_OH) by calculating:
avg_OH = 15/16*avg_OH + 1/16*pckt_OH.
The "SSRC of the packet sender" field indicates the source of the The bit-rate values used in this formats are averaged out over a
request, and the "SSRC of media source" is not used and SHALL be set reasonable timescale. What reasonable timescales are, depends on the
to 0. The SSRC of media sender in the FCI field denotes the media application. However the goal is be able to ignore any burstiness on
very short timescales, below for example 100 ms, introduced by
scheduling or link layer packetization effects.
The media sender MAY use any combination of packet rate and RTP
payload bit-rate to produce a lower media stream bit-rate, as it may
need to address a congestion situation or other limiting factors.
See section 5 . (congestion control) for more discussion.
The ''SSRC of the packet sender'' field indicates the source of the
request, and the ''SSRC of media source'' is not used and SHALL be
set to 0. The SSRC of media sender in the FCI field denotes the media
sender the message applies to. This is useful in the multicast or sender the message applies to. This is useful in the multicast or
translator topologies where each media sender may be addressed in a translator topologies where each media sender may be addressed in a
single TMMBR message using multiple FCIs. single TMMBR message using multiple FCIs.
A TMMBR FCI MAY be repeated in subsequent TMMBR messages if no A TMMBR FCI MAY be repeated in subsequent TMMBR messages if no
applicable TMMBN FCI has been received at the time of transmission of applicable Temporal Maximum Media Stream Bit-Rate Notification
the next RTCP packet. The bit-rate value of a TMMBR FCI MAY be (TMMBN) FCI has been received at the time of transmission of the next
changed from a previous TMMBR message and the next, regardless of the RTCP packet. A TMMBN is applicable if it either indicate the sender
eventual reception of an applicable TMMBN FCI. of the TMMBR as an owner, or contains limitations that are stricter
than one sent in the TMMBR message. The bit-rate value of a TMMBR
FCI MAY be changed from a previous TMMBR message and the next,
regardless of the eventual reception of an applicable TMMBN FCI. The
overhead measurement SHALL be updated to the current value of avg_OH.
Please note that a TMMBN message SHALL be sent by the media sender at A TMMBN message SHALL be sent by the media sender at the earliest
the earliest possible point in time, as a result of any TMMBR possible point in time, as a result of any TMMBR messages received
messages received since the last sending of TMMBN. The TMMBN message since the last sending of TMMBN. The TMMBN message indicates the
indicates the limit and the owner of that limit at the time of the limits and the owners of those limits at the time of the transmission
transmission of the message. The limit is the lowest of the previous of the message. The limits SHALL be set to the set of the stricts
value and all values received in TMMBR FCI's since the last TMMBN was limits of the previous limits and all limits received in TMMBR FCI's
transmitted. since the last TMMBN was transmitted.
A media receiver who is not the owner of the bandwidth limit when A media receiver considering sending a TMMBR, who is not a ''owner''
planning to send a TMMBR, SHOULD request a bandwidth lower than their of a limitation, SHOULD request a limitation stricter than their
knowledge of currently established bandwidth limit for this media knowledge of the currently established limits for this media sender,
sender, or suppres their transmission for TMMBR. The exception to or suppress their transmission of the TMMBR. The exception to the
the above rule is when a receiver either doesn't know the limit or above rule is when a receiver either doesn't know the limit or is
are certain that their local representation of the value is in error. certain that their local representation of the set of limitations are
All received requests for bandwidth limits greater or equal to the in error. All received requests for limits equally or less strict
one currently established are ignored, with the exception of them compared to the ones currently established MUST BE ignored, with the
resulting in the transmission of a TMMBN. A media receiver who is exception of them resulting in the transmission of a TMMBN containg
the owner of the current bandwidth limit, MAY lower the value the current set of limitations. A media receiver who is the owner of
further, raise the value or remove the restriction completely by a current limitation MAY lower the value further, raise the value or
setting the bandwidth limit equal to the session limit. remove the restriction completely by setting the bit-rate part of the
limit equal to the session bit-rate limit.
A limitation tuple LT can be determined to be stricter or not
compared to the current set of limitations if LT is part of the set Y
produced by the algorithm described in Section 3.5.4.2.
Once a session participant receives the TMMBN in response to its Once a session participant receives the TMMBN in response to its
TMMBR, with its own SSRC, it knows that it "owns" the bandwidth TMMBR, with its own SSRC, it knows that it "owns" the bitrate
limitation. Only the "owner" of a bandwidth limitation can raise it limitation. Only the "owner" of a bitrate limitation can raise it or
or reset it to the session limit. reset it to the session limit.
Note that, due to the unreliable nature of transport of TMMBR and Note that, due to the unreliable nature of transport of TMMBR and
TMMBN, the above rules may lead to the sending of TMMBR messages TMMBN, the above rules may lead to the sending of TMMBR messages
disobeying the rules above. Furthermore, in multicast scenarios it disobeying the rules above. Furthermore, in multicast scenarios it
can happen that more than one session participants believes it "owns" can happen that more than one session participants believes it "owns"
the current bandwidth limitation. This is not critical for a number the current bitrate limitation. This is not critical for a number of
of reasons: reasons:
a) If a TMMBR message is lost in transmission, the media sender does a) If a TMMBR message is lost in transmission, the media sender does
not learn about the restrictions imposed on it. However, it also not learn about the restrictions imposed on it. However, it also
does not send a TMMBN message notifying reception of a request it does not send a TMMBN message notifying reception of a request it
has never received. Therefore, no new limit is established, the has never received. Therefore, no new limit is established, the
media receiver sending the more restrictive TMMBR is not the media receiver sending a more restrictive TMMBR is not the owner.
owner. Since this media receiver has not seen a notification Since this media receiver has not seen a notification
corresponding to its request, it is free to re-send it. corresponding to its request, it is free to re-send it.
b) Similarly, if a TMMBN message gets lost, the media receiver that b) Similarly, if a TMMBN message gets lost, the media receiver that
has sent the corresponding TMMBR request does not receive has sent the corresponding TMMBR request does not receive the
acknowledgement. In that case, it is also not the "owner" of the Notification. In that case, it is also not the "owner" of the
restriction and is free to re-send the request. restriction and is free to re-send the request.
c) If multiple competing TMMBR messages are sent by different session c) If multiple competing TMMBR messages are sent by different session
participants, then the resulting TMMBN indicates the lowest participants, then the resulting TMMBN indicates the most
bandwidth requested; the owner is set to the sender of the TMMBR restrictive limits requested including its owners.
with the lowest requested bandwidth value.
TMMBR feedback SHOULD NOT be used if the underlying transport d) If more than one session participant incidently send TMMBR
protocol is capable of providing similar feedback information from messages at the same time and with the same limit, the media
the receiver to the sender. sender selects one of them and addresses it as the ''owner''.
Session-wide, the correct limit is thereby established.
It also important to consider the security risks involved with faked It is also important to consider the security risks involved with
TMMBRs. See security considerations in Section 6. faked TMMBRs. See security considerations in Section 6.
The feedback messages may be used in both multicast and unicast The feedback messages may be used in both multicast and unicast
sessions of any of the specified topologies. sessions of any of the specified topologies.
For sessions with a larger number of participants using the lowest For sessions with a large number of participants using the lowest
common denominator, as required by this mechanism, may not be the common denominator, as required by this mechanism, may not be the
most suitable course of action. Larger session may need to consider most suitable course of action. Large session may need to consider
other ways to support adapted bit-rate to participants, such as other ways to support adapted bit-rate to participants, such as
partitioning the session in different quality tiers, or use some partitioning the session in different quality tiers, or use some
other method of achieving bit-rate scalability. other method of achieving bit-rate scalability.
If the value set by a TMMBR message is expected to be permanent the If the value set by a TMMBR message is expected to be permanent, the
TMMBR setting party is RECOMMENDED to renegotiate the session TMMBR setting party is RECOMMENDED to renegotiate the session
parameters to reflect that using the setup signalling. parameters to reflect that using session setup signaling, e.g. a SIP
re-invite.
An SSRC may time out according to the default rules for RTP session An SSRC may time out according to the default rules for RTP session
participants, i.e. the media sender has not received any RTCP packet participants, i.e. the media sender has not received any RTCP packet
from the owner for the last five regular reporting intervals. An SSRC from the owner for the last five regular reporting intervals. An SSRC
may also leave the session, indicating this through the transmission may also leave the session, indicating this through the transmission
of an RTCP BYE packet or an external signalling channel. In all of of an RTCP BYE packet or an external signaling channel. In all of
these cases the entity is considered to have left the session. In the these cases the entity is considered to have left the session. In the
case the "owner" leaves the session, the value SHALL be set to the case the "owner" leaves the session, the limit SHALL be removed and
session maximum and the transmission of a TMMBN is scheduled. the transmission of a TMMBN is scheduled indicating the remaining
limitations.
4.2.1.2. Message Format 4.2.1.2. Message Format
The Feedback control information (FCI) consists of one or more TMMBR
The Feedback Control Information (FCI) consists of one or more TMMBR
FCI entries with the following syntax: FCI entries with the following syntax:
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC | | SSRC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Maximum bit-rate in units of 128 bits/s | | MMBR Exp | MMBR Mantissa |Measured Overhead|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 1 - Syntax for the TMMBR message Figure 1 - Syntax for the TMMBR message
SSRC: The SSRC value of the media sender that is requested to
obey the new maximum bit-rate).
MMBR Exp (6 bits): The exponential scaling of the mantissa for the
Maximum Media Stream bit-rate value. The value is non
signed integer [0..63].
SSRC: The SSRC value of the target of this specific maximum bit- MMBR Mantissa (17 bits): The mantissa of the Maximum Media Stream
rate request. Bit-rate value as a non-signed integer.
Maximum bit-rate: The temporary maximum media bit-rate value in Measured Overhead (9 bits): The measured average packet overhead
units of 128 bit/s. This provides range from 0 to value in bytes. The measurement SHALL be done according to
549755813888 bits/s (~550 Tbit/s) with a granularity of 128 above description in Section 4.2.1.1.
bits/s.
The maximum media stream bit-rate (MMBR) value in bits per second is
calculated from the MMBR exponent (exp) and mantissa in the following
way:
MMBR = mantissa * 2^exp
This allows for 17 bits of resolution in the range 0 to 131072*2^63
(approximately 1.2*10^24).
The length of the FB message is be set to 2+2*N where N is the number The length of the FB message is be set to 2+2*N where N is the number
of TMMBR FCI entries. of TMMBR FCI entries.
4.2.1.3. Timing Rules 4.2.1.3. Timing Rules
The first transmission of the request message MAY use early or The first transmission of the request message MAY use early or
immediate feedback in cases when timeliness is desirable. Any immediate feedback in cases when timeliness is desirable. Any
repetition of a request message SHOULD use regular RTCP mode for its repetition of a request message SHOULD use regular RTCP mode for its
transmission timing. transmission timing.
4.2.2. Temporary Maximum Media Bit-rate Notification (TMMBN) 4.2.1.4. Handling in Translator and Mixers
The FCI field of the TMMBN Feedback message SHALL contain one TMMBN Media Translators and Mixers will need to receive and respond to
FCI entry. TMMBR messages as they are part of the chain that provides a certain
media stream to the receiver. The mixer or translator may act locally
on the TMMBR request and thus generate a TMMBN to indicate that it
has done so. Alternatively it can forward the request in the case of
a media translator, or generate one of itself in the case of the
mixer. In case it generates a TMMBR, it will need to send a TMMBN
back to the original requestor to indicate that it is handling the
request.
4.2.2. Temporary Maximum Media Stream Bit-rate Notification (TMMBN)
The FCI field of the TMMBN Feedback message may contain zero, one or
more TMMBN FCI entry.
4.2.2.1. Semantics 4.2.2.1. Semantics
This feedback message is used to notify the senders of any TMMBR This feedback message is used to notify the senders of any TMMBR
message that one or more TMMBR messages have been received. It message that one or more TMMBR messages have been received or that a
indicates to all participants the currently employed maximum bit-rate owner has left the session. It indicates to all participants the set
value and the "owner" of the current limitation. The "owner" of a of currently employed limitations and the ''owners'' of those.
limitation is the sender of the last (most restrictive) TMMBR message
received by the media sender.
The "SSRC of the packet sender" field indicates the source of the The ''SSRC of the packet sender'' field indicates the source of the
notification. The "SSRC of media source" SHALL be set to the SSRC of notification. The ''SSRC of media source'' is not used and SHALL be
the media receiver that currently owns the bit-rate limitation. set to 0.
A TMMBN message SHALL be scheduled for transmission after the A TMMBN message SHALL be scheduled for transmission after the
reception of a TMMBR message with a FCI including the session reception of a TMMBR message with a FCI identifying this media
participant's SSRC. Only a single TMMBN SHALL be sent, even if more sender. Only a single TMMBN SHALL be sent, even if more than one
than one TMMBR messages are received between the scheduling of the TMMBR messages are received between the scheduling of the
transmission and the actual transmission of the TMMBN message. The transmission and the actual transmission of the TMMBN message. The
TMMBN message indicates the limit and the owner of that limit at the TMMBN message indicates the limits and their owners at the time of
time of transmitting the message. The limit SHALL be the lowest of transmitting the message. The limits included SHALL be the set of
the existing and all values received in TMMBR messages since the last most restrictive values in the previously established set and
TMMBN was transmitted. The one sending that request SHALL become the received TMMBR messages since the last TMMBN was transmitted.
owner of the limit.
The reception of a TMMBR message with a transmission limit greater or The reception of a TMMBR message with a transmission limit equally or
equal than the current limit SHALL still result in the transmission less restrictive than the set of current limits SHALL still result in
of a TMMBN message. However the limit and owner is not changed, the transmission of a TMMBN message. However the limits and their
unless it was from the same owner, and the current limit and owner is owners are not changed, unless it was from an owner of a limit within
indicated in the TMMBN message. This procedure allows session the current set of limitations. This procedure allows session
participants that haven't seen the last TMMBN message to get a participants that haven't seen the last TMMBN message to get a
correct view of this media sender's state. correct view of this media sender's state.
When a media sender determines an "owner" of a limitation has left When a media sender determines an ''owner'' of a limitation has left
the session, then the current limitation is removed, and the media the session, then that limitation is removed, and the media sender
sender SHALL send a TMMBN message indicating the maximum session SHALL send a TMMBN message indicating the remaining limitations. In
bandwidth. case there are no remaining limitations a TMMBN without any FCI SHALL
be sent to indicate this.
In unicast scenarios (i.e. where a single sender talks to a single
receiver), the aforementioned algorithm to determine ownership
degenerates to the media receiver becoming the ''owner'' as soon as
the media receiver has issued the first TMMBR message.
4.2.2.2. Message Format 4.2.2.2. Message Format
The TMMBN Feedback control information (FCI) entry has the following The Feedback Control Information (FCI) consists of zero, one or more
syntax: TMMBN FCI entries with the following syntax:
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Maximum bit-rate in units of 128 bits/s | | SSRC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MMBR Exp | MMBR Mantissa |Measured Overhead|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 2 - Syntax for the TMMBN message Figure 2 - Syntax for the TMMBR message
Maximum bit-rate: The current temporary maximum media bit-rate SSRC: The SSRC value of the ''owner'' of this limitation.
value in units of 128 bit/s.
The length field value of the FB message SHALL be 3. MMBR Exp (6 bits): The exponential scaling of the mantissa for the
Maximum Media Stream bit-rate value. The value is non-
signed integer [0..63].
MMBR Mantissa (17 bits): The mantissa of the Maximum Media Stream
Bit-rate value as non-signed integer.
Measured Overhead (9 bits): The measured average packet overhead
value in bytes represented as non-signed integer.
Thus the FCI contains blocks indicating the applicable limitations as
the owner followed by the applicable maximum media stream bit-rate
and overhead value.
The length of the FB message is be set to 2+2*N where N is the number
of TMMBR FCI entries.
4.2.2.3. Timing Rules 4.2.2.3. Timing Rules
The acknowledgement SHOULD be sent as soon as allowed by the applied The acknowledgement SHOULD be sent as soon as allowed by the applied
timing rules for the session. Immediate or early feedback mode SHOULD timing rules for the session. Immediate or early feedback mode SHOULD
be used for these messages. be used for these messages.
4.2.2.4. Handling by Translators and Mixers
As discussed in Section 4.2.1.4 mixer or translators may need to
issue TMMBN messages as response to TMMBR messages handled by the
mixer or translator.
4.3. Payload Specific Feedback Messages 4.3. Payload Specific Feedback Messages
Payload-Specific FB messages are identified by the value PT=PSFB Payload-Specific FB messages are identified by the value PT=PSFB
(206) as RTCP packet type. (206) as RTCP packet type (see section 6.1 of RFC 4585 [RFC4585]).
AVPF defines three payload-specific FB messages and one application AVPF defines three payload-specific FB messages and one application
layer FB message. This memo specifies four additional payload layer FB message. This memo specifies four additional payload-
specific feedback messages. All are identified by means of the FMT specific feedback messages. All are identified by means of the FMT
parameter as follows: parameter as follows:
0: unassigned 0: unassigned
1: Picture Loss Indication (PLI) 1: Picture Loss Indication (PLI)
2: Slice Lost Indication (SLI) 2: Slice Lost Indication (SLI)
3: Reference Picture Selection Indication (RPSI) 3: Reference Picture Selection Indication (RPSI)
4: Full Intra Request Command (FIR) 4: Full Intra Request Command (FIR)
5: Temporal-Spatial Trade-off Request (TSTR) 5: Temporal-Spatial Trade-off Request (TSTR)
6: Temporal-Spatial Trade-off Announcement (TSTA) 6: Temporal-Spatial Trade-off Notification (TSTN)
7: Video Back Channel Message (VBCM) 7: Video Back Channel Message (VBCM)
8-14: unassigned 8-14: unassigned
15: Application layer FB message 15: Application layer FB message
16-30: unassigned 16-30: unassigned
31: reserved for future expansion of the number space 31: reserved for future expansion of the number space
The following subsections define the new FCI formats for the payload- The following subsections define the new FCI formats for the payload-
specific FB messages. specific FB messages.
4.3.1. Full Intra Request (FIR) command 4.3.1. Full Intra Request (FIR)
The FIR command FB message is identified by PT=PSFB and FMT=4. The FIR message is identified by PT=PSFB and FMT=4.
There MUST be one or more FIR entry contained in the FCI field. There MUST be one or more FIR entry contained in the FCI field.
4.3.1.1. Semantics 4.3.1.1. Semantics
Upon reception of a FIR message, an encoder MUST send a decoder Upon reception of FIR, the encoder MUST send a Decoder Refresh Point
refresh point (see Section 2.2) as soon as possible. (see Section 2 ..2) as soon as possible.
Note: Currently, video appears to be the only useful application Note: Currently, video appears to be the only useful application
for FIR, as it appears to be the only RTP payloads widely deployed for FIR, as it appears to be the only RTP payloads widely deployed
that relies heavily on media prediction across RTP packet that relies heavily on media prediction across RTP packet
boundaries. However, use of FIR could also reasonably be boundaries. However, use of FIR could also reasonably be
envisioned for other media types that share essential properties envisioned for other media types that share essential properties
with compressed video, namely cross-frame prediction (whatever a with compressed video, namely cross-frame prediction (whatever a
frame may be for that media type). One possible example may be the frame may be for that media type). One possible example may be the
dynamic updates of MPEG-4 scene descriptions. It is suggested that dynamic updates of MPEG-4 scene descriptions. It is suggested that
payload formats for such media types refer to FIR and other message payload formats for such media types refer to FIR and other message
types defined in this specification and in AVPF, instead of types defined in this specification and in AVPF, instead of
creating similar mechanisms in the payload specifications. The creating similar mechanisms in the payload specifications. The
payload specifications may have to explain how the payload specific payload specifications may have to explain how the payload-specific
terminologies map to the video-centric terminology used here. terminologies map to the video-centric terminology used herein.
Note: In environments where the sender has no control over the Note: In environments where the sender has no control over the
codec (e.g. when streaming pre-recorded and pre-coded content), the codec (e.g. when streaming pre-recorded and pre-coded content), the
reaction to this command cannot be specified. One suitable reaction to this command cannot be specified. One suitable
reaction of a sender would be to skip forward in the video bit reaction of a sender would be to skip forward in the video bit
stream to the next decoder refresh point. In other scenarios, it stream to the next decoder refresh point. In other scenarios, it
may be preferable not to react to the command at all, e.g. when may be preferable not to react to the command at all, e.g. when
streaming to a large multicast group. Other reactions may also be streaming to a large multicast group. Other reactions may also be
possible. When deciding on a strategy, a sender could take into possible. When deciding on a strategy, a sender could take into
account factors such as the size of the receiving group, the account factors such as the size of the receiving group, the
"importance" of the sender of the FIR message (however "importance" ''importance'' of the sender of the FIR message (however
may be defined in this specific application), the frequency of ''importance'' may be defined in this specific application), the
decoder refresh points in the content, and others. However a frequency of Decoder Refresh Points in the content, and others.
session which predominately handles pre-coded content shouldn't use However a session which predominately handles pre-coded content is
the FIR at all. not expected to use FIR at all.
The sender MUST consider congestion control as outlined in section 5, The sender MUST consider congestion control as outlined in section 5 .,
which MAY restrict its ability to send a decoder refresh point which MAY restrict its ability to send a decoder refresh point
quickly. quickly.
Note: The relationship between the Picture Loss Indication and FIR Note: The relationship between the Picture Loss Indication and FIR
is as follows. As discussed in section 6.3.1 of AVPF, a Picture is as follows. As discussed in section 6.3.1 of AVPF, a Picture
Loss Indication informs the decoder about the loss of a picture and Loss Indication informs the decoder about the loss of a picture and
hence the likeliness of misalignment of the reference pictures in hence the likeliness of misalignment of the reference pictures in
encoder and decoder. Such a scenario is normally related to losses the encoder and decoder. Such a scenario is normally related to
in an ongoing connection. In point-to-point scenarios, and without losses in an ongoing connection. In point-to-point scenarios, and
the presence of advanced error resilience tools, one possible without the presence of advanced error resilience tools, one
option an encoder has is to send a decoder refresh point. However, possible option of an encoder consists in sending a Decoder Refresh
there are other options including ignoring the PLI, for example if Point. However, there are other options. One example is that the
only one receiver of many has sent a PLI or when the embedded media sender ignores the PLI, because the embedded stream
stream redundancy is likely to clean up the reproduced picture redundancy is likely to clean up the reproduced picture within a
within a reasonable amount of time. The FIR, in contrast, leaves a reasonable amount of time. The FIR, in contrast, leaves a (real-
real-time encoder no choice but to send a decoder refresh point. time) encoder no choice but to send a Decoder Refresh Point. It
It disallows the encoder to take into account any considerations disallows the encoder to take into account any considerations such
such as the ones mentioned above. as the ones mentioned above.
Note: Mandating a maximum delay for completing the sending of a Note: Mandating a maximum delay for completing the sending of a
decoder refresh point would be desirable from an application Decoder Refresh Point would be desirable from an application
viewpoint, but may be problematic from a congestion control point viewpoint, but may be problematic from a congestion control point
of view. "As soon as possible" as mentioned above appears to be a of view. ''As soon as possible'' as mentioned above appears to be
reasonable compromise. a reasonable compromise.
FIR SHALL NOT be sent as a reaction to picture losses - it is FIR SHALL NOT be sent as a reaction to picture losses -- it is
RECOMMENDED to use PLI instead. FIR SHOULD be used only in such RECOMMENDED to use PLI instead. FIR SHOULD be used only in such
situations where not sending a decoder refresh point would render the situations where not sending a decoder refresh point would render the
video unusable for the users. video unusable for the users.
Note: a typical example where sending FIR is adequate is when, in a Note: A typical example where sending FIR is appropriate is when,
multipoint conference, a new user joins the session and no regular in a multipoint conference, a new user joins the session and no
decoder refresh point interval is established. Another example regular Decoder Refresh Point interval is established. Another
would be a video switching MCU that changes streams. Here, example would be a video switching MCU that changes streams. Here,
normally, the MCU issues a FIR to the new sender so to force it to normally, the MCU issues a FIR to the new sender so to force it to
emit a decoder refresh point. The decoder refresh point includes emit a Decoder Refresh Point. The Decoder Refresh Point includes
normally a Freeze Picture Release (defined outside this normally a Freeze Picture Release (defined outside this
specification), which re-starts the rendering process of the specification), which re-starts the rendering process of the
receivers. Both techniques mentioned are commonly used in MCU- receivers. Both techniques mentioned are commonly used in MCU-
based multipoint conferences. based multipoint conferences.
Other RTP payload specifications such as RFC 2032 [RFC2032] already Other RTP payload specifications such as RFC 2032 [RFC2032] already
define a feedback mechanism for certain codecs. An application define a feedback mechanism for certain codecs. An application
supporting both schemes MUST use the feedback mechanism defined in supporting both schemes MUST use the feedback mechanism defined in
this specification when sending feedback. For backward compatibility this specification when sending feedback. For backward compatibility
reasons, such an application SHOULD also be capable to receive and reasons, such an application SHOULD also be capable to receive and
react to the feedback scheme defined in the respective RTP payload react to the feedback scheme defined in the respective RTP payload
format, if this is required by that payload format. format, if this is required by that payload format.
The "SSRC of the packet sender" field indicates the source of the The ''SSRC of the packet sender'' field indicates the source of the
request, and the "SSRC of media source" is not used and SHALL be set request, and the ''SSRC of media source'' is not used and SHALL be
to 0. The SSRC of media sender to which the FIR command applies to is set to 0. The SSRC of media sender to which the FIR command applies
in the FCI. to is in the FCI.
4.3.1.2. Message Format 4.3.1.2. Message Format
Full Intra Request uses one additional FCI field, the content of Full Intra Request uses one additional FCI field, the content of
which is depicted in Figure 3 The length of the FB message MUST be which is depicted in Figure 3 The length of the FB message MUST be
set to 2+2*N, where N is the number of FCI entries. set to 2+2*N, where N is the number of FCI entries.
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC | | SSRC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Seq. nr | Reserved | | Seq. nr | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 3 - Syntax for the FIR message Figure 3 - Syntax for the FIR message
SSRC: The SSRC value of the media sender which is requested to
SSRC: The SSRC value of the media sender of this specific FIR send a Decoder Refresh Point.
command.
Seq. nr: Command sequence number. The sequence number space is Seq. nr: Command sequence number. The sequence number space is
unique for each tuple consisting of the SSRC of command unique for each tuple consisting of the SSRC of command
source and the SSRC of the command target. The sequence source and the SSRC of the command target. The sequence
number SHALL be increased by 1 modulo 256 for each new number SHALL be increased by 1 modulo 256 for each new
command. A repetition SHALL NOT increase the sequence command. A repetition SHALL NOT increase the sequence
number. Initial value is arbitrary. number. Initial value is arbitrary.
Reserved: All bits SHALL be set to 0 and SHALL be ignored on Reserved: All bits SHALL be set to 0 and SHALL be ignored on
reception. reception.
The semantics of this FB message is independent of the RTP payload The semantics of this FB message is independent of the RTP payload
type. type.
4.3.1.3. Timing Rules 4.3.1.3. Timing Rules
The timing follows the rules outlined in section 3 of [RFC4585]. FIR The timing follows the rules outlined in section 3 of [RFC4585]. FIR
commands MAY be used with early or immediate feedback. The FIR commands MAY be used with early or immediate feedback. The FIR
feedback message MAY be repeated. If using immediate feedback mode feedback message MAY be repeated. If using immediate feedback mode
the repetition SHOULD wait at least onee RTT before being sent. In the repetition SHOULD wait at least one RTT before being sent. In
early or regular RTCP mode the repetition is sent in the next regular early or regular RTCP mode the repetition is sent in the next regular
RTCP packet. RTCP packet.
4.3.1.4. Remarks 4.3.1.4. Handling of message in Mixer and Translators
FIR messages typically trigger the sending of full intra or IDR A media translator or a mixer performing media encoding of the
pictures. Both are several times larger then predicted (inter) content for which the session participant has issued a FIR is
pictures. Their size is independent of the time they are generated. responsible for acting upon it. A mixer acting upon a FIR SHOULD NOT
In most environments, especially when employing bandwidth-limited forward the message unaltered, instead it SHOULD issue a FIR itself.
links, the use of an intra picture implies an allowed delay that is a
significant multitude of the typical frame duration. An example: If 4.3.1.5. Remarks
the sending frame rate is 10 fps, and an intra picture is assumed to
be 10 times as big as an inter picture, then a full second of latency In conjunction with video codecs, FIR messages typically trigger the
has to be accepted. In such an environment there is no need for a sending of full intra or IDR pictures. Both are several times larger
particular short delay in sending the FIR message. Hence waiting for then predicted (inter) pictures. Their size is independent of the
the next possible time slot allowed by RTCP timing rules as per time they are generated. In most environments, especially when
[RFC4585] may not have an overly negative impact on the system employing bandwidth-limited links, the use of an intra picture
performance. implies an allowed delay that is a significant multitude of the
typical frame duration. An example: If the sending frame rate is 10
fps, and an intra picture is assumed to be 10 times as big as an
inter picture, then a full second of latency has to be accepted. In
such an environment there is no need for a particularly short delay
in sending the FIR message. Hence waiting for the next possible time
slot allowed by RTCP timing rules as per [RFC4585] may not have an
overly negative impact on the system performance.
4.3.2. Temporal-Spatial Trade-off Request (TSTR) 4.3.2. Temporal-Spatial Trade-off Request (TSTR)
The TSTR FB message is identified by PT=PSFB and FMT=5. The TSTR FB message is identified by PT=PSFB and FMT=5.
There MUST be one or more TSTR entry contained in the FCI field. There MUST be one or more TSTR entry contained in the FCI field.
4.3.2.1. Semantics 4.3.2.1. Semantics
A decoder can suggest the use of a temporal-spatial trade-off by A decoder can suggest the use of a temporal-spatial trade-off by
skipping to change at page 34, line 31 skipping to change at page 41, line 28
adjusting its temporal-spatial trade-off, it SHOULD take into account adjusting its temporal-spatial trade-off, it SHOULD take into account
the received TSTR message for future coding of pictures. A value of the received TSTR message for future coding of pictures. A value of
0 suggests a high spatial quality and a value of 31 suggests a high 0 suggests a high spatial quality and a value of 31 suggests a high
frame rate. The values from 0 to 31 indicate monotonically a desire frame rate. The values from 0 to 31 indicate monotonically a desire
for higher frame rate. Actual values do not correspond to precise for higher frame rate. Actual values do not correspond to precise
values of spatial quality or frame rate. values of spatial quality or frame rate.
The reaction to the reception of more than one TSTR message by a The reaction to the reception of more than one TSTR message by a
media sender from different media receivers is left open to the media sender from different media receivers is left open to the
implementation. The selected trade-off SHALL be communicated to the implementation. The selected trade-off SHALL be communicated to the
media receivers by the means of the TSTA message. media receivers by the means of the TSTN message.
The "SSRC of the packet sender" field indicates the source of the The ''SSRC of the packet sender'' field indicates the source of the
request, and the "SSRC of media source" is not used and SHALL be set request, and the ''SSRC of media source'' is not used and SHALL be
to 0. The SSRC of media sender to which the TSTR applies to is in the set to 0. The SSRC of media sender to which the TSTR applies to is in
FCI entries. the FCI entries.
A TSTR message may contain multiple requests to different media A TSTR message may contain multiple requests to different media
senders, using multiple FCI entries. senders, using multiple FCI entries.
4.3.2.2. Message Format 4.3.2.2. Message Format
The Temporal-Spatial Trade-off Request uses one FCI field, the The Temporal-Spatial Trade-off Request uses one FCI field, the
content of which is depicted in Figure 4. The length of the FB content of which is depicted in Figure 4. The length of the FB
message MUST be set to 2+2*N, where N is the number of FCI entries message MUST be set to 2+2*N, where N is the number of FCI entries
included. included.
skipping to change at page 35, line 15 skipping to change at page 42, line 11
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC | | SSRC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Seq nr. | Reserved | Index | | Seq nr. | Reserved | Index |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 4 - Syntax of the TSTR Figure 4 - Syntax of the TSTR
SSRC: The SSRC value of the target (or the media sender) of this SSRC: The SSRC of the media sender which is requested to apply
specific TSTR request. the tradeoff value in Index.
Seq. nr: Request sequence number. The sequence number space is Seq. nr: Request sequence number. The sequence number space is
unique for each tuple consisting of the SSRC of request unique for each tuple consisting of the SSRC of request
source and the SSRC of the request target. The sequence source and the SSRC of the request target. The sequence
number SHALL be increased by 1 modulo 256 for each new number SHALL be increased by 1 modulo 256 for each new
command. A repetition SHALL NOT increase the sequence command. A repetition SHALL NOT increase the sequence
number. Initial value is arbitrary. number. Initial value is arbitrary.
Index: An integer value between 0 and 31 that indicates the Index: An integer value between 0 and 31 that indicates the
relative trade off that is requested. An index value of 0 relative trade off that is requested. An index value of 0
skipping to change at page 35, line 41 skipping to change at page 42, line 37
reception. reception.
4.3.2.3. Timing Rules 4.3.2.3. Timing Rules
The timing follows the rules outlined in section 3 of [RFC4585]. The timing follows the rules outlined in section 3 of [RFC4585].
This request message is not time critical and SHOULD be sent using This request message is not time critical and SHOULD be sent using
regular RTCP timing. Only if it is known that the user interface regular RTCP timing. Only if it is known that the user interface
requires a quick feedback, the message MAY be sent with early or requires a quick feedback, the message MAY be sent with early or
immediate feedback timing. immediate feedback timing.
4.3.2.4. Remarks 4.3.2.4. Handling of message in Mixers and Translators
Mixer or Media translators that encodes content sent to the session
participant issuing the TSTR SHALL consider the request to determine
if it can fulfill it by changing its own encoding parameters. A media
translator unable to fulfill the request MAY forward the request
unaltered towards the media sender. A Mixer encoding for multiple
session participants will need to consider the joint needs before
generating a TSTR for itself towards the media sender. See also
discussion in Section . 3.5.2.
4.3.2.5. Remarks
The term "spatial quality" does not necessarily refer to the The term "spatial quality" does not necessarily refer to the
resolution, measured by the number of pixels the reconstructed video resolution, measured by the number of pixels the reconstructed video
is using. In fact, in most scenarios the video resolution stays is using. In fact, in most scenarios the video resolution stays
constant during the lifetime of a session. However, all video constant during the lifetime of a session. However, all video
compression standards have means to adjust the spatial quality at a compression standards have means to adjust the spatial quality at a
given resolution, often influenced by the Quantizer Parameter or QP. given resolution, often influenced by the Quantizer Parameter or QP.
A numerically low QP results in a good reconstructed picture quality, A numerically low QP results in a good reconstructed picture quality,
whereas a numerically high QP yields a coarse picture. The typical whereas a numerically high QP yields a coarse picture. The typical
reaction of an encoder to this request is to change its rate control reaction of an encoder to this request is to change its rate control
parameters to use a lower frame rate and a numerically lower (on parameters to use a lower frame rate and a numerically lower (on
average) QP, or vice versa. The precise mapping of Index, frame average) QP, or vice versa. The precise mapping of Index, frame
rate, and QP is intentionally left open here, as it depends on rate, and QP is intentionally left open here, as it depends on
factors such as compression standard employed, spatial resolution, factors such as compression standard employed, spatial resolution,
content, bit rate, and many more. content, bit rate, and many more.
4.3.3. Temporal-Spatial Trade-off Announcement (TSTA) 4.3.3. Temporal-Spatial Trade-off Notification (TSTN)
The TSTA FB message is identified by PT=PSFB and FMT=6. The TSTN message is identified by PT=PSFB and FMT=6.
There SHALL be one or more TSTA contained in the FCI field. There SHALL be one or more TSTN contained in the FCI field.
4.3.3.1. Semantics 4.3.3.1. Semantics
This feedback message is used to acknowledge the reception of a TSTR. This feedback message is used to acknowledge the reception of a TSTR.
A TSTA entry in a TSTA feedback message SHALL be sent for each TSTR A TSTN entry in a TSTN feedback message SHALL be sent for each TSTR
entry targeted to this session participant, i.e. each TSTR received entry targeted to this session participant, i.e. each TSTR received
that in the SSRC field in the entry has the receiving entities SSRC. that in the SSRC field in the entry has the receiving entities SSRC.
A single TSTA message MAY acknowledge multiple requests using A single TSTN message MAY acknowledge multiple requests using
multiple FCI entries. The index value included SHALL be the same in multiple FCI entries. The index value included SHALL be the same in
all FCI's part of the TSTA message. Including a FCI for each all FCI's part of the TSTN message. Including a FCI for each
requestor allows each requesting entity to determine that the media requestor allows each requesting entity to determine that the media
sender targeted have received the request. The announcement SHALL be sender targeted have received the request. The Notification SHALL be
sent also for repetitions received. If the request receiver has sent also for repetitions received. If the request receiver has
received TSTR with several different sequence numbers from a single received TSTR with several different sequence numbers from a single
requestor it SHALL only respond to the request with the highest requestor it SHALL only respond to the request with the highest
(modulo 256) sequence number. (modulo 256) sequence number.
The TSTA SHALL include the Temporal-Spatial Trade-off index that will The TSTN SHALL include the Temporal-Spatial Trade-off index that will
be used as a result of the request. This is not necessarily the same be used as a result of the request. This is not necessarily the same
index as requested, as media sender may need to aggregate requests index as requested, as media sender may need to aggregate requests
from several requesting session participants. It may also have some from several requesting session participants. It may also have some
other policies or rules that limit the selection. other policies or rules that limit the selection.
The "SSRC of the packet sender" field indicates the source of the The ''SSRC of the packet sender'' field indicates the source of the
announcement, and the "SSRC of media source" is not used and SHALL be Notification, and the ''SSRC of media source'' is not used and SHALL
set to 0. The SSRC of the requesting entity to which the announcement be set to 0. The SSRC of the requesting entity to which the
applies to is in the FCI. Notification applies to is in the FCI.
4.3.3.2. Message Format 4.3.3.2. Message Format
The Temporal-Spatial Trade-off Announcement uses one additional FCI The Temporal-Spatial Trade-off Notification uses one additional FCI
field, the content of which is depicted in Figure 5. The length of field, the content of which is depicted in Figure 5. The length of
the FB message MUST be set to 2+2*N, where N is the number of FCI the FB message MUST be set to 2+2*N, where N is the number of FCI
entries. entries.
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC | | SSRC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Seq nr. | Reserved | Index | | Seq nr. | Reserved | Index |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 5 - Syntax of the TSTA Figure 5 - Syntax of the TSTN
SSRC: The SSRC of the source of the TSTA request that is SSRC: The SSRC of the source of the TSTR request which resulted
acknowledged. in this Notification.
Seq. nr: The sequence number value from the TSTA request that is Seq. nr: The sequence number value from the TSTN request that is
being acknowledged. being acknowledged.
Index: The trade-off value the media sender is using henceforth. Index: The trade-off value the media sender is using henceforth.
Reserved: All bits SHALL be set to 0 and SHALL be ignored on Reserved: All bits SHALL be set to 0 and SHALL be ignored on
reception. reception.
Informative note: The returned trade-off value (Index) may differ Informative note: The returned trade-off value (Index) may differ
from the requested one, for example in cases where a media encoder from the requested one, for example in cases where a media encoder
cannot tune its trade-off, or when pre-recorded content is used. cannot tune its trade-off, or when pre-recorded content is used.
4.3.3.3. Timing Rules 4.3.3.3. Timing Rules
The timing follows the rules outlined in section 3 of [RFC4585]. The timing follows the rules outlined in section 3 of [RFC4585].
This acknowledgement message is not extremely time critical and This acknowledgement message is not extremely time critical and
SHOULD be sent using regular RTCP timing. SHOULD be sent using regular RTCP timing.
4.3.3.4. Remarks 4.3.3.4. Handling of message in Mixer and Translators
A Mixer or Translator that act upon a TSTR SHALL also send the
corresponding TSTN. In cases it needs to forward a TSTR itself the
notification message MAY need to be delayed until that request has
been responded to.
4.3.3.5. Remarks
None None
4.3.4. H.271 VideoBackChannelMessage (VBCM) 4.3.4. H.271 VideoBackChannelMessage (VBCM)
The VBCM FB message is identified by PT=PSFB and FMT=7. The VBCM is identified by PT=PSFB and FMT=7.
There MUST be one or more VBCM entry contained in the FCI field. There MUST be one or more VBCM entry contained in the FCI field.
4.3.4.1. Semantics 4.3.4.1. Semantics
The "payload" of VBCM indication carries codec specific, different The "payload" of VBCM indication carries codec-specific, different
types of feedback information. The type of feedback information can types of feedback information. The type of feedback information can
be classified as "status report" such as receiving bit stream be classified as a 'status report' (such as receiving bit stream
without errors, loss of partial or complete picture or block or without errors, or loss of a partial or complete picture or block) or
"update requests" such as complete refresh of the bit stream. 'update requests' (such as complete refresh of the bit stream).
Note: There are possible overlap between the VBCM sub-messages Note: There are possible overlaps between the VBCM sub-
and CCM/AVPF feedback messages, such FIR. Please see section messages and CCM/AVPF feedback messages, such FIR. Please see
3.5.3 for further discussions. section 3 ..5.3 for further discussions.
The different types of feedback sub-messages carried in the VBCM are The different types of feedback sub-messages carried in the VBCM are
indicated by the "payloadType" as defined in [VBCM]. The different indicated by the ''payloadType'' as defined in [VBCM]. The different
sub-message types as defined in [VBCM] are re-produced below for sub-message types as defined in [VBCM] are re-produced below for
convenience. "payloadType", in ITU-T Rec. H.271 terminology, convenience. ''payloadType'', in ITU-T Rec. H.271 terminology,
refers to the sub-type of the H.271 message and should not be refers to the sub-type of the H.271 message and should not be
confused with an RTP payload type. confused with an RTP payload type.
Payload Type Message Content Payload Type Message Content
---------------------------------------------------------------------
0 One or more pictures without detected bitstream error mismatch 0 One or more pictures without detected bitstream error mismatch
1 One or more pictures that are entirely or partially lost 1 One or more pictures that are entirely or partially lost
2 A set of blocks of one picture that is entirely or partially 2 A set of blocks of one picture that is entirely or partially
lost lost
3 CRC for one parameter set 3 CRC for one parameter set
4 CRC for all parameter sets of a certain type 4 CRC for all parameter sets of a certain type
5 A "reset" request indicating that the sender should completely 5 A "reset" request indicating that the sender should completely
refresh the video bitstream as if no prior bitstream data had refresh the video bitstream as if no prior bitstream data had
been received been received
> 5 Reserved for future use by ITU-T > 5 Reserved for future use by ITU-T
skipping to change at page 38, line 47 skipping to change at page 46, line 28
Table 2: H.271 message types Table 2: H.271 message types
The bit string or the "payload" of VBCM message is of variable The bit string or the "payload" of VBCM message is of variable
length and is self-contained and coded in a variable length, binary length and is self-contained and coded in a variable length, binary
format. The media sender necessarily has to be able to parse this format. The media sender necessarily has to be able to parse this
optimized binary format to make use of VBCM messages optimized binary format to make use of VBCM messages
Each of the different types of sub-messages (indicated by Each of the different types of sub-messages (indicated by
payloadType) may have different semantic based on the codec used. payloadType) may have different semantic based on the codec used.
The "SSRC of the packet sender" field indicates the source of the The ''SSRC of the packet sender'' field indicates the source of the
request, and the "SSRC of media source" is not used and SHALL be set request, and the ''SSRC of media source'' is not used and SHALL be
to 0. The SSRC of the media sender to which the VBCM message applies set to 0. The SSRC of the media sender to which the VBCM message
to is in the FCI. applies to is in the FCI.
4.3.4.2. Message Format 4.3.4.2. Message Format
The VBCM indication uses one FCI field and the syntax is depicted in The VBCM indication uses one FCI field and the syntax is depicted in
Figure 6. Figure 6.
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC | | SSRC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Seq. nr |0| Payload Type| Length | | Seq. nr |0| Payload Type| Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| VBCM Octet String.... | Padding | | VBCM Octet String.... | Padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 6 - Syntax for VBCM Message Figure 6 - Syntax for VBCM Message
SSRC: The SSRC value of the media sender that is requested to
SSRC: The SSRC value of the media sender that is target of the instruct its encoder to react to the VBCM message
message, i.e. the media sender whose encoder should to react
to the VBCM message
Seq. nr: Command sequence number. The sequence number space is unique Seq. nr: Command sequence number. The sequence number space is unique
for each tuple consisting of the SSRC of command source and for each tuple consisting of the SSRC of command source and
the SSRC of the command target. The sequence number SHALL be the SSRC of the command target. The sequence number SHALL be
increased by 1 modulo 256 for each new command. A repetition increased by 1 modulo 256 for each new command. A repetition
SHALL NOT increase the sequence number. Initial value is SHALL NOT increase the sequence number. Initial value is
arbitrary. arbitrary.
0: Must be set to 0 and should not be acted upon receiving. 0: Must be set to 0 and should not be acted upon receiving.
skipping to change at page 40, line 5 skipping to change at page 47, line 28
Length: The length of the VBCM octet string in octets exclusive any Length: The length of the VBCM octet string in octets exclusive any
padding octets padding octets
VBCM Octet String: This is the octet string generated by the decoder VBCM Octet String: This is the octet string generated by the decoder
carrying a specific feedback sub-message. It is of variable carrying a specific feedback sub-message. It is of variable
length. length.
Padding: Bytes set to 0 to make up a 32 bit boundary. Padding: Bytes set to 0 to make up a 32 bit boundary.
Timing Rules 4.3.4.3. Timing Rules
The timing follows the rules outlined in section 3 of [RFC4585]. The timing follows the rules outlined in section 3 of [RFC4585]. The
The different sub-message types may have different properties in different sub-message types may have different properties in regards
regards to the timing of messages that should be used. If several to the timing of messages that should be used. If several different
different types are included in the same feedback packet then the types are included in the same feedback packet then the sub-message
sub-message type with the most stringent requiremnts should be type with the most stringent requirements should be followed.
followed.
Remarks 4.3.4.4. Handling of message in Mixer or Translator
Please see section 3.5.3 for the applicability of the VBCM message
in relation to messages in both AVPF and this memo with similar The handling of VBCM in a mixer or translator are sub-message type
dependent.
4.3.4.5. Remarks
Please see section 3.5.3 for the applicability of the VBCM message in
relation to messages in both AVPF and this memo with similar
functionality. functionality.
Note: There has been some discussion whether the payload type field Note: There has been some discussion whether the payload type field
in this message is needed. It would be needed if there were in this message is needed. It would be needed if there were
potentially more than one VBCM-capable RTP payload types in the potentially more than one VBCM-capable RTP payload types in the same
same session, and that the semantics of a given VBCM message session, and that the semantics of a given VBCM message changes from
changes from PT to PT. This appears to be the case. For example, PT to PT. This appears to be the case. For example, the picture
the picture identification mechanism in messages of H.271 type 0 is identification mechanism in messages of H.271 type 0 is fundamentally
fundamentally different between H.263 and H.264 (although both use different between H.263 and H.264 (although both use the same syntax.
the same syntax. Therefore, the payload field is justified here. Therefore, the payload field is justified here. It was further
It was further commented that for TSTS and FIR such a need does not commented that for TSTS and FIR such a need does not exist, because
exist, because the semantics of TSTS and FIR are either loosely the semantics of TSTS and FIR are either loosely enough defined, or
enough defined, or generic enough, to apply to all video payloads generic enough, to apply to all video payloads currently in
currently in existence/envisioned. existence/envisioned.
5. Congestion Control 5. Congestion Control
The correct application of the AVPF timing rules prevents the network The correct application of the AVPF timing rules prevents the network
flooding by feedback messages. Hence, assuming a correct from being flooded by feedback messages. Hence, assuming a correct
implementation, the RTCP channel cannot break its bit-rate commitment implementation, the RTCP channel cannot break its bit-rate commitment
and introduce congestion. and introduce congestion.
The reception of some of the feedback messages modifies the behaviour The reception of some of the feedback messages modifies the behaviour
of the media senders or, more specifically, the media encoders. All of the media senders or, more specifically, the media encoders. All
of these modifications MUST only be performed within the bandwidth of these modifications MUST only be performed within the bandwidth
limits the applied congestion control provides. For example, when limits the applied congestion control provides. For example, when
reacting to a FIR, the unusually high number of packets that form the reacting to a FIR, the unusually high number of packets that form the
decoder refresh point have to be paced in compliance with the decoder refresh point have to be paced in compliance with the
congestion control algorithm, even if the user experience suffers congestion control algorithm, even if the user experience suffers
from a slowly transmitted decoder refresh point. from a slowly transmitted decoder refresh point.
A change of the Temporary Maximum Media Bit-rate value can only A change of the Temporary Maximum Media Stream Bit-rate value can
mitigate congestion, but not cause congestion as long as congestion only mitigate congestion, but not cause congestion as long as
control is also employed. An increase of the value by a request congestion control is also employed. An increase of the value by a
REQUIRES the media sender to use congestion control when increasing request REQUIRES the media sender to use congestion control when
its transmission rate to that value. A reduction of the value results increasing its transmission rate to that value. A reduction of the
in a reduced transmission bit-rate thus reducing the risk for value results in a reduced transmission bit-rate thus reducing the
congestion. risk for congestion.
6. Security Considerations 6. Security Considerations
The defined messages have certain properties that have security The defined messages have certain properties that have security
implications. These must be addressed and taken into account by users implications. These must be addressed and taken into account by users
of this protocol. of this protocol.
The defined setup signalling mechanism is sensitive to modification The defined setup signaling mechanism is sensitive to modification
attacks that can result in session creation with sub-optimal attacks that can result in session creation with sub-optimal
configuration, and, in the worst case, session rejection. To prevent configuration, and, in the worst case, session rejection. To prevent
this type of attack, authentication and integrity protection of the this type of attack, authentication and integrity protection of the
setup signalling is required. setup signaling is required.
Spoofed or maliciously created feedback messages of the type defined Spoofed or maliciously created feedback messages of the type defined
in this specification can have the following implications: in this specification can have the following implications:
a. Severely reduced media bit-rate due to false TMMBR messages a. Severely reduced media bit-rate due to false TMMBR messages
that sets the maximum to a very low value. that sets the maximum to a very low value.
b. The assignment of the ownership of a bit-rate limit with a b. The assignment of the ownership of a bit-rate limit with a
TMMBN message to the wrong participant. Thus potentially TMMBN message to the wrong participant. Thus potentially
freezing the mechanism until a correct TMMBN message reached freezing the mechanism until a correct TMMBN message reached
the participants. the participants.
c. Sending TSTR that result in a video quality different from c. Sending TSTR that result in a video quality different from
the user's desire, rendering the session less useful. the user's desire, rendering the session less useful.
d. Frequent FIR commands will potentially reduce the frame-rate d. Frequent FIR commands will potentially reduce the frame-rate
making the video jerky due to the frequent usage of decoder making the video jerky due to the frequent usage of decoder
refresh points. refresh points.
To prevent these attacks there is need to apply authentication and To prevent these attacks there is a need to apply authentication and
integrity protection of the feedback messages. This can be integrity protection of the feedback messages. This can be
accomplished against group external threats using the RTP profile accomplished against threats external to the current RTP session
that combines SRTP [SRTP] and AVPF into SAVPF [SAVPF]. In the MCU using the RTP profile that combines SRTP [SRTP] and AVPF into SAVPF
cases, separate security contexts and filtering can be applied [SAVPF]. In the Mixer cases, separate security contexts and filtering
between the MCU and the participants thus protecting other MCU users can be applied between the Mixer and the participants thus protecting
from a misbehaving participant. other users on the Mixer from a misbehaving participant.
7. SDP Definitions 7. SDP Definitions
Section 4 of [RFC4585] defines new SDP [RFC2327] attributes that are Section 4 of [RFC4585] defines new SDP [RFC4566] attributes that are
used for the capability exchange of the AVPF commands and used for the capability exchange of the AVPF commands and
indications, such as Reference Picture selection, Picture loss indications, such as Reference Picture selection, Picture loss
indication etc. The defined SDP attribute is known as rtcp-fb and its indication etc. The defined SDP attribute is known as rtcp-fb and its
ABNF is described in section 4.2 of [RFC4585]. In this section we ABNF is described in section 4.2 of [RFC4585]. In this section we
extend the rtcp-fb attribute to include the commands and indications extend the rtcp-fb attribute to include the commands and indications
that are described in this document for codec control protocol. We that are described in this document for codec control protocol. We
also discuss the Offer/Answer implications for the codec control also discuss the Offer/Answer implications for the codec control
commands and indications. commands and indications.
7.1. Extension of rtcp-fb attribute 7.1. Extension of rtcp-fb attribute
As described in [RFC4585], the rtcp-fb attribute is defined to As described in [RFC4585], the rtcp-fb attribute is defined to
indicate the capability of using RTCP feedback. As defined in AVPF indicate the capability of using RTCP feedback. As defined in AVPF
the rtcp-fb attribute must only be used as a media level attribute the rtcp-fb attribute must only be used as a media level attribute
and must not be provided at session level. and must not be provided at session level. All the rules described
All the rules described in [RFC4585] for rtcp-fb attribute relating in [RFC4585] for rtcp-fb attribute relating to payload type and to
to payload type, multiple rtcp-fb attributes in a session description multiple rtcp-fb attributes in a session description also apply to
hold for the new feedback messages for codec control defined in this the new feedback messages defined in this memo.
document.
The ABNF for rtcp-fb attributed as defined in [RFC4585] is The ABNF for rtcp-fb as defined in [RFC4585] is
Rtcp-fb-syntax = "a=rtcp-fb: " rtcp-fb-pt SP rtcp-fb-val CRLF Rtcp-fb-syntax = "a=rtcp-fb: " rtcp-fb-pt SP rtcp-fb-val CRLF
Where rtcp-fb-pt is the payload type and rtcp-fb-val defines the type Where rtcp-fb-pt is the payload type and rtcp-fb-val defines the type
of the feedback message such as ack, nack, trr-int and rtcp-fb-id. of the feedback message such as ack, nack, trr-int and rtcp-fb-id.
For example to indicate the support of feedback of picture loss For example to indicate the support of feedback of picture loss
indication, the sender declares the following in SDP indication, the sender declares the following in SDP
v=0 v=0
o=alice 3203093520 3203093520 IN IP4 host.example.com o=alice 3203093520 3203093520 IN IP4 host.example.com
skipping to change at page 42, line 46 skipping to change at page 50, line 31
a=rtcp-fb:98 nack pli a=rtcp-fb:98 nack pli
In this document we define a new feedback value type called "ccm" In this document we define a new feedback value type called "ccm"
which indicates the support of codec control using RTCP feedback which indicates the support of codec control using RTCP feedback
messages. The "ccm" feedback value should be used with parameters, messages. The "ccm" feedback value should be used with parameters,
which indicates the support of which codec commands the session may which indicates the support of which codec commands the session may
use. In this draft we define four parameters, which can be used with use. In this draft we define four parameters, which can be used with
the ccm feedback value type. the ccm feedback value type.
o "fir" indicates the support of Full Intra Request o "fir" indicates the support of Full Intra Request
o "tmmbr" indicates the support of Temporal Maximum Media Bit-rate o "tmmbr" indicates the support of Temporal Maximum Media Stream
Bit-rate. It has an optional sub parameter to indicate the
session maximum packet rate to be used. If not included it
defaults to infinity.
o "tstr" indicates the support of temporal spatial trade-off o "tstr" indicates the support of temporal spatial trade-off
request. request.
O "vbcm" indicates the support of H.271 video back channel O "vbcm" indicates the support of H.271 video back channel
messages. messages.
In ABNF for rtcp-fb-val defined in [RFC4585], there is a placeholder In ABNF for rtcp-fb-val defined in [RFC4585], there is a placeholder
called rtcp-fb-id to define new feedback types. The ccm is defined as called rtcp-fb-id to define new feedback types. The ccm is defined as
a new feedback type in this document and the ABNF for the parameters a new feedback type in this document and the ABNF for the parameters
for ccm are defined here (please refer section 4.2 of [RFC4585] for for ccm are defined here (please refer section 4.2 of [RFC4585] for
complete ABNF syntax). complete ABNF syntax).
Rtcp-fb-param = SP "app" [SP byte-string] Rtcp-fb-param = SP "app" [SP byte-string]
skipping to change at page 43, line 17 skipping to change at page 51, line 4
In ABNF for rtcp-fb-val defined in [RFC4585], there is a placeholder In ABNF for rtcp-fb-val defined in [RFC4585], there is a placeholder
called rtcp-fb-id to define new feedback types. The ccm is defined as called rtcp-fb-id to define new feedback types. The ccm is defined as
a new feedback type in this document and the ABNF for the parameters a new feedback type in this document and the ABNF for the parameters
for ccm are defined here (please refer section 4.2 of [RFC4585] for for ccm are defined here (please refer section 4.2 of [RFC4585] for
complete ABNF syntax). complete ABNF syntax).
Rtcp-fb-param = SP "app" [SP byte-string] Rtcp-fb-param = SP "app" [SP byte-string]
/ SP rtcp-fb-ccm-param / SP rtcp-fb-ccm-param
/ ; empty / ; empty
rtcp-fb-ccm-param = "ccm" SP ccm-param rtcp-fb-ccm-param = "ccm" SP ccm-param
ccm-param = "fir" ; Full Intra Request ccm-param = "fir" ; Full Intra Request
/ "tmmbr" ; Temporary max media bit rate / "tmmbr" [SP "smaxpr=" MaxPacketRateValue]
; Temporary max media bit rate
/ "tstr" ; Temporal Spatial Trade Off / "tstr" ; Temporal Spatial Trade Off
/ "vbcm" 1*[SP subMessageType] ; H.271 VBCM messages / "vbcm" *(SP subMessageType] ; H.271 VBCM messages
/ token [SP byte-string] / token [SP byte-string]
; for future commands/indications ; for future commands/indications
subMessageType = 1*[integer]; subMessageType = 1*8DIGIT
byte-string = <as defined in section 4.2 of [RFC4585] > byte-string = <as defined in section 4.2 of [RFC4585] >
MaxPacketRateValue = 1*15DIGIT
7.2. Offer-Answer 7.2. Offer-Answer
The Offer/Answer [RFC3264] implications to codec control protocol The Offer/Answer [RFC3264] implications to codec control protocol
feedback messages are similar to as described in [RFC4585]. The feedback messages are similar those described in [RFC4585]. The
offerer MAY indicate the capability to support selected codec offerer MAY indicate the capability to support selected codec
commands and indications. The answerer MUST remove all ccm commands and indications. The answerer MUST remove all ccm parameters
parameters, which it does not understand or does not wish to use in which it does not understand or does not wish to use in this
this particular media session. The answerer MUST NOT add new ccm particular media session. The answerer MUST NOT add new ccm
parameters in addition to what has been offered. The answer is parameters in addition to what has been offered. The answer is
binding for the media session and both offerer and answerer MUST only binding for the media session and both offerer and answerer MUST only
use feedback messages negotiated in this way. use feedback messages negotiated in this way.
The session maximum packet rate parameter part of the TMMBR
indication is declarative and everyone shall use the highest value
indicated in a response. If not present in a offer is SHALL NOT be
included by the answerer.
7.3. Examples 7.3. Examples
Example 1: The following SDP describes a point-to-point video call Example 1: The following SDP describes a point-to-point video call
with H.263 with the originator of the call declaring its capability with H.263 with the originator of the call declaring its capability
to support codec control messages - fir, tstr. The SDP is carried in to support codec control messages - fir, tstr. The SDP is carried in
a high level signalling protocol like SIP a high level signaling protocol like SIP
v=0 v=0
o=alice 3203093520 3203093520 IN IP4 host.example.com o=alice 3203093520 3203093520 IN IP4 host.example.com
s=Point-to-Point call s=Point-to-Point call
c=IN IP4 172.11.1.124 c=IN IP4 172.11.1.124
m=audio 49170 RTP/AVP 0 m=audio 49170 RTP/AVP 0
a=rtpmap:0 PCMU/8000 a=rtpmap:0 PCMU/8000
m=video 51372 RTP/AVPF 98 m=video 51372 RTP/AVPF 98
a=rtpmap:98 H263-1998/90000 a=rtpmap:98 H263-1998/90000
a=rtcp-fb:98 ccm tstr a=rtcp-fb:98 ccm tstr
a=rtcp-fb:98 ccm fir a=rtcp-fb:98 ccm fir
In the above example the sender when it receives a TSTR message from In the above example the sender when it receives a TSTR message from
the remote party can adjust the trade off as indicated in the RTCP the remote party can adjust the trade off as indicated in the RTCP
TSTA feedback message. TSTN feedback message.
Example 2: The following SDP describes a SIP end point joining a Example 2: The following SDP describes a SIP end point joining a
video MCU that is hosting a multiparty video conferencing session. video Mixer that is hosting a multiparty video conferencing session.
The participant supports only the FIR (Full Intra Request) codec The participant supports only the FIR (Full Intra Request) codec
control command and it declares it in its session description. The control command and it declares it in its session description. The
video MCU can send an FIR RTCP feedback message to this end point video Mixer can send an FIR RTCP feedback message to this end point
when it needs to send this participants video to other participants when it needs to send this participants video to other participants
of the conference. of the conference.
v=0 v=0
o=alice 3203093520 3203093520 IN IP4 host.example.com o=alice 3203093520 3203093520 IN IP4 host.example.com
s=Multiparty Video Call s=Multiparty Video Call
c=IN IP4 172.11.1.124 c=IN IP4 172.11.1.124
m=audio 49170 RTP/AVP 0 m=audio 49170 RTP/AVP 0
a=rtpmap:0 PCMU/8000 a=rtpmap:0 PCMU/8000
m=video 51372 RTP/AVPF 98 m=video 51372 RTP/AVPF 98
skipping to change at page 45, line 10 skipping to change at page 52, line 48
v=0 v=0
o=alice 3203093520 3203093520 IN IP4 host.example.com o=alice 3203093520 3203093520 IN IP4 host.example.com
s=Offer/Answer s=Offer/Answer
c=IN IP4 172.11.1.124 c=IN IP4 172.11.1.124
m=audio 49170 RTP/AVP 0 m=audio 49170 RTP/AVP 0
a=rtpmap:0 PCMU/8000 a=rtpmap:0 PCMU/8000
m=video 51372 RTP/AVPF 98 m=video 51372 RTP/AVPF 98
a=rtpmap:98 H263-1998/90000 a=rtpmap:98 H263-1998/90000
a=rtcp-fb:98 ccm tstr a=rtcp-fb:98 ccm tstr
a=rtcp-fb:98 ccm fir a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 ccm tmmbr a=rtcp-fb:* ccm tmmbr smaxpr=120
The answerer only wishes to support FIR and TSTR message as the codec The answerer only wishes to support FIR and TSTR message as the codec
control messages and the answerer SDP is control messages and the answerer SDP is
<---------------- Answer <---------------- Answer
v=0 v=0
o=alice 3203093520 3203093524 IN IP4 otherhost.example.com o=alice 3203093520 3203093524 IN IP4 otherhost.example.com
s=Offer/Answer s=Offer/Answer
c=IN IP4 189.13.1.37 c=IN IP4 189.13.1.37
m=audio 47190 RTP/AVP 0 m=audio 47190 RTP/AVP 0
skipping to change at page 46, line 35 skipping to change at page 54, line 28
For use with "ccm" the following values also needs to be For use with "ccm" the following values also needs to be
registered. registered.
Value name: fir Value name: fir
Long name: Full Intra Request Command Long name: Full Intra Request Command
Usable with: ccm Usable with: ccm
Reference: RFC XXXX Reference: RFC XXXX
Value name: tmmbr Value name: tmmbr
Long name: Temporary Maximum Media Bit-rate Long name: Temporary Maximum Media Stream Bit-rate
Usable with: ccm Usable with: ccm
Reference: RFC XXXX Reference: RFC XXXX
Value name: tstr Value name: tstr
Long name: temporal Spatial Trade Off Long name: temporal Spatial Trade Off
Usable with: ccm Usable with: ccm
Reference: RFC XXXX Reference: RFC XXXX
Value name: vbcm Value name: vbcm
Long name: H.271 video back channel messages Long name: H.271 video back channel messages
Usable with: ccm Usable with: ccm
Reference: RFC XXXX Reference: RFC XXXX
9. Acknowledgements 9. Acknowledgements
The authors would like to thank Andrea Basso, Orit Levin, Nermeen The authors would like to thank Andrea Basso, Orit Levin, Nermeen
Ismail for their work on the requirement and discussion draft Ismail for their work on the requirement and discussion draft
[Basso]. [Basso].
Drafts of this memo were reviewed and extensively commented by Roni
Even, Colin Perkins, Randell Jesup, Keith Lantz, Harikishan Desineni,
Guido Franceschini and others. The authors appreciate these reviews.
Funding for the RFC Editor function is currently provided by the Funding for the RFC Editor function is currently provided by the
Internet Society. Internet Society.
10. References 10. References
10.1. Normative references 10.1. Normative references
[RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., Rey, J., [RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., Rey, J.,
"Extended RTP Profile for Real-Time Transport Control "Extended RTP Profile for Real-Time Transport Control
Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585,
skipping to change at page 48, line 49 skipping to change at page 57, line 4
Norrman, "The Secure Real-time Transport Protocol Norrman, "The Secure Real-time Transport Protocol
(SRTP)", RFC 3711, March 2004. (SRTP)", RFC 3711, March 2004.
[RFC2032] Turletti, T. and C. Huitema, "RTP Payload Format for [RFC2032] Turletti, T. and C. Huitema, "RTP Payload Format for
H.261 Video Streams", RFC 2032, October 1996. H.261 Video Streams", RFC 2032, October 1996.
[SAVPF] J. Ott, E. Carrara, "Extended Secure RTP Profile for [SAVPF] J. Ott, E. Carrara, "Extended Secure RTP Profile for
RTCP-based Feedback (RTP/SAVPF)," draft-ietf-avt-profile- RTCP-based Feedback (RTP/SAVPF)," draft-ietf-avt-profile-
savpf-02.txt, July, 2005. savpf-02.txt, July, 2005.
[RFC3525] Groves, C., Pantaleo, M., Anderson, T., and T. Taylor, [RFC3525] Groves, C., Pantaleo, M., Anderson, T., and T. Taylor,
"Gateway Control Protocol Version 1", RFC 3525, June "Gateway Control Protocol Version 1", RFC 3525, June
2003. 2003.
[RFC3448] M. Handley, S. Floyd, J. Padhye, J. Widmer, "TCP Friendly
Rate Control (TFRC): Protocol Specification", RFC 3448,
[VBCM] ITU-T Rec. H.271, "Video Back Channel Messages", June [VBCM] ITU-T Rec. H.271, "Video Back Channel Messages", June
2006 2006
[RFC3890] Westerlund, M., "A Transport Independent Bandwidth
Modifier for the Session Description Protocol (SDP)", RFC
3890, September 2004.
[RFC4340] Kohler, E., Handley, M., and S. Floyd, "Datagram
Congestion Control Protocol (DCCP)", RFC 4340, March
2006.
[RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
Description Protocol", RFC 4566, July 2006.
[RFC3261] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston,
A., Peterson, J., Sparks, R., Handley, M., and E.
Schooler, "SIP: Session Initiation Protocol", RFC 3261,
June 2002.
11. Authors' Addresses 11. Authors' Addresses
Stephan Wenger Stephan Wenger
Nokia Corporation Nokia Corporation
P.O. Box 100 P.O. Box 100
FIN-33721 Tampere FIN-33721 Tampere
FINLAND FINLAND
Phone: +358-50-486-0637 Phone: +358-50-486-0637
skipping to change at page 49, line 43 skipping to change at page 58, line 13
Bo Burman Bo Burman
Ericsson Research Ericsson Research
Ericsson AB Ericsson AB
SE-164 80 Stockholm, SWEDEN SE-164 80 Stockholm, SWEDEN
Phone: +46 8 7190000 Phone: +46 8 7190000
EMail: bo.burman@ericsson.com EMail: bo.burman@ericsson.com
Full Copyright Statement Full Copyright Statement
Copyright (C) The Internet Society (2006). Copyright (C) The IETF Trust (2007).
This document is subject to the rights, licenses and restrictions This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors contained in BCP 78, and except as set forth therein, the authors
retain all their rights. retain all their rights.
This document and the information contained herein are provided on an This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES,
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT
INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR
PURPOSE.
Intellectual Property Statement Intellectual Property
The IETF takes no position regarding the validity or scope of any The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79. found in BCP 78 and BCP 79.
skipping to change at page 50, line 38 skipping to change at page 59, line 7
such proprietary rights by implementers or users of this such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr. http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at this standard. Please address the information to the IETF at
ietf-ipr@ietf.org. ietf-ipr@ietf.org.
Acknowledgement
Funding for the RFC Editor function is provided by the IETF
Administrative Support Activity (IASA).
RFC Editor Considerations RFC Editor Considerations
The RFC editor is requested to replace all occurrences of XXXX with The RFC editor is requested to replace all occurrences of XXXX with
the RFC number this document receives. the RFC number this document receives.
 End of changes. 238 change blocks. 
700 lines changed or deleted 1083 lines changed or added

This html diff was produced by rfcdiff 1.33. The latest version is available from http://tools.ietf.org/tools/rfcdiff/