--- 1/draft-ietf-rmcat-video-traffic-model-04.txt 2018-07-19 16:13:23.352937065 -0700 +++ 2/draft-ietf-rmcat-video-traffic-model-05.txt 2018-07-19 16:13:23.404938322 -0700 @@ -1,28 +1,28 @@ Network Working Group X. Zhu Internet-Draft S. Mena Intended status: Informational Cisco Systems -Expires: July 22, 2018 Z. Sarker +Expires: January 20, 2019 Z. Sarker Ericsson AB - January 18, 2018 + July 19, 2018 - Modeling Video Traffic Sources for RMCAT Evaluations - draft-ietf-rmcat-video-traffic-model-04 + Video Traffic Models for RTP Congestion Control Evaluations + draft-ietf-rmcat-video-traffic-model-05 Abstract - This document describes two reference video traffic source models for - evaluating RMCAT candidate algorithms. The first model statistically - characterizes the behavior of a live video encoder in response to - changing requests on target video rate. The second model is trace- - driven, and emulates the encoder output based on actual encoded video + This document describes two reference video traffic models for + evaluating RTP congestion control algorithms. The first model + statistically characterizes the behavior of a live video encoder in + response to changing requests on target video rate. The second model + is trace-driven, and emulates the output of actual encoded video frame sizes from a high-resolution test sequence. Both models are designed to strike a balance between simplicity, repeatability, and authenticity in modeling the interactions between a live video traffic source and the congestion control module. Finally, the document describes how both approaches can be combined into a hybrid model. Status of This Memo This Internet-Draft is submitted in full conformance with the @@ -31,21 +31,21 @@ Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." - This Internet-Draft will expire on July 22, 2018. + This Internet-Draft will expire on January 20, 2019. Copyright Notice Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents @@ -69,59 +69,63 @@ 5.4. Rate range limit imposed by video content . . . . . . . . 9 6. A Trace-Driven Model . . . . . . . . . . . . . . . . . . . . 9 6.1. Choosing the video sequence and generating the traces . . 10 6.2. Using the traces in the synthetic codec . . . . . . . . . 11 6.2.1. Main algorithm . . . . . . . . . . . . . . . . . . . 11 6.2.2. Notes to the main algorithm . . . . . . . . . . . . . 13 6.3. Varying frame rate and resolution . . . . . . . . . . . . 13 7. Combining The Two Models . . . . . . . . . . . . . . . . . . 14 8. Implementation Status . . . . . . . . . . . . . . . . . . . . 15 9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 16 - 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 16 - 10.1. Normative References . . . . . . . . . . . . . . . . . . 16 - 10.2. Informative References . . . . . . . . . . . . . . . . . 16 + 10. Security Considerations . . . . . . . . . . . . . . . . . . . 16 + 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 16 + 11.1. Normative References . . . . . . . . . . . . . . . . . . 16 + 11.2. Informative References . . . . . . . . . . . . . . . . . 16 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 17 1. Introduction When evaluating candidate congestion control algorithms designed for real-time interactive media, it is important to account for the characteristics of traffic patterns generated from a live video encoder. Unlike synthetic traffic sources that can conform perfectly to the rate changing requests from the congestion control module, a live video encoder can be sluggish in reacting to such changes. Output rate of a live video encoder also typically deviates from the target rate due to uncertainties in the encoder rate control process. Consequently, end-to-end delay and loss performance of a real-time media flow can be further impacted by rate variations introduced by the live encoder. - On the other hand, evaluation results of a candidate RMCAT algorithm - should mostly reflect performance of the congestion control module, - and somewhat decouple from peculiarities of any specific video codec. - It is also desirable that evaluation tests are repeatable, and be - easily duplicated across different candidate algorithms. + On the other hand, evaluation results of a candidate RTP congestion + control algorithm should mostly reflect performance of the congestion + control module, and somewhat decouple from peculiarities of any + specific video codec. It is also desirable that evaluation tests are + repeatable, and be easily duplicated across different candidate + algorithms. One way to strike a balance between the above considerations is to - evaluate RMCAT algorithms using a synthetic video traffic source - model that captures key characteristics of the behavior of a live - video encoder. To this end, this draft presents two reference - models. The first is based on statistical modeling; the second is - trace-driven. The draft also discusses the pros and cons of each - approach, as well as how both approaches can be combined into a - hybrid model. + evaluate congestion control algorithms using a synthetic video + traffic source model that captures key characteristics of the + behavior of a live video encoder. To this end, this draft presents + two reference models. The first is based on statistical modeling; + the second is trace-driven. The draft also discusses the pros and + cons of each approach, as well as how both approaches can be combined + into a hybrid model. 2. Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", - "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this - document are to be interpreted as described RFC2119 [RFC2119]. + "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and + "OPTIONAL" in this document are to be interpreted as described in BCP + 14 [RFC2119] [RFC8174] when, and only when, they appear in all + capitals, as shown here. 3. Desired Behavior of A Synthetic Video Traffic Model A live video encoder employs encoder rate control to meet a target rate by varying its encoding parameters, such as quantization step size, frame rate, and picture resolution, based on its estimate of the video content (e.g., motion and scene complexity). In practice, however, several factors prevent the output video rate from perfectly conforming to the input target rate. @@ -154,161 +158,161 @@ common characteristics, as outlined below. o Low computational complexity: The model should be computationally lightweight, otherwise it defeats the whole purpose of serving as a substitute for a live video encoder. o Temporal pattern similarity: The individual traffic trace instances generated by the model should mimic the temporal pattern of those from a real video encoder. - o Statistical resemblance: The synthetic traffic should match the - outcome of the real video encoder in terms of statistical + o Statistical resemblance: The synthetic traffic source should match + the outcome of the real video encoder in terms of statistical characteristics, such as the mean, variance, peak, and autocorrelation coefficients of the bitrate. It is also important that the statistical resemblance should hold across different time scales, ranging from tens of milliseconds to sub-seconds. o Wide range of coverage: The model should be easily configurable to cover a wide range of codec behaviors (e.g., with either fast or slow reaction time in live encoder rate control) and video content variations (e.g., ranging from high-motion to low-motion). These distinct behavior features can be characterized via simple statistical modelling, or a trace-driven approach. Section 5 and Section 6 provide an example of each approach, respectively. Section 7 discusses how both models can be combined together. 4. Interactions Between Synthetic Video Traffic Source and Other Components at the Sender - Figure 1 depicts the interactions of the synthetic video encoder with - other components at the sender, such as the application, the - congestion control module, the media packet transport module, etc. - Both reference models, as described later in Section 5 and Section 6, - follow the same set of interactions. + Figure 1 depicts the interactions of the synthetic video traffic + source with other components at the sender, such as the application, + the congestion control module, the media packet transport module, + etc. Both reference models --- as described later in Section 5 and + Section 6 --- follow the same set of interactions. - The synthetic video encoder takes in raw video frames captured by the - camera and then dynamically generates a sequence of encoded video - frames with varying size and interval. These encoded frames are + The synthetic video source dynamically generates a sequence of dummy + video frames with varying size and interval. These dummy frames are processed by other modules in order to transmit the video stream over the network. During the lifetime of a video transmission session, - the synthetic video encoder will typically be required to adapt its + the synthetic video source will typically be required to adapt its encoding bitrate, and sometimes the spatial resolution and frame rate. - In our model, the synthetic video encoder module has a group of + In our model, the synthetic video source module has a group of incoming and outgoing interface calls that allow for interaction with other modules. The following are some of the possible incoming interface calls --- marked as (a) in Figure 1 --- that the synthetic - video encoder may accept. The list is not exhaustive and can be - complemented by other interface calls if deemed necessary. + video traffic source may accept. The list is not exhaustive and can + be complemented by other interface calls if deemed necessary. - o Target rate R_v: target rate request to the encoder, typically - from the congestion control module and updated dynamically over - time. Depending on the congestion control algorithm in use, the - update requests can either be periodic (e.g., once per second), or - on-demand (e.g., only when a drastic bandwidth change over the + o Target rate R_v: target rate request, typically calculated by the + congestion control module and updated dynamically over time. + Depending on the congestion control algorithm in use, the update + requests can either be periodic (e.g., once per second), or on- + demand (e.g., only when a drastic bandwidth change over the network is observed). o Target frame rate FPS: the instantaneous frame rate measured in frames-per-second at a given time. This depends on the native camera capture frame rate as well as the target/preferred frame rate configured by the application or user. - o Frame resolution XY: the 2-dimensional vector indicating the - preferred frame resolution in pixels. Several factors govern the - resolution requested to the synthetic video encoder over time. - Examples of such factors are the capturing resolution of the - native camera; or the current target rate R_v, since very small - resolutions do not make sense with very high bitrates, and vice- - versa. + o Target frame resolution XY: the 2-dimensional vector indicating + the preferred frame resolution in pixels. Several factors govern + the resolution requested to the synthetic video source over time. + Examples of such factors include the capturing resolution of the + native camera and the display size of the destination screen. The + target frame resolution also depends on the current target rate + R_v, since very small resolutions do not make sense with very high + bitrates, and vice-versa. o Instant frame skipping: the request to skip the encoding of one or several captured video frames, for instance when a drastic decrease in available network bandwidth is detected. o On-demand generation of intra (I) frame: the request to encode another I frame to avoid further error propagation at the receiver, if severe packet losses are observed. This request typically comes from the error control module. An example of outgoing interface call --- marked as (b) in Figure 1 - --- is the rate range, that is, the dynamic range of the video - encoder's output rate for the current video contents: [R_min, R_max]. - Here, R_min and R_max are meant to capture the dynamic rate range the - encoder is capable of outputting. This typically depends on the - video content complexity and/or display type (e.g., higher R_max for - video contents with higher motion complexity, or for displays of - higher resolution). Therefore, these values will not change with - R_v, but may change over time if the content is changing. + --- is the rate range [R_min, R_max]. Here, R_min and R_max are + meant to capture the dynamic rate range and actual live video encoder + is capable of generating given the input video content. This + typically depends on the video content complexity and/or display type + (e.g., higher R_max for video contents with higher motion complexity, + or for displays of higher resolution). Therefore, these values will + not change with R_v, but may change over time if the content is + changing. +-------------+ - raw video | | encoded video - frames | Synthetic | frames - ------------> | Video | --------------> - | Encoder | + | | encoded video + | Synthetic | frames + | Video | --------------> + | Source | | | +--------+----+ /|\ | | | -------------------+ +--------------------> interface from interface to other modules (a) other modules (b) Figure 1: Interaction between synthetic video encoder and other modules at the sender 5. A Statistical Reference Model This section describes one simple statistical model of the live video encoder traffic source. Figure 2 summarizes the list of tunable parameters in this statistical model. A more comprehensive survey of popular methods for modeling video traffic source behavior can be found in [Tanwir2013]. - +==============+====================================+================+ + +===========+====================================+================+ | Notation | Parameter Name | Example Value | - +==============+====================================+================+ - | R_v | Target rate request to encoder | 1 Mbps | - +--------------+------------------------------------+----------------+ - | FPS | Target frame rate of encoder output| 30 Hz | - +--------------+------------------------------------+----------------+ + +===========+====================================+================+ + | R_v | Target rate request | 1 Mbps | + +-----------+------------------------------------+----------------+ + | FPS | Target frame rate | 30 Hz | + +-----------+------------------------------------+----------------+ | tau_v | Encoder reaction latency | 0.2 s | - +--------------+------------------------------------+----------------+ + +-----------+------------------------------------+----------------+ | K_d | Burst duration during transient | 8 frames | - +--------------+------------------------------------+----------------+ + +-----------+------------------------------------+----------------+ | K_B | Burst frame size during transient | 13.5 KBytes* | - +--------------+------------------------------------+----------------+ + +-----------+------------------------------------+----------------+ | t0 | Reference frame interval 1/FPS | 33 ms | - +--------------+------------------------------------+----------------+ + +-----------+------------------------------------+----------------+ | B0 | Reference frame size R_v/8/FPS | 4.17 KBytes | - +--------------+------------------------------------+----------------+ + +-----------+------------------------------------+----------------+ | | Scaling parameter of the zero-mean | | | | Laplacian distribution describing | | | SCALE_t | deviations in normalized frame | 0.15 | | | interval (t-t0)/t0 | | - +--------------+------------------------------------+----------------+ + +-----------+------------------------------------+----------------+ | | Scaling parameter of the zero-mean | | | | Laplacian distribution describing | | | SCALE_B | deviations in normalized frame | 0.15 | | | size (B-B0)/B0 | | - +--------------+------------------------------------+----------------+ + +-----------+------------------------------------+----------------+ | R_min | minimum rate supported by video | 150 Kbps | - | | encoder or content activity | | - +--------------+------------------------------------+----------------+ + | | encoder type or content activity | | + +-----------+------------------------------------+----------------+ | R_max | maximum rate supported by video | 1.5 Mbps | - | | encoder or content activity | | - +==============+====================================+================+ + | | encoder type or content activity | | + +===========+====================================+================+ - * Example value of K_B for a video stream encoded at 720p and 30 frames - per second, using H.264/AVC encoder. + * Example value of K_B for a video stream encoded at 720p and + 30 frames per second, using H.264/AVC encoder. Figure 2: List of tunable parameters in a statistical video traffic source model. 5.1. Time-damped response to target rate update While the congestion control module can update its target rate request R_v at any time, the statistical model dictates that the encoder will only react to such changes tau_v seconds after a previous rate transition. In other words, when the encoder has @@ -376,21 +380,21 @@ The output rate R_o is further clipped within the dynamic range [R_min, R_max], which in reality are dictated by scene and motion complexity of the captured video content. In the proposed statistical model, these parameters are specified by the application. 6. A Trace-Driven Model The second approach for modelling a video traffic source is trace- driven. This can be achieved by running an actual live video encoder on a set of chosen raw video sequences and using the encoder's output - traces for constructing a synthetic live encoder. With this + traces for constructing a synthetic video source. With this approach, the recorded video traces naturally exhibit temporal fluctuations around a given target rate request R_v from the congestion control module. The following list summarizes the main steps of this approach: 1. Choose one or more representative raw video sequences. 2. Encode the sequence(s) using an actual live video encoder. Repeat the process for a number of bitrates. Keep only the @@ -398,22 +402,22 @@ 3. Construct a data structure that contains the output of the previous step. The data structure should allow for easy bitrate lookup. 4. Upon a target bitrate request R_v from the controller, look up the closest bitrates among those previously stored. Use the frame size sequences stored for those bitrates to approximate the frame sizes to output. - 5. The output of the synthetic encoder contains "encoded" frames - with zeros as contents but with realistic sizes. + 5. The output of the synthetic video traffic source contains + "encoded" frames with dummy contents but with realistic sizes. In the following, Section 6.1 explains the first three steps (1-3), Section 6.2 elaborates on the remaining two steps (4-5). Finally, Section 6.3 briefly discusses the possibility to extend the trace- driven model for supporting time-varying frame rate and/or time- varying frame resolution. 6.1. Choosing the video sequence and generating the traces The first step is a careful choice of a set of video sequences that @@ -510,21 +514,22 @@ o The variable t_current is an index to the frame size vector stored in Traces[r_current]. It is updated every time a new frame is due. It is assumed that all vectors stored Traces to have the same size, denoted as size_traces. The following equation governs the update of t_current: if t_current < SkipFrames then t_current = t_current + 1 else - t_current = ((t_current+1-SkipFrames) % (size_traces-SkipFrames)) + t_current = ( (t_current + 1 - SkipFrames) + % (size_traces-SkipFrames)) + SkipFrames where operator % denotes modulo, and SkipFrames is a predefined constant that denotes the number of frames to be skipped at the beginning of frame size vectors after t_current has wrapped around. The point of constant SkipFrames is avoiding the effect of periodically sending a large I-frame followed by several smaller- than-average P-frames. A typical value of SkipFrames is 20, although it could be set to 0 if one is interested in studying the effect of sending I-frames periodically. @@ -556,22 +561,22 @@ factor = R_v / R_max framesize = factor * Traces[R_max][t_current] In case b), we set the minimum output size to 1 byte, since the value of factor can be arbitrarily close to 0. 6.2.2. Notes to the main algorithm Note that main algorithm as described above can be further extended - to mimic some additional typical behaviors of a live encoder. Two - examples are given below: + to mimic some additional typical behaviors of a live video encoder. + Two examples are given below: o I-frames on demand: The synthetic codec can be extended to simulate the sending of I-frames on demand, e.g., as a reaction to losses. To implement this extension, the codec's incoming interface (see (a) in Figure 1) is augmented with a new function to request a new I-frame. Upon calling such function, t_current is reset to 0. o Variable step length l between R_min and R_max: In the main algorithm, the step length l is fixed for ease of explanation. @@ -682,30 +687,43 @@ been implemented as a stand-alone, platform-independent traffic source module. This can be easily integrated into network simulation platforms such as [ns-2] and [ns-3], as well as testbeds using a real network. The stand-alone traffic source module is available as an open source implementation at [Syncodecs]. 9. IANA Considerations There are no IANA impacts in this memo. -10. References +10. Security Considerations -10.1. Normative References + It is important to evaluate RTP-based congestion control schemes + using realistic traffic patterns, so as to ensure stable operations + of the network. Therefore, it is RECOMMENDED that candidate RTP- + based congestion control algorithms be tested using the video traffic + models presented in this draft before wide deployment over the + Internet. + +11. References + +11.1. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . -10.2. Informative References + [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC + 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, + May 2017, . + +11.2. Informative References [H264] ITU-T Recommendation H.264, "Advanced video coding for generic audiovisual services", May 2003, . [HEVC] ITU-T Recommendation H.265, "High efficiency video coding", April 2013, . [Hu2010] Hu, H., Ma, Z., and Y. Wang, "Optimization of Spatial,