draft-ietf-rmcat-wireless-tests-11.txt | rfc8869.txt | |||
---|---|---|---|---|
Network Working Group Z. Sarker | Internet Engineering Task Force (IETF) Z. Sarker | |||
Internet-Draft Ericsson AB | Request for Comments: 8869 Ericsson AB | |||
Intended status: Informational X. Zhu | Category: Informational X. Zhu | |||
Expires: September 14, 2020 J. Fu | ISSN: 2070-1721 J. Fu | |||
Cisco Systems | Cisco Systems | |||
March 13, 2020 | January 2021 | |||
Evaluation Test Cases for Interactive Real-Time Media over Wireless | Evaluation Test Cases for Interactive Real-Time Media over Wireless | |||
Networks | Networks | |||
draft-ietf-rmcat-wireless-tests-11 | ||||
Abstract | Abstract | |||
The Real-time Transport Protocol (RTP) is a common transport choice | The Real-time Transport Protocol (RTP) is a common transport choice | |||
for interactive multimedia communication applications. The | for interactive multimedia communication applications. The | |||
performance of these applications typically depends on a well- | performance of these applications typically depends on a well- | |||
functioning congestion control algorithm. To ensure a seamless and | functioning congestion control algorithm. To ensure a seamless and | |||
robust user experience, a well-designed RTP-based congestion control | robust user experience, a well-designed RTP-based congestion control | |||
algorithm should work well across all access network types. This | algorithm should work well across all access network types. This | |||
document describes test cases for evaluating performances of | document describes test cases for evaluating performances of | |||
candidate congestion control algorithms over cellular and Wi-Fi | candidate congestion control algorithms over cellular and Wi-Fi | |||
networks. | networks. | |||
Status of This Memo | Status of This Memo | |||
This Internet-Draft is submitted in full conformance with the | This document is not an Internet Standards Track specification; it is | |||
provisions of BCP 78 and BCP 79. | published for informational purposes. | |||
Internet-Drafts are working documents of the Internet Engineering | ||||
Task Force (IETF). Note that other groups may also distribute | ||||
working documents as Internet-Drafts. The list of current Internet- | ||||
Drafts is at https://datatracker.ietf.org/drafts/current/. | ||||
Internet-Drafts are draft documents valid for a maximum of six months | This document is a product of the Internet Engineering Task Force | |||
and may be updated, replaced, or obsoleted by other documents at any | (IETF). It represents the consensus of the IETF community. It has | |||
time. It is inappropriate to use Internet-Drafts as reference | received public review and has been approved for publication by the | |||
material or to cite them other than as "work in progress." | Internet Engineering Steering Group (IESG). Not all documents | |||
approved by the IESG are candidates for any level of Internet | ||||
Standard; see Section 2 of RFC 7841. | ||||
This Internet-Draft will expire on September 14, 2020. | Information about the current status of this document, any errata, | |||
and how to provide feedback on it may be obtained at | ||||
https://www.rfc-editor.org/info/rfc8869. | ||||
Copyright Notice | Copyright Notice | |||
Copyright (c) 2020 IETF Trust and the persons identified as the | Copyright (c) 2021 IETF Trust and the persons identified as the | |||
document authors. All rights reserved. | document authors. All rights reserved. | |||
This document is subject to BCP 78 and the IETF Trust's Legal | This document is subject to BCP 78 and the IETF Trust's Legal | |||
Provisions Relating to IETF Documents | Provisions Relating to IETF Documents | |||
(https://trustee.ietf.org/license-info) in effect on the date of | (https://trustee.ietf.org/license-info) in effect on the date of | |||
publication of this document. Please review these documents | publication of this document. Please review these documents | |||
carefully, as they describe your rights and restrictions with respect | carefully, as they describe your rights and restrictions with respect | |||
to this document. Code Components extracted from this document must | to this document. Code Components extracted from this document must | |||
include Simplified BSD License text as described in Section 4.e of | include Simplified BSD License text as described in Section 4.e of | |||
the Trust Legal Provisions and are provided without warranty as | the Trust Legal Provisions and are provided without warranty as | |||
described in the Simplified BSD License. | described in the Simplified BSD License. | |||
Table of Contents | Table of Contents | |||
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 | 1. Introduction | |||
2. Cellular Network Specific Test Cases . . . . . . . . . . . . 3 | 2. Cellular Network Specific Test Cases | |||
2.1. Varying Network Load . . . . . . . . . . . . . . . . . . 6 | 2.1. Varying Network Load | |||
2.1.1. Network Connection . . . . . . . . . . . . . . . . . 6 | 2.1.1. Network Connection | |||
2.1.2. Simulation Setup . . . . . . . . . . . . . . . . . . 7 | 2.1.2. Simulation Setup | |||
2.1.3. Expected behavior . . . . . . . . . . . . . . . . . . 9 | 2.1.3. Expected Behavior | |||
2.2. Bad Radio Coverage . . . . . . . . . . . . . . . . . . . 9 | 2.2. Bad Radio Coverage | |||
2.2.1. Network connection . . . . . . . . . . . . . . . . . 9 | 2.2.1. Network Connection | |||
2.2.2. Simulation Setup . . . . . . . . . . . . . . . . . . 9 | 2.2.2. Simulation Setup | |||
2.2.3. Expected behavior . . . . . . . . . . . . . . . . . . 10 | 2.2.3. Expected Behavior | |||
2.3. Desired Evaluation Metrics for cellular test cases . . . 10 | 2.3. Desired Evaluation Metrics for Cellular Test Cases | |||
3. Wi-Fi Networks Specific Test Cases . . . . . . . . . . . . . 10 | 3. Wi-Fi Networks Specific Test Cases | |||
3.1. Bottleneck in Wired Network . . . . . . . . . . . . . . . 12 | 3.1. Bottleneck in Wired Network | |||
3.1.1. Network topology . . . . . . . . . . . . . . . . . . 12 | 3.1.1. Network Topology | |||
3.1.2. Test/simulation setup . . . . . . . . . . . . . . . . 13 | 3.1.2. Test/Simulation Setup | |||
3.1.3. Typical test scenarios . . . . . . . . . . . . . . . 14 | 3.1.3. Typical Test Scenarios | |||
3.1.4. Expected behavior . . . . . . . . . . . . . . . . . . 15 | 3.1.4. Expected Behavior | |||
3.2. Bottleneck in Wi-Fi Network . . . . . . . . . . . . . . . 15 | 3.2. Bottleneck in Wi-Fi Network | |||
3.2.1. Network topology . . . . . . . . . . . . . . . . . . 15 | 3.2.1. Network Topology | |||
3.2.2. Test/simulation setup . . . . . . . . . . . . . . . . 16 | 3.2.2. Test/Simulation Setup | |||
3.2.3. Typical test scenarios . . . . . . . . . . . . . . . 17 | 3.2.3. Typical Test Scenarios | |||
3.2.4. Expected behavior . . . . . . . . . . . . . . . . . . 18 | 3.2.4. Expected Behavior | |||
3.3. Other Potential Test Cases . . . . . . . . . . . . . . . 19 | 3.3. Other Potential Test Cases | |||
3.3.1. EDCA/WMM usage . . . . . . . . . . . . . . . . . . . 19 | 3.3.1. EDCA/WMM usage | |||
3.3.2. Effect of heterogeneous link rates . . . . . . . . . 19 | 3.3.2. Effect of Heterogeneous Link Rates | |||
4. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 | 4. IANA Considerations | |||
5. Security Considerations . . . . . . . . . . . . . . . . . . . 20 | 5. Security Considerations | |||
6. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 20 | 6. References | |||
7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 21 | 6.1. Normative References | |||
8. References . . . . . . . . . . . . . . . . . . . . . . . . . 21 | 6.2. Informative References | |||
8.1. Normative References . . . . . . . . . . . . . . . . . . 21 | Contributors | |||
8.2. Informative References . . . . . . . . . . . . . . . . . 22 | Acknowledgments | |||
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 22 | Authors' Addresses | |||
1. Introduction | 1. Introduction | |||
Wireless networks (both cellular and Wi-Fi [IEEE802.11]) are an | Wireless networks (both cellular and Wi-Fi [IEEE802.11]) are an | |||
integral and increasingly more significant part of the Internet. | integral and increasingly more significant part of the Internet. | |||
Typical application scenarios for interactive multimedia | Typical application scenarios for interactive multimedia | |||
communication over wireless include from video conferencing calls in | communication over wireless include video conferencing calls in a bus | |||
a bus or train as well as live media streaming at home. It is well | or train as well as live media streaming at home. It is well known | |||
known that the characteristics and technical challenges for | that the characteristics and technical challenges for supporting | |||
supporting multimedia services over wireless are very different from | multimedia services over wireless are very different from those of | |||
those of providing the same service over a wired network. Although | providing the same service over a wired network. Although the basic | |||
the basic test cases as defined in [I-D.ietf-rmcat-eval-test] have | test cases as defined in [RFC8867] have covered many common effects | |||
covered many common effects of network impairments for evaluating | of network impairments for evaluating RTP-based congestion control | |||
RTP-based congestion control schemes, they remain to be tested over | schemes, they remain to be tested over characteristics and dynamics | |||
characteristics and dynamics unique to a given wireless environment. | unique to a given wireless environment. For example, in cellular | |||
For example, in cellular networks, the base station maintains | networks, the base station maintains individual queues per radio | |||
individual queues per radio bearer per user hence it leads to a | bearer per user hence it leads to a different nature of interactions | |||
different nature of interactions between traffic flows of different | between traffic flows of different users. This contrasts with a | |||
users. This contrasts with a typical wired network setting where | typical wired network setting where traffic flows from all users | |||
traffic flows from all users share the same queue at the bottleneck. | share the same queue at the bottleneck. Furthermore, user mobility | |||
Furthermore, user mobility patterns in a cellular network differ from | patterns in a cellular network differ from those in a Wi-Fi network. | |||
those in a Wi-Fi network. Therefore, it is important to evaluate the | Therefore, it is important to evaluate the performance of proposed | |||
performance of proposed candidate RTP-based congestion control | candidate RTP-based congestion control solutions over cellular mobile | |||
solutions over cellular mobile networks and over Wi-Fi networks | networks and over Wi-Fi networks respectively. | |||
respectively. | ||||
The draft [I-D.ietf-rmcat-eval-criteria] provides the guideline for | [RFC8868] provides guidelines for evaluating candidate algorithms and | |||
evaluating candidate algorithms and recognizes the importance of | recognizes the importance of testing over wireless access networks. | |||
testing over wireless access networks. However, it does not describe | However, it does not describe any specific test cases for performance | |||
any specific test cases for performance evaluation of candidate | evaluation of candidate algorithms. This document describes test | |||
algorithms. This document describes test cases specifically | cases specifically targeting cellular and Wi-Fi networks. | |||
targeting cellular and Wi-Fi networks. | ||||
2. Cellular Network Specific Test Cases | 2. Cellular Network Specific Test Cases | |||
A cellular environment is more complicated than its wireline | A cellular environment is more complicated than its wireline | |||
counterpart since it seeks to provide services in the context of | counterpart since it seeks to provide services in the context of | |||
variable available bandwidth, location dependencies and user | variable available bandwidth, location dependencies, and user | |||
mobilities at different speeds. In a cellular network, the user may | mobilities at different speeds. In a cellular network, the user may | |||
reach the cell edge which may lead to a significant amount of | reach the cell edge, which may lead to a significant number of | |||
retransmissions to deliver the data from the base station to the | retransmissions to deliver the data from the base station to the | |||
destination and vice versa. These radio links will often act as a | destination and vice versa. These radio links will often act as a | |||
bottleneck for the rest of the network and will eventually lead to | bottleneck for the rest of the network and will eventually lead to | |||
excessive delays or packet drops. An efficient retransmission or | excessive delays or packet drops. An efficient retransmission or | |||
link adaptation mechanism can reduce the packet loss probability but | link adaptation mechanism can reduce the packet loss probability, but | |||
there will remain some packet losses and delay variations. Moreover, | some packet losses and delay variations will remain. Moreover, with | |||
with increased cell load or handover to a congested cell, congestion | increased cell load or handover to a congested cell, congestion in | |||
in the transport network will become even worse. Besides, there | the transport network will become even worse. Besides, there exist | |||
exist certain characteristics that distinguish the cellular network | certain characteristics that distinguish the cellular network from | |||
from other wireless access networks such as Wi-Fi. In a cellular | other wireless access networks such as Wi-Fi. In a cellular network: | |||
network -- | ||||
o The bottleneck is often a shared link with relatively few users. | * The bottleneck is often a shared link with relatively few users. | |||
* The cost per bit over the shared link varies over time and is | - The cost per bit over the shared link varies over time and is | |||
different for different users. | different for different users. | |||
* Leftover/unused resources can be consumed by other greedy | - Leftover/unused resources can be consumed by other greedy | |||
users. | users. | |||
o Queues are always per radio bearer hence each user can have many | * Queues are always per radio bearer, hence each user can have many | |||
such queues. | such queues. | |||
o Users can experience both Inter and Intra Radio Access Technology | * Users can experience both inter- and intra-Radio Access Technology | |||
(RAT) handovers (see [HO-def-3GPP] for the definition of | (RAT) handovers (see [HO-def-3GPP] for the definition of | |||
"handover"). | "handover"). | |||
o Handover between cells or change of serving cells (as described in | * Handover between cells or change of serving cells (as described in | |||
[HO-LTE-3GPP] and [HO-UMTS-3GPP]) might cause user plane | [HO-LTE-3GPP] and [HO-UMTS-3GPP]) might cause user plane | |||
interruptions which can lead to bursts of packet losses, delay | interruptions, which can lead to bursts of packet losses, delay, | |||
and/or jitter. The exact behavior depends on the type of radio | and/or jitter. The exact behavior depends on the type of radio | |||
bearer. Typically, the default best-effort bearers do not | bearer. Typically, the default best-effort bearers do not | |||
generate packet loss, instead, packets are queued up and | generate packet loss, instead, packets are queued up and | |||
transmitted once the handover is completed. | transmitted once the handover is completed. | |||
o The network part decides how much the user can transmit. | * The network part decides how much the user can transmit. | |||
o The cellular network has variable link capacity per user. | * The cellular network has variable link capacity per user. | |||
* It can vary as fast as a period of milliseconds. | - It can vary as fast as a period of milliseconds. | |||
* It depends on many factors (such as distance, speed, | - It depends on many factors (such as distance, speed, | |||
interference, different flows). | interference, different flows). | |||
* It uses complex and smart link adaptation which makes the link | - It uses complex and smart link adaptation, which makes the link | |||
behavior ever more dynamic. | behavior ever more dynamic. | |||
* The scheduling priority depends on the estimated throughput. | - The scheduling priority depends on the estimated throughput. | |||
o Both Quality of Service (QoS) and non-QoS radio bearers can be | * Both Quality of Service (QoS) and non-QoS radio bearers can be | |||
used. | used. | |||
Hence, a real-time communication application operating over a | Hence, a real-time communication application operating over a | |||
cellular network needs to cope with a shared bottleneck link and | cellular network needs to cope with a shared bottleneck link and | |||
variable link capacity, events like handover, non-congestion related | variable link capacity, events like handover, non-congestion-related | |||
loss, abrupt changes in bandwidth (both short term and long term) due | loss, and abrupt changes in bandwidth (both short term and long term) | |||
to handover, network load and bad radio coverage. Even though 3GPP | due to handover, network load, and bad radio coverage. Even though | |||
has defined QoS bearers [QoS-3GPP] to ensure high-quality user | 3GPP has defined QoS bearers [QoS-3GPP] to ensure high-quality user | |||
experience, it is still preferable for real-time applications to | experience, it is still preferable for real-time applications to | |||
behave in an adaptive manner. | behave in an adaptive manner. | |||
Different mobile operators deploy their own cellular networks with | Different mobile operators deploy their own cellular networks with | |||
their own set of network functionalities and policies. Usually, a | their own set of network functionalities and policies. Usually, a | |||
mobile operator network includes a range of radio access technologies | mobile operator network includes a range of radio access technologies | |||
such as 3G and 4G/LTE. Looking at the specifications of such radio | such as 3G and 4G/LTE. Looking at the specifications of such radio | |||
technologies it is evident that only the more recent radio | technologies, it is evident that only the more recent radio | |||
technologies can support the high bandwidth requirements from real- | technologies can support the high bandwidth requirements from real- | |||
time interactive video applications. The future real-time | time interactive video applications. Future real-time interactive | |||
interactive application will impose even greater demand on cellular | applications will impose even greater demand on cellular network | |||
network performance which makes 4G (and beyond) radio technologies | performance, which makes 4G (and beyond) radio technologies more | |||
more suitable for such genre of application. | suitable for such genre of application. | |||
The key factors in defining test cases for cellular networks are: | The key factors in defining test cases for cellular networks are: | |||
o Shared and varying link capacity | * Shared and varying link capacity | |||
o Mobility | * Mobility | |||
o Handover | * Handover | |||
However, these factors are typically highly correlated in a cellular | However, these factors are typically highly correlated in a cellular | |||
network. Therefore, instead of devising separate test cases for | network. Therefore, instead of devising separate test cases for | |||
individual important events, we have divided the test case into two | individual important events, we have divided the test cases into two | |||
categories. It should be noted that the goal of the following test | categories. It should be noted that the goal of the following test | |||
cases is to evaluate the performance of candidate algorithms over the | cases is to evaluate the performance of candidate algorithms over the | |||
radio interface of the cellular network. Hence it is assumed that | radio interface of the cellular network. Hence, it is assumed that | |||
the radio interface is the bottleneck link between the communicating | the radio interface is the bottleneck link between the communicating | |||
peers and that the core network does not introduce any extra | peers and that the core network does not introduce any extra | |||
congestion along the path. Consequently, this draft has kept as out | congestion along the path. Consequently, this document has left out | |||
of scope the combination of multiple access technologies involving | of scope the combination of multiple access technologies involving | |||
both cellular and Wi-Fi users. In this latter case the shared | both cellular and Wi-Fi users. In this latter case, the shared | |||
bottleneck is likely at the wired backhaul link. These test cases | bottleneck is likely at the wired backhaul link. These test cases | |||
further assume a typical real-time telephony scenario where one real- | further assume a typical real-time telephony scenario where one real- | |||
time session consists of one voice stream and one video stream. | time session consists of one voice stream and one video stream. | |||
Even though it is possible to carry out tests over operational | Even though it is possible to carry out tests over operational | |||
cellular networks (e.g., LTE/5G), and actually such tests are already | cellular networks (e.g., LTE/5G), and actually such tests are already | |||
available today, these tests cannot in general be carried out in a | available today, these tests cannot in general be carried out in a | |||
deterministic fashion to ensure repeatability. The main reason is | deterministic fashion to ensure repeatability. The main reason is | |||
that these networks are controlled by cellular operators and there | that these networks are controlled by cellular operators, and there | |||
exist various amounts of competing traffic in the same cell(s). In | exists various amounts of competing traffic in the same cell(s). In | |||
practice, it is only in underground mines that one can carry out near | practice, it is only in underground mines that one can carry out near | |||
deterministic testing. Even there, it is not guaranteed either as | deterministic testing. Even there, it is not guaranteed either as | |||
workers in the mines may carry with them their personal mobile | workers in the mines may carry with them their personal mobile | |||
phones. Furthermore, the underground mining setting may not reflect | phones. Furthermore, the underground mining setting may not reflect | |||
typical usage patterns in an urban setting. We, therefore, recommend | typical usage patterns in an urban setting. We, therefore, recommend | |||
that a cellular network simulator is used for the test cases defined | that a cellular network simulator be used for the test cases defined | |||
in this document, for example -- the LTE simulator in [NS-3]. | in this document, for example -- the LTE simulator in [NS-3]. | |||
2.1. Varying Network Load | 2.1. Varying Network Load | |||
The goal of this test is to evaluate the performance of the candidate | The goal of this test is to evaluate the performance of the candidate | |||
congestion control algorithm under varying network load. The network | congestion control algorithm under varying network load. The network | |||
load variation is created by adding and removing network users a.k.a. | load variation is created by adding and removing network users, | |||
User Equipments (UEs) during the simulation. In this test case, each | a.k.a. User Equipment (UE), during the simulation. In this test | |||
user/UE in the media session is an endpoint following RTP-based | case, each user/UE in the media session is an endpoint following RTP- | |||
congestion control. User arrivals follow a Poisson distribution | based congestion control. User arrivals follow a Poisson | |||
proportional to the length of the call, to keep the number of users | distribution proportional to the length of the call, to keep the | |||
per cell fairly constant during the evaluation period. At the | number of users per cell fairly constant during the evaluation | |||
beginning of the simulation, there should be enough time to warm-up | period. At the beginning of the simulation, there should be enough | |||
the network. This is to avoid running the evaluation in an empty | time to warm up the network. This is to avoid running the evaluation | |||
network where network nodes are having empty buffers, low | in an empty network where network nodes have empty buffers and low | |||
interference at the beginning of the simulation. This network | interference at the beginning of the simulation. This network | |||
initialization period should be excluded from the evaluation period. | initialization period should be excluded from the evaluation period. | |||
Typically, the evaluation period starts 30 seconds after test | Typically, the evaluation period starts 30 seconds after test | |||
initialization. | initialization. | |||
This test case also includes user mobility and some competing | This test case also includes user mobility and some competing | |||
traffic. The latter includes both the same types of flows (with same | traffic. The latter includes both the same types of flows (with same | |||
adaptation algorithms) and different types of flows (with different | adaptation algorithms) and different types of flows (with different | |||
services and congestion control schemes). | services and congestion control schemes). | |||
2.1.1. Network Connection | 2.1.1. Network Connection | |||
Each mobile user is connected to a fixed user. The connection | Each mobile user is connected to a fixed user. The connection | |||
between the mobile user and fixed user consists of a cellular radio | between the mobile user and fixed user consists of a cellular radio | |||
access, an Evolved Packet Core (EPC) and an Internet connection. The | access, an Evolved Packet Core (EPC), and an Internet connection. | |||
mobile user is connected to the EPC using cellular radio access | The mobile user is connected to the EPC using cellular radio access | |||
technology which is further connected to the Internet. At the other | technology, which is further connected to the Internet. At the other | |||
end, the fixed user is connected to the Internet via wired connection | end, the fixed user is connected to the Internet via a wired | |||
with sufficiently high bandwidth, for instance, 10 Gbps, so that the | connection with sufficiently high bandwidth, for instance, 10 Gbps, | |||
system bottleneck is on the cellular radio access interface. The | so that the system bottleneck is on the cellular radio access | |||
wired connection to in this setup does not introduce any network | interface. The wired connection in this setup does not introduce any | |||
impairments to the test; it only adds 10 ms of one-way propagation | network impairments to the test; it only adds 10 ms of one-way | |||
delay. | propagation delay. | |||
The path from the fixed user to the mobile users is defined as | The path from the fixed user to the mobile users is defined as | |||
"Downlink" and the path from the mobile users to the fixed user is | "downlink", and the path from the mobile users to the fixed user is | |||
defined as "Uplink". We assume that only uplink or downlink is | defined as "uplink". We assume that only uplink or downlink is | |||
congested for mobile users. Hence, we recommend that the uplink and | congested for mobile users. Hence, we recommend that the uplink and | |||
downlink simulations are run separately. | downlink simulations are run separately. | |||
uplink | uplink | |||
++))) +--------------------------> | ++))) +--------------------------> | |||
++-+ ((o)) | ++-+ ((o)) | |||
| | / \ +-------+ +------+ +---+ | | | / \ +-------+ +------+ +---+ | |||
+--+ / \----+ +-----+ +----+ | | +--+ / \----+ +-----+ +----+ | | |||
/ \ +-------+ +------+ +---+ | / \ +-------+ +------+ +---+ | |||
UE BS EPC Internet fixed | UE BS EPC Internet fixed | |||
<--------------------------+ | <--------------------------+ | |||
downlink | downlink | |||
Figure 1: Simulation Topology | Figure 1: Simulation Topology | |||
2.1.2. Simulation Setup | 2.1.2. Simulation Setup | |||
The values enclosed within "[ ]" for the following simulation | The values enclosed within "[ ]" for the following simulation | |||
attributes follow the same notion as in [I-D.ietf-rmcat-eval-test]. | attributes follow the same notion as in [RFC8867]. The desired | |||
The desired simulation setup is as follows -- | simulation setup is as follows: | |||
1. Radio environment: | Radio environment: | |||
A. Deployment and propagation model: 3GPP case 1 (see | Deployment and propagation model: 3GPP case 1 (see | |||
[HO-deploy-3GPP]) | [HO-deploy-3GPP]) | |||
B. Antenna: Multiple-Input and Multiple-Output (MIMO), 2D or 3D | Antenna: Multiple-Input and Multiple-Output (MIMO), 2D or 3D | |||
antenna pattern. | antenna pattern | |||
C. Mobility: [3km/h, 30km/h] | Mobility: [3 km/h, 30 km/h] | |||
D. Transmission bandwidth: 10MHz | Transmission bandwidth: 10 MHz | |||
E. Number of cells: multi-cell deployment (3 Cells per Base | Number of cells: multi-cell deployment (3 cells per Base Station | |||
Station (BS) * 7 BS) = 21 cells | (BS) * 7 BS) = 21 cells | |||
F. Cell radius: 166.666 Meters | Cell radius: 166.666 meters | |||
G. Scheduler: Proportional fair with no priority | Scheduler: Proportional fair with no priority | |||
H. Bearer: Default bearer for all traffic. | Bearer: Default bearer for all traffic | |||
I. Active Queue Management (AQM) settings: AQM [on,off] | Active Queue Management (AQM) settings: AQM [on, off] | |||
2. End-to-end Round Trip Time (RTT): [40ms, 150ms] | End-to-end Round Trip Time (RTT): [40 ms, 150 ms] | |||
3. User arrival model: Poisson arrival model | User arrival model: Poisson arrival model | |||
4. User intensity: | User intensity: | |||
* Downlink user intensity: {0.7, 1.4, 2.1, 2.8, 3.5, 4.2, 4.9, | Downlink user intensity: {0.7, 1.4, 2.1, 2.8, 3.5, 4.2, 4.9, 5.6, | |||
5.6, 6.3, 7.0, 7.7, 8.4, 9,1, 9.8, 10.5} | 6.3, 7.0, 7.7, 8.4, 9,1, 9.8, 10.5} | |||
* Uplink user intensity : {0.7, 1.4, 2.1, 2.8, 3.5, 4.2, 4.9, | Uplink user intensity: {0.7, 1.4, 2.1, 2.8, 3.5, 4.2, 4.9, 5.6, | |||
5.6, 6.3, 7.0} | 6.3, 7.0} | |||
5. Simulation duration: 91s | Simulation duration: 91 s | |||
6. Evaluation period: 30s-60s | Evaluation period: 30 s - 60 s | |||
7. Media traffic: | Media traffic: | |||
1. Media type: Video | Media type: Video | |||
a. Media direction: [Uplink, Downlink] | Media direction: [uplink, downlink] | |||
b. Number of Media source per user: One (1) | Number of media sources per user: One (1) | |||
c. Media duration per user: 30s | Media duration per user: 30 s | |||
d. Media source: same as defined in Section 4.3 of | Media source: same as defined in Section 4.3 of [RFC8867] | |||
[I-D.ietf-rmcat-eval-test] | ||||
2. Media Type: Audio | Media type: Audio | |||
a. Media direction: Uplink and Downlink | Media direction: [uplink, downlink] | |||
b. Number of Media source per user: One (1) | Number of media sources per user: One (1) | |||
c. Media duration per user: 30s | Media duration per user: 30 s | |||
d. Media codec: Constant Bit Rate (CBR) | Media codec: Constant Bit Rate (CBR) | |||
e. Media bitrate: 20 Kbps | Media bitrate: 20 Kbps | |||
f. Adaptation: off | Adaptation: off | |||
8. Other traffic models: | Other traffic models: | |||
* Downlink simulation: Maximum of 4Mbps/cell (web browsing or | Downlink simulation: Maximum of 4 Mbps/cell (web browsing or FTP | |||
FTP traffic following default TCP congestion control | traffic following default TCP congestion control [RFC5681]) | |||
[RFC5681]) | ||||
* Unlink simulation: Maximum of 2Mbps/cell (web browsing or FTP | Uplink simulation: Maximum of 2 Mbps/cell (web browsing or FTP | |||
traffic following default TCP congestion control [RFC5681]) | traffic following default TCP congestion control [RFC5681]) | |||
2.1.3. Expected behavior | 2.1.3. Expected Behavior | |||
The investigated congestion control algorithms should result in | The investigated congestion control algorithms should result in | |||
maximum possible network utilization and stability in terms of rate | maximum possible network utilization and stability in terms of rate | |||
variations, lowest possible end to end frame latency, network latency | variations, lowest possible end-to-end frame latency, network | |||
and Packet Loss Rate (PLR) at different cell load levels. | latency, and Packet Loss Rate (PLR) at different cell load levels. | |||
2.2. Bad Radio Coverage | 2.2. Bad Radio Coverage | |||
The goal of this test is to evaluate the performance of candidate | The goal of this test is to evaluate the performance of the candidate | |||
congestion control algorithm when users visit part of the network | congestion control algorithm when users visit part of the network | |||
with bad radio coverage. The scenario is created by using a larger | with bad radio coverage. The scenario is created by using a larger | |||
cell radius than that in the previous test case. In this test case, | cell radius than that in the previous test case. In this test case, | |||
each user/UE in the media session is an endpoint following RTP-based | each user/UE in the media session is an endpoint following RTP-based | |||
congestion control. User arrivals follow a Poisson distribution | congestion control. User arrivals follow a Poisson distribution | |||
proportional to the length of the call, to keep the number of users | proportional to the length of the call, to keep the number of users | |||
per cell fairly constant during the evaluation period. At the | per cell fairly constant during the evaluation period. At the | |||
beginning of the simulation, there should be enough amount of time to | beginning of the simulation, there should be enough time to warm up | |||
warm-up the network. This is to avoid running the evaluation in an | the network. This is to avoid running the evaluation in an empty | |||
empty network where network nodes are having empty buffers, low | network where network nodes have empty buffers and low interference | |||
interference at the beginning of the simulation. This network | at the beginning of the simulation. This network initialization | |||
initialization period should be excluded from the evaluation period. | period should be excluded from the evaluation period. Typically, the | |||
Typically, the evaluation period starts 30 seconds after test | evaluation period starts 30 seconds after test initialization. | |||
initialization. | ||||
This test case also includes user mobility and some competing | This test case also includes user mobility and some competing | |||
traffic. The latter includes the same kind of flows (with same | traffic. The latter includes the same kind of flows (with same | |||
adaptation algorithms). | adaptation algorithms). | |||
2.2.1. Network connection | 2.2.1. Network Connection | |||
Same as defined in Section 2.1.1 | Same as defined in Section 2.1.1. | |||
2.2.2. Simulation Setup | 2.2.2. Simulation Setup | |||
The desired simulation setup is the same as the Varying Network Load | The desired simulation setup is the same as the Varying Network Load | |||
test case defined in Section 2.1 except the following changes: | test case defined in Section 2.1 except for the following changes: | |||
1. Radio environment: Same as defined in Section 2.1.2 except the | Radio environment: Same as defined in Section 2.1.2 except for the | |||
following: | following: | |||
A. Deployment and propagation model: 3GPP case 3 (see | Deployment and propagation model: 3GPP case 3 (see | |||
[HO-deploy-3GPP]) | [HO-deploy-3GPP]) | |||
B. Cell radius: 577.3333 Meters | Cell radius: 577.3333 meters | |||
C. Mobility: 3km/h | Mobility: 3 km/h | |||
2. User intensity = {0.7, 1.4, 2.1, 2.8, 3.5, 4.2, 4.9, 5.6, 6.3, | User intensity: {0.7, 1.4, 2.1, 2.8, 3.5, 4.2, 4.9, 5.6, 6.3, 7.0} | |||
7.0} | ||||
3. Media traffic model: Same as defined in Section 2.1.2 | Media traffic model: Same as defined in Section 2.1.2 | |||
4. Other traffic models: | Other traffic models: | |||
* Downlink simulation: Maximum of 2Mbps/cell (web browsing or | Downlink simulation: Maximum of 2 Mbps/cell (web browsing or FTP | |||
FTP traffic following default TCP congestion control | traffic following default TCP congestion control [RFC5681]) | |||
[RFC5681]) | ||||
* Unlink simulation: Maximum of 1Mbps/cell (web browsing or FTP | Uplink simulation: Maximum of 1 Mbps/cell (web browsing or FTP | |||
traffic following default TCP congestion control [RFC5681]) | traffic following default TCP congestion control [RFC5681]) | |||
2.2.3. Expected behavior | 2.2.3. Expected Behavior | |||
The investigated congestion control algorithms should result in | The investigated congestion control algorithms should result in | |||
maximum possible network utilization and stability in terms of rate | maximum possible network utilization and stability in terms of rate | |||
variations, lowest possible end to end frame latency, network latency | variations, lowest possible end-to-end frame latency, network | |||
and Packet Loss Rate (PLR) at different cell load levels. | latency, and Packet Loss Rate (PLR) at different cell load levels. | |||
2.3. Desired Evaluation Metrics for cellular test cases | 2.3. Desired Evaluation Metrics for Cellular Test Cases | |||
The evaluation criteria document [I-D.ietf-rmcat-eval-criteria] | The evaluation criteria document [RFC8868] defines the metrics to be | |||
defines the metrics to be used to evaluate candidate algorithms. | used to evaluate candidate algorithms. Considering the nature and | |||
Considering the nature and distinction of cellular networks we | distinction of cellular networks, we recommend that at least the | |||
recommend that at least the following metrics be used to evaluate the | following metrics be used to evaluate the performance of the | |||
performance of the candidate algorithms: | candidate algorithms: | |||
o Average cell throughput (for all cells), shows cell utilizations. | * Average cell throughput (for all cells), shows cell utilization. | |||
o Application sending and receiving bitrate, goodput. | * Application sending and receiving bitrate, goodput. | |||
o Packet Loss Rate (PLR). | * Packet Loss Rate (PLR). | |||
o End-to-end Media frame delay. For video, this means the delay | * End-to-end media frame delay. For video, this means the delay | |||
from capture to display. | from capture to display. | |||
o Transport delay. | * Transport delay. | |||
o Algorithm stability in terms of rate variation. | * Algorithm stability in terms of rate variation. | |||
3. Wi-Fi Networks Specific Test Cases | 3. Wi-Fi Networks Specific Test Cases | |||
Given the prevalence of Internet access links over Wi-Fi, it is | Given the prevalence of Internet access links over Wi-Fi, it is | |||
important to evaluate candidate RTP-based congestion control | important to evaluate candidate RTP-based congestion control | |||
solutions over test cases that include Wi-Fi access links. Such | solutions over test cases that include Wi-Fi access links. Such | |||
evaluations should highlight the inherently different characteristics | evaluations should highlight the inherently different characteristics | |||
of Wi-Fi networks in contrast to their wired counterparts: | of Wi-Fi networks in contrast to their wired counterparts: | |||
o The wireless radio channel is subject to interference from nearby | * The wireless radio channel is subject to interference from nearby | |||
transmitters, multipath fading, and shadowing. These effects lead | transmitters, multipath fading, and shadowing. These effects lead | |||
to fluctuations in the link throughput and sometimes an error- | to fluctuations in the link throughput and sometimes an error- | |||
prone communication environment. | prone communication environment. | |||
o Available network bandwidth is not only shared over the air | * Available network bandwidth is not only shared over the air | |||
between concurrent users but also between uplink and downlink | between concurrent users but also between uplink and downlink | |||
traffic due to the half-duplex nature of the wireless transmission | traffic due to the half-duplex nature of the wireless transmission | |||
medium. | medium. | |||
o Packet transmissions over Wi-Fi are susceptible to contentions and | * Packet transmissions over Wi-Fi are susceptible to contentions and | |||
collisions over the air. Consequently, traffic load beyond a | collisions over the air. Consequently, traffic load beyond a | |||
certain utilization level over a Wi-Fi network can introduce | certain utilization level over a Wi-Fi network can introduce | |||
frequent collisions over the air and significant network overhead, | frequent collisions over the air and significant network overhead, | |||
as well as packet drops due to buffer overflow at the | as well as packet drops due to buffer overflow at the | |||
transmitters. This, in turn, leads to excessive delay, | transmitters. This, in turn, leads to excessive delay, | |||
retransmissions, packet losses and lower effective bandwidth for | retransmissions, packet losses, and lower effective bandwidth for | |||
applications. Note further that the collision-induced delay and | applications. Note further that the collision-induced delay and | |||
loss patterns are qualitatively different from those caused by | loss patterns are qualitatively different from those caused by | |||
congestion over a wired connection. | congestion over a wired connection. | |||
o The IEEE 802.11 standard (i.e., Wi-Fi) supports multi-rate | * The IEEE 802.11 standard (i.e., Wi-Fi) supports multi-rate | |||
transmission capabilities by dynamically choosing the most | transmission capabilities by dynamically choosing the most | |||
appropriate modulation and coding scheme (MCS) for the given | appropriate modulation and coding scheme (MCS) for the given | |||
received signal strength. A different choice in the MCS Index | received signal strength. A different choice in the MCS Index | |||
leads to different physical-layer (PHY-layer) link rates and | leads to different physical-layer (PHY-layer) link rates and | |||
consequently different application-layer throughput. | consequently different application-layer throughput. | |||
o The presence of legacy devices (e.g., ones operating only in IEEE | * The presence of legacy devices (e.g., ones operating only in IEEE | |||
802.11b) at a much lower PHY-layer link rate can significantly | 802.11b) at a much lower PHY-layer link rate can significantly | |||
slow down the rest of a modern Wi-Fi network. As discussed in | slow down the rest of a modern Wi-Fi network. As discussed in | |||
[Heusse2003], the main reason for such anomaly is that it takes | [Heusse2003], the main reason for such anomaly is that it takes | |||
much longer to transmit the same packet over a slower link than | much longer to transmit the same packet over a slower link than | |||
over a faster link, thereby consuming a substantial portion of air | over a faster link, thereby consuming a substantial portion of air | |||
time. | time. | |||
o Handover from one Wi-Fi Access Point (AP) to another may lead to | * Handover from one Wi-Fi Access Point (AP) to another may lead to | |||
excessive packet delays and losses during the process. | excessive packet delays and losses during the process. | |||
o IEEE 802.11e has introduced the Enhanced Distributed Channel | * IEEE 802.11e has introduced the Enhanced Distributed Channel | |||
Access (EDCA) mechanism to allow different traffic categories to | Access (EDCA) mechanism to allow different traffic categories to | |||
contend for channel access using different random back-off | contend for channel access using different random back-off | |||
parameters. This mechanism is a mandatory requirement for the Wi- | parameters. This mechanism is a mandatory requirement for the Wi- | |||
Fi Multimedia (WMM) certification in Wi-Fi Alliance. It allows | Fi Multimedia (WMM) certification in Wi-Fi Alliance. It allows | |||
for prioritization of real-time application traffic such as voice | for prioritization of real-time application traffic such as voice | |||
and video over non-urgent data transmissions (e.g., file | and video over non-urgent data transmissions (e.g., file | |||
transfer). | transfer). | |||
In summary, the presence of Wi-Fi access links in different network | In summary, the presence of Wi-Fi access links in different network | |||
topologies can exert different impact on the network performance in | topologies can exert different impacts on the network performance in | |||
terms of application-layer effective throughput, packet loss rate, | terms of application-layer effective throughput, packet loss rate, | |||
and packet delivery delay. These, in turn, will influence the | and packet delivery delay. These, in turn, will influence the | |||
behavior of end-to-end real-time multimedia congestion control. | behavior of end-to-end real-time multimedia congestion control. | |||
Unless otherwise mentioned, the test cases in this section choose the | Unless otherwise mentioned, the test cases in this section choose the | |||
PHY- and MAC-layer parameters based on the IEEE 802.11n Standard. | PHY- and MAC-layer parameters based on the IEEE 802.11n standard. | |||
Statistics collected from enterprise Wi-Fi networks show that the two | Statistics collected from enterprise Wi-Fi networks show that the two | |||
dominant physical modes are 802.11n and 802.11ac, accounting for 41% | dominant physical modes are 802.11n and 802.11ac, accounting for 41% | |||
and 58% of connected devices. As Wi-Fi standards evolve over time -- | and 58% of connected devices, respectively. As Wi-Fi standards | |||
for instance, with the introduction of the emerging Wi-Fi 6 (based on | evolve over time -- for instance, with the introduction of the | |||
IEEE 802.11ax) products -- the PHY- and MAC-layer test case | emerging Wi-Fi 6 (based on IEEE 802.11ax) products -- the PHY- and | |||
specifications need to be updated accordingly to reflect such | MAC-layer test case specifications need to be updated accordingly to | |||
changes. | reflect such changes. | |||
Typically, a Wi-Fi access network connects to a wired infrastructure. | Typically, a Wi-Fi access network connects to a wired infrastructure. | |||
Either the wired or the Wi-Fi segment of the network can be the | Either the wired or the Wi-Fi segment of the network can be the | |||
bottleneck. The following sections describe basic test cases for | bottleneck. The following sections describe basic test cases for | |||
both scenarios separately. The same set of performance metrics as in | both scenarios separately. The same set of performance metrics as in | |||
[I-D.ietf-rmcat-eval-test]) should be collected for each test case. | [RFC8867]) should be collected for each test case. | |||
We recommend to carry out the test cases as defined in this document | We recommend carrying out the test cases as defined in this document | |||
using a simulator, such as [NS-2] or [NS-3]. When feasible, it is | using a simulator, such as [NS-2] or [NS-3]. When feasible, it is | |||
encouraged to perform testbed-based evaluations using Wi-Fi access | encouraged to perform testbed-based evaluations using Wi-Fi access | |||
points and endpoints running up-to-date IEEE 802.11 protocols, such | points and endpoints running up-to-date IEEE 802.11 protocols, such | |||
as 802.11ac and the emerging Wi-Fi 6, so as to verify the viability | as 802.11ac and the emerging Wi-Fi 6, so as to verify the viability | |||
of the candidate schemes. | of the candidate schemes. | |||
3.1. Bottleneck in Wired Network | 3.1. Bottleneck in Wired Network | |||
The test scenarios below are intended to mimic the setup of video | The test scenarios below are intended to mimic the setup of video | |||
conferencing over Wi-Fi connections from the home. Typically, the | conferencing over Wi-Fi connections from the home. Typically, the | |||
Wi-Fi home network is not congested and the bottleneck is present | Wi-Fi home network is not congested, and the bottleneck is present | |||
over the wired home access link. Although it is expected that test | over the wired home access link. Although it is expected that test | |||
evaluation results from this section are similar to those as in | evaluation results from this section are similar to those in | |||
[I-D.ietf-rmcat-eval-test], it is still worthwhile to run through | [RFC8867], it is still worthwhile to run through these tests as | |||
these tests as sanity checks. | sanity checks. | |||
3.1.1. Network topology | 3.1.1. Network Topology | |||
Figure 2 shows the network topology of Wi-Fi test cases. The test | Figure 2 shows the network topology of Wi-Fi test cases. The test | |||
contains multiple mobile nodes (MNs) connected to a common Wi-Fi | contains multiple mobile nodes (MNs) connected to a common Wi-Fi AP | |||
access point (AP) and their corresponding wired clients on fixed | and their corresponding wired clients on fixed nodes (FNs). Each | |||
nodes (FNs). Each connection carries either a RTP-based media flow | connection carries either an RTP-based media flow or a TCP traffic | |||
or a TCP traffic flow. Directions of the flows can be uplink (i.e., | flow. Directions of the flows can be uplink (i.e., from mobile nodes | |||
from mobile nodes to fixed nodes), downlink (i.e., from fixed nodes | to fixed nodes), downlink (i.e., from fixed nodes to mobile nodes), | |||
to mobile nodes), or bi-directional. The total number of | or bidirectional. The total number of uplink/downlink/bidirectional | |||
uplink/downlink/bi-directional flows for RTP-based media traffic and | flows for RTP-based media traffic and TCP traffic are denoted as N | |||
TCP traffic are denoted as N and M, respectively. | and M, respectively. | |||
Uplink | Uplink | |||
+----------------->+ | +----------------->+ | |||
+------+ +------+ | +------+ +------+ | |||
| MN_1 |)))) /=====| FN_1 | | | MN_1 |)))) /=====| FN_1 | | |||
+------+ )) // +------+ | +------+ )) // +------+ | |||
. )) // . | . )) // . | |||
. )) // . | . )) // . | |||
. )) // . | . )) // . | |||
+------+ +----+ +-----+ +------+ | +------+ +----+ +-----+ +------+ | |||
skipping to change at page 13, line 34 ¶ | skipping to change at line 588 ¶ | |||
+----------+ +----+ +-----+ +----------+ | +----------+ +----+ +-----+ +----------+ | |||
. )) \\ . | . )) \\ . | |||
. )) \\ . | . )) \\ . | |||
. )) \\ . | . )) \\ . | |||
+----------+ )) \\ +----------+ | +----------+ )) \\ +----------+ | |||
| MN_tcp_M |))) \=====| FN_tcp_M | | | MN_tcp_M |))) \=====| FN_tcp_M | | |||
+----------+ +----------+ | +----------+ +----------+ | |||
+<-----------------+ | +<-----------------+ | |||
Downlink | Downlink | |||
Figure 2: Network topology for Wi-Fi test cases | Figure 2: Network Topology for Wi-Fi Test Cases | |||
3.1.2. Test/simulation setup | 3.1.2. Test/Simulation Setup | |||
o Test duration: 120s | Test duration: 120 s | |||
o Wi-Fi network characteristics: | Wi-Fi network characteristics: | |||
* Radio propagation model: Log-distance path loss propagation | Radio propagation model: Log-distance path loss propagation model | |||
model (see [NS3WiFi]) | (see [NS3WiFi]) | |||
* PHY- and MAC-layer configuration: IEEE 802.11n | PHY- and MAC-layer configuration: IEEE 802.11n | |||
* MCS Index at 11: 16-QAM 1/2, Raw Data Rate at 52Mbps | MCS Index at 11: Raw data rate at 52 Mbps, 16-QAM (Quadrature | |||
amplitude modulation) and 1/2 coding rate | ||||
o Wired path characteristics: | Wired path characteristics: | |||
* Path capacity: 1Mbps | Path capacity: 1 Mbps | |||
* One-Way propagation delay: 50ms. | One-way propagation delay: 50 ms | |||
* Maximum end-to-end jitter: 30ms | Maximum end-to-end jitter: 30 ms | |||
* Bottleneck queue type: Drop tail. | Bottleneck queue type: Drop tail | |||
* Bottleneck queue size: 300ms. | Bottleneck queue size: 300 ms | |||
* Path loss ratio: 0%. | Path loss ratio: 0% | |||
o Application characteristics: | Application characteristics: | |||
* Media Traffic: | Media traffic: | |||
+ Media type: Video | Media type: Video | |||
+ Media direction: See Section 3.1.3 | Media direction: See Section 3.1.3 | |||
+ Number of media sources (N): See Section 3.1.3 | Number of media sources (N): See Section 3.1.3 | |||
+ Media timeline: | Media timeline: | |||
- Start time: 0s. | Start time: 0 s | |||
- End time: 119s. | End time: 119 s | |||
* Competing traffic: | Competing traffic: | |||
+ Type of sources: long-lived TCP or CBR over UDP | Type of sources: Long-lived TCP or CBR over UDP | |||
+ Traffic direction: See Section 3.1.3 | Traffic direction: See Section 3.1.3 | |||
+ Number of sources (M): See Section 3.1.3 | Number of sources (M): See Section 3.1.3 | |||
+ Congestion control: Default TCP congestion control [RFC5681] | Congestion control: Default TCP congestion control [RFC5681] | |||
or constant-bit-rate (CBR) traffic over UDP. | or CBR traffic over UDP | |||
+ Traffic timeline: See Section 3.1.3 | Traffic timeline: See Section 3.1.3 | |||
3.1.3. Typical test scenarios | 3.1.3. Typical Test Scenarios | |||
o Single uplink RTP-based media flow: N=1 with uplink direction and | Single uplink RTP-based media flow: N=1 with uplink direction and | |||
M=0. | M=0. | |||
o One pair of bi-directional RTP-based media flows: N=2 (i.e., one | One pair of bidirectional RTP-based media flows: N=2 (i.e., one | |||
uplink flow and one downlink flow); M=0. | uplink flow and one downlink flow); M=0. | |||
o One pair of bi-directional RTP-based media flows: N=2; one uplink | One pair of bidirectional RTP-based media flows: N=2; one uplink on- | |||
on-off CBR flow over UDP: M=1 (uplink). The CBR flow has ON time | off CBR flow over UDP: M=1 (uplink). The CBR flow has ON time at | |||
at t=0s-60s and OFF time at t=60s-119s. | t=0s-60s and OFF time at t=60s-119s. | |||
o One pair of bi-directional RTP-based media flows: N=2; one uplink | One pair of bidirectional RTP-based media flows: N=2; one uplink | |||
off-on CBR flow over UDP: M=1 (uplink). The CBR flow has OFF time | off-on CBR flow over UDP: M=1 (uplink). The CBR flow has OFF time | |||
at t=0s-60s and ON time at t=60s-119s. | at t=0s-60s and ON time at t=60s-119s. | |||
o One RTP-based media flow competing against one long-live TCP flow | One RTP-based media flow competing against one long-lived TCP flow | |||
in the uplink direction: N=1 (uplink) and M = 1(uplink). The TCP | in the uplink direction: N=1 (uplink) and M=1 (uplink). The TCP | |||
flow has start time at t=0s and end time at t=119s. | flow has start time at t=0s and end time at t=119s. | |||
3.1.4. Expected behavior | 3.1.4. Expected Behavior | |||
o Single uplink RTP-based media flow: the candidate algorithm is | Single uplink RTP-based media flow: The candidate algorithm is | |||
expected to detect the path capacity constraint, to converge to | expected to detect the path capacity constraint, to converge to | |||
the bottleneck link capacity, and to adapt the flow to avoid | the bottleneck link capacity, and to adapt the flow to avoid | |||
unwanted oscillations when the sending bit rate is approaching the | unwanted oscillations when the sending bit rate is approaching the | |||
bottleneck link capacity. No excessive oscillations in the media | bottleneck link capacity. No excessive oscillations in the media | |||
rate should be present. | rate should be present. | |||
o Bi-directional RTP-based media flows: the candidate algorithm is | Bidirectional RTP-based media flows: The candidate algorithm is | |||
expected to converge to the bottleneck capacity of the wired path | expected to converge to the bottleneck capacity of the wired path | |||
in both directions despite the presence of measurement noise over | in both directions despite the presence of measurement noise over | |||
the Wi-Fi connection. In the presence of background TCP or CBR | the Wi-Fi connection. In the presence of background TCP or CBR | |||
over UDP traffic, the rate of RTP-based media flows should adapt | over UDP traffic, the rate of RTP-based media flows should adapt | |||
promptly to the arrival and departure of background traffic flows. | promptly to the arrival and departure of background traffic flows. | |||
o One RTP-based media flow competing with long-live TCP flow in the | One RTP-based media flow competing with long-lived TCP flow in the | |||
uplink direction: the candidate algorithm is expected to avoid | uplink direction: The candidate algorithm is expected to avoid | |||
congestion collapse and to stabilize at a fair share of the | congestion collapse and to stabilize at a fair share of the | |||
bottleneck link capacity. | bottleneck link capacity. | |||
3.2. Bottleneck in Wi-Fi Network | 3.2. Bottleneck in Wi-Fi Network | |||
The test cases in this section assume that the wired segment along | The test cases in this section assume that the wired segment along | |||
the media path is well-provisioned whereas the bottleneck exists over | the media path is well-provisioned, whereas the bottleneck exists | |||
the Wi-Fi access network. This is to mimic the application scenarios | over the Wi-Fi access network. This is to mimic the application | |||
typically encountered by users in an enterprise environment or at a | scenarios typically encountered by users in an enterprise environment | |||
coffee house. | or at a coffee house. | |||
3.2.1. Network topology | 3.2.1. Network Topology | |||
Same as defined in Section 3.1.1 | Same as defined in Section 3.1.1. | |||
3.2.2. Test/simulation setup | 3.2.2. Test/Simulation Setup | |||
o Test duration: 120s | Test duration: 120 s | |||
o Wi-Fi network characteristics: | Wi-Fi network characteristics: | |||
* Radio propagation model: Log-distance path loss propagation | Radio propagation model: Log-distance path loss propagation model | |||
model (see [NS3WiFi]) | (see [NS3WiFi]) | |||
* PHY- and MAC-layer configuration: IEEE 802.11n | PHY- and MAC-layer configuration: IEEE 802.11n | |||
* MCS Index at 11: 16-QAM 1/2, Raw Data Rate at 52Mbps | MCS Index at 11: Raw data rate at 52 Mbps, 16-QAM (Quadrature | |||
amplitude modulation) and 1/2 coding rate | ||||
o Wired path characteristics: | Wired path characteristics: | |||
* Path capacity: 100Mbps. | Path capacity: 100 Mbps | |||
* One-Way propagation delay: 50ms. | One-Way propagation delay: 50 ms | |||
* Maximum end-to-end jitter: 30ms. | Maximum end-to-end jitter: 30 ms | |||
* Bottleneck queue type: Drop tail. | Bottleneck queue type: Drop tail | |||
* Bottleneck queue size: 300ms. | Bottleneck queue size: 300 ms | |||
* Path loss ratio: 0%. | Path loss ratio: 0% | |||
o Application characteristics: | Application characteristics | |||
* Media Traffic: | Media traffic: | |||
+ Media type: Video | Media type: Video | |||
+ Media direction: See Section 3.2.3. | Media direction: See Section 3.2.3 | |||
+ Number of media sources (N): See Section 3.2.3. | Number of media sources (N): See Section 3.2.3 | |||
+ Media timeline: | Media timeline: | |||
- Start time: 0s. | Start time: 0 s | |||
- End time: 119s. | End time: 119 s | |||
* Competing traffic: | Competing traffic: | |||
+ Type of sources: long-lived TCP or CBR over UDP. | Type of sources: long-lived TCP or CBR over UDP | |||
+ Number of sources (M): See Section 3.2.3. | Number of sources (M): See Section 3.2.3 | |||
+ Traffic direction: See Section 3.2.3. | Traffic direction: See Section 3.2.3 | |||
+ Congestion control: Default TCP congestion control [RFC5681] | Congestion control: Default TCP congestion control [RFC5681] | |||
or constant-bit-rate (CBR) traffic over UDP. | or CBR traffic over UDP | |||
+ Traffic timeline: See Section 3.2.3. | Traffic timeline: See Section 3.2.3 | |||
3.2.3. Typical test scenarios | 3.2.3. Typical Test Scenarios | |||
This section describes a few test scenarios that are deemed as | This section describes a few test scenarios that are deemed as | |||
important for understanding the behavior of a candidate RTP-based | important for understanding the behavior of a candidate RTP-based | |||
congestion control scheme over a Wi-Fi network. | congestion control scheme over a Wi-Fi network. | |||
a. Multiple RTP-based media flows sharing the wireless downlink: | Multiple RTP-based media flows sharing the wireless downlink: N=16 | |||
N=16 (all downlink); M = 0. This test case is for studying the | (all downlink); M=0. This test case is for studying the impact of | |||
impact of contention on the multiple concurrent media flows. For | contention on the multiple concurrent media flows. For an 802.11n | |||
an 802.11n network, given the MCS Index of 11 and the | network, given the MCS Index of 11 and the corresponding link rate | |||
corresponding link rate of 52Mbps, the total application-layer | of 52 Mbps, the total application-layer throughput (assuming | |||
throughput (assuming reasonable distance, low interference and | reasonable distance, low interference, and infrequent contentions | |||
infrequent contentions caused by competing streams) is around | caused by competing streams) is around 20 Mbps. A total of N=16 | |||
20Mbps. A total of N=16 RTP-based media flows (with a maximum | RTP-based media flows (with a maximum rate of 1.5 Mbps each) are | |||
rate of 1.5Mbps each) are expected to saturate the wireless | expected to saturate the wireless interface in this experiment. | |||
interface in this experiment. Evaluation of a given candidate | Evaluation of a given candidate scheme should focus on whether the | |||
scheme should focus on whether the downlink media flows can | downlink media flows can stabilize at a fair share of the total | |||
stabilize at a fair share of the total application-layer | application-layer throughput. | |||
throughput. | ||||
b. Multiple RTP-based media flows sharing the wireless uplink: N = | Multiple RTP-based media flows sharing the wireless uplink: N=16 | |||
16 (all uplink); M = 0. When multiple clients attempt to | (all uplink); M=0. When multiple clients attempt to transmit | |||
transmit media packets uplink over the Wi-Fi network, they | media packets uplink over the Wi-Fi network, they introduce more | |||
introduce more frequent contentions and potential collisions. | frequent contentions and potential collisions. Per-flow | |||
Per-flow throughput is expected to be lower than that in the | throughput is expected to be lower than that in the previous | |||
previous downlink-only scenario. Evaluation of a given candidate | downlink-only scenario. Evaluation of a given candidate scheme | |||
scheme should focus on whether the uplink flows can stabilize at | should focus on whether the uplink flows can stabilize at a fair | |||
a fair share of the total application-layer throughput. | share of the total application-layer throughput. | |||
c. Multiple bi-directional RTP-based media flows: N = 16 (8 uplink | Multiple bidirectional RTP-based media flows: N=16 (8 uplink and 8 | |||
and 8 downlink); M = 0. The goal of this test is to evaluate the | downlink); M=0. The goal of this test is to evaluate the | |||
performance of the candidate scheme in terms of bandwidth | performance of the candidate scheme in terms of bandwidth fairness | |||
fairness between uplink and downlink flows. | between uplink and downlink flows. | |||
d. Multiple bi-directional RTP-based media flows with on-off CBR | Multiple bidirectional RTP-based media flows with on-off CBR | |||
traffic over UDP: N = 16 (8 uplink and 8 downlink); M = 5 | traffic over UDP: N=16 (8 uplink and 8 downlink); M=5 (uplink). The | |||
(uplink). The goal of this test is to evaluate the adaptation | goal of this test is to evaluate the adaptation behavior of the | |||
behavior of the candidate scheme when its available bandwidth | candidate scheme when its available bandwidth changes due to the | |||
changes due to the departure of background traffic. The | departure of background traffic. The background traffic consists | |||
background traffic consists of several (e.g., M=5) CBR flows | of several (e.g., M=5) CBR flows transported over UDP. These | |||
transported over UDP. These background flows are ON at time | background flows are ON at time t=0-60s and OFF at time t=61-120s. | |||
t=0-60s and OFF at time t=61-120s. | ||||
e. Multiple bi-directional RTP-based media flows with off-on CBR | Multiple bidirectional RTP-based media flows with off-on CBR | |||
traffic over UDP: N = 16 (8 uplink and 8 downlink); M = 5 | traffic over UDP: N=16 (8 uplink and 8 downlink); M=5 (uplink). The | |||
(uplink). The goal of this test is to evaluate the adaptation | goal of this test is to evaluate the adaptation behavior of the | |||
behavior of the candidate scheme when its available bandwidth | candidate scheme when its available bandwidth changes due to the | |||
changes due to the arrival of background traffic. The background | arrival of background traffic. The background traffic consists of | |||
traffic consists of several (e.g., M=5) parallel CBR flows | several (e.g., M=5) parallel CBR flows transported over UDP. | |||
transported over UDP. These background flows are OFF at time | These background flows are OFF at time t=0-60s and ON at times | |||
t=0-60s and ON at times t=61-120s. | t=61-120s. | |||
f. Multiple bi-directional RTP-based media flows in the presence of | Multiple bidirectional RTP-based media flows in the presence of | |||
background TCP traffic: N=16 (8 uplink and 8 downlink); M = 5 | background TCP traffic: N=16 (8 uplink and 8 downlink); M=5 | |||
(uplink). The goal of this test is to evaluate how RTP-based | (uplink). The goal of this test is to evaluate how RTP-based | |||
media flows compete against TCP over a congested Wi-Fi network | media flows compete against TCP over a congested Wi-Fi network for | |||
for a given candidate scheme. TCP flows have start time at t=40s | a given candidate scheme. TCP flows have start time at t=40s and | |||
and end time at t=80s. | end time at t=80s. | |||
g. Varying number of RTP-based media flows: A series of tests can be | Varying number of RTP-based media flows: A series of tests can be | |||
carried out for the above test cases with different values of N, | carried out for the above test cases with different values of N, | |||
e.g., N = [4, 8, 12, 16, 20]. The goal of this test is to | e.g., N=[4, 8, 12, 16, 20]. The goal of this test is to evaluate | |||
evaluate how a candidate scheme responds to varying traffic load/ | how a candidate scheme responds to varying traffic load/demand | |||
demand over a congested Wi-Fi network. The start times of the | over a congested Wi-Fi network. The start times of the media | |||
media flows are randomly distributes within a window of t=0-10s; | flows are randomly distributed within a window of t=0-10s; their | |||
their end times are randomly distributed within a window of | end times are randomly distributed within a window of t=110-120s. | |||
t=110-120s. | ||||
3.2.4. Expected behavior | 3.2.4. Expected Behavior | |||
o Multiple downlink RTP-based media flows: each media flow is | Multiple downlink RTP-based media flows: Each media flow is expected | |||
expected to get its fair share of the total bottleneck link | to get its fair share of the total bottleneck link bandwidth. | |||
bandwidth. Overall bandwidth usage should not be significantly | Overall bandwidth usage should not be significantly lower than | |||
lower than that experienced by the same number of concurrent | that experienced by the same number of concurrent downlink TCP | |||
downlink TCP flows. In other words, the behavior of multiple | flows. In other words, the behavior of multiple concurrent TCP | |||
concurrent TCP flows will be used as a performance benchmark for | flows will be used as a performance benchmark for this test | |||
this test scenario. The end-to-end delay and packet loss ratio | scenario. The end-to-end delay and packet loss ratio experienced | |||
experienced by each flow should be within an acceptable range for | by each flow should be within an acceptable range for real-time | |||
real-time multimedia applications. | multimedia applications. | |||
o Multiple uplink RTP-based media flows: overall bandwidth usage by | Multiple uplink RTP-based media flows: Overall bandwidth usage by | |||
all media flows should not be significantly lower than that | all media flows should not be significantly lower than that | |||
experienced by the same number of concurrent uplink TCP flows. In | experienced by the same number of concurrent uplink TCP flows. In | |||
other words, the behavior of multiple concurrent TCP flows will be | other words, the behavior of multiple concurrent TCP flows will be | |||
used as a performance benchmark for this test scenario. | used as a performance benchmark for this test scenario. | |||
o Multiple bi-directional RTP-based media flows with dynamic | Multiple bidirectional RTP-based media flows with dynamic | |||
background traffic carrying CBR flows over UDP: the media flows | background traffic carrying CBR flows over UDP: The media flows are | |||
are expected to adapt in a timely fashion to the changes in | expected to adapt in a timely fashion to the changes in available | |||
available bandwidth introduced by the arrival/departure of | bandwidth introduced by the arrival/departure of background | |||
background traffic. | traffic. | |||
o Multiple bi-directional RTP-based media flows with dynamic | Multiple bidirectional RTP-based media flows with dynamic | |||
background traffic over TCP: during the presence of TCP background | background traffic over TCP: During the presence of TCP background | |||
flows, the overall bandwidth usage by all media flows should not | flows, the overall bandwidth usage by all media flows should not | |||
be significantly lower than those achieved by the same number of | be significantly lower than those achieved by the same number of | |||
bi-directional TCP flows. In other words, the behavior of | bidirectional TCP flows. In other words, the behavior of multiple | |||
multiple concurrent TCP flows will be used as a performance | concurrent TCP flows will be used as a performance benchmark for | |||
benchmark for this test scenario. All downlink media flows are | this test scenario. All downlink media flows are expected to | |||
expected to obtain similar bandwidth as each other. The | obtain similar bandwidth as each other. The throughput of each | |||
throughput of each media flow is expected to decrease upon the | media flow is expected to decrease upon the arrival of TCP | |||
arrival of TCP background traffic and, conversely, increase upon | background traffic and, conversely, increase upon their departure. | |||
their departure. Both reactions should occur in a timely fashion, | Both reactions should occur in a timely fashion, for example, | |||
for example, within 10s of seconds. | within 10s of seconds. | |||
o Varying number of bi-directional RTP-based media flows: the test | Varying number of bidirectional RTP-based media flows: The test | |||
results for varying values of N -- while keeping all other | results for varying values of N -- while keeping all other | |||
parameters constant -- is expected to show steady and stable per- | parameters constant -- is expected to show steady and stable per- | |||
flow throughput for each value of N. The average throughput of | flow throughput for each value of N. The average throughput of | |||
all media flows is expected to stay constant around the maximum | all media flows is expected to stay constant around the maximum | |||
rate when N is small, then gradually decrease with increasing | rate when N is small, then gradually decrease with increasing | |||
value of N till it reaches the minimum allowed rate, beyond which | value of N till it reaches the minimum allowed rate, beyond which | |||
the offered load to the Wi-Fi network exceeds its capacity (i.e., | the offered load to the Wi-Fi network exceeds its capacity (i.e., | |||
with a very large value of N). | with a very large value of N). | |||
3.3. Other Potential Test Cases | 3.3. Other Potential Test Cases | |||
skipping to change at page 19, line 46 ¶ | skipping to change at line 884 ¶ | |||
classes (or Access Categories). RTP-based real-time media flows | classes (or Access Categories). RTP-based real-time media flows | |||
should achieve better performance in terms of lower delay and fewer | should achieve better performance in terms of lower delay and fewer | |||
packet losses with EDCA/WMM enabled when competing against non- | packet losses with EDCA/WMM enabled when competing against non- | |||
interactive background traffic such as file transfers. When most of | interactive background traffic such as file transfers. When most of | |||
the traffic over Wi-Fi is dominated by media, however, turning on WMM | the traffic over Wi-Fi is dominated by media, however, turning on WMM | |||
may degrade performance since all media flows now attempt to access | may degrade performance since all media flows now attempt to access | |||
the wireless transmission medium more aggressively, thereby causing | the wireless transmission medium more aggressively, thereby causing | |||
more frequent collisions and collision-induced losses. This is a | more frequent collisions and collision-induced losses. This is a | |||
topic worthy of further investigation. | topic worthy of further investigation. | |||
3.3.2. Effect of heterogeneous link rates | 3.3.2. Effect of Heterogeneous Link Rates | |||
As discussed in [Heusse2003], the presence of clients operating over | As discussed in [Heusse2003], the presence of clients operating over | |||
slow PHY-layer link rates (e.g., a legacy 802.11b device) connected | slow PHY-layer link rates (e.g., a legacy 802.11b device) connected | |||
to a modern network may adversely impact the overall performance of | to a modern network may adversely impact the overall performance of | |||
the network. Additional test cases can be devised to evaluate the | the network. Additional test cases can be devised to evaluate the | |||
effect of clients with heterogeneous link rates on the performance of | effect of clients with heterogeneous link rates on the performance of | |||
the candidate congestion control algorithm. Such test cases, for | the candidate congestion control algorithm. Such test cases, for | |||
instance, can specify that the PHY-layer link rates for all clients | instance, can specify that the PHY-layer link rates for all clients | |||
span over a wide range (e.g., 2Mbps to 54Mbps) for investigating its | span over a wide range (e.g., 2 Mbps to 54 Mbps) for investigating | |||
effect on the congestion control behavior of the real-time | its effect on the congestion control behavior of the real-time | |||
interactive applications. | interactive applications. | |||
4. IANA Considerations | 4. IANA Considerations | |||
This memo includes no request to IANA. | This document has no IANA actions. | |||
5. Security Considerations | 5. Security Considerations | |||
The security considerations in [I-D.ietf-rmcat-eval-criteria] and the | The security considerations in [RFC8868] and the relevant congestion | |||
relevant congestion control algorithms apply. The principles for | control algorithms apply. The principles for congestion control are | |||
congestion control are described in [RFC2914], and in particular, any | described in [RFC2914], and in particular, any new method must | |||
new method must implement safeguards to avoid congestion collapse of | implement safeguards to avoid congestion collapse of the Internet. | |||
the Internet. | ||||
Given the difficulty of deterministic wireless testing, it is | Given the difficulty of deterministic wireless testing, it is | |||
recommended and expected that the tests described in this document | recommended and expected that the tests described in this document | |||
would be done via simulations. However, in the case where these test | would be done via simulations. However, in the case where these test | |||
cases are carried out in a testbed setting, the evaluation should | cases are carried out in a testbed setting, the evaluation should | |||
take place in a controlled lab environment. In the testbed, the | take place in a controlled lab environment. In the testbed, the | |||
applications, simulators and network nodes ought to be well-behaved | applications, simulators, and network nodes ought to be well-behaved | |||
and should not impact the desired results. It is important to take | and should not impact the desired results. It is important to take | |||
appropriate caution to avoid leaking non-responsive traffic with | appropriate caution to avoid leaking nonresponsive traffic with | |||
unproven congestion avoidance behavior onto the open Internet. | unproven congestion avoidance behavior onto the open Internet. | |||
6. Contributors | 6. References | |||
The following individuals contributed to the design, implementation, | ||||
and verification of the proposed test cases during earlier stages of | ||||
this work. They have helped to validate and substantially improve | ||||
this specification. | ||||
Ingemar Johansson, <ingemar.s.johansson@ericsson.com> of Ericsson AB | ||||
contributing to the description and validation of cellular test cases | ||||
during the earlier stage of this draft. | ||||
Wei-Tian Tan, <dtan2@cisco.com>, of Cisco Systems designed and set up | ||||
a Wi-Fi testbed for evaluating parallel video conferencing streams, | ||||
based upon which proposed Wi-Fi test cases are described. He also | ||||
recommended additional test cases to consider, such as the impact of | ||||
EDCA/WMM usage. | ||||
Michael A. Ramalho, <mar42@cornell.edu> of AcousticComms Consulting | ||||
(previously at Cisco Systems) applied learnings from Cisco's internal | ||||
experimentation to the early versions of the draft. He also worked | ||||
on validating the proposed test cases in a VM-based lab setting. | ||||
7. Acknowledgments | ||||
The authors would like to thank Tomas Frankkila, Magnus Westerlund, | ||||
Kristofer Sandlund, Sergio Mena de la Cruz, and Mirja Kuehlewind for | ||||
their valuable inputs and review comments regarding this draft. | ||||
8. References | ||||
8.1. Normative References | 6.1. Normative References | |||
[HO-deploy-3GPP] | [HO-deploy-3GPP] | |||
TS 25.814, 3GPP., "Physical layer aspects for evolved | 3GPP, "Physical layer aspects for evolved Universal | |||
Universal Terrestrial Radio Access (UTRA)", October 2006, | Terrestrial Radio Access (UTRA)", TS 25.814, October 2006, | |||
<http://www.3gpp.org/ftp/specs/ | <http://www.3gpp.org/ftp/specs/ | |||
archive/25_series/25.814/25814-710.zip>. | archive/25_series/25.814/25814-710.zip>. | |||
[I-D.ietf-rmcat-eval-criteria] | ||||
Singh, V., Ott, J., and S. Holmer, "Evaluating Congestion | ||||
Control for Interactive Real-time Media", draft-ietf- | ||||
rmcat-eval-criteria-13 (work in progress), March 2020. | ||||
[I-D.ietf-rmcat-eval-test] | ||||
Sarker, Z., Singh, V., Zhu, X., and M. Ramalho, "Test | ||||
Cases for Evaluating RMCAT Proposals", draft-ietf-rmcat- | ||||
eval-test-10 (work in progress), May 2019. | ||||
[IEEE802.11] | [IEEE802.11] | |||
IEEE, "Standard for Information technology-- | IEEE, "Standard for Information technology-- | |||
Telecommunications and information exchange between | Telecommunications and information exchange between | |||
systems Local and metropolitan area networks--Specific | systems Local and metropolitan area networks--Specific | |||
requirements Part 11: Wireless LAN Medium Access Control | requirements Part 11: Wireless LAN Medium Access Control | |||
(MAC) and Physical Layer (PHY) Specifications", 2012. | (MAC) and Physical Layer (PHY) Specifications", | |||
IEEE 802.11-2012, | ||||
<https://ieeexplore.ieee.org/document/7786995>. | ||||
[NS3WiFi] "Wi-Fi Channel Model in ns-3 Simulator", | [NS3WiFi] "ns3::YansWifiChannel Class Reference", | |||
<https://www.nsnam.org/doxygen/ | <https://www.nsnam.org/doxygen/ | |||
classns3_1_1_yans_wifi_channel.html>. | classns3_1_1_yans_wifi_channel.html>. | |||
[RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion | [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion | |||
Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, | Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, | |||
<https://www.rfc-editor.org/info/rfc5681>. | <https://www.rfc-editor.org/info/rfc5681>. | |||
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC | [RFC8867] Sarker, Z., Singh, V., Zhu, X., and M. Ramalho, "Test | |||
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, | Cases for Evaluating Congestion Control for Interactive | |||
May 2017, <https://www.rfc-editor.org/info/rfc8174>. | Real-Time Media", RFC 8867, DOI 10.17487/RFC8867, January | |||
2021, <https://www.rfc-editor.org/info/rfc8867>. | ||||
8.2. Informative References | [RFC8868] Singh, V., Ott, J., and S. Holmer, "Evaluating Congestion | |||
Control for Interactive Real-Time Media", RFC 8868, | ||||
DOI 10.17487/RFC8868, January 2021, | ||||
<https://www.rfc-editor.org/info/rfc8868>. | ||||
6.2. Informative References | ||||
[Heusse2003] | [Heusse2003] | |||
Heusse, M., Rousseau, F., Berger-Sabbatel, G., and A. | Heusse, M., Rousseau, F., Berger-Sabbatel, G., and A. | |||
Duda, "Performance anomaly of 802.11b", in Proc. 23th | Duda, "Performance anomaly of 802.11b", IEEE INFOCOM 2003, | |||
Annual Joint Conference of the IEEE Computer and | Twenty-second Annual Joint Conference of the IEEE Computer | |||
Communications Societies, (INFOCOM'03), March 2003. | and Communications Societies, | |||
DOI 10.1109/INFCOM.2003.1208921, March 2003, | ||||
<https://ieeexplore.ieee.org/document/1208921>. | ||||
[HO-def-3GPP] | [HO-def-3GPP] | |||
TR 21.905, 3GPP., "Vocabulary for 3GPP Specifications", | 3GPP, "Vocabulary for 3GPP Specifications", 3GPP | |||
December 2009, <http://www.3gpp.org/ftp/specs/ | TS 21.905, December 2009, <http://www.3gpp.org/ftp/specs/ | |||
archive/21_series/21.905/21905-940.zip>. | archive/21_series/21.905/21905-940.zip>. | |||
[HO-LTE-3GPP] | [HO-LTE-3GPP] | |||
TS 36.331, 3GPP., "E-UTRA- Radio Resource Control (RRC); | 3GPP, "Evolved Universal Terrestrial Radio Access | |||
Protocol specification", December 2011, | (E-UTRA); Radio Resource Control (RRC); Protocol | |||
specification", 3GPP TS 36.331, December 2011, | ||||
<http://www.3gpp.org/ftp/specs/ | <http://www.3gpp.org/ftp/specs/ | |||
archive/36_series/36.331/36331-990.zip>. | archive/36_series/36.331/36331-990.zip>. | |||
[HO-UMTS-3GPP] | [HO-UMTS-3GPP] | |||
TS 25.331, 3GPP., "Radio Resource Control (RRC); Protocol | 3GPP, "Radio Resource Control (RRC); Protocol | |||
specification", December 2011, | specification", 3GPP TS 25.331, December 2011, | |||
<http://www.3gpp.org/ftp/specs/ | <http://www.3gpp.org/ftp/specs/ | |||
archive/25_series/25.331/25331-990.zip>. | archive/25_series/25.331/25331-990.zip>. | |||
[NS-2] "ns-2", December 2014, | [NS-2] "ns-2", December 2014, | |||
<http://nsnam.sourceforge.net/wiki/index.php/Main_Page>. | <http://nsnam.sourceforge.net/wiki/index.php/Main_Page>. | |||
[NS-3] "ns-3 Network Simulator", <https://www.nsnam.org/>. | [NS-3] "ns-3 Network Simulator", <https://www.nsnam.org/>. | |||
[QoS-3GPP] | [QoS-3GPP] 3GPP, "Policy and charging control architecture", 3GPP | |||
TS 23.203, 3GPP., "Policy and charging control | TS 23.203, June 2011, <http://www.3gpp.org/ftp/specs/ | |||
architecture", June 2011, <http://www.3gpp.org/ftp/specs/ | ||||
archive/23_series/23.203/23203-990.zip>. | archive/23_series/23.203/23203-990.zip>. | |||
[RFC2914] Floyd, S., "Congestion Control Principles", BCP 41, | [RFC2914] Floyd, S., "Congestion Control Principles", BCP 41, | |||
RFC 2914, DOI 10.17487/RFC2914, September 2000, | RFC 2914, DOI 10.17487/RFC2914, September 2000, | |||
<https://www.rfc-editor.org/info/rfc2914>. | <https://www.rfc-editor.org/info/rfc2914>. | |||
Contributors | ||||
The following individuals contributed to the design, implementation, | ||||
and verification of the proposed test cases during earlier stages of | ||||
this work. They have helped to validate and substantially improve | ||||
this specification. | ||||
Ingemar Johansson <ingemar.s.johansson@ericsson.com> of Ericsson AB | ||||
contributed to the description and validation of cellular test cases | ||||
during the earlier stage of this document. | ||||
Wei-Tian Tan <dtan2@cisco.com> of Cisco Systems designed and set up a | ||||
Wi-Fi testbed for evaluating parallel video conferencing streams, | ||||
based upon which proposed Wi-Fi test cases are described. He also | ||||
recommended additional test cases to consider, such as the impact of | ||||
EDCA/WMM usage. | ||||
Michael A. Ramalho <mar42@cornell.edu> of AcousticComms Consulting | ||||
(previously at Cisco Systems) applied lessons from Cisco's internal | ||||
experimentation to the draft versions of the document. He also | ||||
worked on validating the proposed test cases in a virtual-machine- | ||||
based lab setting. | ||||
Acknowledgments | ||||
The authors would like to thank Tomas Frankkila, Magnus Westerlund, | ||||
Kristofer Sandlund, Sergio Mena de la Cruz, and Mirja Kühlewind for | ||||
their valuable inputs and review comments regarding this document. | ||||
Authors' Addresses | Authors' Addresses | |||
Zaheduzzaman Sarker | Zaheduzzaman Sarker | |||
Ericsson AB | Ericsson AB | |||
Laboratoriegraend 11 | Torshamnsgatan 23 | |||
Luleae 97753 | SE-164 83 Stockholm | |||
Sweden | Sweden | |||
Phone: +46 107173743 | Phone: +46 10 717 37 43 | |||
Email: zaheduzzaman.sarker@ericsson.com | Email: zaheduzzaman.sarker@ericsson.com | |||
Xiaoqing Zhu | Xiaoqing Zhu | |||
Cisco Systems | Cisco Systems | |||
12515 Research Blvd., Building 4 | Building 4 | |||
Austin, TX 78759 | 12515 Research Blvd | |||
USA | Austin, TX 78759 | |||
United States of America | ||||
Email: xiaoqzhu@cisco.com | Email: xiaoqzhu@cisco.com | |||
Jiantao Fu | Jiantao Fu | |||
Cisco Systems | Cisco Systems | |||
771 Alder Drive | 771 Alder Drive | |||
Milpitas, CA 95035 | Milpitas, CA 95035 | |||
USA | United States of America | |||
Email: jianfu@cisco.com | Email: jianfu@cisco.com | |||
End of changes. 227 change blocks. | ||||
498 lines changed or deleted | 489 lines changed or added | |||
This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/ |