draft-ietf-rmcat-wireless-tests-01.txt   draft-ietf-rmcat-wireless-tests-02.txt 
Network Working Group Z. Sarker Network Working Group Z. Sarker
Internet-Draft I. Johansson Internet-Draft I. Johansson
Intended status: Informational Ericsson AB Intended status: Informational Ericsson AB
Expires: May 8, 2016 X. Zhu Expires: November 8, 2016 X. Zhu
J. Fu J. Fu
W. Tan W. Tan
M. Ramalho M. Ramalho
Cisco Systems Cisco Systems
November 5, 2015 May 7, 2016
Evaluation Test Cases for Interactive Real-Time Media over Wireless Evaluation Test Cases for Interactive Real-Time Media over Wireless
Networks Networks
draft-ietf-rmcat-wireless-tests-01 draft-ietf-rmcat-wireless-tests-02
Abstract Abstract
It is evident that to ensure seamless and robust user experience It is evident that to ensure seamless and robust user experience
across all type of access networks multimedia communication suits across all type of access networks multimedia communication suits
should adapt to the changing network conditions. There is an ongoing should adapt to the changing network conditions. There is an ongoing
effort in IETF RMCAT working group to standardize rate adaptive effort in IETF RMCAT working group to standardize rate adaptive
algorithm(s) to be used in the real-time interactive communication. algorithm(s) to be used in the real-time interactive communication.
In this document test cases are described to evaluate the In this document test cases are described to evaluate the
performances of the proposed endpoint adaptation solutions in LTE performances of the proposed endpoint adaptation solutions in LTE
skipping to change at page 1, line 45 skipping to change at page 1, line 45
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on May 8, 2016. This Internet-Draft will expire on November 8, 2016.
Copyright Notice Copyright Notice
Copyright (c) 2015 IETF Trust and the persons identified as the Copyright (c) 2016 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
skipping to change at page 2, line 38 skipping to change at page 2, line 38
3.2. Bad Radio Coverage . . . . . . . . . . . . . . . . . . . 8 3.2. Bad Radio Coverage . . . . . . . . . . . . . . . . . . . 8
3.2.1. Network connection . . . . . . . . . . . . . . . . . 9 3.2.1. Network connection . . . . . . . . . . . . . . . . . 9
3.2.2. Simulation Setup . . . . . . . . . . . . . . . . . . 9 3.2.2. Simulation Setup . . . . . . . . . . . . . . . . . . 9
3.3. Desired Evaluation Metrics for cellular test cases . . . 10 3.3. Desired Evaluation Metrics for cellular test cases . . . 10
4. Wi-Fi Networks Specific Test Cases . . . . . . . . . . . . . 10 4. Wi-Fi Networks Specific Test Cases . . . . . . . . . . . . . 10
4.1. Bottleneck in Wired Network . . . . . . . . . . . . . . . 12 4.1. Bottleneck in Wired Network . . . . . . . . . . . . . . . 12
4.1.1. Network topology . . . . . . . . . . . . . . . . . . 12 4.1.1. Network topology . . . . . . . . . . . . . . . . . . 12
4.1.2. Test setup . . . . . . . . . . . . . . . . . . . . . 13 4.1.2. Test setup . . . . . . . . . . . . . . . . . . . . . 13
4.1.3. Typical test scenarios . . . . . . . . . . . . . . . 14 4.1.3. Typical test scenarios . . . . . . . . . . . . . . . 14
4.1.4. Expected behavior . . . . . . . . . . . . . . . . . . 14 4.1.4. Expected behavior . . . . . . . . . . . . . . . . . . 14
4.2. Bottleneck in Wi-Fi Network . . . . . . . . . . . . . . . 14 4.2. Bottleneck in Wi-Fi Network . . . . . . . . . . . . . . . 15
4.2.1. Network topology . . . . . . . . . . . . . . . . . . 15 4.2.1. Network topology . . . . . . . . . . . . . . . . . . 15
4.2.2. Test setup . . . . . . . . . . . . . . . . . . . . . 15 4.2.2. Test setup . . . . . . . . . . . . . . . . . . . . . 15
4.2.3. Typical test scenarios . . . . . . . . . . . . . . . 16 4.2.3. Typical test scenarios . . . . . . . . . . . . . . . 16
4.2.4. Expected behavior . . . . . . . . . . . . . . . . . . 17 4.2.4. Expected behavior . . . . . . . . . . . . . . . . . . 17
4.3. Potential Potential Test Cases . . . . . . . . . . . . . 17 4.3. Potential Potential Test Cases . . . . . . . . . . . . . 18
4.3.1. EDCA/WMM usage . . . . . . . . . . . . . . . . . . . 17 4.3.1. EDCA/WMM usage . . . . . . . . . . . . . . . . . . . 18
4.3.2. Legacy 802.11b Effects . . . . . . . . . . . . . . . 17 4.3.2. Legacy 802.11b Effects . . . . . . . . . . . . . . . 18
5. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 18 5. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 18
6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 18 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 18
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 18 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 18
8. Security Considerations . . . . . . . . . . . . . . . . . . . 18 8. Security Considerations . . . . . . . . . . . . . . . . . . . 18
9. References . . . . . . . . . . . . . . . . . . . . . . . . . 18 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 18
9.1. Normative References . . . . . . . . . . . . . . . . . . 18 9.1. Normative References . . . . . . . . . . . . . . . . . . 19
9.2. Informative References . . . . . . . . . . . . . . . . . 19 9.2. Informative References . . . . . . . . . . . . . . . . . 20
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 20 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 20
1. Introduction 1. Introduction
Wireless networks (both cellular and Wi-Fi [IEEE802.11] local area Wireless networks (both cellular and Wi-Fi [IEEE802.11] local area
network) are an integral part of the Internet. Mobile devices network) are an integral part of the Internet. Mobile devices
connected to the wireless networks produces huge amount of media connected to the wireless networks produces huge amount of media
traffic in the Internet. They covers the scenarios of having a video traffic in the Internet. They covers the scenarios of having a video
call in the bus to media consumption sitting on a couch in a living call in the bus to media consumption sitting on a couch in a living
skipping to change at page 10, line 38 skipping to change at page 10, line 38
4. Wi-Fi Networks Specific Test Cases 4. Wi-Fi Networks Specific Test Cases
Given the prevalence of Internet access links over Wi-Fi, it is Given the prevalence of Internet access links over Wi-Fi, it is
important to evaluate candidate RMCAT congestion control solutions important to evaluate candidate RMCAT congestion control solutions
over Wi-Fi test cases. Such evaluations should also highlight the over Wi-Fi test cases. Such evaluations should also highlight the
inherent different characteristics of Wi-Fi networks in contrast to inherent different characteristics of Wi-Fi networks in contrast to
Wired networks: Wired networks:
o The wireless radio channel is subject to interference from nearby o The wireless radio channel is subject to interference from nearby
transmitters, multi-path fading, and shadowing, causing transmitters, multipath fading, and shadowing, causing
fluctuations in link throughput and sometimes an error-prone fluctuations in link throughput and sometimes an error-prone
communication environment communication environment
o Available network bandwidth is not only shared over the air o Available network bandwidth is not only shared over the air
between concurrent users, but also between uplink and downlink between cocurrent users, but also between uplink and downlink
traffic due to the half duplex nature of wireless transmission traffic due to the half duplex nature of wireless transmission
medium. medium.
o Packet transmissions over Wi-Fi are susceptible to contentions and o Packet transmessions over Wi-Fi are susceptible to contentions and
collisions over the air. Consequently, traffic load beyond a collisions over the air. Consequently, traffic load beyond a
certain utilization level over a Wi-Fi network can introduce certain utilization level over a Wi-Fi network can introduce
frequent collisions and significant network overhead. This, in frequent collisions and significant network overhead. This, in
turn, leads to excessive delay, retransmission, loss and lower turn, leads to excessive delay, retransmission, loss and lower
effective bandwidth for applications. effective bandwidth for applications.
o The IEEE 802.11 standard (i.e., Wi-Fi) supports multi-rate o The IEEE 802.11 standard (i.e., Wi-Fi) supports multi-rate
transmission capabilities by dynamically choosing the most transmission capabilities by dynamically choosing the most
appropriate modulation scheme for a given received signal appropriate modulation scheme for a given received singal
strength. A different choice of Physical-layer rate will lead to strength. A different choice of Physical-layer rate will lead to
different application-layer throughput. different application-layer throughput.
o Presence of legacy 802.11b networks can significantly slow down o Presence of legancy 802.11b networks can significantly slow down
the rest of a modern Wi-Fi Network, since it takes longer to the the rest of a modern Wi-Fi Network, since it takes longer to
transmit the same packet over a slower link than over a faster transmit the same packet over a slower link than over a faster
link. [Editor's note: maybe include a reference here instead.] link. [Editor's note: maybe include a reference here instead.]
o Handover from one Wi-Fi Access Point (AP) to another may cause o Handover from one Wi-Fi Access Point (AP) to another may cause
packet delay and loss. packet delay and loss.
o IEEE 802.11e defined EDCA/WMM (Enhanced DCF Channel Access/Wi-Fi o IEEE 802.11e defined EDCA/WMM (Enhanced DCF Channel Access/Wi-Fi
Multi-Media) to give voice and video streams higher priority over Multi-Media) to give voice and video streams higher priority over
pure data applications (e.g., file transfers). pure data applications (e.g., file transfers).
As we can see here, presence of Wi-Fi network in different network As we can see here, presence of Wi-Fi network in different network
topologies and traffic arrival can exert different impact on the topologies and traffic arrival can exert different impact on the
network performance in terms of video transport rate, packet loss and network performance in terms of video transport rate, packet loss and
delay that, in turn, effect end-to-end real-time multimedia delay that, in turn, effect end-to-end real-time multimedia
congestion control. congestion control.
Throughout this draft, unless otherwise mentioned, test cases are Throughout this draft, unless otherwise mentioned, test cases are
described using 802.11g due to its wide availability in network described using 802.11n due to its wide availability in real-world
simulation platform. In practice, however, statistics collected from networks. Statistics collected from enterprise Wi-Fi networks show
enterprise networks show that the dominant physical modes are 802.11n that the dominant physical modes are 802.11n and 802.11ac, accounting
and 802.11ac, accounting for 73.6% and 22.5% of enterprise network for 73.6% and 22.5% of enterprise network users, respectively.
users, respectively. Whenever possible, it is recommended to extend
some of the experiments to 802.11n and 802.11ac, so as to reflect a
more modern Wi-Fi network setting.
Since Wi-Fi network normally connects to a wired infrastructure, Since Wi-Fi network normally connects to a wired infrastructure,
either the wired network or the Wi-Fi network could be the either the wired network or the Wi-Fi network could be the
bottleneck. In the following section, we describe basic test cases bottleneck. In the following section, we describe basic test cases
for both scenarios separately. The same set of performance metrics for both scenarios separately. The same set of performance metrics
as in [I-D.ietf-rmcat-eval-test]) should be collected for each test as in [I-D.ietf-rmcat-eval-test]) should be collected for each test
case. case.
While all test cases described below can be carried out using While all test cases described below can be carried out using
simulations, e.g. based on [ns-2] or [ns-3], it is also recommended simulations, e.g. based on [ns-2] or [ns-3], it is also recommended
to perform testbed-based evaluations using Wi-Fi access points and to perform testbed-based evaluations using Wi-Fi access points and
endpoints running up-to-date IEEE 802.11 protocols. [Editor's Note: endpoints running up-to-date IEEE 802.11 protocols. [Editor's Note:
need to add some more discussions on the pros and cons of simulation- need to add some more discussions on the pros and cons of simulation-
based vs. testbed-based evaluations. It will be good to provide based vs. testbed-based evaluations. Will be good to provide
recommended testbed configurations. ] recommended testbed configurations. ]
4.1. Bottleneck in Wired Network 4.1. Bottleneck in Wired Network
The test scenarios below are intended to mimic the set up of video The test scenarios below are intended to mimic the set up of video
conferencing over Wi-Fi connections from the home. Typically, the conferencing over Wi-Fi connections from the home. Typically, the
Wi-Fi home network is not congested and the bottleneck is present Wi-Fi home network is not congested and the bottleneck is present
over the wired home access link. Although it is expected that test over the wired home access link. Although it is expected that test
evaluation results from this section are similar to those from test evaluation results from this section are similar to those from test
cases defined for wired networks (see [I-D.ietf-rmcat-eval-test]), it cases defined for wired networks (see [I-D.ietf-rmcat-eval-test]), it
skipping to change at page 13, line 14 skipping to change at page 13, line 14
4.1.2. Test setup 4.1.2. Test setup
o Test duration: 120s o Test duration: 120s
o Wi-Fi network characteristics: o Wi-Fi network characteristics:
* Radio propagation model: Log-distance path loss propagation * Radio propagation model: Log-distance path loss propagation
model [NS3WiFi] model [NS3WiFi]
* PHY- and MAC-layer configuration: IEEE 802.11g * PHY- and MAC-layer configuration: IEEE 802.11n
* PHY-layer link rate: 54 Mbps * MCS Index at 11: 16-QAM 1/2, Raw Data Rate@52Mbps
o Wired path characteristics: o Wired path characteristics:
* Path capacity: 1Mbps * Path capacity: 1Mbps
* One-Way propagation delay: 50ms. * One-Way propagation delay: 50ms.
* Maximum end-to-end jitter: 30ms * Maximum end-to-end jitter: 30ms
* Bottleneck queue type: Drop tail. * Bottleneck queue type: Drop tail.
skipping to change at page 13, line 50 skipping to change at page 13, line 50
+ Number of media sources (N): See Section 4.1.3 + Number of media sources (N): See Section 4.1.3
+ Media timeline: + Media timeline:
- Start time: 0s. - Start time: 0s.
- End time: 119s. - End time: 119s.
* Competing traffic: * Competing traffic:
+ Type of sources: long-lived TCP + Type of sources: long-lived TCP or CBR over UDP
+ Traffic direction: See Section 4.1.3 + Traffic direction: See Section 4.1.3
+ Number of sources (M): See Section 4.1.3 + Number of sources (M): See Section 4.1.3
+ Congestion control: Default TCP congestion control [TBD] + Congestion control: Default TCP congestion control [TBD] or
CBR over UDP
+ Traffic timeline:
- Start time: 0s
- End time: 119s + Traffic timeline: See Section 4.1.3
4.1.3. Typical test scenarios 4.1.3. Typical test scenarios
o Single uplink RMCAT flow: N=1 with uplink direction and M=0. o Single uplink RMCAT flow: N=1 with uplink direction and M=0.
o One pair of bi-directional RMCAT flows: N=2 (with one uplink flow o One pair of bi-directional RMCAT flows: N=2 (with one uplink flow
and one downlink flow); M=0. and one downlink flow); M=0.
o One pair of bi-directional RMCAT flows, one on-off CBR over UDP
flow on uplink : N=2 (with one uplink flow and one downlink flow);
M=1 (uplink). CBR flow on time at 0s-60s, off time at 60s-119s
o One pair of bi-directional RMCAT flows, one off-on CBR over UDP
flow on uplink : N=2 (with one uplink flow and one downlink flow);
M=1 (uplink). UDP off time: 0s-60s, on time: 60s-119s
o One RMCAT flow competing against one long-live TCP flow over o One RMCAT flow competing against one long-live TCP flow over
uplink: N=1 (uplink) and M = 1(uplink). uplink: N=1 (uplink) and M = 1(uplink), TCP start time: 0s, end
time: 119s.
4.1.4. Expected behavior 4.1.4. Expected behavior
o Single uplink RMCAT flow: the candidate algorithm is expected to o Single uplink RMCAT flow: the candidate algorithm is expected to
detect the path capacity constraint, converges to bottleneck detect the path capacity constraint, converges to bottleneck
link's capacity and adapt the flow to avoid unwanted oscillation link's capacity and adapt the flow to avoid unwanted oscillation
when the sending bit rate is approaching the bottleneck link's when the sending bit rate is approaching the bottleneck link's
capacity. No excessive rate oscillations. capacity. No excessivie rate oscillations.
o Bi-directional RMCAT flows: It is expected that the candidate o Bi-directional RMCAT flows: It is expected that the candidate
algorithms is able to converge to the bottleneck capacity of the algorithms is able to converge to the bottleneck capacity of the
wired path on both directions despite of the presence of wired path on both directions despite presense of measurment noise
measurement noise over the Wi-Fi connection. over the Wi-Fi connection. In the presence of background TCP or
CBR over UDP traffic, the rate of RMCAT flows should adapt in a
timely manner to changes in the available bottleneck bandwidth.
o One RMCAT flow competing with long-live TCP flow over uplink: the o One RMCAT flow competing with long-live TCP flow over uplink: the
candidate algorithm should be able to avoid congestion collapse, candidate algorithm should be able to avoid congestion collapse,
and stabilize at a fair share of the bottleneck capacity over the and stablize at a fair share of the bottleneck capacity over the
wired path. wired path.
4.2. Bottleneck in Wi-Fi Network 4.2. Bottleneck in Wi-Fi Network
These test cases assume that the wired portion along the media path These test cases assume that the wired portion along the media path
are well-provisioned. The bottleneck is in the Wi-Fi network over are well-provisioned. The bottleneck is in the Wi-Fi network over
wireless. This is to mimic the enterprise/coffee-house scenarios. wireless. This is to mimic the enterprise/coffee-house scenarios.
4.2.1. Network topology 4.2.1. Network topology
skipping to change at page 15, line 18 skipping to change at page 15, line 24
4.2.2. Test setup 4.2.2. Test setup
o Test duration: 120s o Test duration: 120s
o Wi-Fi network characteristics: o Wi-Fi network characteristics:
* Radio propagation model: Log-distance path loss propagation * Radio propagation model: Log-distance path loss propagation
model [NS3WiFi] model [NS3WiFi]
* PHY- and MAC-layer configuration: IEEE 802.11g * PHY- and MAC-layer configuration: IEEE 802.11n
* PHY-layer link rate: 54 Mbps * MCS Index at 11: 16-QAM 1/2, Raw Data Rate at 52Mbps
o Wired path characteristics: o Wired path characteristics:
* Path capacity: 100Mbps * Path capacity: 100Mbps
* One-Way propagation delay: 50ms. * One-Way propagation delay: 50ms.
* Maximum end-to-end jitter: 30ms * Maximum end-to-end jitter: 30ms
* Bottleneck queue type: Drop tail. * Bottleneck queue type: Drop tail.
skipping to change at page 16, line 5 skipping to change at page 16, line 11
+ Number of media sources (N): See Section 4.2.3 + Number of media sources (N): See Section 4.2.3
+ Media timeline: + Media timeline:
- Start time: 0s. - Start time: 0s.
- End time: 119s. - End time: 119s.
* Competing traffic: * Competing traffic:
+ Type of sources: long-lived TCP + Type of sources: long-lived TCP or CBR over UDP
+ Number of sources (M): See Section 4.2.3 + Number of sources (M): See Section 4.2.3
+ Traffic direction: See Section 4.2.3 + Traffic direction: See Section 4.2.3
+ Congestion control: Default TCP congestion control [TBD] + Congestion control: Default TCP congestion control [TBD] or
CBR over UDP
+ Traffic timeline:
- Start time: 0s
- End time: 119s + Traffic timeline: See Section 4.2.3
4.2.3. Typical test scenarios 4.2.3. Typical test scenarios
This sections describes a few specific test scenarios that are deemed This sections describes a few specific test scenarios that are deemed
as important for understanding behavior of a RMCAT candidate solution as important for understanding behavior of a RMCAT candidate solution
over a Wi-Fi network. over a Wi-Fi network.
o Multiple RMCAT Flows Sharing the Wireless Downlink: N=16 (all o Multiple RMCAT Flows Sharing the Wireless Downlink: N=16 (all
downlink); M = 0; This test case is for studying the impact of downlink); M = 0; This test case is for studying the impact of
contention on competing RMCAT flows. Specifications for IEEE contention on competing RMCAT flows. Specifications for IEEE
802.11g with a physical-layer transmission rate of 54 Mbps is 802.11n, MCS Index at 11: 16-QAM 1/2, Raw Data Rate at 52Mbps is
chosen. Note that retransmission and MAC-layer headers and chosen. Note that retransmissions, MAC-layer headers, and control
control packets may be sent at a lower link speed. The total packets may be sent at a lower link speed. The total application-
application-layer throughput (reasonable distance, low layer throughput (reasonable distance, low interference and small
interference and small number of contention stations) for 802.11g number of contention stations) for 802.11n is around 20 Mbps.
is around 20 Mbps. Consequently, a total of N=16 RMCAT flows are Consequently, a total of N=16 RMCAT flows are needed for
needed for saturating the wireless interface in this experiment. saturating the wireless interface in this experiment. Evaluation
Evaluation of a given candidate solution should focus on whether of a given candidate solution should focus on whether downlink
downlink RMCAT flows can stabilize at a fair share of bandwidth. RMCAT flows can stablize at a fair share of bandwidth.
o Multiple RMCAT Flows Sharing the Wireless Uplink: N = 16 (all o Multiple RMCAT Flows Sharing the Wireless Uplink: N = 16 (all
downlink); M = 0; When multiple clients attempt to transmit video downlink); M = 0; When multiple clients attempt to transmit video
packets uplink over the wireless interface, they introduce more packets uplink over the wireless interface, they introduce more
frequent contentions and potentially collisions. Per-flow frequent contentions and potentially collisions. Per-flow
throughput is expected to be lower than that in the previous throughput is expected to be lower than that in the previous
downlink-only scenario. Evaluation of a given candidate solution downlink-only scenario. Evaluation of a given candidate solution
should focus on whether uplink flows can stabilize at a fair share should focus on whether uplink flows can stablize at a fair share
of bandwidth. of bandwidth.
o Multiple Bi-directional RMCAT Flows: N = 16 (8 uplink and 8 o Multiple Bi-directional RMCAT Flows: N = 16 (8 uplink and 8
downlink); M = 0. The goal of this test is to evaluate downlink); M = 0. the goal of this test is to evaluate
performance of the candidate solution in terms of bandwidth performance of the candidate solution in terms of bandwidth
fairness between uplink and downlink flow. fairness between uplink and downlink flow.
o Multiple Bi-directional RMCAT Flows with on-off CBR traffic: N =
16 (8 uplink and 8 downlink); M = 5(uplink). The goal of this
test is to evaluate upgrading performance of the candidate
solution in terms of available bandwidth changes caused by the CBR
uplink flow over UDP. CBR over UDP background flows have on time
0s-60s, and off time 60s-119s
o Multiple Bi-directional RMCAT Flows with off-on CBR traffic: N =
16 (8 uplink and 8 downlink); M = 5(uplink). The goal of this
test is to evaluate upgrading performance of the candidate
solution in terms of available bandwidth changes caused by the CBR
uplink flow over UDP. CBR over UDP background flows have off time
0s-60s, and on time 60s-119s.
o Multiple RMCAT flows in the presence of background TCP traffic: o Multiple RMCAT flows in the presence of background TCP traffic:
the goal of this test is to evaluate how RMCAT flows compete the goal of this test is to evaluate how RMCAT flows compete
against TCP over a congested Wi-Fi network for a given candidate against TCP over a congested Wi-Fi network for a given candidate
solution. [Editor's Note: more detailed description will be added solution. TCP start time: 0s, end time: 119s. [Editor's Note:
in the next version in terms of directoin/number of RMCAT and TCP more detailed description will be added in the next version in
flows. ] terms of directoin/number of RMCAT and TCP flows. ]
o Varying number of RMCAT flows: the goal of this test is to o Varying number of RMCAT flows: the goal of this test is to
evaluate how a candidate RMCAT solution responds to varying evaluate how a candidate RMCAT solution responds to varying
traffic load/demand over a congested Wi-Fi network. [Editor's traffic load/demand over a congested Wi-Fi network. [Editor's
Note: more detailed description will be added in the next version Note: more detailed description will be added in the next version
in terms of arrival/departure pattern of the flows.] in terms of arrival/departure pattern of the flows.]
4.2.4. Expected behavior 4.2.4. Expected behavior
o Multiple downlink RMCAT flows: All RMCAT flows should get fair o Multiple downlink RMCAT flows: All RMCAT flows should get fair
share of the bandwidth. Overall bandwidth usage should be no less share of the bandwidth. Overall bandwidth usage should be no less
than same case with TCP flows (using TCP as performance than same case with TCP flows (using TCP as performance
benchmark). The delay and loss should be within acceptable range benchmark). The delay and loss should be within acceptable range
for real-time multimedia flow. for real-time multimedia flow.
o Multiple uplink RMCAT flows: overall bandwidth usage shared by all o Multiple uplink RMCAT flows: overall bandwidth usage shared by all
RMCAT flows should be no less than those shared by the same number RMCAT flows should be no less than those shared by the same number
of TCP flows (i.e., benchmark performance using TCP flows). of TCP flows (i.e., benchmark performance using TCP flows).
o Multiple bi-directional RMCAT flows: overall bandwidth usage o Multiple bi-directional RMCAT flows with CBR over UDP traffic:
shared by all RMCAT flows should be no less than those shared by RMCAT flows should adapt to the changes in available bandwidth.
the same number of TCP flows (i.e., benchmark performance using
TCP flows). All downlink RMCAT flows are expected to obtain o Multiple bi-directional RMCAT flows with TCP traffic: overall
similar bandwidth with respect to each other. bandwidth usage shared by all RMCAT flows should be no less than
those shared by the same number of TCP flows (i.e., benchmark
performance using TCP flows). All downlink RMCAT flows are
expected to obtain similar bandwidth with respect to each other.
4.3. Potential Potential Test Cases 4.3. Potential Potential Test Cases
4.3.1. EDCA/WMM usage 4.3.1. EDCA/WMM usage
EDCA/WMM is prioritized QoS with four traffic classes (or Access EDCA/WMM is prioritized QoS with four traffic classes (or Access
Categories) with differing priorities. RMCAT flow should have better Categories) with differing priorities. RMCAT flow should have better
performance (lower delay, less loss) with EDCA/WMM enabled when performance (lower delay, less loss) with EDCA/WMM enabled when
competing against non-interactive background traffic (e.g., file competing against non-interactive background traffic (e.g., file
transfers). When most of the traffic over Wi-Fi is dominated by transfers). When most of the traffic over Wi-Fi is dominated by
media, however, turning on WMM may actually degrade performance. media, however, turning on WMM may actually degrade performance.
This is a topic worthy of further investigation. This is a topic worthy of further investigation.
4.3.2. Legacy 802.11b Effects 4.3.2. Legacy 802.11b Effects
When there is 802.11b devices connected to modern 802.11 network, it When there is 802.11b devices connected to modern 802.11 network, it
may affect the performance of the whole network. Additional test may affect the performance of the whole network. Additional test
cases can be added to evaluate the affects of legacy devices on the cases can be added to evaluate the affects of legancy devices on the
performance of RMCAT congestion control algorithm. performance of RMCAT congestion control algorithm.
5. Conclusion 5. Conclusion
This document defines a collection of test cases that are considered This document defines a collection of test cases that are considered
important for cellular and Wi-Fi networks. Moreover, this document important for cellular and Wi-Fi networks. Moreover, this document
also provides a framework for defining additional test cases over also provides a framework for defining additional test cases over
wireless cellular/Wi-Fi networks. wireless cellular/Wi-Fi networks.
6. Acknowledgements 6. Acknowledgements
skipping to change at page 19, line 12 skipping to change at page 19, line 30
<http://www.3gpp.org/ftp/specs/ <http://www.3gpp.org/ftp/specs/
archive/36_series/36.331/36331-990.zip>. archive/36_series/36.331/36331-990.zip>.
[HO-UMTS-3GPP] [HO-UMTS-3GPP]
TS 25.331, 3GPP., "Radio Resource Control (RRC); Protocol TS 25.331, 3GPP., "Radio Resource Control (RRC); Protocol
specification", December 2011, specification", December 2011,
<http://www.3gpp.org/ftp/specs/ <http://www.3gpp.org/ftp/specs/
archive/25_series/25.331/25331-990.zip>. archive/25_series/25.331/25331-990.zip>.
[I-D.ietf-rmcat-eval-criteria] [I-D.ietf-rmcat-eval-criteria]
Singh, V. and J. Ott, "Evaluating Congestion Control for Varun, V., Ott, J., and S. Holmer, "Evaluating Congestion
Interactive Real-time Media", draft-ietf-rmcat-eval- Control for Interactive Real-time Media", draft-ietf-
criteria-04 (work in progress), October 2015. rmcat-eval-criteria-05 (work in progress), March 2016.
[NS3WiFi] "Wi-Fi Channel Model in NS3 Simulator", [NS3WiFi] "Wi-Fi Channel Model in NS3 Simulator",
<https://www.nsnam.org/doxygen/ <https://www.nsnam.org/doxygen/
classns3_1_1_yans_wifi_channel.html>. classns3_1_1_yans_wifi_channel.html>.
[QoS-3GPP] [QoS-3GPP]
TS 23.203, 3GPP., "Policy and charging control TS 23.203, 3GPP., "Policy and charging control
architecture", June 2011, <http://www.3gpp.org/ftp/specs/ architecture", June 2011, <http://www.3gpp.org/ftp/specs/
archive/23_series/23.203/23203-990.zip>. archive/23_series/23.203/23203-990.zip>.
skipping to change at page 19, line 38 skipping to change at page 20, line 13
<http://www.rfc-editor.org/info/rfc2119>. <http://www.rfc-editor.org/info/rfc2119>.
9.2. Informative References 9.2. Informative References
[I-D.ietf-rmcat-cc-requirements] [I-D.ietf-rmcat-cc-requirements]
Jesup, R. and Z. Sarker, "Congestion Control Requirements Jesup, R. and Z. Sarker, "Congestion Control Requirements
for Interactive Real-Time Media", draft-ietf-rmcat-cc- for Interactive Real-Time Media", draft-ietf-rmcat-cc-
requirements-09 (work in progress), December 2014. requirements-09 (work in progress), December 2014.
[I-D.ietf-rmcat-eval-test] [I-D.ietf-rmcat-eval-test]
Sarker, Z., Singh, V., Zhu, X., and M. Ramalho, "Test Sarker, Z., Varun, V., Zhu, X., and M. Ramalho, "Test
Cases for Evaluating RMCAT Proposals", draft-ietf-rmcat- Cases for Evaluating RMCAT Proposals", draft-ietf-rmcat-
eval-test-02 (work in progress), September 2015. eval-test-03 (work in progress), March 2016.
[IEEE802.11] [IEEE802.11]
"Standard for Information technology--Telecommunications "Standard for Information technology--Telecommunications
and information exchange between systems Local and and information exchange between systems Local and
metropolitan area networks--Specific requirements Part 11: metropolitan area networks--Specific requirements Part 11:
Wireless LAN Medium Access Control (MAC) and Physical Wireless LAN Medium Access Control (MAC) and Physical
Layer (PHY) Specifications", 2012. Layer (PHY) Specifications", 2012.
[LTE-simulator] [LTE-simulator]
"NS-3, A discrete-Event Network Simulator", "NS-3, A discrete-Event Network Simulator",
 End of changes. 40 change blocks. 
73 lines changed or deleted 92 lines changed or added

This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/