draft-ietf-rmcat-wireless-tests-02.txt   draft-ietf-rmcat-wireless-tests-03.txt 
Network Working Group Z. Sarker Network Working Group Z. Sarker
Internet-Draft I. Johansson Internet-Draft I. Johansson
Intended status: Informational Ericsson AB Intended status: Informational Ericsson AB
Expires: November 8, 2016 X. Zhu Expires: May 18, 2017 X. Zhu
J. Fu J. Fu
W. Tan W. Tan
M. Ramalho M. Ramalho
Cisco Systems Cisco Systems
May 7, 2016 November 14, 2016
Evaluation Test Cases for Interactive Real-Time Media over Wireless Evaluation Test Cases for Interactive Real-Time Media over Wireless
Networks Networks
draft-ietf-rmcat-wireless-tests-02 draft-ietf-rmcat-wireless-tests-03
Abstract Abstract
It is evident that to ensure seamless and robust user experience There is an ongoing effort in IETF RMCAT working group to standardize
across all type of access networks multimedia communication suits rate adaptation algorithm(s) for real-time interactive communication.
should adapt to the changing network conditions. There is an ongoing To ensure seamless and robust user experience, the proposed rate
effort in IETF RMCAT working group to standardize rate adaptive adaptation algorithm(s) should work well across all access network
algorithm(s) to be used in the real-time interactive communication. types. This document describes test cases for evaluating
In this document test cases are described to evaluate the performances of the proposed rate adaptation solutions over LTE and
performances of the proposed endpoint adaptation solutions in LTE Wi-Fi networks.
networks and Wi-Fi networks. The proposed algorithms should be
evaluated using the test cases defined in this document to select
most optimal solutions.
Status of This Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on November 8, 2016. This Internet-Draft will expire on May 18, 2017.
Copyright Notice Copyright Notice
Copyright (c) 2016 IETF Trust and the persons identified as the Copyright (c) 2016 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 48 skipping to change at page 2, line 43
4.2. Bottleneck in Wi-Fi Network . . . . . . . . . . . . . . . 15 4.2. Bottleneck in Wi-Fi Network . . . . . . . . . . . . . . . 15
4.2.1. Network topology . . . . . . . . . . . . . . . . . . 15 4.2.1. Network topology . . . . . . . . . . . . . . . . . . 15
4.2.2. Test setup . . . . . . . . . . . . . . . . . . . . . 15 4.2.2. Test setup . . . . . . . . . . . . . . . . . . . . . 15
4.2.3. Typical test scenarios . . . . . . . . . . . . . . . 16 4.2.3. Typical test scenarios . . . . . . . . . . . . . . . 16
4.2.4. Expected behavior . . . . . . . . . . . . . . . . . . 17 4.2.4. Expected behavior . . . . . . . . . . . . . . . . . . 17
4.3. Potential Potential Test Cases . . . . . . . . . . . . . 18 4.3. Potential Potential Test Cases . . . . . . . . . . . . . 18
4.3.1. EDCA/WMM usage . . . . . . . . . . . . . . . . . . . 18 4.3.1. EDCA/WMM usage . . . . . . . . . . . . . . . . . . . 18
4.3.2. Legacy 802.11b Effects . . . . . . . . . . . . . . . 18 4.3.2. Legacy 802.11b Effects . . . . . . . . . . . . . . . 18
5. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 18 5. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 18
6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 18 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 18
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 18 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19
8. Security Considerations . . . . . . . . . . . . . . . . . . . 18 8. Security Considerations . . . . . . . . . . . . . . . . . . . 19
9. References . . . . . . . . . . . . . . . . . . . . . . . . . 18 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 19
9.1. Normative References . . . . . . . . . . . . . . . . . . 19 9.1. Normative References . . . . . . . . . . . . . . . . . . 19
9.2. Informative References . . . . . . . . . . . . . . . . . 20 9.2. Informative References . . . . . . . . . . . . . . . . . 20
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 20 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 20
1. Introduction 1. Introduction
Wireless networks (both cellular and Wi-Fi [IEEE802.11] local area Wireless networks (both cellular and Wi-Fi [IEEE802.11] local area
network) are an integral part of the Internet. Mobile devices network) are an integral part of the Internet. Mobile devices
connected to the wireless networks produces huge amount of media connected to the wireless networks generate huge amount of media
traffic in the Internet. They covers the scenarios of having a video traffic in the Internet. Application scenarios range from users
call in the bus to media consumption sitting on a couch in a living having a video call in the bus to media consumption by someone
room. It is a well known fact that the characteristic and challenges sitting on a living room couch. It is well known that the
for offering service over wireless network are very different than characteristics and technical challenges for offering multimedia
providing the same over a wired network. Even though RMCAT basic services over wireless are very different from those of providing the
test cases defines number of test cases that covers lots of effects same service over a wired network. Even though RMCAT basic test
of the impairments visible in the wireless networks but there are cases as defined in [I-D.ietf-rmcat-eval-test] have covered many
characteristics and dynamics those are unique to particular wireless effects of the impairments also visible in wireless networks, there
environment. For example, in the LTE the base station maintains remains characteristics and dynamics unique to a given wireless
queues per radio bearer per user hence it gives different interaction environment. For example, in LTE networks the base station maintains
when all traffic from user share the same queue. Again, the user queues per radio bearer per user hence it leads to a different nature
mobility in a cellular network is different than the user mobility in of interaction from that over the wired network, where traffic from
a Wi-Fi network. Thus, It is important to evaluate the performance all users share the same queue. Furthermore, user mobility in a
of the proposed RMCAT candidates separately in the cellular mobile cellular network is different than user mobility in a Wi-Fi network.
networks and Wi-Fi local networks (IEEE 802.11xx protocol family ). Therefore, It is important to evaluate performance of the proposed
RMCAT candidate solutions separately over cellular mobile networks
and over Wi-Fi local networks (i.e., IEEE 802.11xx protocol family ).
RMCAT evaluation criteria [I-D.ietf-rmcat-eval-criteria] document RMCAT evaluation criteria [I-D.ietf-rmcat-eval-criteria] document
provides the guideline to perform the evaluation on candidate provides the guideline for evaluating candidate algorithms and
algorithms and recognizes wireless networks to be important access recognizes the importance of testing over wireless access networks.
link. However, it does not provides particular test cases to However, it does not describe any specific test cases for evaluating
evaluate the performance of the candidate algorithm. In this performance of the candidate algorithm. This document describes test
document we describe test cases specifically targeting cellular cases specifically targeting cellular networks such as LTE networks
networks such as LTE networks and Wi-Fi local networks. and Wi-Fi local networks.
2. Terminologies 2. Terminologies
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC2119 [RFC2119] document are to be interpreted as described in RFC2119 [RFC2119]
3. Cellular Network Specific Test Cases 3. Cellular Network Specific Test Cases
A cellular environment is more complicated than a wireline ditto A cellular environment is more complicated than a wireline ditto
skipping to change at page 10, line 33 skipping to change at page 10, line 33
from capture to display. from capture to display.
o Transport delay. o Transport delay.
o Algorithm stability in terms of rate variation. o Algorithm stability in terms of rate variation.
4. Wi-Fi Networks Specific Test Cases 4. Wi-Fi Networks Specific Test Cases
Given the prevalence of Internet access links over Wi-Fi, it is Given the prevalence of Internet access links over Wi-Fi, it is
important to evaluate candidate RMCAT congestion control solutions important to evaluate candidate RMCAT congestion control solutions
over Wi-Fi test cases. Such evaluations should also highlight the over test cases that include Wi-Fi access lines. Such evaluations
inherent different characteristics of Wi-Fi networks in contrast to should also highlight the inherent different characteristics of Wi-Fi
Wired networks: networks in contrast to wired networks:
o The wireless radio channel is subject to interference from nearby o The wireless radio channel is subject to interference from nearby
transmitters, multipath fading, and shadowing, causing transmitters, multipath fading, and shadowing, causing
fluctuations in link throughput and sometimes an error-prone fluctuations in link throughput and sometimes an error-prone
communication environment communication environment
o Available network bandwidth is not only shared over the air o Available network bandwidth is not only shared over the air
between cocurrent users, but also between uplink and downlink between cocurrent users, but also between uplink and downlink
traffic due to the half duplex nature of wireless transmission traffic due to the half duplex nature of wireless transmission
medium. medium.
o Packet transmessions over Wi-Fi are susceptible to contentions and o Packet transmissions over Wi-Fi are susceptible to contentions and
collisions over the air. Consequently, traffic load beyond a collisions over the air. Consequently, traffic load beyond a
certain utilization level over a Wi-Fi network can introduce certain utilization level over a Wi-Fi network can introduce
frequent collisions and significant network overhead. This, in frequent collisions and significant network overhead. This, in
turn, leads to excessive delay, retransmission, loss and lower turn, leads to excessive delay, retransmissions, packet losses and
effective bandwidth for applications. lower effective bandwidth for applications.
o The IEEE 802.11 standard (i.e., Wi-Fi) supports multi-rate o The IEEE 802.11 standard (i.e., Wi-Fi) supports multi-rate
transmission capabilities by dynamically choosing the most transmission capabilities by dynamically choosing the most
appropriate modulation scheme for a given received singal appropriate modulation scheme for a given received singal
strength. A different choice of Physical-layer rate will lead to strength. A different choice of physical-layer rate leads to
different application-layer throughput. different application-layer throughput.
o Presence of legancy 802.11b networks can significantly slow down o Presence of legancy 802.11b networks can significantly slow down
the the rest of a modern Wi-Fi Network, since it takes longer to the the rest of a modern Wi-Fi Network, since it takes longer to
transmit the same packet over a slower link than over a faster transmit the same packet over a slower link than over a faster
link. [Editor's note: maybe include a reference here instead.] link. [Editor's note: maybe include a reference here instead.]
o Handover from one Wi-Fi Access Point (AP) to another may cause o Handover from one Wi-Fi Access Point (AP) to another may lead to
packet delay and loss. packet delay and losses during the process.
o IEEE 802.11e defined EDCA/WMM (Enhanced DCF Channel Access/Wi-Fi o IEEE 802.11e defined EDCA/WMM (Enhanced DCF Channel Access/Wi-Fi
Multi-Media) to give voice and video streams higher priority over Multi-Media) to give voice and video streams higher priority over
pure data applications (e.g., file transfers). pure data applications (e.g., file transfers).
As we can see here, presence of Wi-Fi network in different network In summary, presence of Wi-Fi access links in different network
topologies and traffic arrival can exert different impact on the topologies can exert different impact on the network performance in
network performance in terms of video transport rate, packet loss and terms of application-layer effective throughput, packet loss rate,
delay that, in turn, effect end-to-end real-time multimedia and packet delivery delay. These, in turn, influence the behavior of
congestion control. end-to-end real-time multimedia congestion control.
Throughout this draft, unless otherwise mentioned, test cases are Throughout this draft, unless otherwise mentioned, test cases are
described using 802.11n due to its wide availability in real-world described using 802.11n due to its wide availability in real-world
networks. Statistics collected from enterprise Wi-Fi networks show networks. Statistics collected from enterprise Wi-Fi networks show
that the dominant physical modes are 802.11n and 802.11ac, accounting that the dominant physical modes are 802.11n and 802.11ac, accounting
for 73.6% and 22.5% of enterprise network users, respectively. for 73.6% and 22.5% of enterprise network users, respectively.
Since Wi-Fi network normally connects to a wired infrastructure, Typically, a Wi-Fi access network connects to a wired infrastructure.
either the wired network or the Wi-Fi network could be the Either the wired or the Wi-Fi segment of the network could be the
bottleneck. In the following section, we describe basic test cases bottleneck. In the following sections, we describe basic test cases
for both scenarios separately. The same set of performance metrics for both scenarios separately. The same set of performance metrics
as in [I-D.ietf-rmcat-eval-test]) should be collected for each test as in [I-D.ietf-rmcat-eval-test]) should be collected for each test
case. case.
While all test cases described below can be carried out using While all test cases described below can be carried out using
simulations, e.g. based on [ns-2] or [ns-3], it is also recommended simulations, e.g. based on [ns-2] or [ns-3], it is also recommended
to perform testbed-based evaluations using Wi-Fi access points and to perform testbed-based evaluations using Wi-Fi access points and
endpoints running up-to-date IEEE 802.11 protocols. [Editor's Note: endpoints running up-to-date IEEE 802.11 protocols. [Editor's Note:
need to add some more discussions on the pros and cons of simulation- need to add some more discussions on the pros and cons of simulation-
based vs. testbed-based evaluations. Will be good to provide based vs. testbed-based evaluations. Will be good to provide
skipping to change at page 14, line 33 skipping to change at page 14, line 33
flow on uplink : N=2 (with one uplink flow and one downlink flow); flow on uplink : N=2 (with one uplink flow and one downlink flow);
M=1 (uplink). UDP off time: 0s-60s, on time: 60s-119s M=1 (uplink). UDP off time: 0s-60s, on time: 60s-119s
o One RMCAT flow competing against one long-live TCP flow over o One RMCAT flow competing against one long-live TCP flow over
uplink: N=1 (uplink) and M = 1(uplink), TCP start time: 0s, end uplink: N=1 (uplink) and M = 1(uplink), TCP start time: 0s, end
time: 119s. time: 119s.
4.1.4. Expected behavior 4.1.4. Expected behavior
o Single uplink RMCAT flow: the candidate algorithm is expected to o Single uplink RMCAT flow: the candidate algorithm is expected to
detect the path capacity constraint, converges to bottleneck detect the path capacity constraint, to converge to bottleneck
link's capacity and adapt the flow to avoid unwanted oscillation link capacity and to adapt the flow to avoid unwanted oscillation
when the sending bit rate is approaching the bottleneck link's when the sending bit rate is approaching the bottleneck link
capacity. No excessivie rate oscillations. capacity. No excessive rate oscillations should be present.
o Bi-directional RMCAT flows: It is expected that the candidate o Bi-directional RMCAT flows: It is expected that the candidate
algorithms is able to converge to the bottleneck capacity of the algorithm is able to converge to the bottleneck capacity of the
wired path on both directions despite presense of measurment noise wired path on both directions despite presence of measurment noise
over the Wi-Fi connection. In the presence of background TCP or over the Wi-Fi connection. In the presence of background TCP or
CBR over UDP traffic, the rate of RMCAT flows should adapt in a CBR over UDP traffic, the rate of RMCAT flows should adapt in a
timely manner to changes in the available bottleneck bandwidth. timely manner to changes in the available bottleneck bandwidth.
o One RMCAT flow competing with long-live TCP flow over uplink: the o One RMCAT flow competing with long-live TCP flow over uplink: the
candidate algorithm should be able to avoid congestion collapse, candidate algorithm should be able to avoid congestion collapse,
and stablize at a fair share of the bottleneck capacity over the and to stablize at a fair share of the bottleneck link capacity.
wired path.
4.2. Bottleneck in Wi-Fi Network 4.2. Bottleneck in Wi-Fi Network
These test cases assume that the wired portion along the media path These test cases assume that the wired portion along the media path
are well-provisioned. The bottleneck is in the Wi-Fi network over is well-provisioned whereas the bottleneck exists over the Wi-Fi
wireless. This is to mimic the enterprise/coffee-house scenarios. access network. This is to mimic the application scenarios typically
encountered by users in an enterprise environment or at a coffee
house.
4.2.1. Network topology 4.2.1. Network topology
Same as defined in Section 4.1.1 Same as defined in Section 4.1.1
4.2.2. Test setup 4.2.2. Test setup
o Test duration: 120s o Test duration: 120s
o Wi-Fi network characteristics: o Wi-Fi network characteristics:
skipping to change at page 16, line 24 skipping to change at page 16, line 25
+ Traffic direction: See Section 4.2.3 + Traffic direction: See Section 4.2.3
+ Congestion control: Default TCP congestion control [TBD] or + Congestion control: Default TCP congestion control [TBD] or
CBR over UDP CBR over UDP
+ Traffic timeline: See Section 4.2.3 + Traffic timeline: See Section 4.2.3
4.2.3. Typical test scenarios 4.2.3. Typical test scenarios
This sections describes a few specific test scenarios that are deemed This section describes a few test scenarios that are deemed as
as important for understanding behavior of a RMCAT candidate solution important for understanding the behavior of a RMCAT candidate
over a Wi-Fi network. solution over a Wi-Fi network.
o Multiple RMCAT Flows Sharing the Wireless Downlink: N=16 (all o Multiple RMCAT Flows Sharing the Wireless Downlink: N=16 (all
downlink); M = 0; This test case is for studying the impact of downlink); M = 0. This test case is for studying the impact of
contention on competing RMCAT flows. Specifications for IEEE contention on competing RMCAT flows. For an 802.11n network,
802.11n, MCS Index at 11: 16-QAM 1/2, Raw Data Rate at 52Mbps is given the MCS Index of 11 and the corresponding raw data rate of
chosen. Note that retransmissions, MAC-layer headers, and control 52Mbps, the total application-layer throughput (assuming
packets may be sent at a lower link speed. The total application- reasonable distance, low interference and infrequent contentions
layer throughput (reasonable distance, low interference and small caused by competing streams) is around 20Mbps. Consequently, a
number of contention stations) for 802.11n is around 20 Mbps. total of N=16 RMCAT flows are needed to saturate the wireless
Consequently, a total of N=16 RMCAT flows are needed for interface in this experiment. Evaluation of a given candidate
saturating the wireless interface in this experiment. Evaluation solution should focus on whether downlink RMCAT flows can stablize
of a given candidate solution should focus on whether downlink at a fair share of total application-layer throughput.
RMCAT flows can stablize at a fair share of bandwidth.
o Multiple RMCAT Flows Sharing the Wireless Uplink: N = 16 (all o Multiple RMCAT Flows Sharing the Wireless Uplink: N = 16 (all
downlink); M = 0; When multiple clients attempt to transmit video downlink); M = 0. When multiple clients attempt to transmit video
packets uplink over the wireless interface, they introduce more packets uplink over the wireless interface, they introduce more
frequent contentions and potentially collisions. Per-flow frequent contentions and potential collisions. Per-flow
throughput is expected to be lower than that in the previous throughput is expected to be lower than that in the previous
downlink-only scenario. Evaluation of a given candidate solution downlink-only scenario. Evaluation of a given candidate solution
should focus on whether uplink flows can stablize at a fair share should focus on whether uplink flows can stablize at a fair share
of bandwidth. of application-layer throughput.
o Multiple Bi-directional RMCAT Flows: N = 16 (8 uplink and 8 o Multiple Bi-directional RMCAT Flows: N = 16 (8 uplink and 8
downlink); M = 0. the goal of this test is to evaluate downlink); M = 0. The goal of this test is to evaluate
performance of the candidate solution in terms of bandwidth performance of the candidate solution in terms of bandwidth
fairness between uplink and downlink flow. fairness between uplink and downlink flow.
o Multiple Bi-directional RMCAT Flows with on-off CBR traffic: N = o Multiple Bi-directional RMCAT Flows with on-off CBR traffic: N =
16 (8 uplink and 8 downlink); M = 5(uplink). The goal of this 16 (8 uplink and 8 downlink); M = 5(uplink). The goal of this
test is to evaluate upgrading performance of the candidate test is to evaluate adaptation behavior of the candidate solution
solution in terms of available bandwidth changes caused by the CBR when its available bandwidth changes due to departure of
uplink flow over UDP. CBR over UDP background flows have on time background traffic. The background traffic consists of several
0s-60s, and off time 60s-119s (e.g., M=5) CBR flows transported over UDP, which are ON at times
t=0-60s and are OFF at times t=61-120s.
o Multiple Bi-directional RMCAT Flows with off-on CBR traffic: N = o Multiple Bi-directional RMCAT Flows with off-on CBR traffic: N =
16 (8 uplink and 8 downlink); M = 5(uplink). The goal of this 16 (8 uplink and 8 downlink); M = 5(uplink). The goal of this
test is to evaluate upgrading performance of the candidate test is to evaluate adaptation behavior of the candidate solution
solution in terms of available bandwidth changes caused by the CBR when its available bandwidth changes due to arrival of background
uplink flow over UDP. CBR over UDP background flows have off time traffic. The background traffic consists of several (e.g., M=5)
0s-60s, and on time 60s-119s. parallel CBR flows transported over UDP, which are OFF at times
t=0-60s and are ON at times t=61-120s.
o Multiple RMCAT flows in the presence of background TCP traffic: o Multiple RMCAT flows in the presence of background TCP traffic.
the goal of this test is to evaluate how RMCAT flows compete The goal of this test is to evaluate how RMCAT flows compete
against TCP over a congested Wi-Fi network for a given candidate against TCP over a congested Wi-Fi network for a given candidate
solution. TCP start time: 0s, end time: 119s. [Editor's Note: solution. TCP start time: 0s, end time: 119s. [Editor's Note:
more detailed description will be added in the next version in need to add the number of recommended RMCAT and TCP flows]
terms of directoin/number of RMCAT and TCP flows. ]
o Varying number of RMCAT flows: the goal of this test is to o Varying number of RMCAT flows. The goal of this test is to
evaluate how a candidate RMCAT solution responds to varying evaluate how a candidate RMCAT solution responds to varying
traffic load/demand over a congested Wi-Fi network. [Editor's traffic load/demand over a congested Wi-Fi network. [Editor's
Note: more detailed description will be added in the next version Note: need to specify recommended arrival/departure pattern of
in terms of arrival/departure pattern of the flows.] RMCAT flows]
4.2.4. Expected behavior 4.2.4. Expected behavior
o Multiple downlink RMCAT flows: All RMCAT flows should get fair o Multiple downlink RMCAT flows: each RMCAT flow should get its fair
share of the bandwidth. Overall bandwidth usage should be no less share of the total bottleneck link bandwidth. Overall bandwidth
than same case with TCP flows (using TCP as performance usage should not be significantly lower than that experienced by
benchmark). The delay and loss should be within acceptable range the same number of concurrent downlink TCP flows. In other words,
for real-time multimedia flow. the performance of multiple concurrent TCP flows will be used as a
performance benchmark for this test scenario. The end-to-end
delay and packet loss ratio experienced by each flow should be
within acceptable range for real-time multimedia applications.
o Multiple uplink RMCAT flows: overall bandwidth usage shared by all o Multiple uplink RMCAT flows: overall bandwidth usage shared by all
RMCAT flows should be no less than those shared by the same number RMCAT flows should not be significantly lower than that
of TCP flows (i.e., benchmark performance using TCP flows). experienced by the same number of concurrent uplink TCP flows. In
other words, the performance of multiple concurrent TCP flows will
be used as a performance benchmark for this test scenario.
o Multiple bi-directional RMCAT flows with CBR over UDP traffic: o Multiple bi-directional RMCAT flows with dynamic background
RMCAT flows should adapt to the changes in available bandwidth. traffic carry CBR flows over UDP: RMCAT flows should adapt in a
timely fashion to the resulting changes in available bandwidth.
o Multiple bi-directional RMCAT flows with TCP traffic: overall o Multiple bi-directional RMCAT flows with TCP traffic: overall
bandwidth usage shared by all RMCAT flows should be no less than bandwidth usage shared by all RMCAT flows should not be
those shared by the same number of TCP flows (i.e., benchmark significantly lower than those achieved by the same number of bi-
performance using TCP flows). All downlink RMCAT flows are directional TCP flows. In other words, the performance of
multiple concurrent TCP flows will be used as a performance
benchmark for this test scenario. All downlink RMCAT flows are
expected to obtain similar bandwidth with respect to each other. expected to obtain similar bandwidth with respect to each other.
4.3. Potential Potential Test Cases 4.3. Potential Potential Test Cases
4.3.1. EDCA/WMM usage 4.3.1. EDCA/WMM usage
EDCA/WMM is prioritized QoS with four traffic classes (or Access EDCA/WMM is prioritized QoS with four traffic classes (or Access
Categories) with differing priorities. RMCAT flow should have better Categories) with differing priorities. RMCAT flows should achieve
performance (lower delay, less loss) with EDCA/WMM enabled when better performance (i.e., lower delay, fewer packet losses) with
competing against non-interactive background traffic (e.g., file EDCA/WMM enabled when competing against non-interactive background
transfers). When most of the traffic over Wi-Fi is dominated by traffic (e.g., file transfers). When most of the traffic over Wi-Fi
media, however, turning on WMM may actually degrade performance. is dominated by media, however, turning on WMM may actually degrade
This is a topic worthy of further investigation. performance since all media flows now attempt to access the wireless
transmission medium more aggressively, thereby causing more frequent
collisions and collision-induced losses. This is a topic worthy of
further investigation.
4.3.2. Legacy 802.11b Effects 4.3.2. Legacy 802.11b Effects
When there is 802.11b devices connected to modern 802.11 network, it When there is 802.11b devices connected to modern 802.11 network, it
may affect the performance of the whole network. Additional test may affect the performance of the whole network. Additional test
cases can be added to evaluate the affects of legancy devices on the cases can be added to evaluate the affects of legancy devices on the
performance of RMCAT congestion control algorithm. performance of RMCAT congestion control algorithm.
5. Conclusion 5. Conclusion
skipping to change at page 19, line 30 skipping to change at page 19, line 41
<http://www.3gpp.org/ftp/specs/ <http://www.3gpp.org/ftp/specs/
archive/36_series/36.331/36331-990.zip>. archive/36_series/36.331/36331-990.zip>.
[HO-UMTS-3GPP] [HO-UMTS-3GPP]
TS 25.331, 3GPP., "Radio Resource Control (RRC); Protocol TS 25.331, 3GPP., "Radio Resource Control (RRC); Protocol
specification", December 2011, specification", December 2011,
<http://www.3gpp.org/ftp/specs/ <http://www.3gpp.org/ftp/specs/
archive/25_series/25.331/25331-990.zip>. archive/25_series/25.331/25331-990.zip>.
[I-D.ietf-rmcat-eval-criteria] [I-D.ietf-rmcat-eval-criteria]
Varun, V., Ott, J., and S. Holmer, "Evaluating Congestion Singh, V., Ott, J., and S. Holmer, "Evaluating Congestion
Control for Interactive Real-time Media", draft-ietf- Control for Interactive Real-time Media", draft-ietf-
rmcat-eval-criteria-05 (work in progress), March 2016. rmcat-eval-criteria-06 (work in progress), September 2016.
[NS3WiFi] "Wi-Fi Channel Model in NS3 Simulator", [NS3WiFi] "Wi-Fi Channel Model in NS3 Simulator",
<https://www.nsnam.org/doxygen/ <https://www.nsnam.org/doxygen/
classns3_1_1_yans_wifi_channel.html>. classns3_1_1_yans_wifi_channel.html>.
[QoS-3GPP] [QoS-3GPP]
TS 23.203, 3GPP., "Policy and charging control TS 23.203, 3GPP., "Policy and charging control
architecture", June 2011, <http://www.3gpp.org/ftp/specs/ architecture", June 2011, <http://www.3gpp.org/ftp/specs/
archive/23_series/23.203/23203-990.zip>. archive/23_series/23.203/23203-990.zip>.
skipping to change at page 20, line 13 skipping to change at page 20, line 18
<http://www.rfc-editor.org/info/rfc2119>. <http://www.rfc-editor.org/info/rfc2119>.
9.2. Informative References 9.2. Informative References
[I-D.ietf-rmcat-cc-requirements] [I-D.ietf-rmcat-cc-requirements]
Jesup, R. and Z. Sarker, "Congestion Control Requirements Jesup, R. and Z. Sarker, "Congestion Control Requirements
for Interactive Real-Time Media", draft-ietf-rmcat-cc- for Interactive Real-Time Media", draft-ietf-rmcat-cc-
requirements-09 (work in progress), December 2014. requirements-09 (work in progress), December 2014.
[I-D.ietf-rmcat-eval-test] [I-D.ietf-rmcat-eval-test]
Sarker, Z., Varun, V., Zhu, X., and M. Ramalho, "Test Sarker, Z., Singh, V., Zhu, X., and M. Ramalho, "Test
Cases for Evaluating RMCAT Proposals", draft-ietf-rmcat- Cases for Evaluating RMCAT Proposals", draft-ietf-rmcat-
eval-test-03 (work in progress), March 2016. eval-test-04 (work in progress), October 2016.
[IEEE802.11] [IEEE802.11]
"Standard for Information technology--Telecommunications "Standard for Information technology--Telecommunications
and information exchange between systems Local and and information exchange between systems Local and
metropolitan area networks--Specific requirements Part 11: metropolitan area networks--Specific requirements Part 11:
Wireless LAN Medium Access Control (MAC) and Physical Wireless LAN Medium Access Control (MAC) and Physical
Layer (PHY) Specifications", 2012. Layer (PHY) Specifications", 2012.
[LTE-simulator] [LTE-simulator]
"NS-3, A discrete-Event Network Simulator", "NS-3, A discrete-Event Network Simulator",
 End of changes. 41 change blocks. 
122 lines changed or deleted 132 lines changed or added

This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/