draft-ietf-mptcp-experience-06.txt   draft-ietf-mptcp-experience-07.txt 
MPTCP Working Group O. Bonaventure MPTCP Working Group O. Bonaventure
Internet-Draft UCLouvain Internet-Draft UCLouvain
Intended status: Informational C. Paasch Intended status: Informational C. Paasch
Expires: March 2, 2017 Apple, Inc. Expires: April 30, 2017 Apple, Inc.
G. Detal G. Detal
Tessares Tessares
August 29, 2016 October 27, 2016
Use Cases and Operational Experience with Multipath TCP Use Cases and Operational Experience with Multipath TCP
draft-ietf-mptcp-experience-06 draft-ietf-mptcp-experience-07
Abstract Abstract
This document discusses both use cases and operational experience This document discusses both use cases and operational experience
with Multipath TCP in real world networks. It lists several with Multipath TCP in real networks. It lists several prominent use
prominent use cases for which Multipath TCP has been considered and cases where Multipath TCP has been considered and is being used. It
is being used. It also gives insight to some heuristics and also gives insight to some heuristics and decisions that have helped
decisions that have helped to realize these use cases. to realize these use cases and suggests possible improvements.
Status of this Memo Status of this Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on March 2, 2017. This Internet-Draft will expire on April 30, 2017.
Copyright Notice Copyright Notice
Copyright (c) 2016 IETF Trust and the persons identified as the Copyright (c) 2016 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 27 skipping to change at page 2, line 27
3.2. Congestion control . . . . . . . . . . . . . . . . . . . . 13 3.2. Congestion control . . . . . . . . . . . . . . . . . . . . 13
3.3. Subflow management . . . . . . . . . . . . . . . . . . . . 13 3.3. Subflow management . . . . . . . . . . . . . . . . . . . . 13
3.4. Implemented subflow managers . . . . . . . . . . . . . . . 14 3.4. Implemented subflow managers . . . . . . . . . . . . . . . 14
3.5. Subflow destination port . . . . . . . . . . . . . . . . . 16 3.5. Subflow destination port . . . . . . . . . . . . . . . . . 16
3.6. Closing subflows . . . . . . . . . . . . . . . . . . . . . 17 3.6. Closing subflows . . . . . . . . . . . . . . . . . . . . . 17
3.7. Packet schedulers . . . . . . . . . . . . . . . . . . . . 18 3.7. Packet schedulers . . . . . . . . . . . . . . . . . . . . 18
3.8. Segment size selection . . . . . . . . . . . . . . . . . . 19 3.8. Segment size selection . . . . . . . . . . . . . . . . . . 19
3.9. Interactions with the Domain Name System . . . . . . . . . 19 3.9. Interactions with the Domain Name System . . . . . . . . . 19
3.10. Captive portals . . . . . . . . . . . . . . . . . . . . . 20 3.10. Captive portals . . . . . . . . . . . . . . . . . . . . . 20
3.11. Stateless webservers . . . . . . . . . . . . . . . . . . . 21 3.11. Stateless webservers . . . . . . . . . . . . . . . . . . . 21
3.12. Loadbalanced serverfarms . . . . . . . . . . . . . . . . . 22 3.12. Loadbalanced server farms . . . . . . . . . . . . . . . . 22
4. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23 4. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23
5. Security Considerations . . . . . . . . . . . . . . . . . . . 24 5. Security Considerations . . . . . . . . . . . . . . . . . . . 24
6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 25 6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 26
7. Informative References . . . . . . . . . . . . . . . . . . . . 26 7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Appendix A. Changelog . . . . . . . . . . . . . . . . . . . . . . 33 7.1. Normative References . . . . . . . . . . . . . . . . . . . 27
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 35 7.2. Informative References . . . . . . . . . . . . . . . . . . 27
Appendix A. Changelog . . . . . . . . . . . . . . . . . . . . . . 34
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 36
1. Introduction 1. Introduction
Multipath TCP was standardized in [RFC6824] and five independent Multipath TCP was specified in [RFC6824] and five independent
implementations have been developed. As of September 2015, Multipath implementations have been developed. As of November 2016, Multipath
TCP has been or is being implemented on the following platforms: TCP has been or is being implemented on the following platforms:
o Linux kernel [MultipathTCP-Linux] o Linux kernel [MultipathTCP-Linux]
o Apple iOS and MacOS [Apple-MPTCP] o Apple iOS and macOS [Apple-MPTCP]
o Citrix load balancers o Citrix load balancers
o FreeBSD [FreeBSD-MPTCP] o FreeBSD [FreeBSD-MPTCP]
o Oracle o Oracle Solaris
The first three implementations are known to interoperate. The last The first three implementations are known to interoperate. Three of
two are currently being tested and improved against the Linux these implementations are open-source (Linux kernel, FreeBSD and
implementation. Three of these implementations are open-source. Apple's iOS and macOS). Apple's implementation is widely deployed.
Apple's implementation is widely deployed.
Since the publication of [RFC6824], experience has been gathered by Since the publication of [RFC6824] as an experimental RFC, experience
various network researchers and users about the operational issues has been gathered by various network researchers and users about the
that arise when Multipath TCP is used in today's Internet. operational issues that arise when Multipath TCP is used in today's
Internet.
When the MPTCP working group was created, several use cases for When the MPTCP working group was created, several use cases for
Multipath TCP were identified [RFC6182]. Since then, other use cases Multipath TCP were identified [RFC6182]. Since then, other use cases
have been proposed and some have been tested and even deployed. We have been proposed and some have been tested and even deployed. We
describe these use cases in Section 2. describe these use cases in Section 2.
Section 3 focuses on the operational experience with Multipath TCP. Section 3 focuses on the operational experience with Multipath TCP.
Most of this experience comes from the utilisation of the Multipath Most of this experience comes from the utilization of the Multipath
TCP implementation in the Linux kernel [MultipathTCP-Linux]. This TCP implementation in the Linux kernel [MultipathTCP-Linux]. This
open-source implementation has been downloaded and is used by open-source implementation has been downloaded and is used by
thousands of users all over the world. Many of these users have thousands of users all over the world. Many of these users have
provided direct or indirect feedback by writing documents (scientific provided direct or indirect feedback by writing documents (scientific
articles or blog messages) or posting to the mptcp-dev mailing list articles or blog messages) or posting to the mptcp-dev mailing list
(see https://listes-2.sipr.ucl.ac.be/sympa/arc/mptcp-dev ). This (see https://listes-2.sipr.ucl.ac.be/sympa/arc/mptcp-dev ). This
Multipath TCP implementation is actively maintained and continuously Multipath TCP implementation is actively maintained and continuously
improved. It is used on various types of hosts, ranging from improved. It is used on various types of hosts, ranging from
smartphones or embedded routers to high-end servers. smartphones or embedded routers to high-end servers.
The Multipath TCP implementation in the Linux kernel is not, by far, The Multipath TCP implementation in the Linux kernel is not, by far,
the most widespread deployment of Multipath TCP. Since September the most widespread deployment of Multipath TCP. Since September
2013, Multipath TCP is also supported on smartphones and tablets 2013, Multipath TCP is also supported on smartphones and tablets
running iOS7 [IOS7]. There are likely hundreds of millions of since iOS7 [IOS7]. There are likely hundreds of millions of
Multipath TCP enabled devices. However, this particular Multipath Multipath TCP enabled devices. This Multipath TCP implementation is
TCP implementation is currently only used to support a single currently only used to support the Siri voice recognition/control
application. Unfortunately, there is no public information about the application. Some lessons learned from this deployment are described
lessons learned from this large scale deployment. in [IETFJ].
Section 3 is organized as follows. Supporting the middleboxes was Section 3 is organized as follows. Supporting the middleboxes was
one of the difficult issues in designing the Multipath TCP protocol. one of the difficult issues in designing the Multipath TCP protocol.
We explain in Section 3.1 which types of middleboxes the Linux Kernel We explain in Section 3.1 which types of middleboxes the Linux Kernel
implementation of Multipath TCP supports and how it reacts upon implementation of Multipath TCP supports and how it reacts upon
encountering these. Section 3.2 summarises the MPTCP specific encountering these. Section 3.2 summarizes the MPTCP specific
congestion controls that have been implemented. Section 3.3 to congestion controls that have been implemented. Section 3.3 to
Section 3.7 discuss heuristics and issues with respect to subflow Section 3.7 discuss heuristics and issues with respect to subflow
management as well as the scheduling across the subflows. management as well as the scheduling across the subflows.
Section 3.8 explains some problems that occurred with subflows having Section 3.8 explains some problems that occurred with subflows having
different Maximum Segment Size (MSS) values. Section 3.9 presents different Maximum Segment Size (MSS) values. Section 3.9 presents
issues with respect to content delivery networks and suggests a issues with respect to content delivery networks and suggests a
solution to this issue. Finally, Section 3.10 documents an issue solution to this issue. Finally, Section 3.10 documents an issue
with captive portals where MPTCP will behave suboptimally. with captive portals where MPTCP will behave sub optimally.
2. Use cases 2. Use cases
Multipath TCP has been tested in several use cases. There is already Multipath TCP has been tested in several use cases. There is already
an abundant scientific literature on Multipath TCP [MPTCPBIB]. an abundant scientific literature on Multipath TCP [MPTCPBIB].
Several of the papers published in the scientific literature have Several of the papers published in the scientific literature have
identified possible improvements that are worth being discussed here. identified possible improvements that are worth being discussed here.
2.1. Datacenters 2.1. Datacenters
skipping to change at page 5, line 28 skipping to change at page 5, line 28
utilization of Equal Cost Multipath (ECMP) and other load balancing utilization of Equal Cost Multipath (ECMP) and other load balancing
techniques inside the datacenter. Most of the deployed load techniques inside the datacenter. Most of the deployed load
balancing techniques in datacenters rely on hashes computed over the balancing techniques in datacenters rely on hashes computed over the
five tuple. Thus all packets from the same TCP connection follow the five tuple. Thus all packets from the same TCP connection follow the
same path and so are not reordered. The results in [HotNets] same path and so are not reordered. The results in [HotNets]
demonstrate by simulations that Multipath TCP can achieve a better demonstrate by simulations that Multipath TCP can achieve a better
utilization of the available network by using multiple subflows for utilization of the available network by using multiple subflows for
each Multipath TCP session. Although [RFC6182] assumes that at least each Multipath TCP session. Although [RFC6182] assumes that at least
one of the communicating hosts has several IP addresses, [HotNets] one of the communicating hosts has several IP addresses, [HotNets]
demonstrates that Multipath TCP is beneficial when both hosts are demonstrates that Multipath TCP is beneficial when both hosts are
single-homed. This idea is analysed in more details in [SIGCOMM11] single-homed. This idea is analyzed in more details in [SIGCOMM11]
where the Multipath TCP implementation in the Linux kernel is where the Multipath TCP implementation in the Linux kernel is
modified to be able to use several subflows from the same IP address. modified to be able to use several subflows from the same IP address.
Measurements in a public datacenter show the quantitative benefits of Measurements in a public datacenter show the quantitative benefits of
Multipath TCP [SIGCOMM11] in this environment. Multipath TCP [SIGCOMM11] in this environment.
Although ECMP is widely used inside datacenters, this is not the only Although ECMP is widely used inside datacenters, this is not the only
environment where there are different paths between a pair of hosts. environment where there are different paths between a pair of hosts.
ECMP and other load balancing techniques such as Link Aggregation ECMP and other load balancing techniques such as Link Aggregation
Groups (LAG) are widely used in today's networks and having multiple Groups (LAG) are widely used in today's networks and having multiple
paths between a pair of single-homed hosts is becoming the norm paths between a pair of single-homed hosts is becoming the norm
skipping to change at page 7, line 23 skipping to change at page 7, line 23
The single-path mode is slightly different. This mode benefits from The single-path mode is slightly different. This mode benefits from
the break-before-make capability of Multipath TCP. When an MPTCP the break-before-make capability of Multipath TCP. When an MPTCP
session is established, a subflow is created over the WiFi interface. session is established, a subflow is created over the WiFi interface.
No packet is sent over the cellular interface as long as the WiFi No packet is sent over the cellular interface as long as the WiFi
interface remains up [Cellnet12]. This implies that the cellular interface remains up [Cellnet12]. This implies that the cellular
interface can remain idle and battery capacity is preserved. When interface can remain idle and battery capacity is preserved. When
the WiFi interface fails, a new subflow is established over the the WiFi interface fails, a new subflow is established over the
cellular interface in order to preserve the established Multipath TCP cellular interface in order to preserve the established Multipath TCP
sessions. Compared to the backup mode described earlier, sessions. Compared to the backup mode described earlier,
measurements reported in [Cellnet12] indicate that this mode of measurements reported in [Cellnet12] indicate that this mode of
operation is characterised by a throughput drop while the cellular operation is characterized by a throughput drop while the cellular
interface is brought up and the subflows are reestablished. interface is brought up and the subflows are reestablished.
From a protocol viewpoint, [Cellnet12] discusses the problem posed by From a protocol viewpoint, [Cellnet12] discusses the problem posed by
the unreliability of the REMOVE_ADDR option and proposes a small the unreliability of the REMOVE_ADDR option and proposes a small
protocol extension to allow hosts to reliably exchange this option. protocol extension to allow hosts to reliably exchange this option.
It would be useful to analyze packet traces to understand whether the It would be useful to analyze packet traces to understand whether the
unreliability of the REMOVE_ADDR option poses an operational problem unreliability of the REMOVE_ADDR option poses an operational problem
in real deployments. in real deployments.
Another study of the performance of Multipath TCP in wireless Another study of the performance of Multipath TCP in wireless
skipping to change at page 7, line 48 skipping to change at page 7, line 48
congestion control algorithms (LIA, OLIA and Reno - see Section 3.2), congestion control algorithms (LIA, OLIA and Reno - see Section 3.2),
there is no significant performance difference for file sizes smaller there is no significant performance difference for file sizes smaller
than 4MB. than 4MB.
A different study of the performance of Multipath TCP with two A different study of the performance of Multipath TCP with two
wireless networks is presented in [INFOCOM14]. In this study the two wireless networks is presented in [INFOCOM14]. In this study the two
networks had different qualities : a good network and a lossy networks had different qualities : a good network and a lossy
network. When using two paths with different packet loss ratios, the network. When using two paths with different packet loss ratios, the
Multipath TCP congestion control scheme moves traffic away from the Multipath TCP congestion control scheme moves traffic away from the
lossy link that is considered to be congested. However, [INFOCOM14] lossy link that is considered to be congested. However, [INFOCOM14]
documents an interesting scenario that is summarised in Figure 1. documents an interesting scenario that is summarized hereafter.
client ----------- path1 -------- server client ----------- path1 -------- server
| | | |
+--------------- path2 ------------+ +--------------- path2 ------------+
Figure 1: Simple network topology Figure 1: Simple network topology
Initially, the two paths have the same quality and Multipath TCP Initially, the two paths in Figure 1 have the same quality and
distributes the load over both of them. During the transfer, the Multipath TCP distributes the load over both of them. During the
second path becomes lossy, e.g. because the client moves. Multipath transfer, the path2 becomes lossy, e.g. because the client moves.
TCP detects the packet losses and they are retransmitted over the Multipath TCP detects the packet losses and they are retransmitted
first path. This enables the data transfer to continue over the over path1. This enables the data transfer to continue over this
first path. However, the subflow over the second path is still up path. However, the subflow over path2 is still up and transmits one
and transmits one packet from time to time. Although the N packets packet from time to time. Although the N packets have been
have been acknowledged over the first subflow (at the MPTCP level), acknowledged over the first subflow (at the MPTCP level), they have
they have not been acknowledged at the TCP level over the second not been acknowledged at the TCP level over the second subflow. To
subflow. To preserve the continuity of the sequence numbers over the preserve the continuity of the sequence numbers over the second
second subflow, TCP will continue to retransmit these segments until subflow, TCP will continue to retransmit these segments until either
either they are acknowledged or the maximum number of retransmissions they are acknowledged or the maximum number of retransmissions is
is reached. This behavior is clearly inefficient and may lead to reached. This behavior is clearly inefficient and may lead to
blocking since the second subflow will consume window space to be blocking since the second subflow will consume window space to be
able to retransmit these packets. [INFOCOM14] proposes a new able to retransmit these packets. [INFOCOM14] proposes a new
Multipath TCP option to solve this problem. In practice, a new TCP Multipath TCP option to solve this problem. In practice, a new TCP
option is probably not required. When the client detects that the option is probably not required. When the client detects that the
data transmitted over the second subflow has been acknowledged over data transmitted over the second subflow has been acknowledged over
the first subflow, it could decide to terminate the second subflow by the first subflow, it could decide to terminate the second subflow by
sending a RST segment. If the interface associated to this subflow sending a RST segment. If the interface associated to this subflow
is still up, a new subflow could be immediately reestablished. It is still up, a new subflow could be immediately reestablished. It
would then be immediately usable to send new data and would not be would then be immediately usable to send new data and would not be
forced to first retransmit the previously transmitted data. As of forced to first retransmit the previously transmitted data. As of
this writing, this dynamic management of the subflows is not yet this writing, this dynamic management of the subflows is not yet
implemented in the Multipath TCP implementation in the Linux kernel. implemented in the Multipath TCP implementation in the Linux kernel.
Some studies have started to analyse the performance of Multipath TCP Some studies have started to analyze the performance of Multipath TCP
on smartphones with real applications. In contrast with the bulk on smartphones with real applications. In contrast with the bulk
transfers that are used by many publications, real applications do transfers that are used by many publications, many deployed
not exchange huge amounts of data and establish a large number of applications do not exchange huge amounts of data and mainly use
small connections. [COMMAG2016] proposes a software testing small connections. [COMMAG2016] proposes a software testing
framework that allows to automate Android applications to study their framework that allows to automate Android applications to study their
interactions with Multipath TCP. [PAM2016] analyses a one-month interactions with Multipath TCP. [PAM2016] analyses a one-month
packet trace of all the packets exchanged by a dozen of smartphones packet trace of all the packets exchanged by a dozen of smartphones
used by regular users. This analysis reveals that short connections used by regular users. This analysis reveals that short connections
are important on smartphones and that the main benefit of using are important on smartphones and that the main benefit of using
Multipath TCP on smartphones is the ability to perform seamless Multipath TCP on smartphones is the ability to perform seamless
handovers between different wireless networks. Long connections handovers between different wireless networks. Long connections
benefit from these handovers. benefit from these handovers.
skipping to change at page 9, line 29 skipping to change at page 9, line 29
proxy that can convert any Multipath TCP connection into a regular proxy that can convert any Multipath TCP connection into a regular
TCP connection. Multipath TCP-specific proxies have been proposed TCP connection. Multipath TCP-specific proxies have been proposed
[HotMiddlebox13b] [HAMPEL]. [HotMiddlebox13b] [HAMPEL].
Another possibility leverages the SOCKS protocol [RFC1928]. SOCKS is Another possibility leverages the SOCKS protocol [RFC1928]. SOCKS is
often used in enterprise networks to allow clients to reach external often used in enterprise networks to allow clients to reach external
servers. For this, the client opens a TCP connection to the SOCKS servers. For this, the client opens a TCP connection to the SOCKS
server that relays it to the final destination. If both the client server that relays it to the final destination. If both the client
and the SOCKS server use Multipath TCP, but not the final and the SOCKS server use Multipath TCP, but not the final
destination, then Multipath TCP can still be used on the path between destination, then Multipath TCP can still be used on the path between
the client and the SOCKS server. At IETF'93, Korea Telecom announced the clients and the SOCKS server. At IETF'93, Korea Telecom
that they have deployed in June 2015 a commercial service that uses announced that they have deployed in June 2015 a commercial service
Multipath TCP on smartphones. These smartphones access regular TCP that uses Multipath TCP on smartphones. These smartphones access
servers through a SOCKS proxy. This enables them to achieve regular TCP servers through a SOCKS proxy. This enables them to
throughputs of up to 850 Mbps [KT]. achieve throughputs of up to 850 Mbps [KT].
Measurements performed with Android smartphones [Mobicom15] show that Measurements performed with Android smartphones [Mobicom15] show that
popular applications work correctly through a SOCKS proxy and popular applications work correctly through a SOCKS proxy and
Multipath TCP enabled smartphones. Thanks to Multipath TCP, long- Multipath TCP enabled smartphones. Thanks to Multipath TCP, long-
lived connections can be spread over the two available interfaces. lived connections can be spread over the two available interfaces.
However, for short-lived connections, most of the data is sent over However, for short-lived connections, most of the data is sent over
the initial subflow that is created over the interface corresponding the initial subflow that is created over the interface corresponding
to the default route and the second subflow is almost not used to the default route and the second subflow is almost not used
[PAM2016]. [PAM2016].
A second use case is when Multipath TCP is used by middleboxes, A second use case is when Multipath TCP is used by middleboxes,
typically inside access networks. Various network operators are typically inside access networks. Various network operators are
discussing and evaluating solutions for hybrid access networks discussing and evaluating solutions for hybrid access networks
[BBF-WT348]. Such networks arise when a network operator controls [TR-348]. Such networks arise when a network operator controls two
two different access network technologies, e.g. wired and cellular, different access network technologies, e.g. wired and cellular, and
and wants to combine them to improve the bandwidth offered to the wants to combine them to improve the bandwidth offered to the
endusers [I-D.lhwxz-hybrid-access-network-architecture]. Several endusers [I-D.lhwxz-hybrid-access-network-architecture]. Several
solutions are currently investigated for such networks [BBF-WT348]. solutions are currently investigated for such networks [TR-348].
Figure 2 shows the organisation of such a network. When a client Figure 2 shows the organization of such a network. When a client
creates a normal TCP connection, it is intercepted by the Hybrid CPE creates a normal TCP connection, it is intercepted by the Hybrid CPE
(HPCE) that converts it in a Multipath TCP connection so that it can (HPCE) that converts it in a Multipath TCP connection so that it can
use the available access networks (DSL and LTE in the example). The use the available access networks (DSL and LTE in the example). The
Hybrid Access Gateway (HAG) does the opposite to ensure that the Hybrid Access Gateway (HAG) does the opposite to ensure that the
regular server sees a normal TCP connection. Some of the solutions regular server sees a normal TCP connection. Some of the solutions
that are currently discussed for hybrid networks use Multipath TCP on currently discussed for hybrid networks use Multipath TCP on the HCPE
the HCPE and the HAG. Other solutions rely on tunnels between the and the HAG. Other solutions rely on tunnels between the HCPE and
HCPE and the HAG [I-D.lhwxz-gre-notifications-hybrid-access]. the HAG [I-D.lhwxz-gre-notifications-hybrid-access].
client --- HCPE ------ DSL ------- HAG --- internet --- server client --- HCPE ------ DSL ------- HAG --- internet --- server
| | | |
+------- LTE -----------+ +------- LTE -----------+
Figure 2: Hybrid Access Network Figure 2: Hybrid Access Network
3. Operational Experience 3. Operational Experience
3.1. Middlebox interference 3.1. Middlebox interference
skipping to change at page 11, line 40 skipping to change at page 11, line 40
These middlebox interferences have all been included in the MBtest These middlebox interferences have all been included in the MBtest
suite [MBTest]. This test suite is used in [HotMiddlebox13] to suite [MBTest]. This test suite is used in [HotMiddlebox13] to
verify the reaction of the Multipath TCP implementation in the Linux verify the reaction of the Multipath TCP implementation in the Linux
kernel [MultipathTCP-Linux] when faced with middlebox interference. kernel [MultipathTCP-Linux] when faced with middlebox interference.
The test environment used for this evaluation is a dual-homed client The test environment used for this evaluation is a dual-homed client
connected to a single-homed server. The middlebox behavior can be connected to a single-homed server. The middlebox behavior can be
activated on any of the paths. The main results of this analysis are activated on any of the paths. The main results of this analysis are
: :
o the v0.87 Multipath TCP implementation in the Linux kernel is not o the Multipath TCP implementation in the Linux kernel, is not
affected by a middlebox that performs NAT or modifies TCP sequence affected by a middlebox that performs NAT or modifies TCP sequence
numbers numbers
o when a middlebox removes the MP_CAPABLE option from the initial o when a middlebox removes the MP_CAPABLE option from the initial
SYN segment, the v0.87 Multipath TCP implementation in the Linux SYN segment, the Multipath TCP implementation in the Linux kernel
kernel falls back correctly to regular TCP falls back correctly to regular TCP
o when a middlebox removes the DSS option from all data segments, o when a middlebox removes the DSS option from all data segments,
the v0.87 Multipath TCP implementation in the Linux kernel falls the Multipath TCP implementation in the Linux kernel falls back
back correctly to regular TCP correctly to regular TCP
o when a middlebox performs segment coalescing, the v0.87 Multipath o when a middlebox performs segment coalescing, the Multipath TCP
TCP implementation in the Linux kernel is still able to accurately implementation in the Linux kernel is still able to accurately
extract the data corresponding to the indicated mapping extract the data corresponding to the indicated mapping
o when a middlebox performs segment splitting, the v0.87 Multipath o when a middlebox performs segment splitting, the Multipath TCP
TCP implementation in the Linux kernel correctly reassembles the implementation in the Linux kernel correctly reassembles the data
data corresponding to the indicated mapping. [HotMiddlebox13] corresponding to the indicated mapping. [HotMiddlebox13] shows on
shows on figure 4 in section 3.3 a corner case with segment figure 4 in section 3.3 a corner case with segment splitting that
splitting that may lead to a desynchronisation between the two may lead to a desynchronization between the two hosts.
hosts.
The interactions between Multipath TCP and real deployed middleboxes The interactions between Multipath TCP and real deployed middleboxes
is also analyzed in [HotMiddlebox13] and a particular scenario with is also analyzed in [HotMiddlebox13] and a particular scenario with
the FTP application level gateway running on a NAT is described. the FTP application level gateway running on a NAT is described.
Middlebox interference can also be detected by analysing packet Middlebox interference can also be detected by analyzing packet
traces on Multipath TCP enabled servers. A closer look at the traces on Multipath TCP enabled servers. A closer look at the
packets received on the multipath-tcp.org server [TMA2015] shows that packets received on the multipath-tcp.org server [TMA2015] shows that
among the 184,000 Multipath TCP connections, only 125 of them were among the 184,000 Multipath TCP connections, only 125 of them were
falling back to regular TCP. These connections originated from 28 falling back to regular TCP. These connections originated from 28
different client IP addresses. These include 91 HTTP connections and different client IP addresses. These include 91 HTTP connections and
34 FTP connections. The FTP interference is expected and due to 34 FTP connections. The FTP interference is expected since
Application Level Gateways running home routers. The HTTP Application Level Gateways used for FTP modify the TCP payload and
interference appeared only on the direction from server to client and the DSS Checksum detects these modifications. The HTTP interference
could have been caused by transparent proxies deployed in cellular or appeared only on the direction from server to client and could have
enterprise networks. A longer trace is discussed in [COMCOM2016] and been caused by transparent proxies deployed in cellular or enterprise
similar conclusions about the middlebox interference are provided. networks. A longer trace is discussed in [COMCOM2016] and similar
conclusions about the middlebox interference are provided.
From an operational viewpoint, knowing that Multipath TCP can cope From an operational viewpoint, knowing that Multipath TCP can cope
with various types of middlebox interference is important. However, with various types of middlebox interference is important. However,
there are situations where the network operators need to gather there are situations where the network operators need to gather
information about where a particular middlebox interference occurs. information about where a particular middlebox interference occurs.
The tracebox software [tracebox] described in [IMC13a] is an The tracebox software [tracebox] described in [IMC13a] is an
extension of the popular traceroute software that enables network extension of the popular traceroute software that enables network
operators to check at which hop a particular field of the TCP header operators to check at which hop a particular field of the TCP header
(including options) is modified. It has been used by several network (including options) is modified. It has been used by several network
operators to debug various middlebox interference problems. operators to debug various middlebox interference problems.
skipping to change at page 13, line 8 skipping to change at page 13, line 8
Users of the Multipath TCP implementation have reported some Users of the Multipath TCP implementation have reported some
experience with middlebox interference. The strangest scenario has experience with middlebox interference. The strangest scenario has
been a middlebox that accepts the Multipath TCP options in the SYN been a middlebox that accepts the Multipath TCP options in the SYN
segment but later replaces Multipath TCP options with a TCP EOL segment but later replaces Multipath TCP options with a TCP EOL
option [StrangeMbox]. This causes Multipath TCP to perform a option [StrangeMbox]. This causes Multipath TCP to perform a
fallback to regular TCP without any impact on the application. fallback to regular TCP without any impact on the application.
3.2. Congestion control 3.2. Congestion control
Congestion control has been an important challenge for Multipath TCP. Congestion control has been an important challenge for Multipath TCP.
The standardised congestion control scheme for Multipath TCP is The congestion control scheme specified for Multipath TCP is defined
defined in [RFC6356]. A detailed description of this algorithm is in [RFC6356]. A detailed description of this algorithm is provided
provided in [NSDI11]. This congestion control scheme has been in [NSDI11]. This congestion control scheme has been implemented in
implemented in the Linux implementation of Multipath TCP. Linux uses the Linux implementation of Multipath TCP. Linux uses a modular
a modular architecture to support various congestion control schemes. architecture to support various congestion control schemes. This
This architecture is applicable for both regular TCP and Multipath architecture is applicable for both regular TCP and Multipath TCP.
TCP. While the coupled congestion control scheme defined in While the coupled congestion control scheme defined in [RFC6356] is
[RFC6356] is the default congestion control scheme in the Linux the default congestion control scheme in the Linux implementation,
implementation, other congestion control schemes have been added. other congestion control schemes have been added. The second
The second congestion control scheme is OLIA [CONEXT12]. This congestion control scheme is OLIA [CONEXT12]. This congestion
congestion control scheme is also an adaptation of the NewReno single control scheme is also an adaptation of the NewReno single path
path congestion control scheme to support multiple paths. congestion control scheme to support multiple paths. Simulations and
Simulations and measurements have shown that it provides some measurements have shown that it provides some performance benefits
performance benefits compared to the the default congestion control compared to the default congestion control scheme [CONEXT12].
scheme [CONEXT12]. Measurements over a wide range of parameters Measurements over a wide range of parameters reported in [CONEXT13]
reported in [CONEXT13] also indicate some benefits with the OLIA also indicate some benefits with the OLIA congestion control scheme.
congestion control scheme. Recently, a delay-based congestion Recently, a delay-based congestion control scheme has been ported to
control scheme has been ported to the Multipath TCP implementation in the Multipath TCP implementation in the Linux kernel. This
the Linux kernel. This congestion control scheme has been evaluated congestion control scheme has been evaluated by using simulations in
by using simulations in [ICNP12]. The fourth congestion control [ICNP12] and measurements in [PaaschPhD]. The fourth congestion
scheme that has been included in the Linux implementation of control scheme that has been included in the Linux implementation of
Multipath TCP is the BALIA scheme that provides a better balance Multipath TCP is the BALIA scheme that provides a better balance
between TCP friendliness, responsiveness, and window oscillation between TCP friendliness, responsiveness, and window oscillation
[BALIA]. [BALIA].
These different congestion control schemes have been compared in These different congestion control schemes have been compared in
several articles. [CONEXT13] and [PaaschPhD] compare these several articles. [CONEXT13] and [PaaschPhD] compare these
algorithms in an emulated environment. The evaluation showed that algorithms in an emulated environment. The evaluation showed that
the delay-based congestion control scheme is less able to efficiently the delay-based congestion control scheme is less able to efficiently
use the available links than the three other schemes. Reports from use the available links than the three other schemes.
some users indicate that they seem to favor OLIA.
3.3. Subflow management 3.3. Subflow management
The multipath capability of Multipath TCP comes from the utilisation The multipath capability of Multipath TCP comes from the utilization
of one subflow per path. The Multipath TCP architecture [RFC6182] of one subflow per path. The Multipath TCP architecture [RFC6182]
and the protocol specification [RFC6824] define the basic usage of and the protocol specification [RFC6824] define the basic usage of
the subflows and the protocol mechanisms that are required to create the subflows and the protocol mechanisms that are required to create
and terminate them. However, there are no guidelines on how subflows and terminate them. However, there are no guidelines on how subflows
are used during the lifetime of a Multipath TCP session. Most of the are used during the lifetime of a Multipath TCP session. Most of the
published experiments with Multipath TCP have been performed in published experiments with Multipath TCP have been performed in
controlled environments. Still, based on the experience running them controlled environments. Still, based on the experience running them
and discussions on the mptcp-dev mailing list, interesting lessons and discussions on the mptcp-dev mailing list, interesting lessons
have been learned about the management of these subflows. have been learned about the management of these subflows.
skipping to change at page 14, line 41 skipping to change at page 14, line 39
the corresponding interface), it is not always reestablished. There the corresponding interface), it is not always reestablished. There
is ongoing work to enhance the full-mesh path manager to deal with is ongoing work to enhance the full-mesh path manager to deal with
such events. such events.
When the server is multihomed, using the full-mesh subflow manager When the server is multihomed, using the full-mesh subflow manager
may lead to a large number of subflows being established. For may lead to a large number of subflows being established. For
example, consider a dual-homed client connected to a server with example, consider a dual-homed client connected to a server with
three interfaces. In this case, even if the subflows are only three interfaces. In this case, even if the subflows are only
created by the client, 6 subflows will be established. This may be created by the client, 6 subflows will be established. This may be
excessive in some environments, in particular when the client and/or excessive in some environments, in particular when the client and/or
the server have a large number of interfaces. A recent draft has the server have a large number of interfaces. Implementations should
proposed a Multipath TCP option to negotiate the maximum number of limit the number of subflows that are used.
subflows. However, it should be noted that there have been reports
on the mptcp-dev mailing indicating that users rely on Multipath TCP
to aggregate more than four different interfaces. Thus, there is a
need for supporting many interfaces efficiently.
Creating subflows between multihomed clients and servers may Creating subflows between multihomed clients and servers may
sometimes lead to operational issues as observed by discussions on sometimes lead to operational issues as observed by discussions on
the mptcp-dev mailing list. In some cases the network operators the mptcp-dev mailing list. In some cases the network operators
would like to have a better control on how the subflows are created would like to have a better control on how the subflows are created
by Multipath TCP [I-D.boucadair-mptcp-max-subflow]. This might by Multipath TCP [I-D.boucadair-mptcp-max-subflow]. This might
require the definition of policy rules to control the operation of require the definition of policy rules to control the operation of
the subflow manager. The two scenarios below illustrate some of the subflow manager. The two scenarios below illustrate some of
these requirements. these requirements.
skipping to change at page 16, line 40 skipping to change at page 16, line 39
subflows) on the official port number. Using the same destination subflows) on the official port number. Using the same destination
port for all subflows is also useful for operators that rely on the port for all subflows is also useful for operators that rely on the
port numbers to track application usage in their network. port numbers to track application usage in their network.
There have been suggestions from Multipath TCP users to modify the There have been suggestions from Multipath TCP users to modify the
implementation to allow the client to use different destination ports implementation to allow the client to use different destination ports
to reach the server. This suggestion seems mainly motivated by to reach the server. This suggestion seems mainly motivated by
traffic shaping middleboxes that are used in some wireless networks. traffic shaping middleboxes that are used in some wireless networks.
In networks where different shaping rates are associated to different In networks where different shaping rates are associated to different
destination port numbers, this could allow Multipath TCP to reach a destination port numbers, this could allow Multipath TCP to reach a
higher performance. As of this writing, we are not aware of any higher performance. This behavior is valid according to the
implementation of this kind of tweaking. Multipath TCP specification [RFC6824]. An application could used an
enhanced socket API [SOCKET] to behave in this way.
However, from an implementation point-of-view supporting different However, from an implementation point-of-view supporting different
destination ports for the same Multipath TCP connection can cause destination ports for the same Multipath TCP connection can cause
some issues. A legacy implementation of a TCP stack creates a some issues. A legacy implementation of a TCP stack creates a
listening socket to react upon incoming SYN segments. The listening listening socket to react upon incoming SYN segments. The listening
socket is handling the SYN segments that are sent on a specific port socket is handling the SYN segments that are sent on a specific port
number. Demultiplexing incoming segments can thus be done solely by number. Demultiplexing incoming segments can thus be done solely by
looking at the IP addresses and the port numbers. With Multipath TCP looking at the IP addresses and the port numbers. With Multipath TCP
however, incoming SYN segments may have an MP_JOIN option with a however, incoming SYN segments may have an MP_JOIN option with a
different destination port. This means, that all incoming segments different destination port. This means, that all incoming segments
skipping to change at page 19, line 20 skipping to change at page 19, line 20
(Maximum Segment Size) of data per buffer. As the MSS can differ (Maximum Segment Size) of data per buffer. As the MSS can differ
amongst subflows, an MPTCP implementation must select carefully the amongst subflows, an MPTCP implementation must select carefully the
MSS used to generate application data. The Linux kernel MSS used to generate application data. The Linux kernel
implementation had various ways of selecting the MSS: minimum or implementation had various ways of selecting the MSS: minimum or
maximum amongst the different subflows. However, these heuristics of maximum amongst the different subflows. However, these heuristics of
MSS selection can cause significant performance issues in some MSS selection can cause significant performance issues in some
environment. Consider the following example. An MPTCP connection environment. Consider the following example. An MPTCP connection
has two established subflows that respectively use a MSS of 1420 and has two established subflows that respectively use a MSS of 1420 and
1428 bytes. If MPTCP selects the maximum, then the application will 1428 bytes. If MPTCP selects the maximum, then the application will
generate segments of 1428 bytes of data. An MPTCP implementation generate segments of 1428 bytes of data. An MPTCP implementation
will have to split the segment in two (a 1420-byte and 8-byte will have to split the segment in two ( 1420-byte and 8-byte)
segments) when pushing on the subflow with the smallest MSS. The segments when pushing on the subflow with the smallest MSS. The
latter segment will introduce a large overhead as for a single data latter segment will introduce a large overhead as for a single data
segment 2 slots will be used in the congestion window (in packets) segment 2 slots will be used in the congestion window (in packets)
therefore reducing by roughly twice the potential throughput (in therefore reducing by roughly twice the potential throughput (in
bytes/s) of this subflow. Taking the smallest MSS does not solve the bytes/s) of this subflow. Taking the smallest MSS does not solve the
issue as there might be a case where the subflow with the smallest issue as there might be a case where the subflow with the smallest
MSS only sends a few packets therefore reducing the potential MSS only sends a few packets therefore reducing the potential
throughput of the other subflows. throughput of the other subflows.
The Linux implementation recently took another approach [DetalMSS]. The Linux implementation recently took another approach [DetalMSS].
Instead of selecting the minimum and maximum values, it now Instead of selecting the minimum and maximum values, it now
dynamically adapts the MSS based on the contribution of all the dynamically adapts the MSS based on the contribution of all the
subflows to the connection's throughput. For this it computes, for subflows to the connection's throughput. For this it computes, for
each subflow, the potential throughput achieved by selecting each MSS each subflow, the potential throughput achieved by selecting each MSS
value and by taking into account the lost space in the cwnd. It then value and by taking into account the lost space in the congestion
selects the MSS that allows to achieve the highest potential window. It then selects the MSS that allows to achieve the highest
throughput. potential throughput.
Given the prevalence of middleboxes that clamp the MSS, Multipath TCP
implementations must be able to efficiently support subflows with
different MSS values. The strategy described above is a possible
solution to this problem.
3.9. Interactions with the Domain Name System 3.9. Interactions with the Domain Name System
Multihomed clients such as smartphones can send DNS queries over any Multihomed clients such as smartphones can send DNS queries over any
of their interfaces. When a single-homed client performs a DNS of their interfaces. When a single-homed client performs a DNS
query, it receives from its local resolver the best answer for its query, it receives from its local resolver the best answer for its
request. If the client is multihomed, the answer returned to the DNS request. If the client is multihomed, the answer in response to the
query may vary with the interface over which it has been sent. DNS query may vary with the interface over which it has been sent.
cdn1 cdn1
| |
client -- cellular -- internet -- cdn3 client -- cellular -- internet -- cdn3
| | | |
+----- wifi --------+ +----- wifi --------+
| |
cdn2 cdn2
Figure 6: Simple network topology Figure 6: Simple network topology
skipping to change at page 21, line 21 skipping to change at page 21, line 21
The client is attached to two networks : network1 that provides The client is attached to two networks : network1 that provides
limited connectivity and the entire Internet through the second limited connectivity and the entire Internet through the second
network interface. In practice, this scenario corresponds to an open network interface. In practice, this scenario corresponds to an open
WiFi network with a captive portal for network1 and a cellular WiFi network with a captive portal for network1 and a cellular
service for the second interface. On many smartphones, the WiFi service for the second interface. On many smartphones, the WiFi
interface is preferred over the cellular interface. If the interface is preferred over the cellular interface. If the
smartphone learns a default route via both interfaces, it will smartphone learns a default route via both interfaces, it will
typically prefer to use the WiFi interface to send its DNS request typically prefer to use the WiFi interface to send its DNS request
and create the first subflow. This is not optimal with Multipath and create the first subflow. This is not optimal with Multipath
TCP. A better approach would probably be to try a few attempts on TCP. A better approach would probably be to try a few attempts on
the WiFi interface and then try to use the second interface for the the WiFi interface and then, upon failure of these attempts, try to
initial subflow as well. use the second interface for the initial subflow as well.
3.11. Stateless webservers 3.11. Stateless webservers
MPTCP has been designed to interoperate with webservers that benefit MPTCP has been designed to interoperate with webservers that benefit
from SYN-cookies to protect against SYN-flooding attacks [RFC4987]. from SYN-cookies to protect against SYN-flooding attacks [RFC4987].
MPTCP achieves this by echoing the keys negotiated during the MPTCP achieves this by echoing the keys negotiated during the
MP_CAPABLE handshake in the third ACK of the 3-way handshake. MP_CAPABLE handshake in the third ACK of the 3-way handshake.
Reception of this third ACK then allows the server to reconstruct the Reception of this third ACK then allows the server to reconstruct the
state specific to MPTCP. state specific to MPTCP.
skipping to change at page 22, line 9 skipping to change at page 22, line 9
traffic away from the lossy link. It is undesirable that MPTCP has a traffic away from the lossy link. It is undesirable that MPTCP has a
higher chance to fall back to regular TCP in those lossy higher chance to fall back to regular TCP in those lossy
environments. environments.
[I-D.paasch-mptcp-syncookies] discusses this issue and suggests a [I-D.paasch-mptcp-syncookies] discusses this issue and suggests a
modified handshake mechanism that ensures reliable delivery of the modified handshake mechanism that ensures reliable delivery of the
MP_CAPABLE, following the 3-way handshake. This modification will MP_CAPABLE, following the 3-way handshake. This modification will
make MPTCP reliable, even in lossy environments when servers need to make MPTCP reliable, even in lossy environments when servers need to
use SYN-cookies to protect against SYN-flooding attacks. use SYN-cookies to protect against SYN-flooding attacks.
3.12. Loadbalanced serverfarms 3.12. Loadbalanced server farms
Large-scale serverfarms typically deploy thousands of servers behind Large-scale server farms typically deploy thousands of servers behind
a single virtual IP (VIP). Steering traffic to these servers is done a single virtual IP (VIP). Steering traffic to these servers is done
through layer-4 loadbalancers that ensure that a TCP-flow will always through layer-4 load balancers that ensure that a TCP-flow will
be routed to the same server [Presto08]. always be routed to the same server [Presto08].
As Multipath TCP uses multiple different TCP subflows to steer the As Multipath TCP uses multiple different TCP subflows to steer the
traffic across the different paths, loadbalancers need to ensure that traffic across the different paths, load balancers need to ensure
all these subflows are routed to the same server. This implies that that all these subflows are routed to the same server. This implies
the loadbalancers need to track the MPTCP-related state, allowing that the load balancers need to track the MPTCP-related state,
them to parse the token in the MP_JOIN and assign those subflows to allowing them to parse the token in the MP_JOIN and assign those
the appropriate server. However, serverfarms typically deploy subflows to the appropriate server. However, server farms typically
multiple of these loadbalancers for reliability and capacity reasons. deploy several load balancers for reliability and capacity reasons.
As a TCP subflow might get routed to any of these loadbalancers, they As a TCP subflow might get routed to any of these load balancers,
would need to synchronize the MPTCP-related state - a solution that they would need to synchronize the MPTCP-related state - a solution
is not feasible at large scale. that is not feasible at large scale.
The token (carried in the MP_JOIN) contains the information The token (carried in the MP_JOIN) contains the information
indicating which MPTCP-session the subflow belongs to. As the token indicating which MPTCP-session the subflow belongs to. As the token
is a hash of the key, servers are not able to generate the token in is a hash of the key, servers are not able to generate the token in
such a way that the token can provide the necessary information to such a way that the token can provide the necessary information to
the loadbalancers which would allow them to route TCP subflows to the the load balancers, which would allow them to route TCP subflows to
appropriate server. [I-D.paasch-mptcp-loadbalancer] discusses this the appropriate server. [I-D.paasch-mptcp-loadbalancer] discusses
issue in detail and suggests two alternative MP_CAPABLE handshakes to this issue in detail and suggests two alternative MP_CAPABLE
overcome these. As of September 2015, it is not yet clear how MPTCP handshakes to overcome these.
might accomodate such use-case to enable its deployment within
loadbalanced serverfarms.
4. IANA Considerations 4. IANA Considerations
There are no IANA considerations in this informational document. There are no IANA considerations in this informational document.
5. Security Considerations 5. Security Considerations
The security considerations for Multipath TCP have already been This informational document discusses use-cases and operational
documented in [RFC6181], [RFC6182], [RFC6824] and [RFC7430]. experience with Multipath TCP. An extensive analysis of the
remaining security issues in the Multipath TCP specification has been
published in [RFC7430], together with suggestions for possible
solutions.
From a security viewpoint, it is important to note that Multipath
TCP, like other multipath solutions such as SCTP, has the ability to
send packets belonging to a single connection over different paths.
This design feature of Multipath TCP implies that middleboxes that
have been deployed on-path assuming that they would observe all the
packets exchanged for a given connection in both directions may not
function correctly anymore. A typical example are firewalls, IDS or
DPIs deployed in enterprise networks. Those devices expect to
observe all the packets from all TCP connections. With Multipath
TCP, those middleboxes may not observe anymore all packets since some
of them may follow a different path. The two examples below
illustrate typical deployments of such middleboxes. The first
example, Figure 8, shows a Multipath TCP enabled smartphone attached
to both an enterprise and a cellular network. If a Multipath TCP
connection is established by the smartphone towards a server, some of
the packets sent by the smartphone or the server may be transmitted
over the cellular network and thus be invisible for the enterprise
middlebox.
smartphone +----- entreprise net --- MBox----+------ server
| |
+----- cellular net -------------+
Figure 8: Enterprise Middlebox may not observe all packets from
multihomed host
The second example, Figure 9, shows a possible issue when multiple
middleboxes are deployed inside a network. For simplicity, we assume
that network1 is the default IPv4 path while network2 is the default
IPv6 path. A similar issue could occur with per-flow load balancing
such as ECMP [RFC2992]. With regular TCP, all packets from each
connection would either pass through Mbox1 or Mbox2. With Multipath
TCP, the client can easily establish a subflow over network1 and
another over network2 and each middlebox would only observe a part of
the traffic of the end-to-end Multipath TCP connection.
client ----R-- network1 --- MBox1 -----R------------- server
| |
+-- network2 --- MBox2 -----+
Figure 9: Interactions between load balancing and security
Middleboxes
In these two cases, it is possible for an attacker to evade some
security measures operating on the TCP byte stream and implemented on
the middleboxes by controlling the bytes that are actually sent over
each subflow and there are tools that ease those kinds of evasion
[PZ15] [PT14]. This is not a security issue for Multipath TCP itself
since Multipath TCP behaves correctly. However, this demonstrates
the difficulty of enforcing security policies by relying only on on-
path middleboxes instead of enforcing them directly on the endpoints.
6. Acknowledgements 6. Acknowledgements
This work was partially supported by the FP7-Trilogy2 project. We This work was partially supported by the FP7-Trilogy2 project. We
would like to thank all the implementers and users of the Multipath would like to thank all the implementers and users of the Multipath
TCP implementation in the Linux kernel. This document has benefited TCP implementation in the Linux kernel. This document has benefited
from the comments of John Ronan, Yoshifumi Nishida, Phil Eardley, from the comments of John Ronan, Yoshifumi Nishida, Phil Eardley,
Jaehyun Hwang and Mirja Kuehlewind. Jaehyun Hwang, Mirja Kuehlewind, Benoit Claise, Jari Arkko, Qin Wu,
Spencer Dawkins and Ben Campbell.
7. Informative References 7. References
7.1. Normative References
[RFC6182] Ford, A., Raiciu, C., Handley, M., Barre, S., and J.
Iyengar, "Architectural Guidelines for Multipath TCP
Development", RFC 6182, DOI 10.17487/RFC6182, March 2011,
<http://www.rfc-editor.org/info/rfc6182>.
[RFC6824] Ford, A., Raiciu, C., Handley, M., and O. Bonaventure,
"TCP Extensions for Multipath Operation with Multiple
Addresses", RFC 6824, DOI 10.17487/RFC6824, January 2013,
<http://www.rfc-editor.org/info/rfc6824>.
7.2. Informative References
[Apple-MPTCP] [Apple-MPTCP]
Apple, Inc, ., "iOS - Multipath TCP Support in iOS 7", Apple, Inc, ., "iOS - Multipath TCP Support in iOS 7",
n.d., <https://support.apple.com/en-us/HT201373>. n.d., <https://support.apple.com/en-us/HT201373>.
[BALIA] Peng, Q., Walid, A., Hwang, J., and S. Low, "Multipath TCP [BALIA] Peng, Q., Walid, A., Hwang, J., and S. Low, "Multipath TCP
Analysis, Design, and Implementation", IEEE/ACM Trans. on Analysis, Design, and Implementation", IEEE/ACM Trans. on
Networking, Vol. 24, No. 1 , 2016. Networking, Vol. 24, No. 1 , 2016.
[BBF-WT348]
Fabregas (Ed), G., "WT-348 - Hybrid Access for Broadband
Networks", Broadband Forum, contribution bbf2014.1139.04 ,
June 2015.
[CACM14] Paasch, C. and O. Bonaventure, "Multipath TCP", [CACM14] Paasch, C. and O. Bonaventure, "Multipath TCP",
Communications of the ACM, 57(4):51-57 , April 2014, Communications of the ACM, 57(4):51-57 , April 2014,
<http://inl.info.ucl.ac.be/publications/multipath-tcp>. <http://inl.info.ucl.ac.be/publications/multipath-tcp>.
[COMCOM2016] [COMCOM2016]
"Observing real Multipath TCP traffic", Computer Tran, V., De Coninck, Q., Hesmans, B., Sadre, R., and O.
Communications , April 2016, <http://inl.info.ucl.ac.be/ Bonaventure, "Observing real Multipath TCP traffic",
publications/observing-real-multipath-tcp-traffic>. Computer Communications , April 2016, <http://
inl.info.ucl.ac.be/publications/
observing-real-multipath-tcp-traffic>.
[COMMAG2016] [COMMAG2016]
De Coninck, Q., Baerts, M., Hesmans, B., and O. De Coninck, Q., Baerts, M., Hesmans, B., and O.
Bonaventure, "Observing Real Smartphone Applications over Bonaventure, "Observing Real Smartphone Applications over
Multipath TCP", IEEE Communications Magazine , March 2016, Multipath TCP", IEEE Communications Magazine , March 2016,
<http://inl.info.ucl.ac.be/publications/ <http://inl.info.ucl.ac.be/publications/
observing-real-smartphone-applications-over-multipath- observing-real-smartphone-applications-over-multipath-
tcp>. tcp>.
[CONEXT12] [CONEXT12]
skipping to change at page 28, line 41 skipping to change at page 30, line 5
[I-D.paasch-mptcp-syncookies] [I-D.paasch-mptcp-syncookies]
Paasch, C., Biswas, A., and D. Haas, "Making Multipath TCP Paasch, C., Biswas, A., and D. Haas, "Making Multipath TCP
robust for stateless webservers", robust for stateless webservers",
draft-paasch-mptcp-syncookies-02 (work in progress), draft-paasch-mptcp-syncookies-02 (work in progress),
October 2015. October 2015.
[ICNP12] Cao, Y., Xu, M., and X. Fu, "Delay-based congestion [ICNP12] Cao, Y., Xu, M., and X. Fu, "Delay-based congestion
control for multipath TCP", 20th IEEE International control for multipath TCP", 20th IEEE International
Conference on Network Protocols (ICNP) , 2012. Conference on Network Protocols (ICNP) , 2012.
[IETFJ] Bonaventure, O. and S. Seo, "Multipath TCP Deployments,",
IETF Journal, Vol. 12, Issue 2 , November 2016.
[IMC11] Honda, M., Nishida, Y., Raiciu, C., Greenhalgh, A., [IMC11] Honda, M., Nishida, Y., Raiciu, C., Greenhalgh, A.,
Handley, M., and H. Tokuda, "Is it still possible to Handley, M., and H. Tokuda, "Is it still possible to
extend TCP?", Proceedings of the 2011 ACM SIGCOMM extend TCP?", Proceedings of the 2011 ACM SIGCOMM
conference on Internet measurement conference (IMC '11) , conference on Internet measurement conference (IMC '11) ,
2011, <http://doi.acm.org/10.1145/2068816.2068834>. 2011, <http://doi.acm.org/10.1145/2068816.2068834>.
[IMC13a] Detal, G., Hesmans, B., Bonaventure, O., Vanaubel, Y., and [IMC13a] Detal, G., Hesmans, B., Bonaventure, O., Vanaubel, Y., and
B. Donnet, "Revealing Middlebox Interference with B. Donnet, "Revealing Middlebox Interference with
Tracebox", Proceedings of the 2013 ACM SIGCOMM conference Tracebox", Proceedings of the 2013 ACM SIGCOMM conference
on Internet measurement conference , 2013, <http:// on Internet measurement conference , 2013, <http://
skipping to change at page 30, line 27 skipping to change at page 31, line 41
Smartphones", 17th International Passive and Active Smartphones", 17th International Passive and Active
Measurements Conference (PAM2016) , March 2016, <http:// Measurements Conference (PAM2016) , March 2016, <http://
inl.info.ucl.ac.be/publications/ inl.info.ucl.ac.be/publications/
first-analysis-multipath-tcp-smartphones>. first-analysis-multipath-tcp-smartphones>.
[PAMS2014] [PAMS2014]
Arzani, B., Gurney, A., Cheng, S., Guerin, R., and B. Loo, Arzani, B., Gurney, A., Cheng, S., Guerin, R., and B. Loo,
"Impact of Path Selection and Scheduling Policies on MPTCP "Impact of Path Selection and Scheduling Policies on MPTCP
Performance", PAMS2014 , 2014. Performance", PAMS2014 , 2014.
[PT14] Pearce, C. and P. Thomas, "Multipath TCP Breaking Today's
Networks with Tomorrow's Protocols", Proc. Blackhat
Briefings , 2014, <http://www.blackhat.com/docs/us-14/
materials/
us-14-Pearce-Multipath-TCP-Breaking-Todays-Networks-With-
Tomorrows-Protocols-WP.pdf>.
[PZ15] Pearce, C. and S. Zeadally, "Ancillary Impacts of
Multipath TCP on Current and Future Network Security",
IEEE Internet Computing, vol. 19, no. 5, pp. 58-65 , 2015.
[PaaschPhD] [PaaschPhD]
Paasch, C., "Improving Multipath TCP", Ph.D. Thesis , Paasch, C., "Improving Multipath TCP", Ph.D. Thesis ,
November 2014, <http://inl.info.ucl.ac.be/publications/ November 2014, <http://inl.info.ucl.ac.be/publications/
improving-multipath-tcp>. improving-multipath-tcp>.
[Presto08] [Presto08]
Greenberg, A., Lahiri, P., Maltz, D., Parveen, P., and S. Greenberg, A., Lahiri, P., Maltz, D., Parveen, P., and S.
Sengupta, "Towards a Next Generation Data Center Sengupta, "Towards a Next Generation Data Center
Architecture - Scalability and Commoditization", ACM Architecture - Scalability and Commoditization", ACM
PRESTO 2008 , August 2008, PRESTO 2008 , August 2008,
skipping to change at page 30, line 48 skipping to change at page 32, line 26
[RFC1812] Baker, F., Ed., "Requirements for IP Version 4 Routers", [RFC1812] Baker, F., Ed., "Requirements for IP Version 4 Routers",
RFC 1812, DOI 10.17487/RFC1812, June 1995, RFC 1812, DOI 10.17487/RFC1812, June 1995,
<http://www.rfc-editor.org/info/rfc1812>. <http://www.rfc-editor.org/info/rfc1812>.
[RFC1928] Leech, M., Ganis, M., Lee, Y., Kuris, R., Koblas, D., and [RFC1928] Leech, M., Ganis, M., Lee, Y., Kuris, R., Koblas, D., and
L. Jones, "SOCKS Protocol Version 5", RFC 1928, L. Jones, "SOCKS Protocol Version 5", RFC 1928,
DOI 10.17487/RFC1928, March 1996, DOI 10.17487/RFC1928, March 1996,
<http://www.rfc-editor.org/info/rfc1928>. <http://www.rfc-editor.org/info/rfc1928>.
[RFC2992] Hopps, C., "Analysis of an Equal-Cost Multi-Path
Algorithm", RFC 2992, DOI 10.17487/RFC2992, November 2000,
<http://www.rfc-editor.org/info/rfc2992>.
[RFC4987] Eddy, W., "TCP SYN Flooding Attacks and Common [RFC4987] Eddy, W., "TCP SYN Flooding Attacks and Common
Mitigations", RFC 4987, DOI 10.17487/RFC4987, August 2007, Mitigations", RFC 4987, DOI 10.17487/RFC4987, August 2007,
<http://www.rfc-editor.org/info/rfc4987>. <http://www.rfc-editor.org/info/rfc4987>.
[RFC6181] Bagnulo, M., "Threat Analysis for TCP Extensions for
Multipath Operation with Multiple Addresses", RFC 6181,
DOI 10.17487/RFC6181, March 2011,
<http://www.rfc-editor.org/info/rfc6181>.
[RFC6182] Ford, A., Raiciu, C., Handley, M., Barre, S., and J.
Iyengar, "Architectural Guidelines for Multipath TCP
Development", RFC 6182, DOI 10.17487/RFC6182, March 2011,
<http://www.rfc-editor.org/info/rfc6182>.
[RFC6356] Raiciu, C., Handley, M., and D. Wischik, "Coupled [RFC6356] Raiciu, C., Handley, M., and D. Wischik, "Coupled
Congestion Control for Multipath Transport Protocols", Congestion Control for Multipath Transport Protocols",
RFC 6356, DOI 10.17487/RFC6356, October 2011, RFC 6356, DOI 10.17487/RFC6356, October 2011,
<http://www.rfc-editor.org/info/rfc6356>. <http://www.rfc-editor.org/info/rfc6356>.
[RFC6824] Ford, A., Raiciu, C., Handley, M., and O. Bonaventure,
"TCP Extensions for Multipath Operation with Multiple
Addresses", RFC 6824, DOI 10.17487/RFC6824, January 2013,
<http://www.rfc-editor.org/info/rfc6824>.
[RFC7430] Bagnulo, M., Paasch, C., Gont, F., Bonaventure, O., and C. [RFC7430] Bagnulo, M., Paasch, C., Gont, F., Bonaventure, O., and C.
Raiciu, "Analysis of Residual Threats and Possible Fixes Raiciu, "Analysis of Residual Threats and Possible Fixes
for Multipath TCP (MPTCP)", RFC 7430, DOI 10.17487/ for Multipath TCP (MPTCP)", RFC 7430, DOI 10.17487/
RFC7430, July 2015, RFC7430, July 2015,
<http://www.rfc-editor.org/info/rfc7430>. <http://www.rfc-editor.org/info/rfc7430>.
[RFC7871] Contavalli, C., van der Gaast, W., Lawrence, D., and W. [RFC7871] Contavalli, C., van der Gaast, W., Lawrence, D., and W.
Kumari, "Client Subnet in DNS Queries", RFC 7871, Kumari, "Client Subnet in DNS Queries", RFC 7871,
DOI 10.17487/RFC7871, May 2016, DOI 10.17487/RFC7871, May 2016,
<http://www.rfc-editor.org/info/rfc7871>. <http://www.rfc-editor.org/info/rfc7871>.
[SIGCOMM11] [SIGCOMM11]
Raiciu, C., Barre, S., Pluntke, C., Greenhalgh, A., Raiciu, C., Barre, S., Pluntke, C., Greenhalgh, A.,
Wischik, D., and M. Handley, "Improving datacenter Wischik, D., and M. Handley, "Improving datacenter
performance and robustness with multipath TCP", performance and robustness with multipath TCP",
Proceedings of the ACM SIGCOMM 2011 conference , n.d., Proceedings of the ACM SIGCOMM 2011 conference , n.d.,
<http://doi.acm.org/10.1145/2018436.2018467>. <http://doi.acm.org/10.1145/2018436.2018467>.
[SOCKET] Hesmans, B. and O. Bonaventure, "An Enhanced Socket API
for Multipath TCP", Proceedings of the 2016 Applied
Networking Research Workshop , July 2016,
<http://doi.acm.org/10.1145/2959424.2959433>.
[StrangeMbox] [StrangeMbox]
Bonaventure, O., "Multipath TCP through a strange Bonaventure, O., "Multipath TCP through a strange
middlebox", Blog post , January 2015, <http:// middlebox", Blog post , January 2015, <http://
blog.multipath-tcp.org/blog/html/2015/01/30/ blog.multipath-tcp.org/blog/html/2015/01/30/
multipath_tcp_through_a_strange_middlebox.html>. multipath_tcp_through_a_strange_middlebox.html>.
[TMA2015] Hesmans, B., Tran Viet, H., Sadre, R., and O. Bonaventure, [TMA2015] Hesmans, B., Tran Viet, H., Sadre, R., and O. Bonaventure,
"A First Look at Real Multipath TCP Traffic", Traffic "A First Look at Real Multipath TCP Traffic", Traffic
Monitoring and Analysis , 2015, <http:// Monitoring and Analysis , 2015, <http://
inl.info.ucl.ac.be/publications/ inl.info.ucl.ac.be/publications/
first-look-real-multipath-tcp-traffic>. first-look-real-multipath-tcp-traffic>.
[TR-348] Broadband Forum, ., "TR 348 - Hybrid Access Broadband
Network Architecture", July 2016, <https://
www.broadband-forum.org/technical/download/TR-348.pdf>.
[ietf88] Stewart, L., "IETF'88 Meeting minutes of the MPTCP working [ietf88] Stewart, L., "IETF'88 Meeting minutes of the MPTCP working
group", n.d., <http://tools.ietf.org/wg/mptcp/ group", n.d., <http://tools.ietf.org/wg/mptcp/
minutes?item=minutes-88-mptcp.html>. minutes?item=minutes-88-mptcp.html>.
[tracebox] [tracebox]
Detal, G. and O. Tilmans, "tracebox", 2013, Detal, G. and O. Tilmans, "tracebox", 2013,
<http://www.tracebox.org>. <http://www.tracebox.org>.
Appendix A. Changelog Appendix A. Changelog
skipping to change at page 33, line 42 skipping to change at page 34, line 42
* Improved introduction * Improved introduction
* Added details about using SOCKS and Korea Telecom's use-case in * Added details about using SOCKS and Korea Telecom's use-case in
Section 2.3. Section 2.3.
* Added issue around clients caching DNS-results in Section 3.9 * Added issue around clients caching DNS-results in Section 3.9
* Explained issue of MPTCP with stateless webservers Section 3.11 * Explained issue of MPTCP with stateless webservers Section 3.11
* Added description of MPTCP's use behind layer-4 loadbalancers * Added description of MPTCP's use behind layer-4 load balancers
Section 3.12 Section 3.12
* Restructured text and improved writing in some parts * Restructured text and improved writing in some parts
o version ietf-04 : April 2016, answer to last comments o version ietf-04 : April 2016, answer to last comments
* Updated text on measurements with smartphones * Updated text on measurements with smartphones
* Updated conclusion * Updated conclusion
o version ietf-06 : August 2016, answer to AD's review o version ietf-06 : August 2016, answer to AD's review
* removed some expired drafts * removed some expired drafts
* removed conclusion * removed conclusion
o version ietf-07 : September 2016, answer to IESG comments
* small nits
* added more information in the security considerations as
suggested by Stephen Farrel
Authors' Addresses Authors' Addresses
Olivier Bonaventure Olivier Bonaventure
UCLouvain UCLouvain
Email: Olivier.Bonaventure@uclouvain.be Email: Olivier.Bonaventure@uclouvain.be
Christoph Paasch Christoph Paasch
Apple, Inc. Apple, Inc.
 End of changes. 61 change blocks. 
172 lines changed or deleted 259 lines changed or added

This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/