draft-ietf-lwig-tcp-constrained-node-networks-07.txt   draft-ietf-lwig-tcp-constrained-node-networks-08.txt 
LWIG Working Group C. Gomez LWIG Working Group C. Gomez
Internet-Draft UPC Internet-Draft UPC
Intended status: Informational J. Crowcroft Intended status: Informational J. Crowcroft
Expires: September 30, 2019 University of Cambridge Expires: December 6, 2019 University of Cambridge
M. Scharf M. Scharf
Hochschule Esslingen Hochschule Esslingen
March 29, 2019 June 4, 2019
TCP Usage Guidance in the Internet of Things (IoT) TCP Usage Guidance in the Internet of Things (IoT)
draft-ietf-lwig-tcp-constrained-node-networks-07 draft-ietf-lwig-tcp-constrained-node-networks-08
Abstract Abstract
This document provides guidance on how to implement and use the This document provides guidance on how to implement and use the
Transmission Control Protocol (TCP) in Constrained-Node Networks Transmission Control Protocol (TCP) in Constrained-Node Networks
(CNNs), which are a characterstic of the Internet of Things (IoT). (CNNs), which are a characterstic of the Internet of Things (IoT).
Such environments require a lightweight TCP implementation and may Such environments require a lightweight TCP implementation and may
not make use of optional functionality. This document explains a not make use of optional functionality. This document explains a
number of known and deployed techniques to simplify a TCP stack as number of known and deployed techniques to simplify a TCP stack as
well as corresponding tradeoffs. The objective is to help embedded well as corresponding tradeoffs. The objective is to help embedded
skipping to change at page 1, line 40 skipping to change at page 1, line 40
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on September 30, 2019. This Internet-Draft will expire on December 6, 2019.
Copyright Notice Copyright Notice
Copyright (c) 2019 IETF Trust and the persons identified as the Copyright (c) 2019 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 20 skipping to change at page 2, line 20
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Conventions used in this document . . . . . . . . . . . . . . 4 2. Conventions used in this document . . . . . . . . . . . . . . 4
3. Characteristics of CNNs relevant for TCP . . . . . . . . . . 4 3. Characteristics of CNNs relevant for TCP . . . . . . . . . . 4
3.1. Network and link properties . . . . . . . . . . . . . . . 4 3.1. Network and link properties . . . . . . . . . . . . . . . 4
3.2. Usage scenarios . . . . . . . . . . . . . . . . . . . . . 5 3.2. Usage scenarios . . . . . . . . . . . . . . . . . . . . . 5
3.3. Communication and traffic patterns . . . . . . . . . . . 6 3.3. Communication and traffic patterns . . . . . . . . . . . 6
4. TCP implementation and configuration in CNNs . . . . . . . . 6 4. TCP implementation and configuration in CNNs . . . . . . . . 6
4.1. Path properties . . . . . . . . . . . . . . . . . . . . . 7 4.1. Addressing path properties . . . . . . . . . . . . . . . 7
4.1.1. Maximum Segment Size (MSS) . . . . . . . . . . . . . 7 4.1.1. Maximum Segment Size (MSS) . . . . . . . . . . . . . 7
4.1.2. Explicit Congestion Notification (ECN) . . . . . . . 8 4.1.2. Explicit Congestion Notification (ECN) . . . . . . . 8
4.1.3. Explicit loss notifications . . . . . . . . . . . . . 9 4.1.3. Explicit loss notifications . . . . . . . . . . . . . 9
4.2. TCP guidance for single-MSS windows and buffers . . . . . 9 4.2. TCP guidance for single-segment stacks . . . . . . . . . 9
4.2.1. Single-MSS stacks - benefits and issues . . . . . . . 9 4.2.1. Single-segment stacks - benefits and issues . . . . . 9
4.2.2. TCP options for single-MSS stacks . . . . . . . . . . 9 4.2.2. TCP options for single-segment stacks . . . . . . . . 10
4.2.3. Delayed Acknowledgments for single-MSS stacks . . . . 10 4.2.3. Delayed Acknowledgments for single-segment stacks . . 10
4.2.4. RTO estimation for single-MSS stacks . . . . . . . . 11 4.2.4. RTO calculation for single-segment stacks . . . . . . 11
4.3. General recommendations for TCP in CNNs . . . . . . . . . 11 4.3. General recommendations for TCP in CNNs . . . . . . . . . 11
4.3.1. Loss recovery and congestion/flow control . . . . . . 11 4.3.1. Loss recovery and congestion/flow control . . . . . . 12
4.3.1.1. Selective Acknowledgments (SACK) . . . . . . . . 12 4.3.1.1. Selective Acknowledgments (SACK) . . . . . . . . 12
4.3.2. Delayed Acknowledgments . . . . . . . . . . . . . . . 12 4.3.2. Delayed Acknowledgments . . . . . . . . . . . . . . . 13
5. TCP usage recommendations in CNNs . . . . . . . . . . . . . . 13 4.3.3. Initial Window . . . . . . . . . . . . . . . . . . . 13
5.1. TCP connection initiation . . . . . . . . . . . . . . . . 13 5. TCP usage recommendations in CNNs . . . . . . . . . . . . . . 14
5.2. Number of concurrent connections . . . . . . . . . . . . 13 5.1. TCP connection initiation . . . . . . . . . . . . . . . . 14
5.3. TCP connection lifetime . . . . . . . . . . . . . . . . . 13 5.2. Number of concurrent connections . . . . . . . . . . . . 14
6. Security Considerations . . . . . . . . . . . . . . . . . . . 15 5.3. TCP connection lifetime . . . . . . . . . . . . . . . . . 15
7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 16 6. Security Considerations . . . . . . . . . . . . . . . . . . . 17
8. Annex. TCP implementations for constrained devices . . . . . 16 7. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 17
8.1. uIP . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 8. Annex. TCP implementations for constrained devices . . . . . 18
8.2. lwIP . . . . . . . . . . . . . . . . . . . . . . . . . . 17 8.1. uIP . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
8.3. RIOT . . . . . . . . . . . . . . . . . . . . . . . . . . 17 8.2. lwIP . . . . . . . . . . . . . . . . . . . . . . . . . . 19
8.4. TinyOS . . . . . . . . . . . . . . . . . . . . . . . . . 18 8.3. RIOT . . . . . . . . . . . . . . . . . . . . . . . . . . 19
8.5. FreeRTOS . . . . . . . . . . . . . . . . . . . . . . . . 18 8.4. TinyOS . . . . . . . . . . . . . . . . . . . . . . . . . 19
8.6. uC/OS . . . . . . . . . . . . . . . . . . . . . . . . . . 18 8.5. FreeRTOS . . . . . . . . . . . . . . . . . . . . . . . . 20
8.7. Summary . . . . . . . . . . . . . . . . . . . . . . . . . 19 8.6. uC/OS . . . . . . . . . . . . . . . . . . . . . . . . . . 20
9. Annex. Changes compared to previous versions . . . . . . . . 20 8.7. Summary . . . . . . . . . . . . . . . . . . . . . . . . . 20
9.1. Changes between -00 and -01 . . . . . . . . . . . . . . . 20 9. Annex. Changes compared to previous versions . . . . . . . . 22
9.2. Changes between -01 and -02 . . . . . . . . . . . . . . . 20 9.1. Changes between -00 and -01 . . . . . . . . . . . . . . . 22
9.3. Changes between -02 and -03 . . . . . . . . . . . . . . . 20 9.2. Changes between -01 and -02 . . . . . . . . . . . . . . . 22
9.4. Changes between -03 and -04 . . . . . . . . . . . . . . . 21 9.3. Changes between -02 and -03 . . . . . . . . . . . . . . . 22
9.5. Changes between -04 and -05 . . . . . . . . . . . . . . . 21 9.4. Changes between -03 and -04 . . . . . . . . . . . . . . . 23
9.6. Changes between -05 and -06 . . . . . . . . . . . . . . . 21 9.5. Changes between -04 and -05 . . . . . . . . . . . . . . . 23
9.7. Changes between -06 and -07 . . . . . . . . . . . . . . . 21 9.6. Changes between -05 and -06 . . . . . . . . . . . . . . . 23
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 21 9.7. Changes between -06 and -07 . . . . . . . . . . . . . . . 23
10.1. Normative References . . . . . . . . . . . . . . . . . . 21 9.8. Changes between -07 and -08 . . . . . . . . . . . . . . . 23
10.2. Informative References . . . . . . . . . . . . . . . . . 23 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 23
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26 10.1. Normative References . . . . . . . . . . . . . . . . . . 23
10.2. Informative References . . . . . . . . . . . . . . . . . 25
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 29
1. Introduction 1. Introduction
The Internet Protocol suite is being used for connecting Constrained- The Internet Protocol suite is being used for connecting Constrained-
Node Networks (CNNs) to the Internet, enabling the so-called Internet Node Networks (CNNs) to the Internet, enabling the so-called Internet
of Things (IoT) [RFC7228]. In order to meet the requirements that of Things (IoT) [RFC7228]. In order to meet the requirements that
stem from CNNs, the IETF has produced a suite of new protocols stem from CNNs, the IETF has produced a suite of new protocols
specifically designed for such environments (see e.g. [RFC8352]). specifically designed for such environments (see e.g. [RFC8352]).
New IETF protocol stack components include the IPv6 over Low-power New IETF protocol stack components include the IPv6 over Low-power
Wireless Personal Area Networks (6LoWPAN) adaptation layer, the IPv6 Wireless Personal Area Networks (6LoWPAN) adaptation layer
Routing Protocol for Low-power and lossy networks (RPL) routing [RFC4944][RFC6282][RFC6775], the IPv6 Routing Protocol for Low-power
protocol, and the Constrained Application Protocol (CoAP). and lossy networks (RPL) routing protocol [RFC6550], and the
Constrained Application Protocol (CoAP) [RFC7252].
As of the writing, the main current transport layer protocols in IP- As of the writing, the main current transport layer protocols in IP-
based IoT scenarios are UDP and TCP. However, TCP has been based IoT scenarios are UDP and TCP. However, TCP has been
criticized (often, unfairly) as a protocol for the IoT. In fact, criticized (often, unfairly) as a protocol for the IoT. In fact,
some TCP features are not optimal for IoT scenarios, such as some TCP features are not optimal for IoT scenarios, such as
relatively long header size, unsuitability for multicast, and always- relatively long header size, unsuitability for multicast, and always-
confirmed data delivery. However, many typical claims on TCP confirmed data delivery. However, many typical claims on TCP
unsuitability for IoT (e.g. a high complexity, connection-oriented unsuitability for IoT (e.g. a high complexity, connection-oriented
approach incompatibility with radio duty-cycling, and spurious approach incompatibility with radio duty-cycling, and spurious
congestion control activation in wireless links) are not valid, can congestion control activation in wireless links) are not valid, can
be solved, or are also found in well accepted IoT end-to-end be solved, or are also found in well accepted IoT end-to-end
reliability mechanisms (see [IntComp] for a detailed analysis). reliability mechanisms (see [IntComp] for a detailed analysis).
At the application layer, CoAP was developed over UDP [RFC7252]. At the application layer, CoAP was developed over UDP [RFC7252].
However, the integration of some CoAP deployments with existing However, the integration of some CoAP deployments with existing
infrastructure is being challenged by middleboxes such as firewalls, infrastructure is being challenged by middleboxes such as firewalls,
which may limit and even block UDP-based communications. This the which may limit and even block UDP-based communications. This is the
main reason why a CoAP over TCP specification has been developed main reason why a CoAP over TCP specification has been developed
[RFC8323]. [RFC8323].
Other application layer protocols not specifically designed for CNNs Other application layer protocols not specifically designed for CNNs
are also being considered for the IoT space. Some examples include are also being considered for the IoT space. Some examples include
HTTP/2 and even HTTP/1.1, both of which run over TCP by default HTTP/2 and even HTTP/1.1, both of which run over TCP by default
[RFC7230] [RFC7540], and the Extensible Messaging and Presence [RFC7230] [RFC7540], and the Extensible Messaging and Presence
Protocol (XMPP) [RFC6120]. TCP is also used by non-IETF application- Protocol (XMPP) [RFC6120]. TCP is also used by non-IETF application-
layer protocols in the IoT space such as the Message Queue Telemetry layer protocols in the IoT space such as the Message Queue Telemetry
Transport (MQTT) and its lightweight variants. Transport (MQTT) and its lightweight variants.
TCP is a sophisticated transport protocol that includes optional TCP is a sophisticated transport protocol that includes optional
functionality (e.g. TCP options) that may improve performance in functionality (e.g. TCP options) that may improve performance in
some environments. However, many optional TCP extensions require some environments. However, many optional TCP extensions require
complex logic inside the TCP stack and increase the codesize and the complex logic inside the TCP stack and increase the codesize and the
memory requirements. Many TCP extensions are not required for memory requirements. Many TCP extensions are not required for
interoperability with other standard-compliant TCP endpoints. Given interoperability with other standard-compliant TCP endpoints. Given
the limited resources on constrained devices, careful "tuning" of the the limited resources on constrained devices, careful selection of
TCP implementation can make an implementation more lightweight. optional TCP features can make an implementation more lightweight.
This document provides guidance on how to implement and use TCP in This document provides guidance on how to implement and configure
CNNs. The overarching goal is to offer simple measures to allow for TCP, as well as on how TCP is advisable to be used by applications,
lightweight TCP implementation and suitable operation in such in CNNs. The overarching goal is to offer simple measures to allow
for lightweight TCP implementation and suitable operation in such
environments. A TCP implementation following the guidance in this environments. A TCP implementation following the guidance in this
document is intended to be compatible with a TCP endpoint that is document is intended to be compatible with a TCP endpoint that is
compliant to the TCP standards, albeit possibly with a lower compliant to the TCP standards, albeit possibly with a lower
performance. This implies that such a TCP client would always be performance. This implies that such a TCP client would always be
able to connect with a standard-compliant TCP server, and a able to connect with a standard-compliant TCP server, and a
corresponding TCP server would always be able to connect with a corresponding TCP server would always be able to connect with a
standard-compliant TCP client. standard-compliant TCP client.
This document assumes that the reader is familiar with TCP. A This document assumes that the reader is familiar with TCP. A
comprehensive survey of the TCP standards can be found in [RFC7414]. comprehensive survey of the TCP standards can be found in [RFC7414].
skipping to change at page 5, line 26 skipping to change at page 5, line 28
layer technologies that have been characterized as 'lossy', i.e., layer technologies that have been characterized as 'lossy', i.e.,
exhibit a relatively high bit error rate. Dealing with corruption exhibit a relatively high bit error rate. Dealing with corruption
loss is one of the open issues in the Internet [RFC6077]. loss is one of the open issues in the Internet [RFC6077].
3.2. Usage scenarios 3.2. Usage scenarios
There are different deployment and usage scenarios for CNNs. Some There are different deployment and usage scenarios for CNNs. Some
CNNs follow the star topology, whereby one or several hosts are CNNs follow the star topology, whereby one or several hosts are
linked to a central device that acts as a router connecting the CNN linked to a central device that acts as a router connecting the CNN
to the Internet. CNNs may also follow the multihop topology to the Internet. CNNs may also follow the multihop topology
[RFC6606]. One key use case for the use of TCP is a model where [RFC6606].
constrained devices connect to unconstrained servers in the Internet.
But it is also possible that both TCP endpoints run on constrained
devices.
In constrained environments, there can be different types of devices In constrained environments, there can be different types of devices
[RFC7228]. For example, there can be devices with single combined [RFC7228]. For example, there can be devices with single combined
send/receive buffer, devices with a separate send and receive buffer, send/receive buffer, devices with a separate send and receive buffer,
or devices with a pool of multiple send/receive buffers. In the or devices with a pool of multiple send/receive buffers. In the
latter case, it is possible that buffers also be shared for other latter case, it is possible that buffers also be shared for other
protocols. protocols.
When a CNN comprising one or more constrained devices and an One key use case for the use of TCP in CNNs is a model where
unconstrained device communicate over the Internet using TCP, the constrained devices connect to unconstrained servers in the Internet.
communication possibly has to traverse a middlebox (e.g. a firewall, But it is also possible that both TCP endpoints run on constrained
NAT, etc.). Figure 1 illustrates such scenario. Note that the devices. In the first case, communication possibly has to traverse a
scenario is asymmetric, as the unconstrained device will typically middlebox (e.g. a firewall, NAT, etc.). Figure 1 illustrates such
not suffer the severe constraints of the constrained device. The scenario. Note that the scenario is asymmetric, as the unconstrained
unconstrained device is expected to be mains-powered, to have high device will typically not suffer the severe constraints of the
amount of memory and processing power, and to be connected to a constrained device. The unconstrained device is expected to be
resource-rich network. mains-powered, to have high amount of memory and processing power,
and to be connected to a resource-rich network.
Assuming that a majority of constrained devices will correspond to Assuming that a majority of constrained devices will correspond to
sensor nodes, the amount of data traffic sent by constrained devices sensor nodes, the amount of data traffic sent by constrained devices
(e.g. sensor node measurements) is expected to be higher than the (e.g. sensor node measurements) is expected to be higher than the
amount of data traffic in the opposite direction. Nevertheless, amount of data traffic in the opposite direction. Nevertheless,
constrained devices may receive requests (to which they may respond), constrained devices may receive requests (to which they may respond),
commands (for configuration purposes and for constrained devices commands (for configuration purposes and for constrained devices
including actuators) and relatively infrequent firmware/software including actuators) and relatively infrequent firmware/software
updates. updates.
skipping to change at page 7, line 5 skipping to change at page 7, line 5
communicates with an unconstrained device (cf. Figure 1). But it is communicates with an unconstrained device (cf. Figure 1). But it is
also possible that constrained devices communicate amongst also possible that constrained devices communicate amongst
themselves. themselves.
4. TCP implementation and configuration in CNNs 4. TCP implementation and configuration in CNNs
This section explains how a TCP stack can deal with typical This section explains how a TCP stack can deal with typical
constraints in CNN. The guidance in this section relates to the TCP constraints in CNN. The guidance in this section relates to the TCP
implementation and its configuration. implementation and its configuration.
4.1. Path properties 4.1. Addressing path properties
4.1.1. Maximum Segment Size (MSS) 4.1.1. Maximum Segment Size (MSS)
Assuming that IPv6 is used, and for the sake of lightweight Assuming that IPv6 is used, and for the sake of lightweight
implementation and operation, unless applications require handling implementation and operation, unless applications require handling
large data units (i.e. leading to an IPv6 datagram size greater than large data units (i.e. leading to an IPv6 datagram size greater than
1280 bytes), it may be desirable to limit the MTU to 1280 bytes in 1280 bytes), it may be desirable to limit the MTU to 1280 bytes in
order to avoid the need to support Path MTU Discovery [RFC8201]. order to avoid the need to support Path MTU Discovery [RFC8201]. In
addition, an MTU of 1280 bytes avoids incurring IPv6-layer
fragmentation.
An IPv6 datagram size exceeding 1280 bytes can be avoided by setting An IPv6 datagram size exceeding 1280 bytes can be avoided by setting
the TCP MSS not larger than 1220 bytes. This assumes that the remote the TCP MSS not larger than 1220 bytes. This assumes that the remote
sender will use no TCP options, aside from possibly the MSS option, sender will use no TCP options, aside from possibly the MSS option,
which is only used in the initial TCP SYN packet. In order to which is only used in the initial TCP SYN packet.
accommodate unrequested TCP options that may be used by some TCP
implementations, a constrained device may advertise an MSS not larger In order to accommodate unrequested TCP options that may be used by
than 1200 bytes. some TCP implementations, a constrained device may advertise an MSS
smaller than 1220 bytes (e.g. not larger than 1200 bytes). Note
that, in many implementations, TCP options generally consume payload
space instead of increasing datagram size, therefore this suggestion
might be overcautious and its suitability will depend on each
specific scenario.
Note that setting the MTU to 1280 bytes is possible for link layer Note that setting the MTU to 1280 bytes is possible for link layer
technologies in the CNN space, even if some of them are characterized technologies in the CNN space, even if some of them are characterized
by a short data unit payload size, e.g. up to a few tens or hundreds by a short data unit payload size, e.g. up to a few tens or hundreds
of bytes. For example, the maximum frame size in IEEE 802.15.4 is of bytes. For example, the maximum frame size in IEEE 802.15.4 is
127 bytes. 6LoWPAN defined an adaptation layer to support IPv6 over 127 bytes. 6LoWPAN defined an adaptation layer to support IPv6 over
IEEE 802.15.4 networks. The adaptation layer includes a IEEE 802.15.4 networks. The adaptation layer includes a
fragmentation mechanism, since IPv6 requires the layer below to fragmentation mechanism, since IPv6 requires the layer below to
support an MTU of 1280 bytes [RFC2460], while IEEE 802.15.4 lacked support an MTU of 1280 bytes [RFC2460], while IEEE 802.15.4 lacked
fragmentation mechanisms. 6LoWPAN defines an IEEE 802.15.4 link MTU fragmentation mechanisms. 6LoWPAN defines an IEEE 802.15.4 link MTU
skipping to change at page 7, line 48 skipping to change at page 8, line 7
On the other hand, there exist technologies also used in the CNN On the other hand, there exist technologies also used in the CNN
space, such as Master Slave / Token Passing (TP) [RFC8163], space, such as Master Slave / Token Passing (TP) [RFC8163],
Narrowband IoT (NB-IoT) [RFC8376] or IEEE 802.11ah Narrowband IoT (NB-IoT) [RFC8376] or IEEE 802.11ah
[I-D.delcarpio-6lo-wlanah], that do not suffer the same degree of [I-D.delcarpio-6lo-wlanah], that do not suffer the same degree of
frame size limitations as the technologies mentioned above. The MTU frame size limitations as the technologies mentioned above. The MTU
for MS/TP is recommended to be 1500 bytes [RFC8163], the MTU in NB- for MS/TP is recommended to be 1500 bytes [RFC8163], the MTU in NB-
IoT is 1600 bytes, and the maximum frame payload size for IEEE IoT is 1600 bytes, and the maximum frame payload size for IEEE
802.11ah is 7991 bytes. 802.11ah is 7991 bytes.
While IPv6 is the main IP version used in IP-based IoT environments, While many IP-based IoT environments use IPv6, IPv4 can also be in
some IoT scenarios use IPv4. In IPv4, the MTU is 576 bytes. In use. In IPv4, the minimum MTU is 576 bytes. In order to avoid
order to avoid exceeding the IPv4 MTU, the MSS needs to be set to a exceeding the IPv4 MTU, the MSS needs to be set to a value not larger
value not larger than 536 bytes. Similarly to the recommendations than the IPv4 MTU minus 40 bytes. Similarly to the recommendations
given above for IPv6, a constrained device using IPv4 may advertise given above for IPv6, a constrained device using IPv4 may advertise
an MSS not larger than 516 bytes in order to accommodate unrequested an even smaller MSS in order to accommodate unrequested TCP options.
TCP options.
Finally, note that using larger MSS (to a suitable extent) may be Finally, note that using larger MSS (to a suitable extent) may be
beneficial, especially when transferring large payloads, as it beneficial, especially when transferring large payloads, as it
reduces the number of packets (and packet headers) required for a reduces the number of packets (and packet headers) required for a
given payload. given payload.
4.1.2. Explicit Congestion Notification (ECN) 4.1.2. Explicit Congestion Notification (ECN)
Explicit Congestion Notification (ECN) [RFC3168] ECN allows a router Explicit Congestion Notification (ECN) [RFC3168] ECN allows a router
to signal in the IP header of a packet that congestion is arising, to signal in the IP header of a packet that congestion is arising,
skipping to change at page 8, line 35 skipping to change at page 8, line 42
remarkable feature of ECN is that congestion can be signalled without remarkable feature of ECN is that congestion can be signalled without
incurring packet drops (which will lead to retransmissions and incurring packet drops (which will lead to retransmissions and
consumption of limited resources such as energy and bandwitdh). consumption of limited resources such as energy and bandwitdh).
ECN can further reduce packet losses since congestion control ECN can further reduce packet losses since congestion control
measures can be applied earlier [RFC2884]. Less lost packets implies measures can be applied earlier [RFC2884]. Less lost packets implies
that the number of retransmitted segments decreases, which is that the number of retransmitted segments decreases, which is
particularly beneficial in CNNs, where energy and bandwidth resources particularly beneficial in CNNs, where energy and bandwidth resources
are typically limited. Also, it makes sense to try to avoid packet are typically limited. Also, it makes sense to try to avoid packet
drops for transactional workloads with small data sizes, which are drops for transactional workloads with small data sizes, which are
typical for CNNs. In such traffic patterns, it is more difficult to typical for CNNs. In such traffic patterns, it is more difficult and
detect packet loss without retransmission timeouts (e.g., as there often impossible to detect packet loss without retransmission
may be no three duplicate ACKs). Any retransmission timeout slows timeouts (e.g., as there may be no three duplicate ACKs). Any
down the data transfer significantly. In addition, if the retransmission timeout slows down the data transfer significantly.
constrained device uses power saving techniques, a retransmission In addition, if the constrained device uses power saving techniques,
timeout will incur a wake-up action, in contrast to ACK clock- a retransmission timeout will incur a wake-up action, in contrast to
triggered sending. When the congestion window of a TCP sender has a ACK clock- triggered sending. When the congestion window of a TCP
size of one segment, the TCP sender resets the retransmit timer, and sender has a size of one segment and a TCP ACK with an ECN signal
the sender will only be able to send a new packet when the retransmit (ECE flag) arrives at the TCP sender, the TCP sender resets the
timer expires [RFC3168]. Effectively, the TCP sender reduces at that retransmit timer, and the sender will only be able to send a new
moment its sending rate from 1 segment per Round Trip Time (RTT) to 1 packet when the retransmit timer expires. Effectively, the TCP
segment per RTO, which can result in a very low throughput. In sender reduces at that moment its sending rate from 1 segment per
addition to better throughput, ECN can also help reducing latency and Round Trip Time (RTT) to 1 segment per RTO and reduces the sending
jitter. rate further on each ECN signal received in subsequent TCP ACKs.
Otherwise, if an ECN signal is not present in a subsequent TCP ACK
the TCP sender resumes the normal ACK-clocked transmission of
segments [RFC3168].
ECN can be incrementally deployed in the Internet. Guidance on ECN can be incrementally deployed in the Internet. Guidance on
configuration and usage of ECN is provided in [RFC7567]. Given the configuration and usage of ECN is provided in [RFC7567]. Given the
benefits, more and more TCP stacks in the Internet support ECN, and benefits, more and more TCP stacks in the Internet support ECN, and
it specifically makes sense to leverage ECN in controlled it specifically makes sense to leverage ECN in controlled
environments such as CNNs. Note, however, that supporting ECN environments such as CNNs. Note, however, that supporting ECN
increases implementation complexity. increases implementation complexity.
4.1.3. Explicit loss notifications 4.1.3. Explicit loss notifications
There has been a significant body of research on solutions capable of There has been a significant body of research on solutions capable of
explicitly indicating whether a TCP segment loss is due to explicitly indicating whether a TCP segment loss is due to
corruption, in order to avoid activation of congestion control corruption, in order to avoid activation of congestion control
mechanisms [ETEN] [RFC2757]. While such solutions may provide mechanisms [ETEN] [RFC2757]. While such solutions may provide
significant improvement, they have not been widely deployed and significant improvement, they have not been widely deployed and
remain as experimental work. In fact, as of today, the IETF has not remain as experimental work. In fact, as of today, the IETF has not
standardized any such solution. standardized any such solution.
4.2. TCP guidance for single-MSS windows and buffers 4.2. TCP guidance for single-segment stacks
This section discusses TCP stacks that focus on transferring a single This section discusses TCP stacks that allow transferring only a
MSS. More general guidance is provided in Section 4.3. single segment. More general guidance is provided in Section 4.3.
4.2.1. Single-MSS stacks - benefits and issues 4.2.1. Single-segment stacks - benefits and issues
A TCP stack can reduce the memory requirements by advertising a TCP A TCP stack can reduce the memory requirements by advertising a TCP
window size of one MSS, and also transmit at most one MSS of window size of one MSS, and also transmit at most one MSS of
unacknowledged data. In that case, both congestion and flow control unacknowledged data. In that case, both congestion and flow control
implementation is quite simple. Such a small receive and send window implementation are quite simple. Such a small receive and send
may be sufficient for simple message exchanges in the CNN space. window may be sufficient for simple message exchanges in the CNN
However, only using a window of one MSS can significantly affect space. However, only using a window of one MSS can significantly
performance. A stop-and-wait operation results in low throughput for affect performance. A stop-and-wait operation results in low
transfers that exceed the length of one MSS, e.g., a firmware throughput for transfers that exceed the length of one MSS, e.g., a
download. Furthermore, a single-MSS solution relies solely on timer- firmware download. Furthermore, a single-segment solution relies
based loss recovery, therefore missing the performance gain of Fast solely on timer-based loss recovery, therefore missing the
Retransmit and Fast Recovery (which require a larger window size, see performance gain of Fast Retransmit and Fast Recovery (which require
Subsection 4.3.1). a larger window size, see Subsection 4.3.1).
If CoAP is used over TCP with the default setting for NSTART in If CoAP is used over TCP with the default setting for NSTART in
[RFC7252], a CoAP endpoint is not allowed to send a new message to a [RFC7252], a CoAP endpoint is not allowed to send a new message to a
destination until a response for the previous message sent to that destination until a response for the previous message sent to that
destination has been received. This is equivalent to an application- destination has been received. This is equivalent to an application-
layer window size of 1. For this use of CoAP, a maximum TCP window layer window size of 1 data unit. For this use of CoAP, a maximum
of one MSS will be sufficient. TCP window of one MSS may be sufficient, as long as the CoAP message
size does not exceed one MSS.
4.2.2. TCP options for single-MSS stacks 4.2.2. TCP options for single-segment stacks
A TCP implementation needs to support, at a minimum, TCP options 2, 1 A TCP implementation needs to support, at a minimum, TCP options 2, 1
and 0. These are, respectively, the Maximum Segment Size (MSS) and 0. These are, respectively, the Maximum Segment Size (MSS)
option, the No-Operation option, and the End Of Option List marker option, the No-Operation option, and the End Of Option List marker
[RFC0793]. None of these are a substantial burden to support. These [RFC0793]. None of these are a substantial burden to support. These
options are sufficient for interoperability with a standard-compliant options are sufficient for interoperability with a standard-compliant
TCP endpoint, albeit many TCP stacks support additional options and TCP endpoint, albeit many TCP stacks support additional options and
can negotiate their use. A TCP implementation is permitted to can negotiate their use. A TCP implementation is permitted to
silently ignore all other TCP options. silently ignore all other TCP options.
A TCP implementation for a constrained device that uses a single-MSS A TCP implementation for a constrained device that uses a single-
TCP receive or transmit window size may not benefit from supporting segment TCP receive or transmit window size may not benefit from
the following TCP options: Window scale [RFC7323], TCP Timestamps supporting the following TCP options: Window scale [RFC7323], TCP
[RFC7323], Selective Acknowledgments (SACK) and SACK-Permitted Timestamps [RFC7323], Selective Acknowledgments (SACK) and SACK-
[RFC2018]. Also other TCP options may not be required on a Permitted [RFC2018]. Also other TCP options may not be required on a
constrained device with a very lightweight implementation. With constrained device with a very lightweight implementation. With
regard to the Window scale option, note that it is only useful if a regard to the Window scale option, note that it is only useful if a
window size greater than 64 kB is needed. window size greater than 64 kB is needed.
Note that a TCP sender can benefit from the TCP Timestamps option
[RFC7323] in detecting spurious RTOs. The latter are quite likely to
occur in CNN scenarios due to a number of reasons (e.g. route changes
in a multihop scenario, link layer retries, etc.). The header
overhead incurred by the Timestamps option (of up to 12 bytes) needs
to be taken into account.
One potentially relevant TCP option in the context of CNNs is TCP One potentially relevant TCP option in the context of CNNs is TCP
Fast Open (TFO) [RFC7413]. As described in Section 5.3, TFO can be Fast Open (TFO) [RFC7413]. As described in Section 5.3, TFO can be
used to address the problem of traversing middleboxes that perform used to address the problem of traversing middleboxes that perform
early filter state record deletion. early filter state record deletion.
4.2.3. Delayed Acknowledgments for single-MSS stacks 4.2.3. Delayed Acknowledgments for single-segment stacks
TCP Delayed Acknowledgments are meant to reduce the number of ACKs TCP Delayed Acknowledgments are meant to reduce the number of ACKs
sent within a TCP connection, thus reducing network overhead, but sent within a TCP connection, thus reducing network overhead, but
they may increase the time until a sender may receive an ACK. In they may increase the time until a sender may receive an ACK. In
general, usefulness of Delayed ACKs depends heavily on the usage general, usefulness of Delayed ACKs depends heavily on the usage
scenario. There can be interactions with stacks that use single-MSS scenario (see subsection 4.3.2). There can be interactions with
windows. single-segment stacks.
A device that advertises a single-MSS receive window should avoid use
of Delayed ACKs in order to avoid contributing unnecessary delay (of
up to 500 ms) to the RTT [RFC5681], which limits the throughput and
can increase the data delivery time.
A device that can send at most one MSS of data is significantly When traffic is unidirectional, if the sender can send at most one
affected if the receiver uses Delayed ACKs, e.g., if a TCP server or MSS of data or the receiver advertises a receive window not greater
receiver is outside the CNN. One known workaround is to split the than the MSS, Delayed ACKs may unnecessarily contribute delay (up to
data to be sent into two segments of smaller size. A standard 500 ms) to the RTT [RFC5681], which limits the throughput and can
increase data delivery time. Note that, in some cases, it may not be
possible to disable Delayed ACKs. One known workaround is to split
the data to be sent into two segments of smaller size. A standard
compliant TCP receiver will acknowledge the second MSS of data, which compliant TCP receiver will acknowledge the second MSS of data, which
can improve throughput. This "split hack" works if the TCP receiver can improve throughput. However, this 'split hack' may not always
uses Delayed ACKs, but the downside is the overhead of sending two IP work since a TCP receiver is required to acknowledge every second
packets instead of one. full-sized segment, but not two consecutive small segments.
Furthermore, the overhead of sending two IP packets instead of one is
another downside of the 'split hack'.
Similar issues happen when the sender uses the Nagle algorithm. Similar issues happen when the sender uses the Nagle algorithm.
Disabling the algorithm will not have impact if the sender can only Disabling the algorithm will not have impact if the sender can only
handle stop-and-wait operation. handle stop-and-wait operation.
4.2.4. RTO estimation for single-MSS stacks For request-response traffic, when the receiver uses Delayed ACKs, a
response to a data message can piggyback an ACK, as long as the
latter is sent before the Delayed ACK timer expires, thus avoiding
unnecessary pure ACKs. Disabling Delayed ACKs at the sender allows
an immediate ACK for the data segment carrying the response.
The Retransmission Timeout (RTO) estimation is one of the fundamental 4.2.4. RTO calculation for single-segment stacks
TCP algorithms. There is a fundamental trade-off: A short,
aggressive RTO behavior reduces wait time before retransmissions, but
it also increases the probability of spurious timeouts. The latter
lead to unnecessary waste of potentially scarce resources in CNNs
such as energy and bandwidth. In contrast, a conservative timeout
can result in long error recovery times and thus needlessly delay
data delivery.
[RFC6298] describes the standard TCP RTO algorithm. If a TCP sender The Retransmission Timeout (RTO) calculation is one of the
uses very small window size, and it cannot use Fast Retransmit/Fast fundamental TCP algorithms [RFC6298]. There is a fundamental trade-
Recovery or SACK, the Retransmission Timeout (RTO) algorithm has a off: A short, aggressive RTO behavior reduces wait time before
larger impact on performance than for a more powerful TCP stack. In retransmissions, but it also increases the probability of spurious
that case, RTO algorithm tuning may be considered, although careful timeouts. The latter lead to unnecessary waste of potentially scarce
assessment of possible drawbacks is recommended resources in CNNs such as energy and bandwidth. In contrast, a
[I-D.ietf-tcpm-rto-consider]. conservative timeout can result in long error recovery times and thus
needlessly delay data delivery.
If a TCP sender uses a very small window size, and it cannot benefit
from Fast Retransmit/Fast Recovery or SACK, the RTO algorithm has a
large impact on performance. In that case, RTO algorithm tuning may
be considered, although careful assessment of possible drawbacks is
recommended [I-D.ietf-tcpm-rto-consider].
As an example, an adaptive RTO algorithm for CoAP over UDP has been As an example, an adaptive RTO algorithm for CoAP over UDP has been
defined [I-D.ietf-core-cocoa] that has been found to perform well in defined that has been found to perform well in CNN scenarios
CNN scenarios [Commag]. [Commag].
4.3. General recommendations for TCP in CNNs 4.3. General recommendations for TCP in CNNs
This section summarizes some widely used techniques to improve TCP, This section summarizes some widely used techniques to improve TCP,
with a focus on their use in CNNs. The TCP extensions discussed here with a focus on their use in CNNs. The TCP extensions discussed here
are useful in a wide range of network scenarios, including CNNs. are useful in a wide range of network scenarios, including CNNs.
This section is not comprehensive. A comprehensive survey of TCP This section is not comprehensive. A comprehensive survey of TCP
extensions is published in [RFC7414]. extensions is published in [RFC7414].
4.3.1. Loss recovery and congestion/flow control 4.3.1. Loss recovery and congestion/flow control
Devices that have enough memory to allow a larger (i.e. more than 3 Devices that have enough memory to allow a larger (i.e. more than 3
MSS of data) TCP window size can leverage a more efficient loss MSS of data) TCP window size can leverage a more efficient loss
recovery than the timer-based approach used for smaller TCP window recovery than the timer-based approach used for smaller TCP window
size (see Subsection 3.2.1) by using Fast Retransmit and Fast size (see Subsection 3.2.1) by using Fast Retransmit and Fast
Recovery [RFC5681], at the expense of slightly greater complexity and Recovery [RFC5681], at the expense of slightly greater complexity and
skipping to change at page 11, line 43 skipping to change at page 12, line 15
This section is not comprehensive. A comprehensive survey of TCP This section is not comprehensive. A comprehensive survey of TCP
extensions is published in [RFC7414]. extensions is published in [RFC7414].
4.3.1. Loss recovery and congestion/flow control 4.3.1. Loss recovery and congestion/flow control
Devices that have enough memory to allow a larger (i.e. more than 3 Devices that have enough memory to allow a larger (i.e. more than 3
MSS of data) TCP window size can leverage a more efficient loss MSS of data) TCP window size can leverage a more efficient loss
recovery than the timer-based approach used for smaller TCP window recovery than the timer-based approach used for smaller TCP window
size (see Subsection 3.2.1) by using Fast Retransmit and Fast size (see Subsection 3.2.1) by using Fast Retransmit and Fast
Recovery [RFC5681], at the expense of slightly greater complexity and Recovery [RFC5681], at the expense of slightly greater complexity and
TCB size. Assuming that Delayed ACKs are used by the receiver, the TCB size. Assuming that Delayed ACKs are used by the receiver, a
mentioned algorithms work efficiently for window sizes of at least 5 window size of up to 5 MSS is required for Fast Retransmit and Fast
MSS: If in a given TCP transmission of segments 1, 2, 3, 4, 5, and 6 Recovery to work efficiently: If in a given TCP transmission of
the segment 2 gets lost, the sender should get an ACK for segment 1 segments 1, 2, 3, 4, 5, and 6 segment 2 gets lost, and the ACK for
when 3 arrives and duplicate acknowledgements when 4, 5, and 6 segment 1 is held by the Delayed ACK timer, then the sender should
arrive. It will retransmit segment 2 when the third duplicate ACK get an ACK for segment 1 when 3 arrives and duplicate ACKs when
arrives. In order to have segment 2, 3, 4, 5, and 6 sent, the window segments 4, 5, and 6 arrive. It will retransmit segment 2 when the
has to be at least 5 MSS. With an MSS of 1220 byte, a buffer of the third duplicate ACK arrives. In order to have segments 2, 3, 4, 5,
size of 5 MSS would require 6100 bytes. and 6 sent, the window has to be of at least 5 MSS. With an MSS of
1220 bytes, a buffer of a size of 5 MSS would require 6100 bytes.
For bulk data transfers further TCP improvements may also be useful, Further TCP improvements such as Limited Transmit [RFC3042] may also
such as limited transmit [RFC3042]. be useful for any transfer that has more than one segment in flight.
Small transfers tend to benefit more from Limited Transmit, because
they are more likely to not receive enough duplicate ACKs. Assuming
the example in the previous paragraph, Limited Transmit allows
sending 5 MSS with a congestion window (cwnd) of 3 segments, plus two
additional segments for each one of the first two duplicate ACKs.
When a multiple-segment window is used, the receiver will need to
manage the reception of possible out-of-order received segments,
requiring sufficient buffer space.
4.3.1.1. Selective Acknowledgments (SACK) 4.3.1.1. Selective Acknowledgments (SACK)
If a device with less severe memory and processing constraints can If a device with less severe memory and processing constraints can
afford advertising a TCP window size of several MSS, it makes sense afford advertising a TCP window size of several MSS, it makes sense
to support the SACK option to improve performance. SACK allows a to support the SACK option to improve performance. SACK allows a
data receiver to inform the data sender of non-contiguous data blocks data receiver to inform the data sender of non-contiguous data blocks
received, thus a sender (having previously sent the SACK-Permitted received, thus a sender (having previously sent the SACK-Permitted
option) can avoid performing unnecessary retransmissions, saving option) can avoid performing unnecessary retransmissions, saving
energy and bandwidth, as well as reducing latency. SACK is energy and bandwidth, as well as reducing latency. In addition, SACK
particularly useful for bulk data transfers. The receiver supporting often allows for faster loss recovery when there is more than one
SACK will need to manage the reception of possible out-of-order lost segment in a window of data, since with SACK recovery requires
received segments, requiring sufficient buffer space. SACK adds less RTTs. SACK is particularly useful for bulk data transfers. A
8*n+2 bytes to the TCP header, where n denotes the number of data receiver supporting SACK will need to keep track of the SACK blocks
blocks received, up to 4 blocks. For a low number of out-of-order that need to be received. The sender will also need to keep track of
segments, the header overhead penalty of SACK is compensated by which data segments need to be resent after learning which data
avoiding unnecessary retransmissions. blocks are missing at the receiver. SACK adds 8*n+2 bytes to the TCP
header, where n denotes the number of data blocks received, up to 4
blocks. For a low number of out-of-order segments, the header
overhead penalty of SACK is compensated by avoiding unnecessary
retransmissions. When the sender discovers the data blocks that have
already been received, it needs to also store the necessary state to
avoid unnecessary retransmission of data segments that have already
been received.
4.3.2. Delayed Acknowledgments 4.3.2. Delayed Acknowledgments
For certain traffic patterns, Delayed ACKs may have a detrimental For certain traffic patterns, Delayed ACKs may have a detrimental
effect, as already noted in Section 4.2.3. Advanced TCP stacks may effect, as already noted in Section 4.2.3. Advanced TCP stacks may
use heuristics to determine the maximum delay for an ACK. For CNNs, use heuristics to determine the maximum delay for an ACK. For CNNs,
the recommendation depends on the expected communication patterns. the recommendation depends on the expected communication patterns.
When traffic over a CNN is expected to mostly be unidirectional When traffic over a CNN is expected to mostly be unidirectional
messages with a size typically up to one MSS, and the time between messages with a size typically up to one MSS, and the time between
two consecutive message transmissions is greater than the delayed ACK two consecutive message transmissions is greater than the Delayed ACK
timeout, it may make sense to use a small timeout or disable delayed timeout, it may make sense to use a small timeout or disable Delayed
ACKs at the receiver. This avoids incurring additional delay, as ACKs at the receiver. This avoids incurring additional delay, as
well as the energy consumption of the sender (which might e.g. keep well as the energy consumption of the sender (which might e.g. keep
its radio interface in receive mode) during that time. Note that its radio interface in receive mode) during that time. Note that
disabling delayed ACKs may only be possible if the peer device is disabling Delayed ACKs may only be possible if the peer device is
administered by the same entity managing the constrained device. For administered by the same entity managing the constrained device. For
request-response traffic, enabling delayed ACKs is recommended, in request-response traffic, enabling Delayed ACKs is recommended, in
order to allow combining a response with the ACK into a single order to allow combining a response with the ACK into a single
segment, thus increasing efficiency. segment, thus increasing efficiency. In this case, disabling Delayed
ACKs at the sender allows an immediate ACK for the data segment
carrying the response.
In contrast, Delayed ACKs allow to reduce the number of ACKs in bulk In contrast, Delayed ACKs allow to reduce the number of ACKs in bulk
transfer type of traffic, e.g. for firmware/software updates or for transfer type of traffic, e.g. for firmware/software updates or for
transferring larger data units containing a batch of sensor readings. transferring larger data units containing a batch of sensor readings.
Note that, in many scenarios, the peer that a constrained device Note that, in many scenarios, the peer that a constrained device
communicates with will be a general purpose system that communicates communicates with will be a general purpose system that communicates
with both constrained and unconstrained devices. Since delayed ACKs with both constrained and unconstrained devices. Since delayed ACKs
are often configured through system-wide parameters, delayed ACKs are often configured through system-wide parameters, delayed ACKs
behavior at the peer will be the same regardless of the nature of the behavior at the peer will be the same regardless of the nature of the
endpoints it talks to. Such a peer will typically have delayed ACKs endpoints it talks to. Such a peer will typically have delayed ACKs
enabled. enabled.
4.3.3. Initial Window
RFC 5681 specifies a TCP Initial Window (IW) of roughly 4 kB
[RFC5681]. Subsequently, RFC 6928 defined an experimental new value
for the IW, which in practice will result in an IW of 10 MSS
[RFC6928]. The latter is nowadays used in many TCP implementations.
Note that a 10-MSS IW was recommended for resource-rich environments
(e.g. broadband environments), which are significantly different from
CNNs. In CNNs, many application layer data units are relatively
small (e.g. below one MSS). However, larger objects (e.g. large
files containing sensor readings, firmware updates, etc.) may also
need to be transferred in CNNs. If such a large object is
transferred in CNNs, with an IW setting of 10 MSS, there is
significant buffer overflow risk. In order to avoid such problem, in
CNNs the IW needs to be carefully set, based on device and network
resource constraints. In many cases, a safe IW setting will be
smaller than 10 MSS.
5. TCP usage recommendations in CNNs 5. TCP usage recommendations in CNNs
This section discusses how a TCP stack can be used by applications This section discusses how TCP can be used by applications that are
that are developed for CNN scenarios. These remarks are by and large developed for CNN scenarios. These remarks are by and large
independent of how TCP is exactly implemented. independent of how TCP is exactly implemented.
5.1. TCP connection initiation 5.1. TCP connection initiation
In the constrained device to unconstrained device scenario In the constrained device to unconstrained device scenario
illustrated above, a TCP connection is typically initiated by the illustrated above, a TCP connection is typically initiated by the
constrained device, in order for this device to support possible constrained device, in order for this device to support possible
sleep periods to save energy. sleep periods to save energy.
5.2. Number of concurrent connections 5.2. Number of concurrent connections
skipping to change at page 13, line 36 skipping to change at page 14, line 47
the internal TCP implementation, each connection may result in the internal TCP implementation, each connection may result in
further memory overhead, and connections may compete for scarce further memory overhead, and connections may compete for scarce
resources (e.g. further memory overhead for send and receive buffers, resources (e.g. further memory overhead for send and receive buffers,
etc). etc).
A careful application design may try to keep the number of concurrent A careful application design may try to keep the number of concurrent
connections as small as possible. A client can for instance limit connections as small as possible. A client can for instance limit
the number of simultaneous open connections that it maintains to a the number of simultaneous open connections that it maintains to a
given server. Multiple connections could for instance be used to given server. Multiple connections could for instance be used to
avoid the "head-of-line blocking" problem in an application transfer. avoid the "head-of-line blocking" problem in an application transfer.
However, in addition to comsuming resources, using multiple However, in addition to consuming resources, using multiple
connections can also cause undesirable side effects in congested connections can also cause undesirable side effects in congested
networks. For example, the HTTP/1.1 specification encourages clients networks. For example, the HTTP/1.1 specification encourages clients
to be conservative when opening multiple connections [RFC7230]. to be conservative when opening multiple connections [RFC7230].
Furthermore, each new connection will start with a 3-way handshake, Furthermore, each new connection will start with a 3-way handshake,
therefore increasing message overhead. therefore increasing message overhead.
Being conservative when opening multiple TCP connections is of Being conservative when opening multiple TCP connections is of
particular importance in Constrained-Node Networks. particular importance in Constrained-Node Networks.
5.3. TCP connection lifetime 5.3. TCP connection lifetime
In order to minimize message overhead, it makes sense to keep a TCP In order to minimize message overhead, it makes sense to keep a TCP
connection open as long as the two TCP endpoints have more data to connection open as long as the two TCP endpoints have more data to
skipping to change at page 14, line 7 skipping to change at page 15, line 20
5.3. TCP connection lifetime 5.3. TCP connection lifetime
In order to minimize message overhead, it makes sense to keep a TCP In order to minimize message overhead, it makes sense to keep a TCP
connection open as long as the two TCP endpoints have more data to connection open as long as the two TCP endpoints have more data to
send. If applications exchange data rather infrequently, i.e., if send. If applications exchange data rather infrequently, i.e., if
TCP connections would stay idle for a long time, the idle time can TCP connections would stay idle for a long time, the idle time can
result in problems. For instance, certain middleboxes such as result in problems. For instance, certain middleboxes such as
firewalls or NAT devices are known to delete state records after an firewalls or NAT devices are known to delete state records after an
inactivity interval. RFC 5382 specifies a minimum value for such inactivity interval. RFC 5382 specifies a minimum value for such
interval of 124 minutes. A mean TCP NAT binding timeout of 386 interval of 124 minutes. Measurement studies have reported that TCP
minutes has been reported, while in some cases, inactivity timeouts NAT binding timeouts are highly variable across devices, with a
are in the order of a few minutes [HomeGateway]. The timeout median around 60 minutes, the shortest timeout being around 2
duration used by a middlebox implementation may not be known to the minutes, and more than 50% of the devices with a timeout shorter than
TCP endpoints. the aforementioned minimum timeout of 124 minutes [HomeGateway]. The
timeout duration used by a middlebox implementation may not be known
to the TCP endpoints.
In CNNs, such middleboxes may e.g. be present at the boundary between In CNNs, such middleboxes may e.g. be present at the boundary between
the CNN and other networks. If the middlebox can be optimized for the CNN and other networks. If the middlebox can be optimized for
CNN use cases, it makes sense to increase the initial value for CNN use cases, it makes sense to increase the initial value for
filter state inactivity timers to avoid problems with idle filter state inactivity timers to avoid problems with idle
connections. Apart from that, this problem can be dealt with by connections. Apart from that, this problem can be dealt with by
different connection handling strategies, each having pros and cons. different connection handling strategies, each having pros and cons.
One approach for infrequent data transfer is to use short-lived TCP One approach for infrequent data transfer is to use short-lived TCP
connections. Instead of trying to maintain a TCP connection for long connections. Instead of trying to maintain a TCP connection for long
skipping to change at page 15, line 14 skipping to change at page 16, line 26
Another approach is to use long-lived TCP connections with Another approach is to use long-lived TCP connections with
application-layer heartbeat messages. Various application protocols application-layer heartbeat messages. Various application protocols
support such heartbeat messages (e.g. CoAP over TCP [RFC8323]). support such heartbeat messages (e.g. CoAP over TCP [RFC8323]).
Periodic application-layer heartbeats can prevent early filter state Periodic application-layer heartbeats can prevent early filter state
record deletion in middleboxes. If the TCP binding timeout for a record deletion in middleboxes. If the TCP binding timeout for a
middlebox to be traversed by a given connection is known, middlebox middlebox to be traversed by a given connection is known, middlebox
filter state deletion will be avoided if the heartbeat period is filter state deletion will be avoided if the heartbeat period is
lower than the middlebox TCP binding timeout. Otherwise, the lower than the middlebox TCP binding timeout. Otherwise, the
implementer needs to take into account that middlebox TCP binding implementer needs to take into account that middlebox TCP binding
timeouts fall in a wide range of possible values [HomeGateway]. One timeouts fall in a wide range of possible values [HomeGateway], and
specific advantage of Heartbeat messages is that they also allow it may be hard to find a proper heartbeat period for application-
layer heartbeat messages.
One specific advantage of Heartbeat messages is that they also allow
aliveness checks at the application level. In general, it makes aliveness checks at the application level. In general, it makes
sense to realize aliveness checks at the highest protocol layer sense to realize aliveness checks at the highest protocol layer
possible that is meaningful to the application, in order to maximize possible that is meaningful to the application, in order to maximize
the depth of the aliveness check. In addition, timely detection of a the depth of the aliveness check. In addition, timely detection of a
dead peer may allow savings in terms of TCB memory use. However, the dead peer may allow savings in terms of TCB memory use. However, the
transmission of heartbeat messages consumes resources. This aspect transmission of heartbeat messages consumes resources. This aspect
needs to be assessed carefully, considering the characteristics of needs to be assessed carefully, considering the characteristics of
each specific CNN. each specific CNN.
A TCP implementation may also be able to send "keep-alive" segments A TCP implementation may also be able to send "keep-alive" segments
skipping to change at page 16, line 6 skipping to change at page 17, line 24
has a size of 16-20 bytes. has a size of 16-20 bytes.
For the mechanisms discussed in this document, the corresponding For the mechanisms discussed in this document, the corresponding
considerations apply. For instance, if TFO is used, the security considerations apply. For instance, if TFO is used, the security
considerations of [RFC7413] apply. considerations of [RFC7413] apply.
Constrained devices are expected to support smaller TCP window sizes Constrained devices are expected to support smaller TCP window sizes
than less limited devices. In such conditions, segment than less limited devices. In such conditions, segment
retransmission triggered by RTO expiration is expected to be retransmission triggered by RTO expiration is expected to be
relatively frequent, due to lack of (enough) duplicate ACKs, relatively frequent, due to lack of (enough) duplicate ACKs,
especially when a constrained device uses a single-MSS window size. especially when a constrained device uses a single-segment
For this reason, constrained devices running TCP may appear as implementation. For this reason, constrained devices running TCP may
particularly appealing victims of the so-called "shrew" Denial of appear as particularly appealing victims of the so-called "shrew"
Service (DoS) attack [shrew], whereby one or more sources generate a Denial of Service (DoS) attack [shrew], whereby one or more sources
packet spike targetted to coincide with consecutive RTO-expiration- generate a packet spike targetted to coincide with consecutive RTO-
triggered retry attempts of a victim node. Note that the attack may expiration-triggered retry attempts of a victim node. Note that the
be performed by Internet-connected devices, including constrained attack may be performed by Internet-connected devices, including
devices in the same CNN as the victim, as well as remote ones. constrained devices in the same CNN as the victim, as well as remote
Mitigation techniques include RTO randomization and attack blocking ones. Mitigation techniques include RTO randomization and attack
by routers able to detect shrew attacks based on their traffic blocking by routers able to detect shrew attacks based on their
pattern. traffic pattern.
7. Acknowledgments 7. Acknowledgments
Carles Gomez has been funded in part by the Spanish Government Carles Gomez has been funded in part by the Spanish Government
(Ministerio de Educacion, Cultura y Deporte) through the Jose (Ministerio de Educacion, Cultura y Deporte) through the Jose
Castillejo grants CAS15/00336 and and CAS18/00170, and by European Castillejo grants CAS15/00336 and and CAS18/00170, and by European
Regional Development Fund (ERDF) and the Spanish Government through Regional Development Fund (ERDF) and the Spanish Government through
project TEC2016-79988-P, AEI/FEDER, UE. Part of his contribution to project TEC2016-79988-P, AEI/FEDER, UE. Part of his contribution to
this work has been carried out during his stays as a visiting scholar this work has been carried out during his stays as a visiting scholar
at the Computer Laboratory of the University of Cambridge. at the Computer Laboratory of the University of Cambridge.
The authors appreciate the feedback received for this document. The The authors appreciate the feedback received for this document. The
following folks provided comments that helped improve the document: following folks provided comments that helped improve the document:
Carsten Bormann, Zhen Cao, Wei Genyu, Ari Keranen, Abhijan Carsten Bormann, Zhen Cao, Wei Genyu, Ari Keranen, Abhijan
Bhattacharyya, Andres Arcia-Moret, Yoshifumi Nishida, Joe Touch, Fred Bhattacharyya, Andres Arcia-Moret, Yoshifumi Nishida, Joe Touch, Fred
Baker, Nik Sultana, Kerry Lynn, Erik Nordmark, Markku Kojo, Hannes Baker, Nik Sultana, Kerry Lynn, Erik Nordmark, Markku Kojo, Hannes
Tschofenig, David Black, Yoshifumi Nishida, Ilpo Jarvinen, Emmanuel Tschofenig, David Black, Yoshifumi Nishida, Ilpo Jarvinen, Emmanuel
Baccelli, and Stuart Cheshire. Simon Brummer provided details, and Baccelli, Stuart Cheshire, Gorry Fairhurst, and Ingemar Johansson.
kindly performed RAM and ROM usage measurements, on the RIOT TCP
implementation. Xavi Vilajosana provided details on the OpenWSN TCP Simon Brummer provided details, and kindly performed RAM and ROM
implementation. Rahul Jadhav kindly performed code size measurements usage measurements, on the RIOT TCP implementation. Xavi Vilajosana
on the Contiki-NG and lwIP 2.1.2 TCP implementations. He also provided details on the OpenWSN TCP implementation. Rahul Jadhav
provided details on the uIP TCP implementation. kindly performed code size measurements on the Contiki-NG and lwIP
2.1.2 TCP implementations. He also provided details on the uIP TCP
implementation.
8. Annex. TCP implementations for constrained devices 8. Annex. TCP implementations for constrained devices
This section overviews the main features of TCP implementations for This section overviews the main features of TCP implementations for
constrained devices. The survey is limited to open source stacks constrained devices. The survey is limited to open source stacks
with small footprint. It is not meant to be all-encompassing. For with small footprint. It is not meant to be all-encompassing. For
more powerful embedded systems (e.g., with 32-bit processors), there more powerful embedded systems (e.g., with 32-bit processors), there
are further stacks that comprehensively implement TCP. On the other are further stacks that comprehensively implement TCP. On the other
hand, please be aware that this Annex is based on information hand, please be aware that this Annex is based on information
available as of the writing. available as of the writing.
skipping to change at page 18, line 4 skipping to change at page 19, line 26
fast retransmit and fast recovery. SACK and Window Scale support has fast retransmit and fast recovery. SACK and Window Scale support has
been recently added to lwIP. been recently added to lwIP.
8.3. RIOT 8.3. RIOT
The RIOT TCP implementation (called GNRC TCP) has been designed for The RIOT TCP implementation (called GNRC TCP) has been designed for
Class 1 devices [RFC 7228]. The main target platforms are 8- and Class 1 devices [RFC 7228]. The main target platforms are 8- and
16-bit microcontrollers, with 32-bit platforms also supported. GNRC 16-bit microcontrollers, with 32-bit platforms also supported. GNRC
TCP offers a similar function set as uIP, but it provides and TCP offers a similar function set as uIP, but it provides and
maintains an independent receive buffer for each connection. In maintains an independent receive buffer for each connection. In
contrast to uIP, retransmission is also handled by GNRC TCP. GNRC contrast to uIP, retransmission is also handled by GNRC TCP. For
TCP uses a single-MSS window size, which simplifies the simplicity, GNRC TCP uses a single-segment implementation. The
implementation. The application programmer does not need to know application programmer does not need to know anything about the TCP
anything about the TCP internals, therefore GNRC TCP can be seen as a internals, therefore GNRC TCP can be seen as a user-friendly uIP TCP
user-friendly uIP TCP implementation. implementation.
The MSS is set on connections establishment and cannot be changed The MSS is set on connections establishment and cannot be changed
during connection lifetime. GNRC TCP allows multiple connections in during connection lifetime. GNRC TCP allows multiple connections in
parallel, but each TCB must be allocated somewhere in the system. By parallel, but each TCB must be allocated somewhere in the system. By
default there is only enough memory allocated for a single TCP default there is only enough memory allocated for a single TCP
connection, but it can be increased at compile time if the user needs connection, but it can be increased at compile time if the user needs
multiple parallel connections. multiple parallel connections.
The RIOT TCP implementation offers an optional POSIX socket wrapper The RIOT TCP implementation offers an optional POSIX socket wrapper
that enables POSIX compliance, if needed. that enables POSIX compliance, if needed.
skipping to change at page 18, line 40 skipping to change at page 20, line 13
Instead, it will immediately dispatch new, in-order data to the Instead, it will immediately dispatch new, in-order data to the
application and otherwise drop the segment. A send buffer is application and otherwise drop the segment. A send buffer is
provided by the application. Multiple TCP connections are possible. provided by the application. Multiple TCP connections are possible.
Recently there has been little further work on the stack. Recently there has been little further work on the stack.
8.5. FreeRTOS 8.5. FreeRTOS
FreeRTOS is a real-time operating system kernel for embedded devices FreeRTOS is a real-time operating system kernel for embedded devices
that is supported by 16- and 32-bit microprocessors. Its TCP that is supported by 16- and 32-bit microprocessors. Its TCP
implementation is based on multiple-segment window size, although a implementation is based on multiple-segment window size, although a
'Tiny-TCP' option, which is a single-MSS variant, can be enabled. 'Tiny-TCP' option, which is a single-segment variant, can be enabled.
Delayed ACKs are supported, with a 20-ms Delayed ACK timer as a Delayed ACKs are supported, with a 20-ms Delayed ACK timer as a
technique intended 'to gain performance'. technique intended 'to gain performance'.
8.6. uC/OS 8.6. uC/OS
uC/OS is a real-time operating system kernel for embedded devices, uC/OS is a real-time operating system kernel for embedded devices,
which is maintained by Micrium. uC/OS is intended for 8-, 16- and which is maintained by Micrium. uC/OS is intended for 8-, 16- and
32-bit microprocessors. The uC/OS TCP implementation supports a 32-bit microprocessors. The uC/OS TCP implementation supports a
multiple-segment window size. multiple-segment window size.
skipping to change at page 21, line 39 skipping to change at page 23, line 39
Emmanuel Baccelli Emmanuel Baccelli
9.6. Changes between -05 and -06 9.6. Changes between -05 and -06
o Incorporated suggestions by Stuart Cheshire o Incorporated suggestions by Stuart Cheshire
9.7. Changes between -06 and -07 9.7. Changes between -06 and -07
o Addressed comments by Gorry Fairhurst o Addressed comments by Gorry Fairhurst
9.8. Changes between -07 and -08
o Addressed WGLC comments by Ilpo Jarvinen, Markku Kojo and Ingemar
Johansson throughout the document, including the addition of a new
subsection on Initial Window considerations.
10. References 10. References
10.1. Normative References 10.1. Normative References
[RFC0793] Postel, J., "Transmission Control Protocol", STD 7, [RFC0793] Postel, J., "Transmission Control Protocol", STD 7,
RFC 793, DOI 10.17487/RFC0793, September 1981, RFC 793, DOI 10.17487/RFC0793, September 1981,
<https://www.rfc-editor.org/info/rfc793>. <https://www.rfc-editor.org/info/rfc793>.
[RFC1122] Braden, R., Ed., "Requirements for Internet Hosts - [RFC1122] Braden, R., Ed., "Requirements for Internet Hosts -
Communication Layers", STD 3, RFC 1122, Communication Layers", STD 3, RFC 1122,
skipping to change at page 22, line 48 skipping to change at page 25, line 5
[RFC5925] Touch, J., Mankin, A., and R. Bonica, "The TCP [RFC5925] Touch, J., Mankin, A., and R. Bonica, "The TCP
Authentication Option", RFC 5925, DOI 10.17487/RFC5925, Authentication Option", RFC 5925, DOI 10.17487/RFC5925,
June 2010, <https://www.rfc-editor.org/info/rfc5925>. June 2010, <https://www.rfc-editor.org/info/rfc5925>.
[RFC6298] Paxson, V., Allman, M., Chu, J., and M. Sargent, [RFC6298] Paxson, V., Allman, M., Chu, J., and M. Sargent,
"Computing TCP's Retransmission Timer", RFC 6298, "Computing TCP's Retransmission Timer", RFC 6298,
DOI 10.17487/RFC6298, June 2011, DOI 10.17487/RFC6298, June 2011,
<https://www.rfc-editor.org/info/rfc6298>. <https://www.rfc-editor.org/info/rfc6298>.
[RFC6928] Chu, J., Dukkipati, N., Cheng, Y., and M. Mathis,
"Increasing TCP's Initial Window", RFC 6928,
DOI 10.17487/RFC6928, April 2013,
<https://www.rfc-editor.org/info/rfc6928>.
[RFC7228] Bormann, C., Ersue, M., and A. Keranen, "Terminology for [RFC7228] Bormann, C., Ersue, M., and A. Keranen, "Terminology for
Constrained-Node Networks", RFC 7228, Constrained-Node Networks", RFC 7228,
DOI 10.17487/RFC7228, May 2014, DOI 10.17487/RFC7228, May 2014,
<https://www.rfc-editor.org/info/rfc7228>. <https://www.rfc-editor.org/info/rfc7228>.
[RFC7323] Borman, D., Braden, B., Jacobson, V., and R. [RFC7323] Borman, D., Braden, B., Jacobson, V., and R.
Scheffenegger, Ed., "TCP Extensions for High Performance", Scheffenegger, Ed., "TCP Extensions for High Performance",
RFC 7323, DOI 10.17487/RFC7323, September 2014, RFC 7323, DOI 10.17487/RFC7323, September 2014,
<https://www.rfc-editor.org/info/rfc7323>. <https://www.rfc-editor.org/info/rfc7323>.
skipping to change at page 23, line 41 skipping to change at page 26, line 5
Haetoenen, S., Nyrhinen, A., Eggert, L., Strowes, S., Haetoenen, S., Nyrhinen, A., Eggert, L., Strowes, S.,
Sarolahti, P., and M. Kojo, "An Experimental Study of Home Sarolahti, P., and M. Kojo, "An Experimental Study of Home
Gateway Characteristics", Proceedings of the 10th ACM Gateway Characteristics", Proceedings of the 10th ACM
SIGCOMM conference on Internet measurement 2010. SIGCOMM conference on Internet measurement 2010.
[I-D.delcarpio-6lo-wlanah] [I-D.delcarpio-6lo-wlanah]
Vega, L., Robles, I., and R. Morabito, "IPv6 over Vega, L., Robles, I., and R. Morabito, "IPv6 over
802.11ah", draft-delcarpio-6lo-wlanah-01 (work in 802.11ah", draft-delcarpio-6lo-wlanah-01 (work in
progress), October 2015. progress), October 2015.
[I-D.ietf-core-cocoa]
Bormann, C., Betzler, A., Gomez, C., and I. Demirkol,
"CoAP Simple Congestion Control/Advanced", draft-ietf-
core-cocoa-03 (work in progress), February 2018.
[I-D.ietf-tcpm-rto-consider] [I-D.ietf-tcpm-rto-consider]
Allman, M., "Retransmission Timeout Requirements", draft- Allman, M., "Retransmission Timeout Requirements", draft-
ietf-tcpm-rto-consider-08 (work in progress), February ietf-tcpm-rto-consider-08 (work in progress), February
2019. 2019.
[IntComp] C. Gomez, A. Arcia-Moret, J. Crowcroft, "TCP in the [IntComp] C. Gomez, A. Arcia-Moret, J. Crowcroft, "TCP in the
Internet of Things: from ostracism to prominence", IEEE Internet of Things: from ostracism to prominence", IEEE
Internet Computing, January-February 2018. Internet Computing, January-February 2018.
[RFC2757] Montenegro, G., Dawkins, S., Kojo, M., Magret, V., and N. [RFC2757] Montenegro, G., Dawkins, S., Kojo, M., Magret, V., and N.
skipping to change at page 24, line 45 skipping to change at page 27, line 5
[RFC6092] Woodyatt, J., Ed., "Recommended Simple Security [RFC6092] Woodyatt, J., Ed., "Recommended Simple Security
Capabilities in Customer Premises Equipment (CPE) for Capabilities in Customer Premises Equipment (CPE) for
Providing Residential IPv6 Internet Service", RFC 6092, Providing Residential IPv6 Internet Service", RFC 6092,
DOI 10.17487/RFC6092, January 2011, DOI 10.17487/RFC6092, January 2011,
<https://www.rfc-editor.org/info/rfc6092>. <https://www.rfc-editor.org/info/rfc6092>.
[RFC6120] Saint-Andre, P., "Extensible Messaging and Presence [RFC6120] Saint-Andre, P., "Extensible Messaging and Presence
Protocol (XMPP): Core", RFC 6120, DOI 10.17487/RFC6120, Protocol (XMPP): Core", RFC 6120, DOI 10.17487/RFC6120,
March 2011, <https://www.rfc-editor.org/info/rfc6120>. March 2011, <https://www.rfc-editor.org/info/rfc6120>.
[RFC6282] Hui, J., Ed. and P. Thubert, "Compression Format for IPv6
Datagrams over IEEE 802.15.4-Based Networks", RFC 6282,
DOI 10.17487/RFC6282, September 2011,
<https://www.rfc-editor.org/info/rfc6282>.
[RFC6550] Winter, T., Ed., Thubert, P., Ed., Brandt, A., Hui, J.,
Kelsey, R., Levis, P., Pister, K., Struik, R., Vasseur,
JP., and R. Alexander, "RPL: IPv6 Routing Protocol for
Low-Power and Lossy Networks", RFC 6550,
DOI 10.17487/RFC6550, March 2012,
<https://www.rfc-editor.org/info/rfc6550>.
[RFC6606] Kim, E., Kaspar, D., Gomez, C., and C. Bormann, "Problem [RFC6606] Kim, E., Kaspar, D., Gomez, C., and C. Bormann, "Problem
Statement and Requirements for IPv6 over Low-Power Statement and Requirements for IPv6 over Low-Power
Wireless Personal Area Network (6LoWPAN) Routing", Wireless Personal Area Network (6LoWPAN) Routing",
RFC 6606, DOI 10.17487/RFC6606, May 2012, RFC 6606, DOI 10.17487/RFC6606, May 2012,
<https://www.rfc-editor.org/info/rfc6606>. <https://www.rfc-editor.org/info/rfc6606>.
[RFC6775] Shelby, Z., Ed., Chakrabarti, S., Nordmark, E., and C.
Bormann, "Neighbor Discovery Optimization for IPv6 over
Low-Power Wireless Personal Area Networks (6LoWPANs)",
RFC 6775, DOI 10.17487/RFC6775, November 2012,
<https://www.rfc-editor.org/info/rfc6775>.
[RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer [RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer
Protocol (HTTP/1.1): Message Syntax and Routing", Protocol (HTTP/1.1): Message Syntax and Routing",
RFC 7230, DOI 10.17487/RFC7230, June 2014, RFC 7230, DOI 10.17487/RFC7230, June 2014,
<https://www.rfc-editor.org/info/rfc7230>. <https://www.rfc-editor.org/info/rfc7230>.
[RFC7252] Shelby, Z., Hartke, K., and C. Bormann, "The Constrained [RFC7252] Shelby, Z., Hartke, K., and C. Bormann, "The Constrained
Application Protocol (CoAP)", RFC 7252, Application Protocol (CoAP)", RFC 7252,
DOI 10.17487/RFC7252, June 2014, DOI 10.17487/RFC7252, June 2014,
<https://www.rfc-editor.org/info/rfc7252>. <https://www.rfc-editor.org/info/rfc7252>.
 End of changes. 59 change blocks. 
202 lines changed or deleted 296 lines changed or added

This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/