--- 1/draft-ietf-intarea-frag-fragile-06.txt 2019-01-30 13:13:11.446585390 -0800 +++ 2/draft-ietf-intarea-frag-fragile-07.txt 2019-01-30 13:13:11.502586760 -0800 @@ -1,27 +1,27 @@ Internet Area WG R. Bonica Internet-Draft Juniper Networks Intended status: Best Current Practice F. Baker -Expires: August 2, 2019 Unaffiliated +Expires: August 3, 2019 Unaffiliated G. Huston APNIC R. Hinden Check Point Software O. Troan Cisco F. Gont SI6 Networks - January 29, 2019 + January 30, 2019 IP Fragmentation Considered Fragile - draft-ietf-intarea-frag-fragile-06 + draft-ietf-intarea-frag-fragile-07 Abstract This document describes IP fragmentation and explains how it reduces the reliability of Internet communication. This document also proposes alternatives to IP fragmentation and provides recommendations for developers and network operators. Status of This Memo @@ -32,21 +32,21 @@ Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." - This Internet-Draft will expire on August 2, 2019. + This Internet-Draft will expire on August 3, 2019. Copyright Notice Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents @@ -61,47 +61,48 @@ 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 2. IP Fragmentation . . . . . . . . . . . . . . . . . . . . . . 3 2.1. Links, Paths, MTU and PMTU . . . . . . . . . . . . . . . 3 2.2. Fragmentation Procedures . . . . . . . . . . . . . . . . 5 2.3. Upper-Layer Reliance on IP Fragmentation . . . . . . . . 6 3. Requirements Language . . . . . . . . . . . . . . . . . . . . 7 4. Reduced Reliability . . . . . . . . . . . . . . . . . . . . . 7 4.1. Policy-Based Routing . . . . . . . . . . . . . . . . . . 7 4.2. Network Address Translation (NAT) . . . . . . . . . . . . 8 4.3. Stateless Firewalls . . . . . . . . . . . . . . . . . . . 9 - 4.4. Stateless Load Balancers . . . . . . . . . . . . . . . . 9 - 4.5. Equal Cost Multipath (ECMP) . . . . . . . . . . . . . . . 10 - 4.6. IPv4 Reassembly Errors at High Data Rates . . . . . . . . 10 - 4.7. Security Vulnerabilities . . . . . . . . . . . . . . . . 10 - 4.8. PMTU Blackholing Due to ICMP Loss . . . . . . . . . . . . 11 - 4.8.1. Transient Loss . . . . . . . . . . . . . . . . . . . 12 - 4.8.2. Incorrect Implementation of Security Policy . . . . . 12 - 4.8.3. Persistent Loss Caused By Anycast . . . . . . . . . . 13 - 4.9. Blackholing Due To Filtering or Loss . . . . . . . . . . 13 + 4.4. Equal Cost Multipath, Link Aggregate Groups and Stateless + Load-Balancers . . . . . . . . . . . . . . . . . . . . . 9 + 4.5. IPv4 Reassembly Errors at High Data Rates . . . . . . . . 10 + 4.6. Security Vulnerabilities . . . . . . . . . . . . . . . . 10 + 4.7. PMTU Blackholing Due to ICMP Loss . . . . . . . . . . . . 11 + 4.7.1. Transient Loss . . . . . . . . . . . . . . . . . . . 12 + 4.7.2. Incorrect Implementation of Security Policy . . . . . 12 + 4.7.3. Persistent Loss Caused By Anycast . . . . . . . . . . 13 + 4.8. Blackholing Due To Filtering or Loss . . . . . . . . . . 13 5. Alternatives to IP Fragmentation . . . . . . . . . . . . . . 14 5.1. Transport Layer Solutions . . . . . . . . . . . . . . . . 14 5.2. Application Layer Solutions . . . . . . . . . . . . . . . 15 6. Applications That Rely on IPv6 Fragmentation . . . . . . . . 16 - 6.1. DNS . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 + 6.1. DNS . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 6.2. OSPF . . . . . . . . . . . . . . . . . . . . . . . . . . 17 6.3. Packet-in-Packet Encapsulations . . . . . . . . . . . . . 17 - 6.4. Licklider Transmission Protocol (LTP) . . . . . . . . . . 17 + 6.4. Licklider Transmission Protocol (LTP) . . . . . . . . . . 18 7. Recommendations . . . . . . . . . . . . . . . . . . . . . . . 18 7.1. For Application and Protocol Developers . . . . . . . . . 18 7.2. For System Developers . . . . . . . . . . . . . . . . . . 18 - 7.3. For Middle Box Developers . . . . . . . . . . . . . . . . 18 - 7.4. For Network Operators . . . . . . . . . . . . . . . . . . 19 - 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 19 - 9. Security Considerations . . . . . . . . . . . . . . . . . . . 19 - 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 19 - 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 19 - 11.1. Normative References . . . . . . . . . . . . . . . . . . 19 + 7.3. For Middle Box Developers . . . . . . . . . . . . . . . . 19 + 7.4. For ECMP, LAG and Load-Balancer Developers And Operators 19 + 7.5. For Network Operators . . . . . . . . . . . . . . . . . . 19 + 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20 + 9. Security Considerations . . . . . . . . . . . . . . . . . . . 20 + 10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 20 + 11. References . . . . . . . . . . . . . . . . . . . . . . . . . 20 + 11.1. Normative References . . . . . . . . . . . . . . . . . . 20 11.2. Informative References . . . . . . . . . . . . . . . . . 21 Appendix A. Contributors' Address . . . . . . . . . . . . . . . 24 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 24 1. Introduction Operational experience [Kent] [Huston] [RFC7872] reveals that IP fragmentation reduces the reliability of Internet communication. This document describes IP fragmentation and explains how it reduces the reliability of Internet communication. This document also @@ -369,21 +370,21 @@ o The Destination IP Address and Destination Port on each inbound packet. A+P [RFC6346] and Carrier Grade NAT (CGN) [RFC6888] are two common NAT strategies. In both approaches the NAT device must virtually reassemble fragmented packets in order to translate and forward each fragment. Virtual reassembly in the network is problematic, because it is computationally expensive and because it is prone to attacks - (Section 4.7). + (Section 4.6). 4.3. Stateless Firewalls IP fragmentation causes problems for stateless firewalls whose rules include TCP and UDP ports. Because port information is not available in the trailing fragments the firewall is limited to the following options: o Accept all trailing fragments, possibly admitting certain classes of attack. @@ -391,77 +392,85 @@ o Block all trailing fragments, possibly blocking legitimate traffic. Neither option is attractive. This problem does not occur in stateful firewalls or Network Address Translation (NAT) devices. Such devices maintain state so that they can afford identical treatment to each fragment that belongs to a packet. -4.4. Stateless Load Balancers +4.4. Equal Cost Multipath, Link Aggregate Groups and Stateless Load- + Balancers - IP fragmentation causes problems for stateless load balancers. In - order to assign a packet or packet fragment to a link, the load- - balancer executes an algorithm. The following paragraphs describe a - commonly deployed load-balancing algorithm. + IP fragmentation causes problems for Equal Cost Multipath (ECMP), + Link Aggregate Groups (LAG) and other stateless load-balancing + technologies. In order to assign a packet or packet fragment to a + link, an intermediate node executes a hash (i.e., load-balancing) + algorithm. The following paragraphs describe a commonly deployed + hash algorithm. If the packet or packet fragment contains a transport-layer header, - the load balancing algorithm accepts the following 5-tuple as input: + the algorithm accepts the following 5-tuple as input: o IP Source Address. o IP Destination Address. o IPv4 Protocol or IPv6 Next Header. o transport-layer source port. o transport-layer destination port. If the packet or packet fragment does not contain a transport-layer - header, the load balancing algorithm accepts only the following - 3-tuple as input: + header, the algorithm accepts only the following 3-tuple as input: o IP Source Address. o IP Destination Address. o IPv4 Protocol or IPv6 Next Header. Therefore, non-fragmented packets belonging to a flow can be assigned to one link while fragmented packets belonging to the same flow can be divided between that link and another. This can cause suboptimal - load balancing. + load-balancing. -4.5. Equal Cost Multipath (ECMP) + [RFC6438] offers a partial solution to this problem for IPv6 devices + only. According to [RFC6438]: - IP fragmentation causes problems for routers that support Equal Cost - Multipath (ECMP). Many routers that support ECMP execute the - algorithm described in Section 4.4. Therefore, the exhibit they same - problematic behaviors described in Section 4.4. + "At intermediate routers that perform load distribution, the hash + algorithm used to determine the outgoing component-link in an ECMP + and/or LAG toward the next hop MUST minimally include the 3-tuple + {dest addr, source addr, flow label} and MAY also include the + remaining components of the 5-tuple." -4.6. IPv4 Reassembly Errors at High Data Rates + If the algorithm includes only the 3-tuple {dest addr, source addr, + flow label}, it will assign all fragments belonging to a packet to + the same link. + +4.5. IPv4 Reassembly Errors at High Data Rates IPv4 fragmentation is not sufficiently robust for use under some conditions in today's Internet. At high data rates, the 16-bit IP identification field is not large enough to prevent frequent incorrectly assembled IP fragments, and the TCP and UDP checksums are insufficient to prevent the resulting corrupted datagrams from being delivered to higher protocol layers. [RFC4963] describes some easily reproduced experiments demonstrating the problem, and discusses some of the operational implications of these observations. These reassembly issues are not easily reproducible in IPv6 because the IPv6 identification field is 32 bits long. -4.7. Security Vulnerabilities +4.6. Security Vulnerabilities Security researchers have documented several attacks that exploit IP fragmentation. The following are examples: o Overlapping fragment attacks [RFC1858][RFC3128][RFC5722] o Resource exhaustion attacks (such as the Rose Attack) o Attacks based on predictable fragment identification values [RFC7739] @@ -502,63 +511,63 @@ for an attacker to forge malicious IP fragments that would cause the reassembly procedure for legitimate packets to fail. NIDS aims at identifying malicious activity by analyzing network traffic. Ambiguity in the possible result of the fragment reassembly process may allow an attacker to evade these systems. Many of these systems try to mitigate some of these evasion techniques (e.g. By computing all possible outcomes of the fragment reassembly process, at the expense of increased processing requirements). -4.8. PMTU Blackholing Due to ICMP Loss +4.7. PMTU Blackholing Due to ICMP Loss As mentioned in Section 2.3, upper-layer protocols can be configured to rely on PMTUD. Because PMTUD relies upon the network to deliver ICMP PTB messages, those protocols also rely on the networks to deliver ICMP PTB messages. According to [RFC4890], ICMP PTB messages must not be filtered. However, ICMP PTB delivery is not reliable. It is subject to both transient and persistent loss. Transient loss of ICMP PTB messages can cause transient PMTU black holes. When the conditions contributing to transient loss abate, the network regains its ability to deliver ICMP PTB messages and connectivity between the source and destination nodes is restored. - Section 4.8.1 of this document describes conditions that lead to + Section 4.7.1 of this document describes conditions that lead to transient loss of ICMP PTB messages. Persistent loss of ICMP PTB messages can cause persistent black - holes. Section 4.8.2 and Section 4.8.3 of this document describe + holes. Section 4.7.2 and Section 4.7.3 of this document describe conditions that lead to persistent loss of ICMP PTB messages. The problem described in this section is specific to PMTUD. It does not occur when the upper-layer protocol obtains its PMTU estimate from PLPMTUD or from any other source. -4.8.1. Transient Loss +4.7.1. Transient Loss The following factors can contribute to transient loss of ICMP PTB messages: o Network congestion. o Packet corruption. o Transient routing loops. o ICMP rate limiting. The effect of rate limiting may be severe, as RFC 4443 recommends strict rate limiting of IPv6 traffic. -4.8.2. Incorrect Implementation of Security Policy +4.7.2. Incorrect Implementation of Security Policy Incorrect implementation of security policy can cause persistent loss of ICMP PTB messages. Assume that a Customer Premise Equipment (CPE) router implements the following zone-based security policy: o Allow any traffic to flow from the inside zone to the outside zone. @@ -574,42 +583,42 @@ allows the ICMP PTB to flow from the outside zone to the inside zone. If not, the implementation discards the ICMP PTB message. When a incorrect implementation of the above-mentioned security policy receives an ICMP PTB message, it discards the packet because its source address is not associated with an existing flow. The security policy described above is implemented incorrectly on many consumer CPE routers. -4.8.3. Persistent Loss Caused By Anycast +4.7.3. Persistent Loss Caused By Anycast Anycast can cause persistent loss of ICMP PTB messages. Consider the example below: A DNS client sends a request to an anycast address. The network routes that DNS request to the nearest instance of that anycast address (i.e., a DNS Server). The DNS server generates a response and sends it back to the DNS client. While the response does not exceed the DNS server's PMTU estimate, it does exceed the actual PMTU. A downstream router drops the packet and sends an ICMP PTB message the packet's source (i.e., the anycast address). The network routes the ICMP PTB message to the anycast instance closest to the downstream router. That anycast instance may not be the DNS server that originated the DNS response. It may be another DNS server with the same anycast address. The DNS server that originated the response may never receive the ICMP PTB message and may never update its PMTU estimate. -4.9. Blackholing Due To Filtering or Loss +4.8. Blackholing Due To Filtering or Loss In RFC 7872, researchers sampled Internet paths to determine whether they would convey packets that contain IPv6 extension headers. Sampled paths terminated at popular Internet sites (e.g., popular web, mail and DNS servers). The study revealed that at least 28% of the sampled paths did not convey packets containing the IPv6 Fragment extension header. In most cases, fragments were dropped in the destination autonomous system. In other cases, the fragments were dropped in transit @@ -623,21 +632,21 @@ Possible causes follow: o Hardware inability to process fragmented packets. o Failure to change vendor defaults. o Unintentional misconfiguration. o Intentional configuration (e.g., network operators consciously chooses to drop IPv6 fragments in order to address the issues - raised in Section 4.1 through Section 4.8, above.) + raised in Section 4.1 through Section 4.7, above.) 5. Alternatives to IP Fragmentation 5.1. Transport Layer Solutions The Transport Control Protocol (TCP) [RFC0793]) can be operated in a mode that does not require IP fragmentation. Applications submit a stream of data to TCP. TCP divides that stream of data into segments, with no segment exceeding the TCP Maximum @@ -855,29 +864,44 @@ must maintain state in order to achieve this goal. Price and performance considerations frequently motivate network operators to deploy stateless middle boxes. These stateless middle boxes may perform sub-optimally, process IP fragments in a manner that is not compliant with RFC 791 or RFC 8200, or even discard IP fragments completely. Such behaviors are NOT RECOMMENDED. If a middleboxes implements non-standard behavior with respect to IP fragmentation, then that behavior MUST be clearly documented. -7.4. For Network Operators +7.4. For ECMP, LAG and Load-Balancer Developers And Operators + + In their default configuration, when the IPv6 Flow Label is not equal + to zero, IPv6 devices that implement ECMP, LAG or other load- + balancing technologies SHOULD accept only the following fields as + input to their hash algorithm: + + o IP Source Address. + + o IP Destination Address. + + o Flow Label. + + Operators SHOULD deploy these devices in their default configuration. + +7.5. For Network Operators Operators MUST ensure proper PMTUD operation in their network, including making sure the network generates PTB packets when dropping packets too large compared to outgoing interface MTU. As per RFC 4890, network operators MUST NOT filter ICMPv6 PTB messages unless they are known to be forged or otherwise - illegitimate. As stated in Section 4.8, filtering ICMPv6 PTB packets + illegitimate. As stated in Section 4.7, filtering ICMPv6 PTB packets causes PMTUD to fail. Many upper-layer protocols rely on PMTUD. As per RFC 8200, network operators MUST NOT deploy IPv6 links whose MTU is less than 1280 bytes. Network operators SHOULD NOT filter IP fragments if they originated at a domain name server or are destined for a domain name server. 8. IANA Considerations @@ -1065,20 +1089,25 @@ [RFC5927] Gont, F., "ICMP Attacks against TCP", RFC 5927, DOI 10.17487/RFC5927, July 2010, . [RFC6346] Bush, R., Ed., "The Address plus Port (A+P) Approach to the IPv4 Address Shortage", RFC 6346, DOI 10.17487/RFC6346, August 2011, . + [RFC6438] Carpenter, B. and S. Amante, "Using the IPv6 Flow Label + for Equal Cost Multipath Routing and Link Aggregation in + Tunnels", RFC 6438, DOI 10.17487/RFC6438, November 2011, + . + [RFC6888] Perreault, S., Ed., Yamagata, I., Miyakawa, S., Nakagawa, A., and H. Ashida, "Common Requirements for Carrier-Grade NATs (CGNs)", BCP 127, RFC 6888, DOI 10.17487/RFC6888, April 2013, . [RFC7588] Bonica, R., Pignataro, C., and J. Touch, "A Widely Deployed Solution to the Generic Routing Encapsulation (GRE) Fragmentation Problem", RFC 7588, DOI 10.17487/RFC7588, July 2015, .