draft-ietf-rtgwg-net2cloud-gap-analysis-06.txt   draft-ietf-rtgwg-net2cloud-gap-analysis-07.txt 
Network Working Group L. Dunbar Network Working Group L. Dunbar
Internet Draft Futurewei Internet Draft Futurewei
Intended status: Informational A. Malis Intended status: Informational A. Malis
Expires: November 1, 2020 Independent Expires: January 26, 2021 Malis Consulting
C. Jacquenet C. Jacquenet
Orange Orange
May 1, 2020 July 26, 2020
Networks Connecting to Hybrid Cloud DCs: Gap Analysis Networks Connecting to Hybrid Cloud DCs: Gap Analysis
draft-ietf-rtgwg-net2cloud-gap-analysis-06 draft-ietf-rtgwg-net2cloud-gap-analysis-07
Abstract Abstract
This document analyzes the technical gaps that may affect the This document analyzes the IETF routing area technical gaps that may
dynamic connection to workloads and applications hosted in hybrid affect the dynamic connection to workloads and applications hosted
Cloud Data Centers from enterprise premises. in hybrid Cloud Data Centers from enterprise premises.
Status of this Memo Status of this Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet- other groups may also distribute working documents as Internet-
Drafts. Drafts.
skipping to change at page 1, line 41 skipping to change at page 1, line 40
months and may be updated, replaced, or obsoleted by other documents months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as at any time. It is inappropriate to use Internet-Drafts as
reference material or to cite them other than as "work in progress." reference material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html http://www.ietf.org/shadow.html
This Internet-Draft will expire on November 1, 2020. This Internet-Draft will expire on January 26, 2009.
Copyright Notice Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 25 skipping to change at page 2, line 25
respect to this document. Code Components extracted from this respect to this document. Code Components extracted from this
document must include Simplified BSD License text as described in document must include Simplified BSD License text as described in
Section 4.e of the Trust Legal Provisions and are provided without Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Simplified BSD License. warranty as described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction...................................................3 1. Introduction...................................................3
2. Conventions used in this document..............................3 2. Conventions used in this document..............................3
3. Gap Analysis for Accessing Cloud Resources.....................4 3. Gap Analysis for Accessing Cloud Resources.....................4
3.1. Multiple PEs connecting to virtual CPEs in Cloud DCs......7 3.1. Multiple PEs connecting to virtual CPEs in Cloud DCs......6
3.2. Access Control for workloads in the Cloud DCs.............7 3.2. Access Control for workloads in the Cloud DCs.............6
3.3. NAT Traversal.............................................8 3.3. NAT Traversal.............................................7
3.4. BGP between PEs and remote CPEs via Internet..............8 3.4. BGP between PEs and remote CPEs via Internet..............7
3.5. Multicast traffic from/to the remote edges................9 3.5. Multicast traffic from/to the remote edges................8
4. Gap Analysis of Overlay Edge Node's WAN Port Management........9 4. Gap Analysis of Traffic over Multiple Underlay Networks........9
5. Aggregating VPN paths and Internet paths......................11 5. Aggregating VPN paths and Internet paths......................10
5.1. Control Plane for Overlay over Heterogeneous Networks....12 5.1. Control Plane for Cloud Access via Heterogeneous Networks11
5.2. Using BGP UPDATE Messages................................13 5.2. Using BGP UPDATE Messages................................12
5.2.1. Lacking SD-WAN Segments Identifier..................13 5.2.1. Lack ways to differentiate traffic in Cloud DCs.....12
5.2.2. Missing attributes in Tunnel-Encap..................13 5.2.2. Miss attributes in Tunnel-Encap.....................12
5.3. SECURE-L3VPN/EVPN........................................15 5.3. SECURE-EVPN/BGP-EDGE-DISCOVERY...........................12
5.4. Preventing attacks from Internet-facing ports............16 5.4. SECURE-L3VPN.............................................13
6. Gap Summary...................................................16 5.5. Preventing attacks from Internet-facing ports............14
7. Manageability Considerations..................................17 6. Gap Summary...................................................14
8. Security Considerations.......................................17 7. Manageability Considerations..................................15
9. IANA Considerations...........................................18 8. Security Considerations.......................................16
10. References...................................................18 9. IANA Considerations...........................................16
10.1. Normative References....................................18 10. References...................................................16
10.2. Informative References..................................18 10.1. Normative References....................................16
11. Acknowledgments..............................................19 10.2. Informative References..................................16
11. Acknowledgments..............................................17
1. Introduction 1. Introduction
[Net2Cloud-Problem] describes the problems enterprises face today [Net2Cloud-Problem] describes the problems enterprises face today
when interconnecting their branch offices with dynamic workloads when interconnecting their branch offices with dynamic workloads
hosted in third party data centers (a.k.a. Cloud DCs). In hosted in third party data centers (a.k.a. Cloud DCs). In
particular, this document analyzes the routing protocols to identify particular, this document analyzes the available routing protocols
whether there are any gaps that may impede such interconnection to identify whether there are any gaps that may impede such
which may for example justify additional specification effort to interconnection which may for example justify additional
define proper protocol extensions. specification effort to define proper protocol extensions.
For the sake of readability, an edge, an endpoint, C-PE, or CPE are For the sake of readability, an edge, C-PE, or CPE are used
used interchangeably throughout this document. More precisely: interchangeably throughout this document. More precisely:
. Edge: may include multiple devices (virtual or physical); . Edge: may include multiple devices (virtual or physical);
. endpoint: refers to a WAN port of device located in the edge;
. C-PE: provider-owned edge, e.g. for SECURE-EVPN's PE-based . C-PE: provider-owned edge, e.g. for SECURE-EVPN's PE-based
BGP/MPLS VPN, where PE is the edge node; BGP/MPLS VPN, where PE is the edge node;
. CPE: device located in enterprise premises. . CPE: device located in enterprise premises.
2. Conventions used in this document 2. Conventions used in this document
Cloud DC: Third party Data Centers that usually host applications Cloud DC: Third party Data Centers that usually host applications
and workload owned by different organizations or and workload owned by different organizations or
tenants. tenants.
Controller: Used interchangeably with Overlay controller to manage Controller: Used interchangeably with Overlay controller to manage
overlay path creation/deletion and monitor the path overlay path creation/deletion and monitor the path
conditions between sites. conditions between sites.
CPE-Based VPN: Virtual Private Network designed and deployed from CPE-Based VPN: Virtual Private Network designed and deployed from
CPEs. This is to differentiate from most commonly used CPEs. This is to differentiate from most commonly used
PE-based VPNs a la RFC 4364. PE-based VPNs a la RFC 4364.
OnPrem: On Premises data centers and branch offices OnPrem: On Premises data centers and branch offices
SDWAN: Software Defined Wide Area Network, "SDWAN" refers to
the solutions of pooling WAN bandwidth from multiple
underlay networks to get better WAN bandwidth
management, visibility & control. When the underlay is a
private network, traffic may be forwarded without any
additional encryption; when the underlay networks are
public, such as the Internet, some traffic needs to be
encrypted when passing through (depending on user-
provided policies).
3. Gap Analysis for Accessing Cloud Resources 3. Gap Analysis for Accessing Cloud Resources
Because of the ephemeral property of the selected Cloud DCs for Because of the ephemeral property of the selected Cloud DCs for
specific workloads/Apps, an enterprise or its network service specific workloads/Apps, an enterprise or its network service
provider may not have direct physical connections to the Cloud DCs provider may not have direct physical connections to the Cloud DCs
that are optimal for hosting the enterprise's specific that are optimal for hosting the enterprise's specific
workloads/Apps. Under those circumstances, an overlay network design workloads/Apps. Under those circumstances, an overlay network design
can be an option to interconnect the enterprise's on-premises data can be an option to interconnect the enterprise's on-premises data
centers & branch offices to its desired Cloud DCs. centers & branch offices to its desired Cloud DCs.
However, overlay paths established over the public Internet can have However, overlay paths established over the public Internet can have
unpredictable performance, especially over long distances and across unpredictable performance, especially over long distances.
operators' domains. Therefore, it is highly desirable to minimize Therefore, it is highly desirable to minimize the distance or the
the distance or the number of segments that traffic had to be number of segments that traffic had to be forwarded over the public
forwarded over the public Internet. Internet.
The Metro Ethernet Forum's Cloud Service Architecture [MEF-Cloud] The Metro Ethernet Forum's Cloud Service Architecture [MEF-Cloud]
also describes a use case of network operators using Overlay paths also describes a use case of network operators using Overlay paths
over an LTE network or the public Internet for the last mile access over an LTE network or the public Internet for the last mile access
where the VPN service providers cannot always provide the required where the VPN service providers cannot always provide the required
physical infrastructure. physical infrastructure.
In these scenarios, some overlay edge nodes may not be directly In some scenarios, some overlay edge nodes may not be directly
attached to the PEs that participate to the delivery and the attached to the PEs that participate to the delivery and the
operation of the enterprise's VPN. operation of the enterprise's VPN.
When using an overlay network to connect the enterprise's sites to When using an overlay network to connect the enterprise's sites to
the workloads hosted in Cloud DCs, the existing C-PEs at the workloads hosted in Cloud DCs, the existing C-PEs at
enterprise's sites have to be upgraded to connect to the said enterprise's sites have to be upgraded to connect to the said
overlay network. If the workloads hosted in Cloud DCs need to be overlay network. If the workloads hosted in Cloud DCs need to be
connected to many sites, the upgrade process can be very expensive. connected to many sites, the upgrade process can be very expensive.
[Net2Cloud-Problem] describes a hybrid network approach that extends [Net2Cloud-Problem] describes a hybrid network approach that extends
skipping to change at page 8, line 16 skipping to change at page 7, line 16
Cloud Resources could be improved by having automated and reliable Cloud Resources could be improved by having automated and reliable
tools to map the user-friendly (natural language) rules into machine tools to map the user-friendly (natural language) rules into machine
readable policies and to provide interfaces for enterprises to self- readable policies and to provide interfaces for enterprises to self-
manage policy enforcement points for their own workloads. manage policy enforcement points for their own workloads.
3.3. NAT Traversal 3.3. NAT Traversal
Cloud DCs that only assign private IPv4 addresses to the Cloud DCs that only assign private IPv4 addresses to the
instantiated workloads assume that traffic to/from the workload instantiated workloads assume that traffic to/from the workload
usually needs to traverse NATs. usually needs to traverse NATs.
An overlay edge node can solicit a STUN (Session Traversal of UDP
Through Network Address Translation, [RFC3489]) Server to get the There is no automatic way for an enterprise's network controller to
be informed of the NAT properties for its workloads in Cloud DCs
One potential solution could be utilizing the messages sent during
initialization of an IKE VPN when NAT Traversal option is enabled.
There are some inherent problems while sending IPSec packets through
NAT devices. One way to overcome these problems is to encapsulate
IPSec packets in UDP. To do this effectively, there is a discovery
phase in IKE (Phase1) that tries to determine if either of the IPSec
gateways is behind a NAT device. If a NAT device is found, IPSec-
over-UDP is proposed during IPSec (Phase 2) negotiation. If there is
no NAT device detected, IPSec is used
Another potential solution could be allowing the virtual CPE in
Cloud DCs to solicit a STUN (Session Traversal of UDP Through
Network Address Translation, [RFC3489]) Server to get the
information about the NAT property, the public IP addresses and port information about the NAT property, the public IP addresses and port
numbers so that such information can be communicated to the relevant numbers so that such information can be communicated to the relevant
peers. peers.
3.4. BGP between PEs and remote CPEs via Internet 3.4. BGP between PEs and remote CPEs via Internet
Even though an EBGP (external BGP) Multi-Hop design can be used to Even though an EBGP (external BGP) Multi-Hop design can be used to
connect peers that are not directly connected to each other, there connect peers that are not directly connected to each other, there
are still some issues about extending BGP from MPLS VPN PEs to are still some issues about extending BGP from MPLS VPN PEs to
remote CPEs via any access path (e.g., Internet). remote CPEs in cloud DCs via any access path (e.g., Internet).
The path between the remote CPEs and VPN PEs that maintain VPN The path between the remote CPEs and VPN PEs that maintain VPN
routes can traverse untrusted segments. routes can traverse untrusted segments.
EBGP Multi-hop design requires configuration on both peers, either EBGP Multi-hop design requires configuration on both peers, either
manually or via NETCONF from a controller. To use EBGP between a PE manually or via NETCONF from a controller. To use EBGP between a PE
and remote CPEs, the PE has to be manually configured with the and remote CPEs, the PE has to be manually configured with the
"next-hop" set to the IP address of the CPEs. When remote CPEs, "next-hop" set to the IP address of the CPEs. When remote CPEs,
especially remote virtualized CPEs are dynamically instantiated or especially remote virtualized CPEs are dynamically instantiated or
removed, the configuration of Multi-Hop EBGP on the PE has to be removed, the configuration of Multi-Hop EBGP on the PE has to be
skipping to change at page 9, line 21 skipping to change at page 8, line 36
when connecting via the public Internet. when connecting via the public Internet.
Remote CPEs, if instantiated in Cloud DCs might have to traverse Remote CPEs, if instantiated in Cloud DCs might have to traverse
NATs to reach PEs. It is not clear how BGP can be used between NATs to reach PEs. It is not clear how BGP can be used between
devices located beyond the NAT and the devices located behind the devices located beyond the NAT and the devices located behind the
NAT. It is not clear how to configure the Next Hop on the PEs to NAT. It is not clear how to configure the Next Hop on the PEs to
reach private IPv4 addresses. reach private IPv4 addresses.
3.5. Multicast traffic from/to the remote edges 3.5. Multicast traffic from/to the remote edges
Among the multiple floating PEs that are reachable from a remote Among the multiple floating PEs that are reachable from a remote CPE
CPE, multicast traffic sent by the remote CPE towards the MPLS VPN in a Cloud DC, multicast traffic sent by the remote CPE towards the
can be forwarded back to the remote CPE due to the PE receiving the MPLS VPN can be forwarded back to the remote CPE due to the PE
multicast packets forwarding the multicast/broadcast frame to other receiving the multicast packets forwarding the multicast/broadcast
PEs that in turn send to all attached CPEs. This process may cause frame to other PEs that in turn send to all attached CPEs. This
traffic loops. process may cause traffic loops.
This problem can be solved by selecting one floating PE as the CPE's This problem can be solved by selecting one floating PE as the CPE's
Designated Forwarder, similar to TRILL's Appointed Forwarders Designated Forwarder, similar to TRILL's Appointed Forwarders
[RFC6325]. [RFC6325].
BGP/MPLS VPNs do not have features like TRILL's Appointed BGP/MPLS VPNs do not have features like TRILL's Appointed
Forwarders. Forwarders.
4. Gap Analysis of Overlay Edge Node's WAN Port Management 4. Gap Analysis of Traffic over Multiple Underlay Networks
Very often the Hybrid Cloud DCs are interconnected by overlay
networks that arch over many different types of networks, such as
VPN, public Internet, wireless and wired infrastructures, etc.
Sometimes the enterprises' VPN providers do not have direct access
to the Cloud DCs that host some specific applications or workloads
operated by the enterprise.
Under those circumstances, the overlay network's edge nodes can have
WAN ports facing networks provided by different ISPs, some of these
networks may not be trustable, some others can be trusted like VPNs
(to some extent), etc.
If all WAN ports of an edge node are facing an untrusted network, Very often the Hybrid Cloud DCs are interconnected by multiple types
then all sensitive data to/from this edge node have to be encrypted, of underlay networks, such as VPN, public Internet, wireless and
usually by means of IPsec tunnels which can be terminated at the WAN wired infrastructures, etc. Sometimes the enterprises' VPN providers
port address, at the edge node's loopback address if the loopback do not have direct access to the Cloud DCs that host some specific
address is routable in the wide area network, or even at the ingress applications or workloads operated by the enterprise.
ports of the edge node.
If an edge node has some WAN ports facing trusted networks and When reached by an untrusted network, all sensitive data to/from
others facing untrusted networks, sensitive data can be forwarded this virtual CPE have to be encrypted, usually by means of IPsec
through ports facing the trusted networks natively (i.e., without tunnels. When reached by a trusted direct connect paths, sensitive
encryption) and forwarded through ports facing untrusted networks data can be forwarded without encryption for better performance.
assuming encryption. To achieve this flexibility of sending traffic
either encrypted or not encrypted depending on egress WAN ports, it
is necessary to have the IPsec tunnels terminated at the WAN ports
facing the untrusted networks.
In order to establish peer-wise secure encrypted communications If a virtual CPE in Cloud DC can be reached by both trusted and
among those WAN ports of two edge nodes, it is necessary for the untrusted paths, better performance can be achieved to have a mixed
edge nodes (peers) to be informed of the WAN port properties. encrypted and unencrypted traffic depending which paths the traffic
is forwarded. However, there is no appropriate control plane
protocol to achieve this automatically.
Some of those overlay networks (such as some deployed SDWAN Some networks achieve the IPsec tunnel automation by using the
networks) use the modified NHRP protocol [RFC2332] to register WAN modified NHRP protocol [RFC2332] to register network facing ports of
ports of the edge nodes with their Controller (or NHRP server), the edge nodes with their Controller (or NHRP server), which then
which then maps a private VPN address to a public IP address of the maps a private VPN address to a public IP address of the destination
destination node/port. DSVPN [DSVPN] or DMVPN [DMVPN] are used to node/port. DSVPN [DSVPN] or DMVPN [DMVPN] are used to establish
establish tunnels between WAN ports of SDWAN edge nodes. tunnels between WAN ports of SDWAN edge nodes.
NHRP was originally intended for ATM address resolution, and as a NHRP was originally intended for ATM address resolution, and as a
result, it misses many attributes that are necessary for dynamic result, it misses many attributes that are necessary for dynamic
endpoint C-PE registration to the controller, such as: virtual C-PE registration to the controller, such as:
- Interworking with the MPLS VPN control plane. An overlay edge can - Interworking with the MPLS VPN control plane. An overlay edge can
have some ports facing the MPLS VPN network over which packets can have some ports facing the MPLS VPN network over which packets can
be forwarded without any encryption and some ports facing the be forwarded without any encryption and some ports facing the
public Internet over which sensitive traffic needs to be public Internet over which sensitive traffic needs to be
encrypted. encrypted.
- Scalability: NHRP/DSVPN/DMVPN work fine with small numbers of edge - Scalability: NHRP/DSVPN/DMVPN work fine with small numbers of edge
nodes. When a network has more than 100 nodes, these protocols do nodes. When a network has more than 100 nodes, these protocols do
not scale well. not scale well.
- NHRP does not have the IPsec attributes, which are needed for - NHRP does not have the IPsec attributes, which are needed for
peers to build Security Associations over the public Internet. peers to build Security Associations over the public Internet.
- NHRP messages do not have any field to encode the C-PE supported - NHRP messages do not have any field to encode the C-PE supported
encapsulation types, such as IPsec-GRE or IPsec-VxLAN. encapsulation types, such as IPsec-GRE or IPsec-VxLAN.
- NHRP messages do not have any field to encode C-PE Location - NHRP messages do not have any field to encode C-PE Location
identifiers, such as Site Identifier, System ID, and/or Port ID. identifiers, such as Site Identifier, System ID, and/or Port ID.
- NHRP messages do not have any field to describe the gateway(s) to - NHRP messages do not have any field to describe the gateway(s) to
skipping to change at page 11, line 23 skipping to change at page 10, line 24
identifiers, such as Site Identifier, System ID, and/or Port ID. identifiers, such as Site Identifier, System ID, and/or Port ID.
- NHRP messages do not have any field to describe the gateway(s) to - NHRP messages do not have any field to describe the gateway(s) to
which the C-PE is attached. When a C-PE is instantiated in a Cloud which the C-PE is attached. When a C-PE is instantiated in a Cloud
DC, it is desirable for the C-PE's owner to be informed about how DC, it is desirable for the C-PE's owner to be informed about how
and where the C-PE is attached. and where the C-PE is attached.
- NHRP messages do not have any field to describe C-PE's NAT - NHRP messages do not have any field to describe C-PE's NAT
properties if the C-PE is using private IPv4 addresses, such as properties if the C-PE is using private IPv4 addresses, such as
the NAT type, Private address, Public address, Private port, the NAT type, Private address, Public address, Private port,
Public port, etc. Public port, etc.
[BGP-SDWAN-PORT] describes how to use BGP to distribute SDWAN edge
properties to peers. SDWAN is an overlay network with specific
properties, such as application-based forwarding, augmented
transport, and user specified policies. There is a need to extend
the protocol to register WAN port properties of an edge node to the
overlay controller, which then propagates the information to other
overlay edge nodes that are authenticated and authorized to
communicate with them.
5. Aggregating VPN paths and Internet paths 5. Aggregating VPN paths and Internet paths
Most likely, enterprises (especially the largest ones) already have Most likely, enterprises (especially the largest ones) already have
their C-PEs interconnected by VPN service providers, based upon VPN their C-PEs interconnected by VPNs, based upon VPN techniques like
techniques like EVPN, L2VPN, or L3VPN, and which can be lead to PE- EVPN, L2VPN, or L3VPN. Their VPN providers might have direct
based or CPE-based VPN service designs. The commonly used PE-based paths/links to the Cloud DCs that host their workloads and
BGP/MPLS VPNs have C-PEs directly attached to PEs, the communication applications.
between C-PEs and PEs is considered as secure as they are connected
by direct physical links albeit there could be routes leaking or
unauthorized routes being injected. MP-BGP can be used to learn &
distribute routes among C-PEs, but sometimes routes among C-PEs are
statically configured on the C-PEs.
For enterprises already interconnected by VPNs, if there are short When there is short term high traffic volume that can't justify
term high traffic volume that can't justify increasing the VPNs increasing the VPNs capacity, enterprises can utilize public
capacity, it is desirable for the CPE to aggregate the bandwidth internet to reach their Cloud vCPEs. Then it is necessary for the
that pertains to VPN paths and Internet paths by adding ports that vCPEs to communicate with the controller on how traffic is
connect the CPE to the public Internet. Under this scenario, which distributed among multiple heterogeneous underlay networks and to
is referred to as the Overlay scenario throughout this document, it manage secure tunnels over untrusted networks.
is necessary for the C-PEs to manage and communicate with the
controller on how traffic is distributed among multiple
heterogeneous underlay networks, and also to manage secure tunnels
over untrusted networks.
When using NHRP for WAN port registration purposes, C-PEs need to
run two separate control planes: EVPN&BGP for CPE-based VPNs, and
NHRP & DSVPN/DMVPN for ports connected to the Internet. Two separate
control planes not only add complexity to C-PEs, but also increase
operational costs.
+---+ +---+
+--------------|RR |----------+ +--------------|RR |----------+
/ Untrusted +-+-+ \ / Untrusted +-+-+ \
/ \ / +----------------------
/ \ / | \ Cloud DC
+----+ +---------+ packets encrypted over +------+ +----+ +----+ +---------+ packets encrypted over | +------+ +----+
| TN3|--| A1-----+ Untrusted +------ B1 |--| TN1| | TN3|--| A1-----+ Untrusted +-------+ |--| TN1|
+----+ | C-PE A2-\ | C-PE | +----+ +----+ | C-PE A2-\ | | vCPE | +----+
+----+ | A A3--+--+ +---+---B2 B | +----+ +----+ | A A3--+--+ +---+ | | B | +----+
| TN2|--| | |PE+--------------+PE |---B3 |--| TN3| | TN2|--| | |PE+--------------+PE |---+ |--| TN3|
+----+ +---------+ +--+ trusted +---+ +------+ +----+ +----+ +---------+ +--+ trusted +---+ | +------+ +----+
| WAN | | WAN | |
+----+ +---------+ +--+ packets +---+ +------+ +----+ +----+ +---------+ +--+ packets +---+ | +------+ +----+
| TN1|--| C1--|PE| go natively |PE |-- D1 |--| TN1| | TN1|--| C1--|PE| go natively |PE |---+ |--| TN1|
+----+ | C-PE C2--+--+ without encry+---+ | C-PE | +----+ +----+ | C-PE C2--+--+ without encry+---+ | | vCPE | +----+
| C | +--------------+ | D | | C | +--------------+ | | D |
| | | | | | | | |
+----+ | C3--| without encrypt over | | +----+ +----+ | C3--| without encrypt over | | | +----+
| TN2|--| C4--+---- Untrusted --+------D2 |--| TN2| | TN2|--| C4--+---- Untrusted --+------+ |--| TN2|
+----+ +---------+ +------+ +----+ +----+ +---------+ | +------+ +----+
Figure 2: CPEs interconnected by VPN paths and Internet Paths |
+------------------------
Figure 2: vCPEs reached by Hybrid Paths
5.1. Control Plane for Overlay over Heterogeneous Networks 5.1. Control Plane for Cloud Access via Heterogeneous Networks
As described in [BGP-SDWAN-Usage], the Control Plane for Overlay The Control Plane for managing applications and workloads in cloud
network over heterogeneous networks has three distinct properties: DCs reachable by heterogeneous networks need to include the
following properties:
- WAN Port Property registration to the Overlay Controller. - vCPE in a cloud DCs needs to communicate with its controller of
o To inform the Overlay controller and authorized peers of the properties of the directly connected underlay networks.
the WAN port properties of the Edge nodes. When the WAN
ports are assigned private IPv4 addresses, this step can
register the type of NAT that translates these addresses
into public ones.
- Controller-facilitated IPsec SA management and NAT information - Need Controller-facilitated IPsec SA attributes and NAT
distribution information distribution
o The Overlay controller facilitates and manages the IPsec o The controller facilitates and manages the peer
configuration and peer authentication for all IPsec authentication for all IPsec tunnels terminated at the
tunnels terminated at the edge nodes. vCPEs.
- Establishing and Managing the topology and reachability for - Establishing and Managing the topology and reachability for
services attached to the client ports of overlay edge nodes. services attached to the vCPEs in Cloud DCs.
o This is for the overlay layer's route distribution, so o This is for the overlay layer's route distribution, so
that a C-PE can populate its overlay routing table with that a vCPE can populate its overlay routing table with
entries that identify the next hop for reaching a specific entries that identify the next hop for reaching a specific
route/service attached to remote nodes. [SECURE-EVPN] route/service attached to the vCPEs.
describes EVPN and other options.
5.2. Using BGP UPDATE Messages 5.2. Using BGP UPDATE Messages
5.2.1. Lacking SD-WAN Segments Identifier 5.2.1. Lack ways to differentiate traffic in Cloud DCs
There could be multiple SD-WAN networks with their edge nodes One enterprise can have different types of applications in one Cloud
exchanging BGP UPDATE messages with the BGP RR. The multiple SD-WAN DC. Some can be production applications, some can be testing
networks could have common underlay networks. Therefore, it is very applications, and some can belong to one specific departments. The
important to have an identifier to differentiate BGP UPDATE messages traffic to/from different applications might need to traverse
belonging to different SD-WAN networks (or sometimes called SD-WAN different network paths or need to be differentiated by Control
Segmentations). Today's BGP doesn't have this feature yet, unless plane and data plane.
there are multiple BGP instances and their corresponding RRs.
5.2.2. Missing attributes in Tunnel-Encap BGP already has built-in mechanisms, like Route Target, to
differentiate different VPNs. But Route Target (RT) is for MPLS
based VPNs, therefore RT is not appropriate to directly apply to
virtual paths laid over mixed VPNs, IPsec or public underlay
networks.
5.2.2. Miss attributes in Tunnel-Encap
[Tunnel-Encap] describes the BGP UPDATE Tunnel Path Attribute that [Tunnel-Encap] describes the BGP UPDATE Tunnel Path Attribute that
advertises endpoints' tunnel encapsulation capabilities for the advertises endpoints' tunnel encapsulation capabilities for the
respective attached client routes encoded in the MP-NLRI Path respective attached client routes encoded in the MP-NLRI Path
Attribute. The receivers of the BGP UPDATE can use any of the Attribute. The receivers of the BGP UPDATE can use any of the
supported encapsulations encoded in the Tunnel Path Attribute for supported encapsulations encoded in the Tunnel Path Attribute for
the routes encoded in the MP-NLRI Path Attribute. the routes encoded in the MP-NLRI Path Attribute.
Here are some of the issues raised by the use of [Tunnel-Encap] to Here are some of the issues raised by using [Tunnel-Encap] to
distribute Edge WAN port properties: distribute the property of client routes be carried by mixed of
hybrid networks:
- [Tunnel-Encap] doesn't have the encoding to describe the NAT - [Tunnel-Encap] doesn't have encoding methods to advertise that a
information for WAN ports that are assigned private IPv4 addresses route can be carried by mixed of IPsec tunnels and other already
yet. The NAT information needs to be propagated to the trusted supported tunnels.
peers such as the virtual C-PEs instantiated in public Cloud DCs
via Controllers.
- The mechanism defined in [Tunnel-Encap] does not facilitate the - The mechanism defined in [Tunnel-Encap] does not facilitate the
exchange of IPsec SA-specific parameters independently from exchange of IPsec SA-specific attributes.
advertising the attached clients' routes, even after adding a new
IPsec tunnel type.
[Tunnel-Encap] requires all tunnels updates to be associated with
routes. There can be many client routes associated with an IPsec
tunnel established between two C-PEs' WAN ports; the corresponding
destination prefixes (as announced by the aforementioned routes)
may also be reached through the VPN underlay without any
encryption.
The establishment of an IPsec tunnel can fail, e.g., because the 5.3. SECURE-EVPN/BGP-EDGE-DISCOVERY
two endpoints support different encryption algorithms. Multiple
negotiation messages that carry the IPsec SA parameters between
two end-points may be exchanged. This is why it is cleaner to
separate the establishment of an IPsec SA association between two
end-points from the policies enforced to map routes to a specific
IPSec SA.
If C-PEs need to establish a WAN Port-based IPsec SA, the [SECURE-EVPN] describes a solution that utilize BGP as control plane
information encoded in the Tunnel Path Attribute should only apply for the Scenario #1 described in [BGP-SDWAN-Usage]. It relies upon a
to the WAN ports and should be independent from the clients' BGP cluster design to facilitate the key and policy exchange among
routes. PE devices to create private pair-wise IPsec Security Associations.
[Secure-EVPN] attaches all the IPsec SA information to the actual
client routes.
In addition, the Overlay IPsec SA Tunnel is very likely to be [BGP-Edge-DISCOVERY] proposes BGP UPDATEs from client routers only
established before clients' routes are attached. include the IPsec SA identifiers (ID) to reference the IPsec SA
attributes being advertised by separate Underlay Property BGP UPDATE
messages. If a client route can be encrypted by multiple IPsec SAs,
then multiple IPsec SA IDs are included in the Tunnel-Encap Path
attribute for the client route.
- When an overlay network spans across large geographic regions [BGP-Edge-DISCOVERY] proposes detailed IPsec SA attributes are
(such as countries or continents), one C-PE in one region may not advertised in a separate BGP UPDATE for the underlay networks.
even be aware of remote CPEs in other regions that it needs to
communicate. Therefore, the distribution of the Overlay Edge WAN
ports information need to be restricted to the authorized peers.
5.3. SECURE-L3VPN/EVPN [Secure-EVPN] and [BGP-Edge-Discovery] differs in the information
included in the client routes. [Secure-EVPN] attaches all the IPsec
SA information to the actual client routes, whereas the [BGP-Edge-
Discovery] only includes the IPsec SA IDs for the client routes. The
IPsec SA IDs used by [BGP-Edge-Discovery] is pointing to the SA-
Information which are advertised separately, with all the SA-
Information attached to routes which describe the SDWAN underlay,
such as WAN Ports or Node address.
[SECURE-L3VPN] describes a method to enrich BGP/MPLS VPN [RFC4364] 5.4. SECURE-L3VPN
[SECURE-L3VPN] describes a method to enrich BGP/MPLS VPN [RFC4364]
capabilities to allow some PEs to connect to other PEs via public capabilities to allow some PEs to connect to other PEs via public
networks. [SECURE-L3VPN] introduces the concept of Red Interface & networks. [SECURE-L3VPN] introduces the concept of Red Interface &
Black Interface used by PEs, where the RED interfaces are used to Black Interface used by PEs, where the RED interfaces are used to
forward traffic into the VPN, and the Black Interfaces are used forward traffic into the VPN, and the Black Interfaces are used
between WAN ports through which only IPsec-formatted packets are between WAN ports through which only IPsec-formatted packets are
forwarded to the Internet or to any other backbone network, thereby forwarded to the Internet or to any other backbone network, thereby
eliminating the need for MPLS transport in the backbone. eliminating the need for MPLS transport in the backbone.
[SECURE-L3VPN] assumes PEs use MPLS over IPsec when sending traffic [SECURE-L3VPN] assumes PEs use MPLS over IPsec when sending traffic
through the Black Interfaces. through the Black Interfaces.
[SECURE-EVPN] describes a solution where point-to-multipoint BGP [SECURE-L3VPN] is useful, but it misses the aspects of aggregating
signaling is used in the control plane for the Scenario #1 described VPN and Internet underlays. In addition:
in [BGP-SDWAN-Usage]. It relies upon a BGP cluster design to
facilitate the key and policy exchange among PE devices to create
private pair-wise IPsec Security Associations without IKEv2 point-
to-point signaling or any other direct peer-to-peer session
establishment messages.
Both [SECURE-L3VPN] and [SECURE-EVPN] are useful, however, they both
miss the aspects of aggregating VPN and Internet underlays. In
summary:
- Both documents assume that an IPsec tunnel is associated with
client traffic. Regardless of which WAN ports the traffic egress
from the edge, the client traffic associated with IPsec is always
encrypted. Within the context of an overlay architecture that
relies upon minimizing resource used for encryption, traffic sent
from an edge node can be encrypted once and forwarded through a
WAN port towards an untrusted network, but can also remain
unencrypted and be forwarded at different times through a WAN port
to the BGP/MPLS VPN.
- The [SECURE-L3VPN] assumes that a CPE "registers" with the RR. - The [SECURE-L3VPN] assumes that a CPE "registers" with the RR.
However, it does not say how. It assumes that the remote CPEs are However, it does not say how. It assumes that the remote CPEs are
pre-configured with the IPsec SA manually. For overlay networks to pre-configured with the IPsec SA manually. For overlay networks to
connect Hybrid Cloud DCs, Zero Touch Provisioning is expected. connect Hybrid Cloud DCs, Zero Touch Provisioning is expected.
Manual configuration is not an option. Manual configuration is not an option.
- The [SECURE-L3VPN] assumes that C-PEs and RRs are connected via an - The [SECURE-L3VPN] assumes that C-PEs and RRs are connected via an
IPsec tunnel. For management channel, TLS/DTLS is more economical IPsec tunnel. For management channel, TLS/DTLS is more economical
than IPsec. The following assumption made by [SECURE-L3VPN] can be than IPsec. The following assumption made by [SECURE-L3VPN] can be
difficult to meet in the environment where zero touch provisioning difficult to meet in the environment where zero touch provisioning
is expected: is expected:
A CPE must also be provisioned with whatever additional A CPE must also be provisioned with whatever additional
information is needed in order to set up an IPsec SA with information is needed in order to set up an IPsec SA with
each of the red RRs each of the red RRs
- IPsec requires periodic refreshment of the keys. The draft does - IPsec requires periodic refreshment of the keys. The [SECURE-
not provide any information about how to synchronize the L3VPN] does not provide any information about how to synchronize
refreshment among multiple nodes. the refreshment among multiple nodes.
- IPsec usually sends configuration parameters to two endpoints only - IPsec usually sends configuration parameters to two endpoints only
and lets these endpoints negotiate the key. The [SECURE-L3VPN] and lets these endpoints negotiate the key. The [SECURE-L3VPN]
assumes that the RR is responsible for creating/managing the key assumes that the RR is responsible for creating/managing the key
for all endpoints. When one endpoint is compromised, all other for all endpoints. When one endpoint is compromised, all other
connections may be impacted. connections may be impacted.
5.4. Preventing attacks from Internet-facing ports 5.5. Preventing attacks from Internet-facing ports
When C-PEs have Internet-facing ports, additional security risks are When C-PEs have Internet-facing ports, additional security risks are
raised. raised.
To mitigate security risks, in addition to requiring Anti-DDoS To mitigate security risks, in addition to requiring Anti-DDoS
features on C-PEs, it is necessary for C-PEs to support means to features on C-PEs, it is necessary for C-PEs to support means to
determine whether traffic sent by remote peers is legitimate to determine whether traffic sent by remote peers is legitimate to
prevent spoofing attacks, in particular. prevent spoofing attacks, in particular.
6. Gap Summary 6. Gap Summary
Here is the summary of the technical gaps discussed in this Here is the summary of the technical gaps discussed in this
document: document:
- For Accessing Cloud Resources - For Accessing Cloud Resources
a) When a remote vCPE can be reached by multiple PEs of one a) Traffic Path Management: when a remote vCPE can be reached by
provider VPN network, it is not straightforward to designate multiple PEs of one provider VPN network, it is not
which egress PE to the remote vCPE based on applications straightforward to designate which egress PE to the remote
b) Need automated and reliable tools to map the user-friendly vCPE based on applications or performance.
(natural language) access rules into machine readable b) NAT Traversal: There is no automatic way for an enterprise's
policies and to provide interfaces for enterprises to self- network controller to be informed of the NAT properties for
manage policy enforcement points for their own workloads. its workloads in Cloud DCs.
c) NAT Traversal. An enterprise's network controller needs to be c) There is no loop prevention for the multicast traffic to/from
informed of the NAT properties for its workloads in Cloud remote vCPE in Cloud DCs.
DCs. If the workloads are attached to the enterprise's own
vCPEs instantiated in the Cloud DCs, the task can be
achieved.
d) The multicast traffic to/from remote vCPE needs a feature
like Appointed Forwarder specified by TRILL to prevent
multicast data frames from looping around.
e) BGP between PEs and remote CPEs via untrusted networks.
f) Traffic Path Management
- Overlay Edge Node's WAN Port Management: BGP UPDATE propagate Needs a feature like Appointed Forwarder specified by TRILL to
client's routes information, but don't distinguish network facing prevent multicast data frames from looping around.
ports.
- Aggregating VPN paths and Internet paths d) BGP between PEs and remote CPEs via untrusted networks.
a) Control Plane for Overlay over Heterogeneous Networks is not - Missing control plane to manage the propagation of the property of
clear. networks connected to the virtual nodes in Cloud DCs.
b) BGP UPDATE Messages missing properties:
- Lacking SD-WAN Segments Identifier BGP UPDATE propagate client's routes information, but don't
- Missing attributes in Tunnel-Encap distinguish underlay networks.
c) SECURE-L3VPN/EVPN is not enough - Issues of aggregating traffic over private paths and Internet
d) Missing clear methods in preventing attacks from Internet- paths
a) Control plane messages for different overlay segmentations
needs to be differentiated. User traffic belonging to
different segmentations need to be differentiated.
b) BGP Tunnel Encap doesn't have ways to indicate a route or
prefix that can be carried by both IPsec tunnels and VPN
tunnels
c) Missing clear methods in preventing attacks from Internet-
facing ports facing ports
7. Manageability Considerations 7. Manageability Considerations
Zero touch provisioning of overlay networks to interconnect Hybrid Zero touch provisioning of overlay networks to interconnect Hybrid
Clouds is highly desired. It is necessary for a newly powered up Clouds is highly desired. It is necessary for a newly powered up
edge node to establish a secure connection (by means of TLS, DTLS, edge node to establish a secure connection (by means of TLS, DTLS,
etc.) with its controller. etc.) with its controller.
8. Security Considerations 8. Security Considerations
skipping to change at page 18, line 31 skipping to change at page 16, line 36
10.2. Informative References 10.2. Informative References
[RFC8192] S. Hares, et al, "Interface to Network Security Functions [RFC8192] S. Hares, et al, "Interface to Network Security Functions
(I2NSF) Problem Statement and Use Cases", July 2017 (I2NSF) Problem Statement and Use Cases", July 2017
[RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation Subsequent [RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation Subsequent
Address Family Identifier (SAFI) and the BGP Tunnel Address Family Identifier (SAFI) and the BGP Tunnel
Encapsulation Attribute", April 2009. Encapsulation Attribute", April 2009.
[BGP-SDWAN-PORT]L. Dunbar, et al, "Subsequent Address Family [BGP-EDGE-DISCOVERY] L. Dunbar, et al, "BGP UPDATE for SDWAN Edge
Indicator for SDWAN Ports", draft-dunbar-idr-sdwan-port- Discovery ", draft-dunbar-idr-sdwan-edge-discovery-00,
safi-00, Work-in-progress, March 2019. Work-in-progress, July 2020.
[BGP-SDWAN-Usage] L. Dunbar, et al, "Framework of Using BGP for [BGP-SDWAN-Usage] L. Dunbar, et al, "BGP Usage for SDWAN Overlay
SDWAN Overlay Networks", draft-dunbar-idr-sdwan-framework- Networks ", draft-dunbar-bess-bgp-sdwan-usage-08, work-in-
00, work-in-progress, Feb 2019. progress, July 2020.
[Tunnel-Encap]E. Rosen, et al, "The BGP Tunnel Encapsulation [Tunnel-Encap] K. Patel, et al, "The BGP Tunnel Encapsulation
Attribute", draft-ietf-idr-tunnel-encaps-10, July 2018. Attribute", draft-ietf-idr-tunnel-encaps-17, July 2020.
[SECURE-EVPN A. Sajassi, et al, draft-sajassi-bess-secure-evpn-01, [SECURE-EVPN] A. Sajassi, et al, draft-sajassi-bess-secure-evpn-01,
work in progress, March 2019. work in progress, March 2019.
[SECURE-L3VPN] E. Rosen, "Provide Secure Layer L3VPNs over Public [SECURE-L3VPN] E. Rosen, "Provide Secure Layer L3VPNs over Public
Infrastructure", draft-rosen-bess-secure-l3vpn-00, work- Infrastructure", draft-rosen-bess-secure-l3vpn-00, work-
in-progress, July 2018 in-progress, July 2018
[DMVPN] Dynamic Multi-point VPN: [DMVPN] Dynamic Multi-point VPN:
https://www.cisco.com/c/en/us/products/security/dynamic- https://www.cisco.com/c/en/us/products/security/dynamic-
multipoint-vpn-dmvpn/index.html multipoint-vpn-dmvpn/index.html
skipping to change at page 20, line 12 skipping to change at page 18, line 12
This document was prepared using 2-Word-v2.0.template.dot. This document was prepared using 2-Word-v2.0.template.dot.
Authors' Addresses Authors' Addresses
Linda Dunbar Linda Dunbar
Futurewei Futurewei
Email: ldunbar@futurewei.com Email: ldunbar@futurewei.com
Andrew G. Malis Andrew G. Malis
Independent Malis Consulting
Email: agmalis@gmail.com Email: agmalis@gmail.com
Christian Jacquenet Christian Jacquenet
Orange Orange
Rennes, 35000 Rennes, 35000
France France
Email: Christian.jacquenet@orange.com Email: Christian.jacquenet@orange.com
 End of changes. 60 change blocks. 
275 lines changed or deleted 219 lines changed or added

This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/