Network Working Group                                    L. Dunbar
Internet Draft                                           Futurewei
Intended status: Informational                            A. Malis
Expires: September 18, November 1, 2020                              Independent

                                                       C. Jacquenet
                                                             Orange
                                                     March 18,
                                                       May 1, 2020

           Networks Connecting to Hybrid Cloud DCs: Gap Analysis
                draft-ietf-rtgwg-net2cloud-gap-analysis-05
                draft-ietf-rtgwg-net2cloud-gap-analysis-06

Abstract

   This document analyzes the technical gaps that may affect the
   dynamic connection to workloads and applications hosted in hybrid
   Cloud Data Centers from enterprise premises.

Status of this Memo

   This Internet-Draft is submitted in full conformance with the
   provisions of BCP 78 and BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other documents
   at any time.  It is inappropriate to use Internet-Drafts as
   reference material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html

   This Internet-Draft will expire on September 18, November 1, 2020.

xxx, et al.             Expires September 18, 2020              [Page
1]

Copyright Notice

   Copyright (c) 2020 IETF Trust and the persons identified as the
   document authors. All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document. Please review these documents
   carefully, as they describe your rights and restrictions with
   respect to this document. Code Components extracted from this
   document must include Simplified BSD License text as described in
   Section 4.e of the Trust Legal Provisions and are provided without
   warranty as described in the Simplified BSD License.

Table of Contents

   1. Introduction...................................................3
   2. Conventions used in this document..............................3
   3. Gap Analysis for Accessing Cloud Resources.....................4
      3.1. Multiple PEs connecting to virtual CPEs in Cloud DCs......7
      3.2. Access Control for workloads in the Cloud DCs.............7
      3.3. NAT Traversal.............................................8
      3.4. BGP between PEs and remote CPEs via Internet..............8
      3.5. Multicast traffic from/to the remote edges................9
   4. Gap Analysis of Overlay Edge Node's WAN Port Management........4 Management........9
   5. Aggregating VPN paths and Internet paths.......................6 paths......................11
      5.1. Control Plane for Overlay over Heterogeneous Networks.....7 Networks....12
      5.2. Using BGP UPDATE Messages.................................8 Messages................................13
         5.2.1. Lacking SD-WAN Segments Identifier...................8 Identifier..................13
         5.2.2. Missing attributes in Tunnel-Encap...................8 Tunnel-Encap..................13
      5.3. SECURE-L3VPN/EVPN.........................................9 SECURE-L3VPN/EVPN........................................15
      5.4. Preventing attacks from Internet-facing ports............11 ports............16
   6. C-PEs not directly connected to VPN PEs.......................11
      6.1. Floating PEs to connect to Remote CPEs...................14
      6.2. NAT Traversal............................................14
      6.3. Complexity of using BGP between PEs and remote CPEs via
      Internet......................................................14
      6.4. Designated Forwarder to the remote edges.................15
      6.5. Traffic Path Management..................................16 Gap Summary...................................................16
   7. Manageability Considerations..................................16 Considerations..................................17
   8. Security Considerations.......................................16 Considerations.......................................17
   9. IANA Considerations...........................................17 Considerations...........................................18
   10. References...................................................17 References...................................................18
      10.1. Normative References....................................17 References....................................18
      10.2. Informative References..................................17 References..................................18
   11. Acknowledgments..............................................18 Acknowledgments..............................................19

1. Introduction

   [Net2Cloud-Problem] describes the problems enterprises face today
   when interconnecting their branch offices with dynamic workloads
   hosted in third party data centers (a.k.a. Cloud DCs). In
   particular, this document analyzes the routing protocols to identify
   whether there are any gaps that may impede such interconnection
   which may for example justify additional specification effort to
   define proper protocol extensions.

   For the sake of readability, an edge, an endpoint, C-PE, or CPE are
   used interchangeably throughout this document. More precisely:

     . Edge: may include multiple devices (virtual or physical);
     . endpoint: refers to a WAN port of device located in the edge;
     . C-PE: provider-owned edge, e.g. for SECURE-EVPN's PE-based
        BGP/MPLS VPN, where PE is the edge node;
     . CPE: device located in enterprise premises.

2. Conventions used in this document

   Cloud DC:   Third party Data Centers that usually host applications
               and workload owned by different organizations or
               tenants.

   Controller: Used interchangeably with Overlay controller to manage
               overlay path creation/deletion and monitor the path
               conditions between sites.

   CPE-Based VPN: Virtual Private Network designed and deployed from
               CPEs. This is to differentiate from most commonly used
               PE-based VPNs a la RFC 4364.

   OnPrem:     On Premises data centers and branch offices

   SDWAN:      Software Defined Wide Area Network, "SDWAN" refers to
               the solutions of pooling WAN bandwidth from multiple
               underlay networks to get better WAN bandwidth
               management, visibility & control. When the underlay is a
               private network, traffic may be forwarded without any
               additional encryption; when the underlay networks are
               public, such as the Internet, some traffic needs to be
               encrypted when passing through (depending on user-
               provided policies).

3. Gap Analysis for Accessing Cloud Resources

   Many problems described in the [Net2Cloud-Problem] are not in the
   scope

   Because of IETF, let alone IETF Routing area. This document primarily
   focuses on the gap analysis for protocols in IETF Routing area.

4. Gap Analysis ephemeral property of Overlay Edge Node's WAN Port Management

   Very often the Hybrid selected Cloud DCs are interconnected by overlay
   networks that arch over many different types of networks, such as
   VPN, public Internet, wireless and wired infrastructures, etc.
   Sometimes the enterprises' VPN providers do for
   specific workloads/Apps, an enterprise or its network service
   provider may not have direct access physical connections to the Cloud DCs
   that host some specific applications or workloads
   operated by are optimal for hosting the enterprise. enterprise's specific
   workloads/Apps. Under those circumstances, the an overlay network's edge nodes can have
   WAN ports facing networks provided by different ISPs, some of these
   networks may not be trustable, some others network design
   can be trusted like VPNs
   (to some extent), etc.

   If all WAN ports of an edge node are facing an untrusted network,
   then all sensitive option to interconnect the enterprise's on-premises data to/from this edge node have
   centers & branch offices to be encrypted,
   usually by means of IPsec tunnels which its desired Cloud DCs.

   However, overlay paths established over the public Internet can be terminated at have
   unpredictable performance, especially over long distances and across
   operators' domains. Therefore, it is highly desirable to minimize
   the WAN
   port address, at the edge node's loopback address if the loopback
   address is routable in the wide area network, distance or even at the ingress
   ports number of the edge node.

   If an edge node has some WAN ports facing trusted networks and
   others facing untrusted networks, sensitive data can segments that traffic had to be
   forwarded
   through ports facing over the trusted networks natively (i.e., without
   encryption) and forwarded through ports facing untrusted networks
   assuming encryption. To achieve this flexibility public Internet.

   The Metro Ethernet Forum's Cloud Service Architecture [MEF-Cloud]
   also describes a use case of sending traffic
   either encrypted network operators using Overlay paths
   over an LTE network or not encrypted depending on egress WAN ports, it
   is necessary to have the IPsec tunnels terminated at public Internet for the WAN ports
   facing last mile access
   where the untrusted networks.

   In order to establish peer-wise secure encrypted communications
   among those WAN ports of two edge nodes, it is necessary for VPN service providers cannot always provide the required
   physical infrastructure.

   In these scenarios, some overlay edge nodes (peers) to may not be informed of directly
   attached to the WAN port properties.

   Some PEs that participate to the delivery and the
   operation of those overlay networks (such as some deployed SDWAN
   networks) use the modified NHRP protocol [RFC2332] enterprise's VPN.

   When using an overlay network to register WAN
   ports of connect the edge nodes with their Controller (or NHRP server),
   which then maps a private VPN address enterprise's sites to a public IP address of
   the
   destination node/port. DSVPN [DSVPN] or DMVPN [DMVPN] are used workloads hosted in Cloud DCs, the existing C-PEs at
   enterprise's sites have to
   establish tunnels between WAN ports of SDWAN edge nodes.

   NHRP was originally intended for ATM address resolution, and as a
   result, it misses many attributes that are necessary for dynamic
   endpoint C-PE registration be upgraded to connect to the controller, such as:

   - Interworking with the MPLS VPN control plane. An said
   overlay edge can
     have some ports facing network.  If the MPLS VPN network over which packets can workloads hosted in Cloud DCs need to be forwarded without any encryption and some ports facing the
     public Internet over which sensitive traffic needs
   connected to many sites, the upgrade process can be
     encrypted.
   - Scalability: NHRP/DSVPN/DMVPN work fine with small numbers of edge
     nodes. When very expensive.

   [Net2Cloud-Problem] describes a hybrid network has more than 100 nodes, these protocols do
     not scale well.
   - NHRP does not have approach that extends
   the IPsec attributes, which are needed for
     peers existing MPLS-based VPNs to build Security Associations the Cloud DC Workloads over the public Internet.
   - NHRP messages do
   access paths that are not have any field under the VPN provider's control. To make
   it work properly, a small number of the PEs of the BGP/MPLS VPN can
   be designated to connect to encode the C-PE supported
     encapsulation types, such remote workloads via secure IPsec
   tunnels.  Those designated PEs are shown as IPsec-GRE fPE (floating PE or IPsec-VxLAN.
   - NHRP messages do not have any field to encode C-PE Location
     identifiers, such as Site Identifier, System ID, and/or Port ID.
   - NHRP messages do not have any field to describe
   smart PE) in Figure 3. Once the gateway(s) to
     which secure IPsec tunnels are
   established, the C-PE is attached. When a C-PE is instantiated workloads hosted in a Cloud
     DC, it is desirable for the C-PE's owner to DCs can be informed about how
     and where the C-PE is attached.
   - NHRP messages do not have any field to describe C-PE's NAT
     properties if reached by the C-PE is using private IPv4 addresses, such as
   enterprise's VPN without upgrading all of the NAT type, Private address, Public address, Private port,
     Public port, etc.

   [BGP-SDWAN-PORT] describes how to use BGP enterprise's CPEs. The
   only CPE that needs to distribute SDWAN edge
   properties connect to peers.  SDWAN is an the overlay network with specific
   properties, such as application-based forwarding, augmented
   transport, and user specified policies. There is would be a need
   virtualized CPE instantiated within the cloud DC.

   +--------+                                             +--------+
   | Host-a +--+                                     +----| Host-b |
   |        |  |                                    (')   |        |
   +--------+  |           +-----------+           (   )  +--------+
               |  +-+--+  ++-+        ++-+  +--+-+  (_)
               |  | CPE|--|PE|        |PE+--+ CPE|   |
               +--|    |  |  |        |  |  |    |---+
                  +-+--+  ++-+        ++-+  +----+
                   /       |           |
                  /        |  MPLS   +-+---+    +--+-++--------+
          +------+-+       | Network |fPE-1|    |CPE || Host   |
          | Host   |       |         |     |- --|    ||   d    |
          |   c    |       +-----+   +-+---+    +--+-++--------+
          +--------+       |fPE-2|-----+
                           +---+-+    (|)
                              (|)     (|) Overlay
                              (|)     (|) over any access
                              +=\======+=========+
                             //   \    | Cloud DC \\
                            //      \ ++-----+       \\
                                      +      |
                                      | vCPE |
                                      +-+----+
                            ----+-------+-------+-----
                                |               |
                            +---+----+      +---+----+
                            | Remote |      | Remote |
                            | App-1  |      | App-2  |
                            +--------+      +--------+

                    Figure 1: VPN Extension to extend Cloud DC

   In Figure 1, the protocol to register WAN port properties of an edge node optimal Cloud DC to host the
   overlay controller, which then propagates workloads (as a
   function of the information to proximity, capacity, pricing, or any other
   overlay edge nodes that are authenticated and authorized to
   communicate with them.

5. Aggregating VPN paths and Internet paths

   Most likely, enterprises (especially criteria
   chosen by the largest ones) already enterprises) does not have
   their C-PEs interconnected by VPN service providers, based upon VPN
   techniques like EVPN, L2VPN, or L3VPN, and which can be lead a direct connection to PE-
   based or CPE-based the
   PEs of the NGP/MPLS VPN service designs. The commonly used PE-based that interconnects the enterprise's sites.

3.1. Multiple PEs connecting to virtual CPEs in Cloud DCs

   To extend BGP/MPLS VPNs have C-PEs directly attached to PEs, the communication
   between C-PEs and PEs virtual CPEs in Cloud DCs, it is considered as
   necessary to establish secure tunnels (such as they are connected
   by direct physical links albeit there could be routes leaking or
   unauthorized routes being injected. MP-BGP IPsec tunnels)
   between the PEs and the vCPEs.

   Even though a set of PEs can be used to learn &
   distribute routes among C-PEs, but sometimes routes among C-PEs are
   statically configured on the C-PEs.

   For enterprises already interconnected by VPNs, if manually selected for a specific
   cloud data center, there are short
   term high traffic volume that can't justify increasing no standard protocols for those PEs to
   interact with the VPNs
   capacity, it vCPEs instantiated in the third party cloud data
   centers over unsecure networks. The interaction includes exchanging
   performance, route information, etc..

   When there is desirable more than one PE available for use (as there should be
   for resiliency purposes or because of the CPE need to aggregate the bandwidth
   that pertains support multiple
   cloud DCs geographically scattered), it is not straightforward to VPN paths and Internet paths by adding ports
   designate an egress PE to remote vCPEs based on applications.  It
   might not be possible for PEs to recognize all applications because
   too much traffic traversing the PEs.

   When there are multiple floating PEs that
   connect have established IPsec
   tunnels with a remote CPE, the remove CPE can forward outbound
   traffic to the public Internet. Under this scenario, optimal PE, which
   is referred in turn forwards traffic to as egress
   PEs to reach the Overlay scenario throughout this document, final destinations. However, it is necessary not
   straightforward for the C-PEs ingress PE to manage and communicate with select which egress PEs to
   send traffic. For example, in Figure 1:

     - fPE-1 is the
   controller on how traffic optimal PE for communication between App-1 <->
     Host-a due to latency, pricing or other criteria.

     - fPE-2 is distributed among the optimal PE for communication between App-1 <->
     Host-b.

3.2. Access Control for workloads in the Cloud DCs

   There is widespread diffusion of access policy for Cloud Resource,
   some of which is not easy for verification and validation. Because
   there are multiple
   heterogeneous underlay networks, parties involved in accessing Cloud Resources,
   policy enforcement points are not easily visible for policy
   refinement, monitoring, and also to manage secure tunnels
   over untrusted networks.

   When using NHRP testing.

   The current state of the art for WAN port registration purposes, C-PEs need to
   run two separate control planes: EVPN&BGP specifying access policies for CPE-based VPNs,
   Cloud Resources could be improved by having automated and
   NHRP & DSVPN/DMVPN for ports connected reliable
   tools to map the Internet. Two separate
   control planes not only add complexity user-friendly (natural language) rules into machine
   readable policies and to C-PEs, but also increase
   operational costs.

                                       +---+
                        +--------------|RR |----------+
                       /  Untrusted    +-+-+           \
                      /                                 \
                     /                                   \
     +----+  +---------+  packets encrypted over     +------+  +----+
     | TN3|--|         A1-----+ Untrusted    +------ B1     |--| TN1|
     +----+  | C-PE    A2-\                          | C-PE |  +----+
     +----+  |  A      A3--+--+              +---+---B2  B  |  +----+
     | TN2|--|         |   |PE+--------------+PE |---B3     |--| TN3|
     +----+  +---------+   +--+   trusted    +---+   +------+  +----+
                              |      WAN     |
     +----+  +---------+   +--+   packets    +---+   +------+  +----+
     | TN1|--|         C1--|PE| go natively  |PE |-- D1     |--| TN1|
     +----+  | C-PE    C2--+--+ without encry+---+   | C-PE |  +----+
             |  C      |      +--------------+       |  D   |
             |         |                             |      |
     +----+  |         C3--|  without encrypt over   |      |  +----+
     | TN2|--|         C4--+---- Untrusted  --+------D2     |--| TN2|
     +----+  +---------+                             +------+  +----+
       Figure 1: CPEs interconnected by VPN paths and Internet Paths

 5.1. Control Plane provide interfaces for Overlay over Heterogeneous Networks

   As described in [BGP-SDWAN-Usage], the Control Plane enterprises to self-
   manage policy enforcement points for Overlay
   network over heterogeneous networks has three distinct properties:

     - WAN Port Property registration their own workloads.

3.3. NAT Traversal

   Cloud DCs that only assign private IPv4 addresses to the Overlay Controller.
          o To inform the Overlay controller and authorized peers of
   instantiated workloads assume that traffic to/from the WAN port properties workload
   usually needs to traverse NATs.
   An overlay edge node can solicit a STUN (Session Traversal of UDP
   Through Network Address Translation, [RFC3489]) Server to get the Edge nodes. When the WAN
             ports are assigned private IPv4 addresses, this step can
             register
   information about the type of NAT that translates these addresses
             into property, the public ones.

     - Controller-facilitated IPsec SA management IP addresses and NAT port
   numbers so that such information
        distribution
          o The Overlay controller facilitates and manages the IPsec
             configuration and peer authentication for all IPsec
             tunnels terminated at the edge nodes.

     - Establishing and Managing the topology and reachability for
        services attached to the client ports of overlay edge nodes.

          o This is for the overlay layer's route distribution, so
             that a C-PE can populate its overlay routing table with
             entries that identify the next hop for reaching a specific
             route/service attached to remote nodes. [SECURE-EVPN]
             describes EVPN and other options.

 5.2. Using BGP UPDATE Messages

 5.2.1. Lacking SD-WAN Segments Identifier

   There could be multiple SD-WAN networks with their edge nodes
   exchanging BGP UPDATE messages with communicated to the relevant
   peers.

3.4. BGP RR. The multiple SD-WAN
   networks could have common underlay networks. Therefore, it is very
   important to have between PEs and remote CPEs via Internet

   Even though an identifier EBGP (external BGP) Multi-Hop design can be used to differentiate BGP UPDATE messages
   belonging
   connect peers that are not directly connected to different SD-WAN networks (or sometimes called SD-WAN
   Segmentations). Today's BGP doesn't have this feature yet, unless each other, there
   are multiple still some issues about extending BGP instances and their corresponding RRs.

 5.2.2. Missing attributes in Tunnel-Encap

   [Tunnel-Encap] describes from MPLS VPN PEs to
   remote CPEs via any access path (e.g., Internet).

   The path between the BGP UPDATE Tunnel Path Attribute remote CPEs and VPN PEs that
   advertises endpoints' tunnel encapsulation capabilities for the
   respective attached client maintain VPN
   routes encoded in the MP-NLRI Path
   Attribute. The receivers of the BGP UPDATE can traverse untrusted segments.

   EBGP Multi-hop design requires configuration on both peers, either
   manually or via NETCONF from a controller. To use any of the
   supported encapsulations encoded in EBGP between a PE
   and remote CPEs, the Tunnel Path Attribute for PE has to be manually configured with the routes encoded in
   "next-hop" set to the MP-NLRI Path Attribute.

   Here are some IP address of the issues raised by CPEs. When remote CPEs,
   especially remote virtualized CPEs are dynamically instantiated or
   removed, the use configuration of [Tunnel-Encap] to
   distribute Edge WAN port properties:

   - [Tunnel-Encap] doesn't have the encoding to describe Multi-Hop EBGP on the NAT
     information for WAN ports that are assigned private IPv4 addresses
     yet. The NAT information needs PE has to be propagated to the trusted
     peers such as the virtual C-PEs instantiated
   changed accordingly.

     Egress peering engineering (EPE) is not sufficient. Running BGP on
     virtualized CPEs in public Cloud DCs
     via Controllers.
   - The mechanism defined in [Tunnel-Encap] does not facilitate the
     exchange of IPsec SA-specific parameters independently from
     advertising the attached clients' routes, even after adding a new
     IPsec tunnel type.
     [Tunnel-Encap] requires all GRE tunnels updates to be associated with
     routes. There can be many client routes associated with an IPsec
     tunnel
     established between two C-PEs' WAN ports; the corresponding
     destination prefixes (as announced by the aforementioned routes)
     may also be reached through first, which requires the remote CPEs to support
     address and key management capabilities. RFC 7024 (Virtual Hub &
     Spoke) and Hierarchical VPN underlay without any
     encryption.

     The establishment of an IPsec tunnel can fail, e.g., because the
     two endpoints do not support different encryption algorithms. Multiple
     negotiation messages that carry the IPsec SA parameters between
     two end-points may be exchanged. This is why it required
     properties.

     Also, there is cleaner a need for a mechanism to
     separate the establishment of an IPsec SA association between two
     end-points from the policies enforced to map routes automatically trigger
     configuration changes on PEs when remote CPEs' are instantiated or
     moved (leading to an IP address change) or deleted.

     EBGP Multi-hop design does not include a specific
     IPSec SA.

     If C-PEs security mechanism by
     default. The PE and remote CPEs need to establish a WAN Port-based IPsec SA, secure communication channels
     when connecting via the
     information encoded public Internet.

   Remote CPEs, if instantiated in the Tunnel Path Attribute should only apply Cloud DCs might have to traverse
   NATs to reach PEs. It is not clear how BGP can be used between
   devices located beyond the WAN ports NAT and should be independent from the clients'
     routes.

     In addition, devices located behind the Overlay IPsec SA Tunnel
   NAT. It is very likely to be
     established before clients' routes are attached.

   - When an overlay network spans across large geographic regions
     (such as countries or continents), one C-PE in one region may not
     even be aware of remote CPEs in other regions that it needs clear how to
     communicate.  Therefore, configure the distribution of Next Hop on the Overlay Edge WAN
     ports information need to be restricted PEs to
   reach private IPv4 addresses.

3.5. Multicast traffic from/to the authorized peers.

5.3. SECURE-L3VPN/EVPN

   [SECURE-L3VPN] describes remote edges

   Among the multiple floating PEs that are reachable from a method to enrich BGP/MPLS remote
   CPE, multicast traffic sent by the remote CPE towards the MPLS VPN [RFC4364]
   capabilities
   can be forwarded back to allow some PEs the remote CPE due to connect the PE receiving the
   multicast packets forwarding the multicast/broadcast frame to other
   PEs via public
   networks. [SECURE-L3VPN] introduces the concept of Red Interface &
   Black Interface used that in turn send to all attached CPEs. This process may cause
   traffic loops.

   This problem can be solved by PEs, where selecting one floating PE as the RED interfaces are used CPE's
   Designated Forwarder, similar to
   forward traffic into TRILL's Appointed Forwarders
   [RFC6325].

   BGP/MPLS VPNs do not have features like TRILL's Appointed
   Forwarders.

4. Gap Analysis of Overlay Edge Node's WAN Port Management

   Very often the Hybrid Cloud DCs are interconnected by overlay
   networks that arch over many different types of networks, such as
   VPN, public Internet, wireless and wired infrastructures, etc.
   Sometimes the Black Interfaces are used
   between WAN ports through which only IPsec-formatted packets are
   forwarded enterprises' VPN providers do not have direct access
   to the Internet Cloud DCs that host some specific applications or to any other backbone network, thereby
   eliminating workloads
   operated by the need for MPLS transport in enterprise.

   Under those circumstances, the backbone.

   [SECURE-L3VPN] assumes PEs use MPLS over overlay network's edge nodes can have
   WAN ports facing networks provided by different ISPs, some of these
   networks may not be trustable, some others can be trusted like VPNs
   (to some extent), etc.

   If all WAN ports of an edge node are facing an untrusted network,
   then all sensitive data to/from this edge node have to be encrypted,
   usually by means of IPsec when sending traffic
   through tunnels which can be terminated at the Black Interfaces.

   [SECURE-EVPN] describes a solution where point-to-multipoint BGP
   signaling WAN
   port address, at the edge node's loopback address if the loopback
   address is used routable in the control plane for the Scenario #1 described
   in [BGP-SDWAN-Usage]. It relies upon a BGP cluster design to
   facilitate the key and policy exchange among PE devices to create
   private pair-wise IPsec Security Associations without IKEv2 point-
   to-point signaling wide area network, or any other direct peer-to-peer session
   establishment messages.

   Both [SECURE-L3VPN] and [SECURE-EVPN] are useful, however, they both
   miss even at the aspects of aggregating VPN and Internet underlays. In
   summary:

   - Both documents assume that an IPsec tunnel is associated with
     client traffic. Regardless of which WAN ingress
   ports the traffic egress
     from the edge, the client traffic associated with IPsec is always
     encrypted. Within the context of an overlay architecture that
     relies upon minimizing resource used for encryption, traffic sent
     from the edge node.

   If an edge node has some WAN ports facing trusted networks and
   others facing untrusted networks, sensitive data can be encrypted once and forwarded
   through a
     WAN port towards an untrusted network, but can also remain
     unencrypted ports facing the trusted networks natively (i.e., without
   encryption) and be forwarded at different times through a ports facing untrusted networks
   assuming encryption. To achieve this flexibility of sending traffic
   either encrypted or not encrypted depending on egress WAN port ports, it
   is necessary to have the BGP/MPLS VPN.

   - The [SECURE-L3VPN] assumes that a CPE "registers" with the RR.
     However, it does not say how. It assumes that IPsec tunnels terminated at the remote CPEs are
     pre-configured with WAN ports
   facing the IPsec SA manually. For overlay networks untrusted networks.

   In order to
     connect Hybrid Cloud DCs, Zero Touch Provisioning establish peer-wise secure encrypted communications
   among those WAN ports of two edge nodes, it is expected.
     Manual configuration is not an option.

   - The [SECURE-L3VPN] assumes that C-PEs and RRs are connected via an
     IPsec tunnel. For management channel, TLS/DTLS is more economical
     than IPsec. The following assumption made by [SECURE-L3VPN] can be
     difficult to meet in necessary for the environment where zero touch provisioning
     is expected:
          A CPE must also be provisioned with whatever additional
          information is needed in order
   edge nodes (peers) to set up an IPsec SA with
          each be informed of the red RRs

   - IPsec requires periodic refreshment WAN port properties.

   Some of those overlay networks (such as some deployed SDWAN
   networks) use the keys. The draft does
     not provide any information about how modified NHRP protocol [RFC2332] to synchronize register WAN
   ports of the
     refreshment among multiple nodes.
   - IPsec usually sends configuration parameters edge nodes with their Controller (or NHRP server),
   which then maps a private VPN address to two endpoints only
     and lets these endpoints negotiate the key. The [SECURE-L3VPN]
     assumes that the RR is responsible for creating/managing a public IP address of the key
     for all endpoints. When one endpoint is compromised, all other
     connections may be impacted.

5.4. Preventing attacks from Internet-facing ports

   When C-PEs have Internet-facing ports, additional security risks
   destination node/port. DSVPN [DSVPN] or DMVPN [DMVPN] are
   raised.

   To mitigate security risks, in addition used to requiring Anti-DDoS
   features on C-PEs,
   establish tunnels between WAN ports of SDWAN edge nodes.

   NHRP was originally intended for ATM address resolution, and as a
   result, it is misses many attributes that are necessary for C-PEs to support means dynamic
   endpoint C-PE registration to
   determine whether traffic sent by remote peers is legitimate to
   prevent spoofing attacks, in particular.

6. C-PEs not directly connected to VPN PEs

   Because of the ephemeral property of controller, such as:

   - Interworking with the selected Cloud DCs for
   specific workloads/Apps, an enterprise or its network service
   provider may not MPLS VPN control plane. An overlay edge can
     have direct physical connections to the Cloud DCs
   that are optimal for hosting some ports facing the enterprise's specific
   workloads/Apps. Under those circumstances, an overlay MPLS VPN network design over which packets can
     be an option to interconnect the enterprise's on-premises data
   centers & branch offices to its desired Cloud DCs.

   However, overlay paths established over forwarded without any encryption and some ports facing the
     public Internet can have
   unpredictable performance, especially over long distances and across
   operators' domains. Therefore, it is highly desirable to minimize
   the distance or the number of segments that which sensitive traffic had needs to be
   forwarded over the public Internet.

   The Metro Ethernet Forum's Cloud Service Architecture [MEF-Cloud]
   also describes a use case
     encrypted.

   - Scalability: NHRP/DSVPN/DMVPN work fine with small numbers of network operators using Overlay paths
   over edge
     nodes. When a LTE network or the public Internet for the last mile access
   where the VPN service providers cannot always provide the required
   physical infrastructure.

   In has more than 100 nodes, these scenarios, some overlay edge nodes may protocols do
     not be directly
   attached to scale well.
   - NHRP does not have the PEs that participate IPsec attributes, which are needed for
     peers to build Security Associations over the delivery and the
   operation of public Internet.
   - NHRP messages do not have any field to encode the enterprise's VPN.

   When using an overlay network C-PE supported
     encapsulation types, such as IPsec-GRE or IPsec-VxLAN.
   - NHRP messages do not have any field to connect encode C-PE Location
     identifiers, such as Site Identifier, System ID, and/or Port ID.
   - NHRP messages do not have any field to describe the enterprise's sites gateway(s) to
     which the workloads hosted C-PE is attached. When a C-PE is instantiated in a Cloud DCs,
     DC, it is desirable for the corresponding C-PEs have C-PE's owner to be upgraded to connect informed about how
     and where the C-PE is attached.
   - NHRP messages do not have any field to describe C-PE's NAT
     properties if the said overlay network.  If C-PE is using private IPv4 addresses, such as
     the
   workloads hosted in Cloud DCs need NAT type, Private address, Public address, Private port,
     Public port, etc.

   [BGP-SDWAN-PORT] describes how to be connected use BGP to many sites,
   the upgrade process can be very expensive.

   [Net2Cloud-Problem] describes a hybrid distribute SDWAN edge
   properties to peers.  SDWAN is an overlay network approach that extends with specific
   properties, such as application-based forwarding, augmented
   transport, and user specified policies. There is a need to extend
   the existing MPLS-based VPNs protocol to register WAN port properties of an edge node to the Cloud DC Workloads over
   overlay controller, which then propagates the
   access paths information to other
   overlay edge nodes that are not under the authenticated and authorized to
   communicate with them.

5. Aggregating VPN provider's control. To make
   it work properly, a small number of the PEs of paths and Internet paths

   Most likely, enterprises (especially the BGP/MPLS largest ones) already have
   their C-PEs interconnected by VPN can
   be designated to connect to service providers, based upon VPN
   techniques like EVPN, L2VPN, or L3VPN, and which can be lead to PE-
   based or CPE-based VPN service designs. The commonly used PE-based
   BGP/MPLS VPNs have C-PEs directly attached to PEs, the remote workloads via secure IPsec
   tunnels.  Those designated communication
   between C-PEs and PEs are shown is considered as fPE (floating PE or
   smart PE) in Figure 3. Once the secure IPsec tunnels as they are
   established, the workloads hosted in Cloud DCs connected
   by direct physical links albeit there could be routes leaking or
   unauthorized routes being injected. MP-BGP can be reached used to learn &
   distribute routes among C-PEs, but sometimes routes among C-PEs are
   statically configured on the C-PEs.

   For enterprises already interconnected by VPNs, if there are short
   term high traffic volume that can't justify increasing the
   enterprise's VPN without upgrading all of VPNs
   capacity, it is desirable for the enterprise's CPEs. The
   only CPE to aggregate the bandwidth
   that needs pertains to VPN paths and Internet paths by adding ports that
   connect to the overlay network would be a
   virtualized CPE instantiated within to the cloud DC.

   +--------+                                             +--------+
   | Host-a +--+                                     +----| Host-b |
   |        |  |                                    (')   |        |
   +--------+  |           +-----------+           (   )  +--------+
               |  +-+--+  ++-+        ++-+  +--+-+  (_)
               |  | CPE|--|PE|        |PE+--+ CPE|   |
               +--|    |  |  |        |  |  |    |---+
                  +-+--+  ++-+        ++-+  +----+
                   /       |           |
                  /        |  MPLS   +-+---+    +--+-++--------+
          +------+-+       | Network |fPE-1|    |CPE || Host   |
          | Host   |       |         |     |- --|    ||   d    |
          |   c    |       +-----+   +-+---+    +--+-++--------+
          +--------+       |fPE-2|-----+
                           +---+-+    (|)
                              (|)     (|) Overlay
                              (|)     (|) public Internet. Under this scenario, which
   is referred to as the Overlay scenario throughout this document, it
   is necessary for the C-PEs to manage and communicate with the
   controller on how traffic is distributed among multiple
   heterogeneous underlay networks, and also to manage secure tunnels
   over any access
                              +=\======+=========+
                             // untrusted networks.

   When using NHRP for WAN port registration purposes, C-PEs need to
   run two separate control planes: EVPN&BGP for CPE-based VPNs, and
   NHRP & DSVPN/DMVPN for ports connected to the Internet. Two separate
   control planes not only add complexity to C-PEs, but also increase
   operational costs.
                                       +---+
                        +--------------|RR |----------+
                       /  Untrusted    +-+-+           \    | Cloud DC \\
                            //
                      /                                 \ ++-----+       \\
                                      +Remote|
                     /                                   \
     +----+  +---------+  packets encrypted over     +------+  +----+
     |  CPE TN3|--|         A1-----+ Untrusted    +------ B1     |--| TN1|
     +----+  |
                                      +-+----+
                            ----+-------+-------+----- C-PE    A2-\                          | C-PE |
                            +---+----+      +---+----+  +----+
     +----+  | Remote  A      A3--+--+              +---+---B2  B  |  +----+
     | Remote TN2|--|         |   |PE+--------------+PE |---B3     |--| TN3|
     +----+  +---------+   +--+   trusted    +---+   +------+  +----+
                              | App-1      WAN     |
     +----+  +---------+   +--+   packets    +---+   +------+  +----+
     | App-2 TN1|--|         C1--|PE| go natively  |PE |-- D1     |--| TN1|
     +----+  |
                            +--------+      +--------+ C-PE    C2--+--+ without encry+---+   | C-PE |  +----+
             |  C      |      +--------------+       |  D   |
             |         |                             |      |
     +----+  |         C3--|  without encrypt over   |      |  +----+
     | TN2|--|         C4--+---- Untrusted  --+------D2     |--| TN2|
     +----+  +---------+                             +------+  +----+
       Figure 3: 2: CPEs interconnected by VPN Extension paths and Internet Paths

 5.1. Control Plane for Overlay over Heterogeneous Networks

   As described in [BGP-SDWAN-Usage], the Control Plane for Overlay
   network over heterogeneous networks has three distinct properties:

     - WAN Port Property registration to Cloud DC

   In Figure 3, the optimal Cloud DC to host Overlay Controller.
          o To inform the Overlay controller and authorized peers of
             the WAN port properties of the Edge nodes. When the WAN
             ports are assigned private IPv4 addresses, this step can
             register the type of NAT that translates these addresses
             into public ones.

     - Controller-facilitated IPsec SA management and NAT information
        distribution
          o The Overlay controller facilitates and manages the IPsec
             configuration and peer authentication for all IPsec
             tunnels terminated at the edge nodes.

     - Establishing and Managing the topology and reachability for
        services attached to the client ports of overlay edge nodes.
          o This is for the overlay layer's route distribution, so
             that a C-PE can populate its overlay routing table with
             entries that identify the next hop for reaching a specific
             route/service attached to remote nodes. [SECURE-EVPN]
             describes EVPN and other options.

 5.2. Using BGP UPDATE Messages

 5.2.1. Lacking SD-WAN Segments Identifier

   There could be multiple SD-WAN networks with their edge nodes
   exchanging BGP UPDATE messages with the BGP RR. The multiple SD-WAN
   networks could have common underlay networks. Therefore, it is very
   important to have an identifier to differentiate BGP UPDATE messages
   belonging to different SD-WAN networks (or sometimes called SD-WAN
   Segmentations). Today's BGP doesn't have this feature yet, unless
   there are multiple BGP instances and their corresponding RRs.

 5.2.2. Missing attributes in Tunnel-Encap

   [Tunnel-Encap] describes the BGP UPDATE Tunnel Path Attribute that
   advertises endpoints' tunnel encapsulation capabilities for the workloads (as a
   function
   respective attached client routes encoded in the MP-NLRI Path
   Attribute. The receivers of the proximity, capacity, pricing, or BGP UPDATE can use any other criteria
   chosen of the
   supported encapsulations encoded in the Tunnel Path Attribute for
   the routes encoded in the MP-NLRI Path Attribute.

   Here are some of the issues raised by the enterprises) use of [Tunnel-Encap] to
   distribute Edge WAN port properties:

   - [Tunnel-Encap] doesn't have the encoding to describe the NAT
     information for WAN ports that are assigned private IPv4 addresses
     yet. The NAT information needs to be propagated to the trusted
     peers such as the virtual C-PEs instantiated in public Cloud DCs
     via Controllers.
   - The mechanism defined in [Tunnel-Encap] does not have facilitate the
     exchange of IPsec SA-specific parameters independently from
     advertising the attached clients' routes, even after adding a direct connection new
     IPsec tunnel type.
     [Tunnel-Encap] requires all tunnels updates to be associated with
     routes. There can be many client routes associated with an IPsec
     tunnel established between two C-PEs' WAN ports; the
   PEs of corresponding
     destination prefixes (as announced by the aforementioned routes)
     may also be reached through the NGP/MPLS VPN underlay without any
     encryption.

     The establishment of an IPsec tunnel can fail, e.g., because the
     two endpoints support different encryption algorithms. Multiple
     negotiation messages that interconnects carry  the enterprise's sites.

6.1. Floating PEs to connect to Remote CPEs

   To extend BGP/MPLS VPNs to remote CPEs, IPsec SA parameters between
     two end-points may be exchanged. This is why it is necessary cleaner to establish
   secure tunnels (such as
     separate the establishment of an IPsec tunnels) SA association between two
     end-points from the Floating PEs and
   the remote CPEs.

   Even though a set of PEs can be manually selected policies enforced to map routes to act as the
   floating PEs for a specific cloud data center, there are no standard
   protocols for those PEs
     IPSec SA.

     If C-PEs need to interact with establish a WAN Port-based IPsec SA, the remote CPEs (most
   likely virtualized) instantiated
     information encoded in the third party cloud data
   centers (e.g., Tunnel Path Attribute should only apply
     to exchange performance or route information).

   When there is more than one fPE available for use (as there the WAN ports and should be for resiliency purposes or because of independent from the need to support
   multiple cloud DCs geographically scattered), it clients'
     routes.

     In addition, the Overlay IPsec SA Tunnel is not
   straightforward very likely to designate be
     established before clients' routes are attached.

   - When an egress fPE to remote CPEs based on
   applications.  There is too much applications' traffic traversing
   PEs, and it is overlay network spans across large geographic regions
     (such as countries or continents), one C-PE in one region may not feasible for PEs to recognize applications from
   the payload
     even be aware of packets.

6.2. NAT Traversal

   Cloud DCs that only assign private IPv4 addresses to the
   instantiated workloads assume remote CPEs in other regions that traffic to/from the workload
   usually it needs to traverse NATs.
   An overlay edge node can solicit a STUN (Session Traversal of UDP
   Through Network Address Translation, [RFC3489]) Server to get the
   information about the NAT property,
     communicate.  Therefore, the public IP addresses and port
   numbers so that such distribution of the Overlay Edge WAN
     ports information can need to be communicated restricted to the relevant authorized peers.

6.3. Complexity of using BGP between

5.3. SECURE-L3VPN/EVPN

   [SECURE-L3VPN] describes a method to enrich BGP/MPLS VPN [RFC4364]
   capabilities to allow some PEs and remote CPEs via Internet

   Even though an EBGP (external BGP) Multi-Hop design can be used to connect peers that to other PEs via public
   networks. [SECURE-L3VPN] introduces the concept of Red Interface &
   Black Interface used by PEs, where the RED interfaces are not directly connected used to each other, there
   forward traffic into the VPN, and the Black Interfaces are still some issues about extending BGP from MPLS VPN PEs used
   between WAN ports through which only IPsec-formatted packets are
   forwarded to the Internet or to
   remote CPEs via any access path (e.g., Internet).

   The path between other backbone network, thereby
   eliminating the remote CPEs and VPN need for MPLS transport in the backbone.

   [SECURE-L3VPN] assumes PEs that maintain VPN
   routes may very well traverse untrusted nodes.

   EBGP Multi-hop design requires configuration on both peers, either
   manually or via NETCONF from a controller. To use EBGP between MPLS over IPsec when sending traffic
   through the Black Interfaces.

   [SECURE-EVPN] describes a PE
   and remote CPEs, solution where point-to-multipoint BGP
   signaling is used in the PE has to be manually configured with control plane for the
   "next-hop" set Scenario #1 described
   in [BGP-SDWAN-Usage]. It relies upon a BGP cluster design to
   facilitate the IP address of the CPEs. When remote CPEs,
   especially remote virtualized CPEs key and policy exchange among PE devices to create
   private pair-wise IPsec Security Associations without IKEv2 point-
   to-point signaling or any other direct peer-to-peer session
   establishment messages.

   Both [SECURE-L3VPN] and [SECURE-EVPN] are dynamically instantiated or
   removed, useful, however, they both
   miss the configuration aspects of Multi-Hop EBGP on the PE has to be
   changed accordingly.

     Egress peering engineering (EPE) aggregating VPN and Internet underlays. In
   summary:

   - Both documents assume that an IPsec tunnel is not sufficient. Running BGP on
     virtualized CPEs in Cloud DCs requires GRE tunnels to be
     established first, associated with
     client traffic. Regardless of which requires WAN ports the remote CPEs to support
     address and key management capabilities. RFC 7024 (Virtual Hub &
     Spoke) and Hierarchical VPN do not support traffic egress
     from the required
     properties.

     Also, there edge, the client traffic associated with IPsec is a need always
     encrypted. Within the context of an overlay architecture that
     relies upon minimizing resource used for encryption, traffic sent
     from an edge node can be encrypted once and forwarded through a mechanism to automatically trigger
     configuration changes on PEs when remote CPEs' are instantiated or
     moved (leading to
     WAN port towards an IP address change) or deleted.

     EBGP Multi-hop design does not include untrusted network, but can also remain
     unencrypted and be forwarded at different times through a security mechanism by
     default. WAN port
     to the BGP/MPLS VPN.

   - The PE and [SECURE-L3VPN] assumes that a CPE "registers" with the RR.
     However, it does not say how. It assumes that the remote CPEs need secure communication channels
     when connecting via are
     pre-configured with the public Internet.

   Remote CPEs, if instantiated in Cloud DCs might have to traverse
   NATs IPsec SA manually. For overlay networks to reach PEs. It
     connect Hybrid Cloud DCs, Zero Touch Provisioning is expected.
     Manual configuration is not clear how BGP an option.

   - The [SECURE-L3VPN] assumes that C-PEs and RRs are connected via an
     IPsec tunnel. For management channel, TLS/DTLS is more economical
     than IPsec. The following assumption made by [SECURE-L3VPN] can be used between
   devices located beyond
     difficult to meet in the NAT and environment where zero touch provisioning
     is expected:
          A CPE must also be provisioned with whatever additional
          information is needed in order to set up an IPsec SA with
          each of the devices located behind red RRs

   - IPsec requires periodic refreshment of the
   NAT. It is keys. The draft does
     not clear provide any information about how to configure the Next Hop on synchronize the PEs to
   reach private IPv4 addresses.

6.4. Designated Forwarder
     refreshment among multiple nodes.
   - IPsec usually sends configuration parameters to two endpoints only
     and lets these endpoints negotiate the remote edges

   Among the multiple floating PEs key. The [SECURE-L3VPN]
     assumes that are reachable from a remote
   CPE, multicast traffic sent by the remote CPE towards RR is responsible for creating/managing the MPLS VPN
   can key
     for all endpoints. When one endpoint is compromised, all other
     connections may be forwarded back to the remote CPE due impacted.

5.4. Preventing attacks from Internet-facing ports

   When C-PEs have Internet-facing ports, additional security risks are
   raised.

   To mitigate security risks, in addition to the PE receiving the
   multicast packets forwarding the multicast/broadcast frame requiring Anti-DDoS
   features on C-PEs, it is necessary for C-PEs to other
   PEs that in turn send support means to all attached CPEs. This process may cause
   determine whether traffic loops.

   This problem sent by remote peers is legitimate to
   prevent spoofing attacks, in particular.

6. Gap Summary

   Here is the summary of the technical gaps discussed in this
   document:

   - For Accessing Cloud Resources

       a) When a remote vCPE can be solved reached by selecting multiple PEs of one floating
          provider VPN network, it is not straightforward to designate
          which egress PE as the CPE's
   Designated Forwarder, similar to TRILL's Appointed Forwarders
   [RFC6325].

   BGP/MPLS VPNs do not have features like TRILL's Appointed
   Forwarders.

6.5. Traffic Path Management

   When there are multiple floating PEs that have established IPsec
   tunnels with a remote CPE, the latter can forward outbound traffic remote vCPE based on applications
       b) Need automated and reliable tools to map the Designated Forwarder PE, which in turn forwards traffic to
   egress PEs user-friendly
          (natural language) access rules into machine readable
          policies and then to provide interfaces for enterprises to self-
          manage policy enforcement points for their own workloads.
       c) NAT Traversal. An enterprise's network controller needs to be
          informed of the final destinations. However, it is not
   straightforward NAT properties for its workloads in Cloud
          DCs. If the egress PE workloads are attached to send back the return traffic to enterprise's own
          vCPEs instantiated in the Designated Cloud DCs, the task can be
          achieved.
       d) The multicast traffic to/from remote vCPE needs a feature
          like Appointed Forwarder PE.

   As Figure 3: specified by TRILL to prevent
          multicast data frames from looping around.
       e) BGP between PEs and remote CPEs via untrusted networks.
       f) Traffic Path Management

   - fPE-1 is DF Overlay Edge Node's WAN Port Management: BGP UPDATE propagate
   client's routes information, but don't distinguish network facing
   ports.

   - Aggregating VPN paths and Internet paths

       a) Control Plane for communication between App-1 <-> Host-a due to
   latency, pricing or other criteria. Overlay over Heterogeneous Networks is not
          clear.
       b) BGP UPDATE Messages missing properties:

             - fPE-2  Lacking SD-WAN Segments Identifier
             -  Missing attributes in Tunnel-Encap

       c) SECURE-L3VPN/EVPN is DF for communication between App-1 <-> Host-b. not enough
       d) Missing clear methods in preventing attacks from Internet-
          facing ports

7. Manageability Considerations

     Zero touch provisioning of overlay networks to interconnect Hybrid
     Clouds is highly desired. It is necessary for a newly powered up
     edge node to establish a secure connection (by means of TLS, DTLS,
     etc.) with its controller.

8. Security Considerations

     Cloud Services are built upon shared infrastructures, therefore
     not secure by nature.

     Secure user identity management, authentication, and access
     control mechanisms are important. Developing appropriate security
     measurements can enhance the confidence needed by enterprises to
     fully take advantage of Cloud Services.

9. IANA Considerations

   This document requires no IANA actions. RFC Editor: Please remove
   this section before publication.

10. References

10.1. Normative References

   [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
             Requirement Levels", BCP 14, RFC 2119, March 1997.

10.2. Informative References

   [RFC8192] S. Hares, et al, "Interface to Network Security Functions
             (I2NSF) Problem Statement and Use Cases", July 2017

   [RFC5521] P. Mohapatra, E. Rosen, "The BGP Encapsulation Subsequent
             Address Family Identifier (SAFI) and the BGP Tunnel
             Encapsulation Attribute", April 2009.

   [BGP-SDWAN-PORT]L. Dunbar, et al, "Subsequent Address Family
             Indicator for SDWAN Ports", draft-dunbar-idr-sdwan-port-
             safi-00, Work-in-progress, March 2019.

   [BGP-SDWAN-Usage] L. Dunbar, et al, "Framework of Using BGP for
             SDWAN Overlay Networks", draft-dunbar-idr-sdwan-framework-
             00, work-in-progress, Feb 2019.

   [Tunnel-Encap]E. Rosen, et al, "The BGP Tunnel Encapsulation
             Attribute", draft-ietf-idr-tunnel-encaps-10, July 2018.

   [SECURE-EVPN A. Sajassi, et al, draft-sajassi-bess-secure-evpn-01,
             work in progress, March 2019.

   [SECURE-L3VPN] E. Rosen, "Provide Secure Layer L3VPNs over Public
             Infrastructure", draft-rosen-bess-secure-l3vpn-00, work-
             in-progress, July 2018

   [DMVPN] Dynamic Multi-point VPN:
             https://www.cisco.com/c/en/us/products/security/dynamic-
             multipoint-vpn-dmvpn/index.html

   [DSVPN] Dynamic Smart VPN:
             http://forum.huawei.com/enterprise/en/thread-390771-1-
             1.html

   [ITU-T-X1036] ITU-T Recommendation X.1036, "Framework for creation,
             storage, distribution and enforcement of policies for
             network security", Nov 2007.

    [Net2Cloud-Problem] L. Dunbar and A. Malis, "Seamless Interconnect
             Underlay to Cloud Overlay Problem Statement", draft-dm-
             net2cloud-problem-statement-02, June 2018

11. Acknowledgments

   Acknowledgements to John Drake for his review and contributions.
   Many thanks to John Scudder for stimulating the clarification
   discussion on the Tunnel-Encap draft so that our gap analysis can be
   more accurate.

   This document was prepared using 2-Word-v2.0.template.dot.

Authors' Addresses

   Linda Dunbar
   Futurewei
   Email: ldunbar@futurewei.com

   Andrew G. Malis
   Independent
   Email: agmalis@gmail.com

   Christian Jacquenet
   Orange
   Rennes, 35000
   France
   Email: Christian.jacquenet@orange.com