draft-ietf-nvo3-evpn-applicability-02.txt   draft-ietf-nvo3-evpn-applicability-03.txt 
NVO3 Workgroup J. Rabadan, Ed. NVO3 Workgroup J. Rabadan, Ed.
Internet Draft M. Bocci Internet-Draft M. Bocci
Intended status: Informational Nokia Intended status: Informational Nokia
Expires: May 6, 2021 S. Boutros
S. Boutros Ciena
VMware
A. Sajassi A. Sajassi
Cisco Cisco
November 2, 2020
Expires: January 9, 2020 July 8, 2019
Applicability of EVPN to NVO3 Networks Applicability of EVPN to NVO3 Networks
draft-ietf-nvo3-evpn-applicability-02 draft-ietf-nvo3-evpn-applicability-03
Abstract Abstract
In NVO3 networks, Network Virtualization Edge (NVE) devices sit at In NVO3 networks, Network Virtualization Edge (NVE) devices sit at
the edge of the underlay network and provide Layer-2 and Layer-3 the edge of the underlay network and provide Layer-2 and Layer-3
connectivity among Tenant Systems (TSes) of the same tenant. The NVEs connectivity among Tenant Systems (TSes) of the same tenant. The
need to build and maintain mapping tables so that they can deliver NVEs need to build and maintain mapping tables so that they can
encapsulated packets to their intended destination NVE(s). While deliver encapsulated packets to their intended destination NVE(s).
there are different options to create and disseminate the mapping While there are different options to create and disseminate the
table entries, NVEs may exchange that information directly among mapping table entries, NVEs may exchange that information directly
themselves via a control-plane protocol, such as EVPN. EVPN provides among themselves via a control-plane protocol, such as EVPN. EVPN
an efficient, flexible and unified control-plane option that can be provides an efficient, flexible and unified control-plane option that
used for Layer-2 and Layer-3 Virtual Network (VN) service can be used for Layer-2 and Layer-3 Virtual Network (VN) service
connectivity. This document describes the applicability of EVPN to connectivity. This document describes the applicability of EVPN to
NVO3 networks and how EVPN solves the challenges in those networks. NVO3 networks and how EVPN solves the challenges in those networks.
Status of this Memo Status of This Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF). Note that other groups may also distribute
other groups may also distribute working documents as Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at This Internet-Draft will expire on May 6, 2021.
http://www.ietf.org/shadow.html
This Internet-Draft will expire on January 9, 2020.
Copyright Notice Copyright Notice
Copyright (c) 2019 IETF Trust and the persons identified as the Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. EVPN and NVO3 Terminology . . . . . . . . . . . . . . . . . . . 3 2. EVPN and NVO3 Terminology . . . . . . . . . . . . . . . . . . 3
3. Why Is EVPN Needed In NVO3 Networks? . . . . . . . . . . . . . 6 3. Why Is EVPN Needed In NVO3 Networks? . . . . . . . . . . . . 6
4. Applicability of EVPN to NVO3 Networks . . . . . . . . . . . . 8 4. Applicability of EVPN to NVO3 Networks . . . . . . . . . . . 8
4.1. EVPN Route Types used in NVO3 Networks . . . . . . . . . . 8 4.1. EVPN Route Types used in NVO3 Networks . . . . . . . . . 8
4.2. EVPN Basic Applicability For Layer-2 Services . . . . . . . 9 4.2. EVPN Basic Applicability For Layer-2 Services . . . . . . 9
4.2.1. Auto-Discovery and Auto-Provisioning of ES, 4.2.1. Auto-Discovery and Auto-Provisioning . . . . . . . . 10
Multi-Homing PEs and NVE services . . . . . . . . . . . 10 4.2.2. Remote NVE Auto-Discovery . . . . . . . . . . . . . . 12
4.2.2. Remote NVE Auto-Discovery . . . . . . . . . . . . . . . 11 4.2.3. Distribution Of Tenant MAC and IP Information . . . . 12
4.2.3. Distribution Of Tenant MAC and IP Information . . . . . 12 4.3. EVPN Basic Applicability for Layer-3 Services . . . . . . 13
4.3. EVPN Basic Applicability for Layer-3 Services . . . . . . . 13 4.4. EVPN as a Control Plane for NVO3 Encapsulations and
4.4. EVPN as a Control Plane for NVO3 Encapsulations and GENEVE . . . . . . . . . . . . . . . . . . . . . . . . . 15
GENEVE . . . . . . . . . . . . . . . . . . . . . . . . . . 15 4.5. EVPN OAM and application to NVO3 . . . . . . . . . . . . 16
4.5. EVPN OAM and application to NVO3 . . . . . . . . . . . . . 16 4.6. EVPN as the control plane for NVO3 security . . . . . . . 16
4.6. EVPN as the control plane for NVO3 security . . . . . . . . 16 4.7. Advanced EVPN Features For NVO3 Networks . . . . . . . . 16
4.7. Advanced EVPN Features For NVO3 Networks . . . . . . . . . 16 4.7.1. Virtual Machine (VM) Mobility . . . . . . . . . . . . 16
4.7.1. Virtual Machine (VM) Mobility . . . . . . . . . . . . . 16 4.7.2. MAC Protection, Duplication Detection and Loop
4.7.2. MAC Protection, Duplication Detection and Loop Protection . . . . . . . . . . . . . . . . . . . . . 17
Protection . . . . . . . . . . . . . . . . . . . . . . 17 4.7.3. Reduction/Optimization of BUM Traffic In Layer-2
4.7.3. Reduction/Optimization of BUM Traffic In Layer-2 Services . . . . . . . . . . . . . . . . . . . . . . 17
Services . . . . . . . . . . . . . . . . . . . . . . . 17 4.7.4. Ingress Replication (IR) Optimization For BUM Traffic 18
4.7.5. EVPN Multi-homing . . . . . . . . . . . . . . . . . . 19
4.7.4. Ingress Replication (IR) Optimization For BUM Traffic . 18 4.7.6. EVPN Recursive Resolution for Inter-Subnet Unicast
4.7.5. EVPN Multi-homing . . . . . . . . . . . . . . . . . . . 19 Forwarding . . . . . . . . . . . . . . . . . . . . . 20
4.7.6. EVPN Recursive Resolution for Inter-Subnet Unicast 4.7.7. EVPN Optimized Inter-Subnet Multicast Forwarding . . 21
Forwarding . . . . . . . . . . . . . . . . . . . . . . 20 4.7.8. Data Center Interconnect (DCI) . . . . . . . . . . . 21
4.7.7. EVPN Optimized Inter-Subnet Multicast Forwarding . . . 21 5. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.7.8. Data Center Interconnect (DCI) . . . . . . . . . . . . 21 6. Conventions used in this document . . . . . . . . . . . . . . 22
5. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 22 7. Security Considerations . . . . . . . . . . . . . . . . . . . 22
6. Conventions used in this document . . . . . . . . . . . . . . . 22 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23
7. Security Considerations . . . . . . . . . . . . . . . . . . . . 22 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 23
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 23 9.1. Normative References . . . . . . . . . . . . . . . . . . 23
9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 23 9.2. Informative References . . . . . . . . . . . . . . . . . 23
9.1 Normative References . . . . . . . . . . . . . . . . . . . . 23 Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . 26
9.2 Informative References . . . . . . . . . . . . . . . . . . . 23 Appendix B. Contributors . . . . . . . . . . . . . . . . . . . . 26
10. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 26 Appendix C. Authors' Addresses . . . . . . . . . . . . . . . . . 26
11. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 26 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 26
12. Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 26
1. Introduction 1. Introduction
In NVO3 networks, Network Virtualization Edge (NVE) devices sit at In NVO3 networks, Network Virtualization Edge (NVE) devices sit at
the edge of the underlay network and provide Layer-2 and Layer-3 the edge of the underlay network and provide Layer-2 and Layer-3
connectivity among Tenant Systems (TSes) of the same tenant. The NVEs connectivity among Tenant Systems (TSes) of the same tenant. The
need to build and maintain mapping tables so that they can deliver NVEs need to build and maintain mapping tables so that they can
encapsulated packets to their intended destination NVE(s). While deliver encapsulated packets to their intended destination NVE(s).
there are different options to create and disseminate the mapping While there are different options to create and disseminate the
table entries, NVEs may exchange that information directly among mapping table entries, NVEs may exchange that information directly
themselves via a control-plane protocol, such as EVPN. EVPN provides among themselves via a control-plane protocol, such as EVPN. EVPN
an efficient, flexible and unified control-plane option that can be provides an efficient, flexible and unified control-plane option that
used for Layer-2 and Layer-3 Virtual Network (VN) service can be used for Layer-2 and Layer-3 Virtual Network (VN) service
connectivity. connectivity.
In this document, we assume that the EVPN control-plane module In this document, we assume that the EVPN control-plane module
resides in the NVEs. The NVEs can be virtual switches in hypervisors, resides in the NVEs. The NVEs can be virtual switches in
TOR/Leaf switches or Data Center Gateways. As described in [RFC7365], hypervisors, TOR/Leaf switches or Data Center Gateways. As described
Network Virtualization Authorities (NVAs) may be used to provide the in [RFC7365], Network Virtualization Authorities (NVAs) may be used
forwarding information to the NVEs, and in that case, EVPN could be to provide the forwarding information to the NVEs, and in that case,
used to disseminate the information across multiple federated NVAs. EVPN could be used to disseminate the information across multiple
The applicability of EVPN would then be similar to the one described federated NVAs. The applicability of EVPN would then be similar to
in this document. However, for simplicity, the description assumes the one described in this document. However, for simplicity, the
control-plane communication among NVE(s). description assumes control-plane communication among NVE(s).
2. EVPN and NVO3 Terminology 2. EVPN and NVO3 Terminology
o EVPN: Ethernet Virtual Private Networks, as described in [RFC7432]. o EVPN: Ethernet Virtual Private Networks, as described in
[RFC7432].
o PE: Provider Edge router. o PE: Provider Edge router.
o NVO3 or Overlay tunnels: Network Virtualization Over Layer-3 o NVO3 or Overlay tunnels: Network Virtualization Over Layer-3
tunnels. In this document, NVO3 tunnels or simply Overlay tunnels tunnels. In this document, NVO3 tunnels or simply Overlay tunnels
will be used interchangeably. Both terms refer to a way to will be used interchangeably. Both terms refer to a way to
encapsulate tenant frames or packets into IP packets whose IP encapsulate tenant frames or packets into IP packets whose IP
Source Addresses (SA) or Destination Addresses (DA) belong to the Source Addresses (SA) or Destination Addresses (DA) belong to the
underlay IP address space, and identify NVEs connected to the same underlay IP address space, and identify NVEs connected to the same
underlay network. Examples of NVO3 tunnel encapsulations are VXLAN underlay network. Examples of NVO3 tunnel encapsulations are
[RFC7348], [GENEVE] or MPLSoUDP [RFC7510]. VXLAN [RFC7348], [I-D.ietf-nvo3-geneve] or MPLSoUDP [RFC7510].
o VXLAN: Virtual eXtensible Local Area Network, an NVO3 encapsulation o VXLAN: Virtual eXtensible Local Area Network, an NVO3
defined in [RFC7348]. encapsulation defined in [RFC7348].
o GENEVE: Generic Network Virtualization Encapsulation, an NVO3 o GENEVE: Generic Network Virtualization Encapsulation, an NVO3
encapsulation defined in [GENEVE]. encapsulation defined in [I-D.ietf-nvo3-geneve].
o CLOS: a multistage network topology described in [CLOS1953], where o CLOS: a multistage network topology described in [CLOS1953], where
all the edge switches (or Leafs) are connected to all the core all the edge switches (or Leafs) are connected to all the core
switches (or Spines). Typically used in Data Centers nowadays. switches (or Spines). Typically used in Data Centers nowadays.
o ECMP: Equal Cost Multi-Path. o ECMP: Equal Cost Multi-Path.
o NVE: Network Virtualization Edge is a network entity that sits at o NVE: Network Virtualization Edge is a network entity that sits at
the edge of an underlay network and implements L2 and/or L3 network the edge of an underlay network and implements L2 and/or L3
virtualization functions. The network-facing side of the NVE uses network virtualization functions. The network-facing side of the
the underlying L3 network to tunnel tenant frames to and from other NVE uses the underlying L3 network to tunnel tenant frames to and
NVEs. The tenant-facing side of the NVE sends and receives Ethernet from other NVEs. The tenant-facing side of the NVE sends and
frames to and from individual Tenant Systems. In this document, an receives Ethernet frames to and from individual Tenant Systems.
NVE could be implemented as a virtual switch within a hypervisor, a In this document, an NVE could be implemented as a virtual switch
switch or a router, and runs EVPN in the control-plane. within a hypervisor, a switch or a router, and runs EVPN in the
control-plane.
o EVI: or EVPN Instance. It is a Layer-2 Virtual Network that uses an o EVI: or EVPN Instance. It is a Layer-2 Virtual Network that uses
EVPN control-plane to exchange reachability information among the an EVPN control-plane to exchange reachability information among
member NVEs. It corresponds to a set of MAC-VRFs of the same the member NVEs. It corresponds to a set of MAC-VRFs of the same
tenant. See MAC-VRF in this section. tenant. See MAC-VRF in this section.
o BD: or Broadcast Domain, it corresponds to a tenant IP subnet. If o BD: or Broadcast Domain, it corresponds to a tenant IP subnet. If
no suppression techniques are used, a BUM frame that is injected in no suppression techniques are used, a BUM frame that is injected
a BD will reach all the NVEs that are attached to that BD. An EVI in a BD will reach all the NVEs that are attached to that BD. An
may contain one or multiple BDs depending on the service model EVI may contain one or multiple BDs depending on the service model
[RFC7432]. This document will use the term BD to refer to a tenant [RFC7432]. This document will use the term BD to refer to a
subnet. tenant subnet.
o EVPN VLAN-based service model: it refers to one of the three o EVPN VLAN-based service model: one of the three service models
service models defined in [RFC7432]. It is characterized as a BD defined in [RFC7432]. It is characterized as a BD that uses a
that uses a single VLAN per physical access port to attach tenant single VLAN per physical access port to attach tenant traffic to
traffic to the BD. In this service model, there is only one BD per the BD. In this service model, there is only one BD per EVI.
EVI.
o EVPN VLAN-bundle service model: similar to VLAN-based but uses a o EVPN VLAN-bundle service model: similar to VLAN-based but uses a
bundle of VLANs per physical port to attach tenant traffic to the bundle of VLANs per physical port to attach tenant traffic to the
BD. As in VLAN-based, in this model there is a single BD per EVI. BD. As in VLAN-based, in this model there is a single BD per EVI.
o EVPN VLAN-aware bundle service model: similar to the VLAN-bundle o EVPN VLAN-aware bundle service model: similar to the VLAN-bundle
model but each individual VLAN value is mapped to a different BD. model but each individual VLAN value is mapped to a different BD.
In this model there are multiple BDs per EVI for a given tenant. In this model there are multiple BDs per EVI for a given tenant.
Each BD is identified by an "Ethernet Tag", that is a control-plane Each BD is identified by an "Ethernet Tag", that is a control-
value that identifies the routes for the BD within the EVI. plane value that identifies the routes for the BD within the EVI.
o IP-VRF: an IP Virtual Routing and Forwarding table, as defined in o IP-VRF: an IP Virtual Routing and Forwarding table, as defined in
[RFC4364]. It stores IP Prefixes that are part of the tenant's IP [RFC4364]. It stores IP Prefixes that are part of the tenant's IP
space, and are distributed among NVEs of the same tenant by EVPN. space, and are distributed among NVEs of the same tenant by EVPN.
Route-Distinghisher (RD) and Route-Target(s) (RTs) are required Route-Distinghisher (RD) and Route-Target(s) (RTs) are required
properties of an IP-VRF. An IP-VRF is instantiated in an NVE for a properties of an IP-VRF. An IP-VRF is instantiated in an NVE for
given tenant, if the NVE is attached to multiple subnets of the a given tenant, if the NVE is attached to multiple subnets of the
tenant and local inter-subnet-forwarding is required across those tenant and local inter-subnet-forwarding is required across those
subnets. subnets.
o MAC-VRF: a MAC Virtual Routing and Forwarding table, as defined in o MAC-VRF: a MAC Virtual Routing and Forwarding table, as defined in
[RFC7432]. The instantiation of an EVI (EVPN Instance) in an NVE. [RFC7432]. The instantiation of an EVI (EVPN Instance) in an NVE.
Route-distinghisher (RD) and Route-Target(s) (RTs) are required Route-distinghisher (RD) and Route-Target(s) (RTs) are required
properties of a MAC-VRF and they are normally different than the properties of a MAC-VRF and they are normally different than the
ones defined in the associated IP-VRF (if the MAC-VRF has an IRB ones defined in the associated IP-VRF (if the MAC-VRF has an IRB
interface). interface).
o BT: a Bridge Table, as defined in [RFC7432]. A BT is the o BT: a Bridge Table, as defined in [RFC7432]. A BT is the
instantiation of a BD in an NVE. When there is a single BD on a instantiation of a BD in an NVE. When there is a single BD on a
given EVI, the MAC-VRF is equivalent to the BT on that NVE. given EVI, the MAC-VRF is equivalent to the BT on that NVE.
o AC: Attachment Circuit or logical interface associated to a given o AC: Attachment Circuit or logical interface associated to a given
BT. To determine the AC on which a packet arrived, the NVE will BT. To determine the AC on which a packet arrived, the NVE will
examine the physical/logical port and/or VLAN tags (where the VLAN examine the physical/logical port and/or VLAN tags (where the VLAN
tags can be individual c-tags, s-tags or ranges of both). tags can be individual c-tags, s-tags or ranges of both).
o IRB: Integrated Routing and Bridging interface. It refers to the o IRB: Integrated Routing and Bridging interface. It refers to the
logical interface that connects a BD instance (or a BT) to an IP- logical interface that connects a BD instance (or a BT) to an IP-
VRF and allows to forward packets with destination in a different VRF and allows to forward packets with destination in a different
subnet. subnet.
o ES: Ethernet Segment. When a Tenant System (TS) is connected to one o ES: Ethernet Segment. When a Tenant System (TS) is connected to
or more NVEs via a set of Ethernet links, then that set of links is one or more NVEs via a set of Ethernet links, then that set of
referred to as an 'Ethernet segment'. Each ES is represented by a links is referred to as an 'Ethernet segment'. Each ES is
unique Ethernet Segment Identifier (ESI) in the NVO3 network and represented by a unique Ethernet Segment Identifier (ESI) in the
the ESI is used in EVPN routes that are specific to that ES. NVO3 network and the ESI is used in EVPN routes that are specific
to that ES.
o DF and NDF: they refer to Designated Forwarder and Non-Designated o DF and NDF: they refer to Designated Forwarder and Non-Designated
Forwarder, which are the roles that a given PE can have in a given Forwarder, which are the roles that a given PE can have in a given
ES. ES.
o VNI: Virtual Network Identifier. Irrespective of the NVO3 o VNI: Virtual Network Identifier. Irrespective of the NVO3
encapsulation, the tunnel header always includes a VNI that is encapsulation, the tunnel header always includes a VNI that is
added at the ingress NVE (based on the mapping table lookup) and added at the ingress NVE (based on the mapping table lookup) and
identifies the BT at the egress NVE. This VNI is called VNI in identifies the BT at the egress NVE. This VNI is called VNI in
VXLAN or GENEVE, VSID in nvGRE or Label in MPLSoGRE or MPLSoUDP. VXLAN or GENEVE, VSID in nvGRE or Label in MPLSoGRE or MPLSoUDP.
This document will refer to VNI as a generic Virtual Network This document will refer to VNI as a generic Virtual Network
Identifier for any NVO3 encapsulation. Identifier for any NVO3 encapsulation.
o BUM: Broadcast, Unknown unicast and Multicast frames. o BUM: Broadcast, Unknown unicast and Multicast frames.
o SA and DA: they refer to Source Address and Destination Address. o SA and DA: Source Address and Destination Address. They are used
They are used along with MAC or IP, e.g. IP SA or MAC DA. along with MAC or IP, e.g. IP SA or MAC DA.
o RT and RD: they refer to Route Target and Route Distinguisher. o RT and RD: Route Target and Route Distinguisher.
o PTA: Provider Multicast Service Interface Tunnel Attribute. o PTA: Provider Multicast Service Interface Tunnel Attribute.
o RT-1, RT-2, RT-3, etc.: they refer to Route Type followed by the o RT-1, RT-2, RT-3, etc.: they refer to Route Type followed by the
type number as defined in the IANA registry for EVPN route types. type number as defined in the IANA registry for EVPN route types.
o TS: Tenant System. o TS: Tenant System.
o ARP and ND: they refer to Address Resolution Protocol and Neighbor o ARP and ND: Address Resolution Protocol and Neighbor Discovery
Discovery protocol. protocol.
o Ethernet Tag: Used to represent a BD that is configured on a given o Ethernet Tag: Used to represent a BD that is configured on a given
ES for the purpose of DF election. Note that any of the following ES for the purpose of DF election. Note that any of the following
may be used to represent a BD: VIDs (including Q-in-Q tags), may be used to represent a BD: VIDs (including Q-in-Q tags),
configured IDs, VNIs (Virtual Extensible Local Area Network (VXLAN) configured IDs, VNIs (Virtual Extensible Local Area Network
Network Identifiers), normalized VIDs, I-SIDs (Service Instance (VXLAN) Network Identifiers), normalized VIDs, I-SIDs (Service
Identifiers), etc., as long as the representation of the BDs is Instance Identifiers), etc., as long as the representation of the
configured consistently across the multihomed PEs attached to that BDs is configured consistently across the multihomed PEs attached
ES. The Ethernet Tag value MUST be different from zero. to that ES. The Ethernet Tag value MUST be different from zero.
3. Why Is EVPN Needed In NVO3 Networks? 3. Why Is EVPN Needed In NVO3 Networks?
Data Centers have adopted NVO3 architectures mostly due to the issues Data Centers have adopted NVO3 architectures mostly due to the issues
discussed in [RFC7364]. The architecture of a Data Center is nowadays discussed in [RFC7364]. The architecture of a Data Center is
based on a CLOS design, where every Leaf is connected to a layer of nowadays based on a CLOS design, where every Leaf is connected to a
Spines, and there is a number of ECMP paths between any two leaf layer of Spines, and there is a number of ECMP paths between any two
nodes. All the links between Leaf and Spine nodes are routed links, leaf nodes. All the links between Leaf and Spine nodes are routed
forming what we also know as an underlay IP Fabric. The underlay IP links, forming what we also know as an underlay IP Fabric. The
Fabric does not have issues with loops or flooding (like old Spanning underlay IP Fabric does not have issues with loops or flooding (like
Tree Data Center designs did), convergence is fast and ECMP provides old Spanning Tree Data Center designs did), convergence is fast and
a fairly optimal bandwidth utilization on all the links. ECMP provides a fairly optimal bandwidth utilization on all the
links.
On this architecture and as discussed by [RFC7364] multi-tenant On this architecture and as discussed by [RFC7364] multi-tenant
intra-subnet and inter-subnet connectivity services are provided by intra-subnet and inter-subnet connectivity services are provided by
NVO3 tunnels, being VXLAN [RFC7348] or [GENEVE] two examples of such NVO3 tunnels, being VXLAN [RFC7348] or [I-D.ietf-nvo3-geneve] two
tunnels. examples of such tunnels.
Why is a control-plane protocol along with NVO3 tunnels required? Why is a control-plane protocol along with NVO3 tunnels required?
There are three main reasons: There are three main reasons:
a) Auto-discovery of the remote NVEs that are attached to the same a. Auto-discovery of the remote NVEs that are attached to the same
VPN instance (Layer-2 and/or Layer-3) as the ingress NVE is. VPN instance (Layer-2 and/or Layer-3) as the ingress NVE is.
b) Dissemination of the MAC/IP host information so that mapping b. Dissemination of the MAC/IP host information so that mapping
tables can be populated on the remote NVEs. tables can be populated on the remote NVEs.
c) Advanced features such as MAC Mobility, MAC Protection, BUM and c. Advanced features such as MAC Mobility, MAC Protection, BUM and
ARP/ND traffic reduction/suppression, Multi-homing, Prefix ARP/ND traffic reduction/suppression, Multi-homing, Prefix
Independent Convergence (PIC) like functionality, Fast Independent Convergence (PIC) like functionality, Fast
Convergence, etc. Convergence, etc.
A possible approach to achieve points (a) and (b) above for A possible approach to achieve points (a) and (b) above for
multipoint Ethernet services, is "Flood and Learn". "Flood and Learn" multipoint Ethernet services, is "Flood and Learn". "Flood and
refers to not using a specific control-plane on the NVEs, but rather Learn" refers to not using a specific control-plane on the NVEs, but
"Flood" BUM traffic from the ingress NVE to all the egress NVEs rather "Flood" BUM traffic from the ingress NVE to all the egress
attached to the same BD. The egress NVEs may then use data path MAC NVEs attached to the same BD. The egress NVEs may then use data path
SA "Learning" on the frames received over the NVO3 tunnels. When the MAC SA "Learning" on the frames received over the NVO3 tunnels. When
destination host replies back and the frames arrive at the NVE that the destination host replies back and the frames arrive at the NVE
initially flooded BUM frames, the NVE will also "Learn" the MAC SA of that initially flooded BUM frames, the NVE will also "Learn" the MAC
the frame encapsulated on the NVO3 tunnel. This approach has the SA of the frame encapsulated on the NVO3 tunnel. This approach has
following drawbacks: the following drawbacks:
o In order to Flood a given BUM frame, the ingress NVE must know the o In order to Flood a given BUM frame, the ingress NVE must know the
IP addresses of the remote NVEs attached to the same BD. This may IP addresses of the remote NVEs attached to the same BD. This may
be done as follows: be done as follows:
- The remote tunnel IP addresses can be statically provisioned on - The remote tunnel IP addresses can be statically provisioned on
the ingress NVE. If the ingress NVE receives a BUM frame for the the ingress NVE. If the ingress NVE receives a BUM frame for
BD on an ingress AC, it will do ingress replication and will send the BD on an ingress AC, it will do ingress replication and
the frame to all the configured egress NVE IP DAs in the BD. will send the frame to all the configured egress NVE IP DAs in
the BD.
- All the NVEs attached to the same BD can subscribe to an underlay - All the NVEs attached to the same BD can subscribe to an
IP Multicast Group that is dedicated to that BD. When an ingress underlay IP Multicast Group that is dedicated to that BD. When
NVE receives a BUM frame on an ingress AC, it will send a single an ingress NVE receives a BUM frame on an ingress AC, it will
copy of the frame encapsulated into an NVO3 tunnel, using the send a single copy of the frame encapsulated into an NVO3
multicast address as IP DA of the tunnel. This solution requires tunnel, using the multicast address as IP DA of the tunnel.
PIM in the underlay network and the association of individual BDs This solution requires PIM in the underlay network and the
to underlay IP multicast groups. association of individual BDs to underlay IP multicast groups.
o "Flood and Learn" solves the issues of auto-discovery and learning o "Flood and Learn" solves the issues of auto-discovery and learning
of the MAC to VNI/tunnel IP mapping on the NVEs for a given BD. of the MAC to VNI/tunnel IP mapping on the NVEs for a given BD.
However, it does not provide a solution for advanced features and However, it does not provide a solution for advanced features and
it does not scale well. it does not scale well.
EVPN provides a unified control-plane that solves the NVE auto- EVPN provides a unified control-plane that solves the NVE auto-
discovery, tenant MAP/IP dissemination and advanced features in a discovery, tenant MAP/IP dissemination and advanced features in a
scalable way and keeping the independence of the underlay IP Fabric, scalable way and keeping the independence of the underlay IP Fabric,
i.e. there is no need to enable PIM in the underlay network and i.e. there is no need to enable PIM in the underlay network and
maintain multicast states for tenant BDs. maintain multicast states for tenant BDs.
Section 4 describes how to apply EVPN to meet the control-plane Section 4 describes how to apply EVPN to meet the control-plane
requirements in an NVO3 network. requirements in an NVO3 network.
4. Applicability of EVPN to NVO3 Networks 4. Applicability of EVPN to NVO3 Networks
This section discusses the applicability of EVPN to NVO3 networks. This section discusses the applicability of EVPN to NVO3 networks.
The intend is not to provide a comprehensive explanation of the The intend is not to provide a comprehensive explanation of the
protocol itself but give an introduction and point at the protocol itself but give an introduction and point at the
corresponding reference document, so that the reader can easily find corresponding reference document, so that the reader can easily find
more details if needed. more details if needed.
4.1. EVPN Route Types used in NVO3 Networks 4.1. EVPN Route Types used in NVO3 Networks
EVPN supports multiple Route Types and each type has a different EVPN supports multiple Route Types and each type has a different
function. For convenience, Table 1 shows a summary of all the function. For convenience, Table 1 shows a summary of all the
existing EVPN route types and its usage. We will refer to these route existing EVPN route types and its usage. We will refer to these
types as RT-x throughout the rest of the document, where x is the route types as RT-x throughout the rest of the document, where x is
type number included in the first column of Table 1. the type number included in the first column of Table 1.
+----+------------------------+-------------------------------------+ +------+---------------------+--------------------------------------+
|Type|Description |Usage | | Type | Description | Usage |
+----+------------------------+-------------------------------------+ +------+---------------------+--------------------------------------+
|1 |Ethernet Auto-Discovery |Multi-homing: | | 1 | Ethernet Auto- | Multi-homing: Per-ES: Mass |
| | | Per-ES: Mass withdrawal | | | Discovery | withdrawal Per-EVI: aliasing/backup |
| | | Per-EVI: aliasing/backup | +------+---------------------+--------------------------------------+
+----+------------------------+-------------------------------------+ | 2 | MAC/IP | Host MAC/IP dissemination Supports |
|2 |MAC/IP Advertisement |Host MAC/IP dissemination | | | Advertisement | MAC mobility and protection |
| | |Supports MAC mobility and protection | +------+---------------------+--------------------------------------+
+----+------------------------+-------------------------------------+ | 3 | Inclusive Multicast | NVE discovery and BUM flooding tree |
|3 |Inclusive Multicast |NVE discovery and BUM flooding tree | | | Ethernet Tag | setup |
| |Ethernet Tag |setup | +------+---------------------+--------------------------------------+
+----+------------------------+-------------------------------------+ | 4 | Ethernet Segment | Multi-homing: ES auto-discovery and |
|4 |Ethernet Segment |Multi-homing: ES auto-discovery and | | | | DF Election |
| | |DF Election | +------+---------------------+--------------------------------------+
+----+------------------------+-------------------------------------+ | 5 | IP Prefix | IP Prefix dissemination |
|5 |IP Prefix |IP Prefix dissemination | +------+---------------------+--------------------------------------+
+----+------------------------+-------------------------------------+ | 6 | Selective Multicast | Indicate interest for a multicast |
|6 |Selective Multicast |Indicate interest for a multicast | | | Ethernet Tag | S,G or *,G |
| |Ethernet Tag |S,G or *,G | +------+---------------------+--------------------------------------+
+----+------------------------+-------------------------------------+ | 7 | Multicast Join | Multi-homing: S,G or *,G state synch |
|7 |Multicast Join Synch |Multi-homing: S,G or *,G state synch | | | Synch | |
+----+------------------------+-------------------------------------+ +------+---------------------+--------------------------------------+
|8 |Multicast Leave Synch |Multi-homing: S,G or *,G leave synch | | 8 | Multicast Leave | Multi-homing: S,G or *,G leave synch |
+----+------------------------+-------------------------------------+ | | Synch | |
|9 |Per-Region I-PMSI A-D |BUM tree creation across regions | +------+---------------------+--------------------------------------+
+----+------------------------+-------------------------------------+ | 9 | Per-Region I-PMSI | BUM tree creation across regions |
|10 |S-PMSI A-D |Multicast tree for S,G or *,G states | | | A-D | |
+----+------------------------+-------------------------------------+ +------+---------------------+--------------------------------------+
|11 |Leaf A-D |Used for responses to explicit | | 10 | S-PMSI A-D | Multicast tree for S,G or *,G states |
| | |tracking | +------+---------------------+--------------------------------------+
+----+------------------------+-------------------------------------+ | 11 | Leaf A-D | Used for responses to explicit |
| | | tracking |
+------+---------------------+--------------------------------------+
Table 1 EVPN route types Table 1: EVPN route types
4.2. EVPN Basic Applicability For Layer-2 Services 4.2. EVPN Basic Applicability For Layer-2 Services
Although the applicability of EVPN to NVO3 networks spans multiple Although the applicability of EVPN to NVO3 networks spans multiple
documents, EVPN's baseline specification is [RFC7432]. [RFC7432] documents, EVPN's baseline specification is [RFC7432]. [RFC7432]
allows multipoint layer-2 VPNs to be operated as [RFC4364] IP-VPNs, allows multipoint layer-2 VPNs to be operated as [RFC4364] IP-VPNs,
where MACs and the information to setup flooding trees are where MACs and the information to setup flooding trees are
distributed by MP-BGP. Based on [RFC7432], [RFC8365] describes how to distributed by MP-BGP. Based on [RFC7432], [RFC8365] describes how
use EVPN to deliver Layer-2 services specifically in NVO3 Networks. to use EVPN to deliver Layer-2 services specifically in NVO3
Networks.
Figure 1 represents a Layer-2 service deployed with an EVPN BD in an Figure 1 represents a Layer-2 service deployed with an EVPN BD in an
NVO3 network. NVO3 network.
+--TS2---+ +--TS2---+
* | Single-Active * | Single-Active
* | ESI-1 * | ESI-1
+----+ +----+ +----+ +----+
|BD1 | |BD1 | |BD1 | |BD1 |
+-------------| |--| |-----------+ +-------------| |--| |-----------+
skipping to change at page 10, line 25 skipping to change at page 10, line 28
+-------------+ RT-2 | | | +-------------+ RT-2 | | |
| +-MAC-VRF1+ | +-------+ +----+ | | +-MAC-VRF1+ | +-------+ +----+ |
| | +----+ | | |MAC1 | NVE5 TS3 | | +----+ | | |MAC1 | NVE5 TS3
TS1--------|BD1 | | | |IP1 | +----+ | TS1--------|BD1 | | | |IP1 | +----+ |
MAC1 | | +----+ | | |Label L|---> | BD1|=====+ MAC1 | | +----+ | | |Label L|---> | BD1|=====+
IP1 | +---------+ | |NH IP-A| | | All-Active IP1 | +---------+ | |NH IP-A| | | All-Active
| Hypervisor | +-------+ +----+ ESI-2 | Hypervisor | +-------+ +----+ ESI-2
+-------------+ | +-------------+ |
+--------------------------------------+ +--------------------------------------+
Figure 1 EVPN for L2 in an NVO3 Network - example Figure 1: EVPN for L2 in an NVO3 Network - example
In a simple NVO3 network, such as the example of Figure 1, these are In a simple NVO3 network, such as the example of Figure 1, these are
the basic constructs that EVPN uses for Layer-2 services (or Layer-2 the basic constructs that EVPN uses for Layer-2 services (or Layer-2
Virtual Networks): Virtual Networks):
o BD1 is an EVPN Broadcast Domain for a given tenant and TS1, TS2 and o BD1 is an EVPN Broadcast Domain for a given tenant and TS1, TS2
TS3 are connected to it. The five represented NVEs are attached to and TS3 are connected to it. The five represented NVEs are
BD1 and are connected to the same underlay IP network. That is, attached to BD1 and are connected to the same underlay IP network.
each NVE learns the remote NVEs' loopback addresses via underlay That is, each NVE learns the remote NVEs' loopback addresses via
routing protocol. underlay routing protocol.
o NVE1 is deployed as a virtual switch in a Hypervisor with IP-A as o NVE1 is deployed as a virtual switch in a Hypervisor with IP-A as
underlay loopback IP address. The rest of the NVEs in Figure 1 are underlay loopback IP address. The rest of the NVEs in Figure 1
physical switches and TS2/TS3 are multi-homed to them. TS1 is a are physical switches and TS2/TS3 are multi-homed to them. TS1 is
virtual machine, identified by MAC1 and IP1. TS2 and TS3 are a virtual machine, identified by MAC1 and IP1. TS2 and TS3 are
physically dual-connected to NVEs, hence they are normally not physically dual-connected to NVEs, hence they are normally not
considered virtual machines. considered virtual machines.
4.2.1. Auto-Discovery and Auto-Provisioning of ES, Multi-Homing PEs and 4.2.1. Auto-Discovery and Auto-Provisioning
NVE services
Auto-discovery is one of the basic capabilities of EVPN. The Auto-discovery is one of the basic capabilities of EVPN. The
provisioning of EVPN components in NVEs is significantly automated, provisioning of EVPN components in NVEs is significantly automated,
simplifying the deployment of services and minimizing manual simplifying the deployment of services and minimizing manual
operations that are prone to human error. operations that are prone to human error.
These are some of the Auto-Discovery and Auto-Provisioning These are some of the Auto-Discovery and Auto-Provisioning
capabilities available in EVPN: capabilities available in EVPN:
o Automation on Ethernet Segments (ES): an ES is defined as a group o Automation on Ethernet Segments (ES): an ES is defined as a group
of NVEs that are attached to the same TS or network. An ES is of NVEs that are attached to the same TS or network. An ES is
identified by an Ethernet Segment Identifier (ESI) in the control identified by an Ethernet Segment Identifier (ESI) in the control
plane, but neither the ESI nor the NVEs that share the same ES are plane, but neither the ESI nor the NVEs that share the same ES are
required to be manually provisioned in the local NVE: required to be manually provisioned in the local NVE:
- If the multi-homed TS or network are running protocols such as - If the multi-homed TS or network are running protocols such as
LACP (Link Aggregation Control Protocol), MSTP (Multiple-instance LACP (Link Aggregation Control Protocol), MSTP (Multiple-
Spanning Tree Protocol), G.8032, etc. and all the NVEs in the ES instance Spanning Tree Protocol), G.8032, etc. and all the NVEs
can listen to the protocol PDUs to uniquely identify the multi- in the ES can listen to the protocol PDUs to uniquely identify
homed TS/network, then the ESI can be "auto-sensed" or "auto- the multi- homed TS/network, then the ESI can be "auto-sensed"
provisioned" following the guidelines in [RFC7432] section 5. or "auto-provisioned" following the guidelines in [RFC7432]
section 5. The ESI can also be auto-derived out of other
parameters that are common to all NVEs attached to the same ES.
- As described in [RFC7432], EVPN can also auto-derive the BGP - As described in [RFC7432], EVPN can also auto-derive the BGP
parameters required to advertise the presence of a local ES in parameters required to advertise the presence of a local ES in
the control plane (RT and RD). Local ESes are advertised using the control plane (RT and RD). Local ESes are advertised using
RT-4s and the ESI-import Route-Target used by RT-4s can be auto- RT-4s and the ESI-import Route-Target used by RT-4s can be
derived based on the procedures of [RFC7432], section 7.6. auto-derived based on the procedures of [RFC7432], section 7.6.
- By listening to other RT-4s that match the local ESI and import - By listening to other RT-4s that match the local ESI and import
RT, an NVE can also auto-discover the other NVEs participating in RT, an NVE can also auto-discover the other NVEs participating
the multi-homing for the ES. in the multi-homing for the ES.
- Once the NVE has auto-discovered all the NVEs attached to the - Once the NVE has auto-discovered all the NVEs attached to the
same ES, the NVE can automatically perform the DF Election same ES, the NVE can automatically perform the DF Election
algorithm (which determines the NVE that will forward traffic to algorithm (which determines the NVE that will forward traffic
the multi-homed TS/network). EVPN guarantees that all the NVEs in to the multi-homed TS/network). EVPN guarantees that all the
the ES have a consistent DF Election. NVEs in the ES have a consistent DF Election.
o Auto-provisioning of services: when deploying a Layer-2 Service for o Auto-provisioning of services: when deploying a Layer-2 Service
a tenant in an NVO3 network, all the NVEs attached to the same for a tenant in an NVO3 network, all the NVEs attached to the same
subnet must be configured with a MAC-VRF and the BD for the subnet, subnet must be configured with a MAC-VRF and the BD for the
as well as certain parameters for them. Note that, if the EVPN subnet, as well as certain parameters for them. Note that, if the
service model is VLAN-based or VLAN-bundle, implementations do not EVPN service model is VLAN-based or VLAN-bundle, implementations
normally have a specific provisioning for the BD (since it is in do not normally have a specific provisioning for the BD (since it
that case the same construct as the MAC-VRF). EVPN allows auto- is in that case the same construct as the MAC-VRF). EVPN allows
deriving as many MAC-VRF parameters as possible. As an example, the auto-deriving as many MAC-VRF parameters as possible. As an
MAC-VRF's RT and RD for the EVPN routes may be auto-derived. example, the MAC-VRF's RT and RD for the EVPN routes may be auto-
Section 5.1.2.1 in [RFC8365] specifies how to auto-derive a MAC- derived. Section 5.1.2.1 in [RFC8365] specifies how to auto-
VRF's RT as long as VLAN-based service model is implemented. derive a MAC-VRF's RT as long as VLAN-based service model is
[RFC7432] specifies how to auto-derive the RD. implemented. [RFC7432] specifies how to auto-derive the RD.
4.2.2. Remote NVE Auto-Discovery 4.2.2. Remote NVE Auto-Discovery
Auto-discovery via MP-BGP is used to discover the remote NVEs Auto-discovery via MP-BGP is used to discover the remote NVEs
attached to a given BD, NVEs participating in a given redundancy attached to a given BD, NVEs participating in a given redundancy
group, the tunnel encapsulation types supported by an NVE, etc. group, the tunnel encapsulation types supported by an NVE, etc.
In particular, when a new MAC-VRF and BD are enabled, the NVE will In particular, when a new MAC-VRF and BD are enabled, the NVE will
advertise a new RT-3. Besides other fields, the RT-3 will encode the advertise a new RT-3. Besides other fields, the RT-3 will encode the
IP address of the advertising NVE, the Ethernet Tag (which is zero in IP address of the advertising NVE, the Ethernet Tag (which is zero in
case of VLAN-based and VLAN-bundle models) and also a PMSI Tunnel case of VLAN-based and VLAN-bundle models) and also a PMSI Tunnel
Attribute (PTA) that indicates the information about the intended way Attribute (PTA) that indicates the information about the intended way
to deliver BUM traffic for the BD. to deliver BUM traffic for the BD.
In the example of Figure 1, when MAC-VRF1/BD1 are enabled, NVE1 will In the example of Figure 1, when MAC-VRF1/BD1 are enabled, NVE1 will
send an RT-3 including its own IP address, Ethernet-Tag for BD1 and send an RT-3 including its own IP address, Ethernet-Tag for BD1 and
the PTA. Assuming Ingress Replication (IR), the RT-3 will include an the PTA. Assuming Ingress Replication (IR), the RT-3 will include an
identification for IR in the PTA and the VNI the NVEs must use to identification for IR in the PTA and the VNI the NVEs must use to
send BUM traffic to the advertising NVE. The other NVEs in the BD, send BUM traffic to the advertising NVE. The other NVEs in the BD,
will import the RT-3 and will add NVE1's IP address to the flooding will import the RT-3 and will add NVE1's IP address to the flooding
list for BD1. Note that the RT-3 is also sent with a BGP list for BD1. Note that the RT-3 is also sent with a BGP
encapsulation attribute [TUNNEL-ENCAP] that indicates what NVO3 encapsulation attribute [I-D.ietf-idr-tunnel-encaps] that indicates
encapsulation the remote NVEs should use when sending BUM traffic to what NVO3 encapsulation the remote NVEs should use when sending BUM
NVE1. traffic to NVE1.
Refer to [RFC7432] for more information about the RT-3 and forwarding Refer to [RFC7432] for more information about the RT-3 and forwarding
of BUM traffic, and to [RFC8365] for its considerations on NVO3 of BUM traffic, and to [RFC8365] for its considerations on NVO3
networks. networks.
4.2.3. Distribution Of Tenant MAC and IP Information 4.2.3. Distribution Of Tenant MAC and IP Information
Tenant MAC/IP information is advertised to remote NVEs using RT-2s. Tenant MAC/IP information is advertised to remote NVEs using RT-2s.
Following the example of Figure 1: Following the example of Figure 1:
o In a given EVPN BD, TSes' MAC addresses are first learned at the o In a given EVPN BD, TSes' MAC addresses are first learned at the
NVE they are attached to, via data path or management plane NVE they are attached to, via data path or management plane
learning. In Figure 1 we assume NVE1 learns MAC1/IP1 in the learning. In Figure 1 we assume NVE1 learns MAC1/IP1 in the
management plane (for instance, via Cloud Management System) since management plane (for instance, via Cloud Management System) since
the NVE is a virtual switch. NVE2, NVE3, NVE4 and NVE4 are TOR/Leaf the NVE is a virtual switch. NVE2, NVE3, NVE4 and NVE4 are TOR/
switches and they normally learn MAC addresses via data path. Leaf switches and they normally learn MAC addresses via data path.
o Once NVE1's BD1 learns MAC1/IP1, NVE1 advertises that information o Once NVE1's BD1 learns MAC1/IP1, NVE1 advertises that information
along with a VNI and Next Hop IP-A in an RT-2. The EVPN routes are along with a VNI and Next Hop IP-A in an RT-2. The EVPN routes
advertised using the RD/RTs of the MAC-VRF where the BD belongs. are advertised using the RD/RTs of the MAC-VRF where the BD
All the NVEs in BD1 learn local MAC/IP addresses and advertise them belongs. All the NVEs in BD1 learn local MAC/IP addresses and
in RT-2 routes in a similar way. advertise them in RT-2 routes in a similar way.
o The remote NVEs can then add MAC1 to their mapping table for BD1 o The remote NVEs can then add MAC1 to their mapping table for BD1
(BT). For instance, when TS3 sends frames to NVE4 with MAC DA = (BT). For instance, when TS3 sends frames to NVE4 with MAC DA =
MAC1, NVE4 does a MAC lookup on the BT that yields IP-A and Label MAC1, NVE4 does a MAC lookup on the BT that yields IP-A and Label
L. NVE4 can then encapsulate the frame into an NVO3 tunnel with IP- L. NVE4 can then encapsulate the frame into an NVO3 tunnel with
A as the tunnel IP DA and L as the Virtual Network Identifier. Note IP-A as the tunnel IP DA and L as the Virtual Network Identifier.
that the RT-2 may also contain the host's IP address (as in the Note that the RT-2 may also contain the host's IP address (as in
example of Figure 1). While the MAC of the received RT-2 is the example of Figure 1). While the MAC of the received RT-2 is
installed in the BT, the IP address may be installed in the Proxy- installed in the BT, the IP address may be installed in the Proxy-
ARP/ND table (if enabled) or in the ARP/IP-VRF tables if the BD has ARP/ND table (if enabled) or in the ARP/IP-VRF tables if the BD
an IRB. See section 4.7.3. to see more information about Proxy- has an IRB. See Section 4.7.3 to see more information about
ARP/ND and section 4.3. for more details about IRB and Layer-3 Proxy-ARP/ND and Section 4.3. for more details about IRB and
services. Layer-3 services.
Refer to [RFC7432] and [RFC8365] for more information about the RT-2 Refer to [RFC7432] and [RFC8365] for more information about the RT-2
and forwarding of known unicast traffic. and forwarding of known unicast traffic.
4.3. EVPN Basic Applicability for Layer-3 Services 4.3. EVPN Basic Applicability for Layer-3 Services
[IP-PREFIX] and [INTER-SUBNET] are the reference documents that [I-D.ietf-bess-evpn-prefix-advertisement] and
describe how EVPN can be used for Layer-3 services. Inter Subnet [I-D.ietf-bess-evpn-inter-subnet-forwarding] are the reference
Forwarding in EVPN networks is implemented via IRB interfaces between documents that describe how EVPN can be used for Layer-3 services.
BDs and IP-VRFs. As discussed, an EVPN BD corresponds to an IP Inter Subnet Forwarding in EVPN networks is implemented via IRB
subnet. When IP packets generated in a BD are destined to a different interfaces between BDs and IP-VRFs. As discussed, an EVPN BD
subnet (different BD) of the same tenant, the packets are sent to the corresponds to an IP subnet. When IP packets generated in a BD are
IRB attached to local BD in the source NVE. As discussed in [INTER- destined to a different subnet (different BD) of the same tenant, the
SUBNET], depending on how the IP packets are forwarded between the packets are sent to the IRB attached to local BD in the source NVE.
ingress NVE and the egress NVE, there are two forwarding models: As discussed in [I-D.ietf-bess-evpn-inter-subnet-forwarding],
Asymmetric and Symmetric. depending on how the IP packets are forwarded between the ingress NVE
and the egress NVE, there are two forwarding models: Asymmetric and
Symmetric.
The Asymmetric model is illustrated in the example of Figure 2 and it The Asymmetric model is illustrated in the example of Figure 2 and it
requires the configuration of all the BDs of the tenant in all the requires the configuration of all the BDs of the tenant in all the
NVEs attached to the same tenant. In that way, there is no need to NVEs attached to the same tenant. In that way, there is no need to
advertise IP Prefixes between NVEs since all the NVEs are attached to advertise IP Prefixes between NVEs since all the NVEs are attached to
all the subnets. It is called Asymmetric because the ingress and all the subnets. It is called Asymmetric because the ingress and
egress NVEs do not perform the same number of lookups in the data egress NVEs do not perform the same number of lookups in the data
plane. In Figure 2, if TS1 and TS2 are in different subnets, and TS1 plane. In Figure 2, if TS1 and TS2 are in different subnets, and TS1
sends IP packets to TS2, the following lookups are required in the sends IP packets to TS2, the following lookups are required in the
data path: a MAC lookup (on BD1's table), an IP lookup (on the IP- data path: a MAC lookup (on BD1's table), an IP lookup (on the IP-
VRF) and a MAC lookup (on BD2's table) at the ingress NVE1 and then VRF) and a MAC lookup (on BD2's table) at the ingress NVE1 and then
only a MAC lookup at the egress NVE. The two IP-VRFs in Figure 2 are only a MAC lookup at the egress NVE. The two IP-VRFs in Figure 2 are
not connected by tunnels and all the connectivity between the NVEs is not connected by tunnels and all the connectivity between the NVEs is
done based on tunnels between the BDs. done based on tunnels between the BDs.
+-------------------------------------+ +-------------------------------------+
| EVPN NVO3 | | EVPN NVO3 |
| | | |
NVE1 NVE2 NVE1 NVE2
+--------------------+ +--------------------+ +--------------------+ +--------------------+
| +---+IRB +------+ | | +------+IRB +---+ | | +---+IRB +------+ | | +------+IRB +---+ |
TS1-----|BD1|----|IP-VRF| | | |IP-VRF|----|BD1| | TS1-----|BD1|----|IP-VRF| | | |IP-VRF|----|BD1| |
| +---+ | | | | | | +---+ | | +---+ | | | | | | +---+ |
| +---+ | | | | | | +---+ | | +---+ | | | | | | +---+ |
| |BD2|----| | | | | |----|BD2|----TS2 | |BD2|----| | | | | |----|BD2|----TS2
| +---+IRB +------+ | | +------+IRB +---+ | | +---+IRB +------+ | | +------+IRB +---+ |
+--------------------+ +--------------------+ +--------------------+ +--------------------+
| | | |
+-------------------------------------+ +-------------------------------------+
Figure 2 EVPN for L3 in an NVO3 Network - Asymmetric model Figure 2: EVPN for L3 in an NVO3 Network - Asymmetric model
In the Symmetric model, depicted in Figure 3, the same number of data In the Symmetric model, depicted in Figure 3, the same number of data
path lookups is needed at the ingress and egress NVEs. For example, path lookups is needed at the ingress and egress NVEs. For example,
if TS1 sends IP packets to TS3, the following data path lookups are if TS1 sends IP packets to TS3, the following data path lookups are
required: a MAC lookup at NVE1's BD1 table, an IP lookup at NVE1's required: a MAC lookup at NVE1's BD1 table, an IP lookup at NVE1's
IP-VRF and then IP lookup and MAC lookup at NVE2's IP-VRF and BD3 IP-VRF and then IP lookup and MAC lookup at NVE2's IP-VRF and BD3
respectively. In the Symmetric model, the Inter Subnet connectivity respectively. In the Symmetric model, the Inter Subnet connectivity
between NVEs is done based on tunnels between the IP-VRFs. between NVEs is done based on tunnels between the IP-VRFs.
+-------------------------------------+ +-------------------------------------+
| EVPN NVO3 | | EVPN NVO3 |
| | | |
NVE1 NVE2 NVE1 NVE2
+--------------------+ +--------------------+ +--------------------+ +--------------------+
| +---+IRB +------+ | | +------+IRB +---+ | | +---+IRB +------+ | | +------+IRB +---+ |
TS1-----|BD1|----|IP-VRF| | | |IP-VRF|----|BD3|-----TS3 TS1-----|BD1|----|IP-VRF| | | |IP-VRF|----|BD3|-----TS3
| +---+ | | | | | | +---+ | | +---+ | | | | | | +---+ |
| +---+IRB | | | | +------+ | | +---+IRB | | | | +------+ |
TS2-----|BD2|----| | | +--------------------+ TS2-----|BD2|----| | | +--------------------+
| +---+ +------+ | | | +---+ +------+ | |
+--------------------+ | +--------------------+ |
| | | |
+-------------------------------------+ +-------------------------------------+
Figure 3 EVPN for L3 in an NVO3 Network - Symmetric model Figure 3: EVPN for L3 in an NVO3 Network - Symmetric model
The Symmetric model scales better than the Asymmetric model because The Symmetric model scales better than the Asymmetric model because
it does not require the NVEs to be attached to all the tenant's it does not require the NVEs to be attached to all the tenant's
subnets. However, it requires the use of NVO3 tunnels on the IP-VRFs subnets. However, it requires the use of NVO3 tunnels on the IP-VRFs
and the exchange of IP Prefixes between the NVEs in the control and the exchange of IP Prefixes between the NVEs in the control
plane. EVPN uses RT-2 and RT-5 routes for the exchange of host IP plane. EVPN uses RT-2 and RT-5 routes for the exchange of host IP
routes (in the case of RT-2 and RT-5) and IP Prefixes (RT-5s) of any routes (in the case of RT-2 and RT-5) and IP Prefixes (RT-5s) of any
length. As an example, in Figure 3, NVE2 needs to advertise TS3's length. As an example, in Figure 3, NVE2 needs to advertise TS3's
host route and/or TS3's subnet, so that the IP lookup on NVE1's IP- host route and/or TS3's subnet, so that the IP lookup on NVE1's IP-
VRF succeeds. VRF succeeds.
[INTER-SUBNET] specifies the use of RT-2s for the advertisement of [I-D.ietf-bess-evpn-inter-subnet-forwarding] specifies the use of RT-
host routes. Section 4.4.1 in [IP-PREFIX] specifies the use of RT-5s 2s for the advertisement of host routes. Section 4.4.1 in
[I-D.ietf-bess-evpn-prefix-advertisement] specifies the use of RT-5s
for the advertisement of IP Prefixes in an "Interface-less IP-VRF-to- for the advertisement of IP Prefixes in an "Interface-less IP-VRF-to-
IP-VRF Model". The Symmetric model for host routes can be implemented IP-VRF Model". The Symmetric model for host routes can be
following either approach: implemented following either approach:
a. [INTER-SUBNET] uses RT-2s to convey the information to populate a. [I-D.ietf-bess-evpn-inter-subnet-forwarding] uses RT-2s to convey
L2, ARP/ND and L3 FIB tables in the remote NVE. For instance, in the information to populate L2, ARP/ND and L3 FIB tables in the
Figure 3, NVE2 would advertise a RT-2 with TS3's IP and MAC remote NVE. For instance, in Figure 3, NVE2 would advertise a
addresses, and including two labels/VNIs: a label-3/VNI-3 that RT-2 with TS3's IP and MAC addresses, and including two labels/
identifies BD3 for MAC lookup (that would be used for L2 traffic VNIs: a label-3/VNI-3 that identifies BD3 for MAC lookup (that
in case NVE1 was attached to BD3 too) and a label-1/VNI-1 that would be used for L2 traffic in case NVE1 was attached to BD3
identifies the IP-VRF for IP lookup (and will be used for L3 too) and a label-1/VNI-1 that identifies the IP-VRF for IP lookup
traffic). NVE1 imports the RT-2 and installs TS3's IP in the IP- (and will be used for L3 traffic). NVE1 imports the RT-2 and
VRF route table with label-1/VNI-1. Traffic from e.g., TS2 to TS3, installs TS3's IP in the IP- VRF route table with label-1/VNI-1.
will be encapsulated with label-1/VNI-1 and forwarded to NVE2. Traffic from e.g., TS2 to TS3, will be encapsulated with label-1/
VNI-1 and forwarded to NVE2.
b. [IP-PREFIX] uses RT-2s to convey the information to populate the b. [I-D.ietf-bess-evpn-prefix-advertisement] uses RT-2s to convey
L2 FIB and ARP/ND tables, and RT-5s to populate the IP-VRF L3 FIB the information to populate the L2 FIB and ARP/ND tables, and RT-
table. For instance, in Figure 3, NVE2 would advertise a RT-2 5s to populate the IP-VRF L3 FIB table. For instance, in
including TS3's MAC and IP addresses with a single label-3/VNI-3. Figure 3, NVE2 would advertise a RT-2 including TS3's MAC and IP
In this example, this RT-2 wouldn't be imported by NVE1 because addresses with a single label-3/VNI-3. In this example, this
NVE1 is not attached to BD3. In addition, NVE2 would advertise a RT-2 wouldn't be imported by NVE1 because NVE1 is not attached to
RT-5 with TS3's IP address and label-1/VNI-1. This RT-5 would be BD3. In addition, NVE2 would advertise a RT-5 with TS3's IP
imported by NVE1's IP-VRF and the host route installed in the L3 address and label-1/VNI-1. This RT-5 would be imported by NVE1's
FIB associated to label-1/VNI-1. Traffic from TS2 to TS3 would be IP-VRF and the host route installed in the L3 FIB associated to
encapsulated with label-1/VNI-1. label-1/VNI-1. Traffic from TS2 to TS3 would be encapsulated
with label-1/VNI-1.
4.4. EVPN as a Control Plane for NVO3 Encapsulations and GENEVE 4.4. EVPN as a Control Plane for NVO3 Encapsulations and GENEVE
[RFC8365] describes how to use EVPN for NVO3 encapsulations, such us [RFC8365] describes how to use EVPN for NVO3 encapsulations, such us
VXLAN, nvGRE or MPLSoGRE. The procedures can be easily applicable to VXLAN, nvGRE or MPLSoGRE. The procedures can be easily applicable to
any other NVO3 encapsulation, in particular GENEVE. any other NVO3 encapsulation, in particular GENEVE.
The NVO3 working group has been working on different data plane The NVO3 working group has been working on different data plane
encapsulations. The Generic Network Virtualization Encapsulation encapsulations. The Generic Network Virtualization Encapsulation
[GENEVE] has been recommended to be the proposed standard for NVO3 [I-D.ietf-nvo3-geneve] has been recommended to be the proposed
Encapsulation. The EVPN control plane can signal the GENEVE standard for NVO3 Encapsulation. The EVPN control plane can signal
encapsulation type in the BGP Tunnel Encapsulation Extended Community the GENEVE encapsulation type in the BGP Tunnel Encapsulation
(see [TUNNEL-ENCAP]). Extended Community (see [I-D.ietf-idr-tunnel-encaps]).
The NVO3 encapsulation design team has made a recommendation in The NVO3 encapsulation design team has made a recommendation in
[NVO3-ENCAP] for a control plane to: [I-D.ietf-nvo3-encap] for a control plane to:
1- Negotiate a subset of GENEVE option TLVs that can be carried on a 1. Negotiate a subset of GENEVE option TLVs that can be carried on a
GENEVE tunnel GENEVE tunnel
2- Enforce an order for GENEVE option TLVs and 2. Enforce an order for GENEVE option TLVs and
3- Limit the total number of options that could be carried on a 3. Limit the total number of options that could be carried on a
GENEVE tunnel. GENEVE tunnel.
The EVPN control plane can easily extend the BGP Tunnel Encapsulation The EVPN control plane can easily extend the BGP Tunnel Encapsulation
Attribute sub-TLV [TUNNEL-ENCAP] to specify the GENEVE tunnel options Attribute sub-TLV [I-D.ietf-idr-tunnel-encaps] to specify the GENEVE
that can be received or transmitted over a GENEVE tunnels by a given tunnel options that can be received or transmitted over a GENEVE
NVE. [EVPN-GENEVE] describes the EVPN control plane extensions to tunnels by a given NVE. [I-D.boutros-bess-evpn-geneve] describes the
support GENEVE. EVPN control plane extensions to support GENEVE.
4.5. EVPN OAM and application to NVO3 4.5. EVPN OAM and application to NVO3
EVPN OAM (as in [EVPN-LSP-PING]) defines mechanisms to detect data EVPN OAM (as in [I-D.ietf-bess-evpn-lsp-ping]) defines mechanisms to
plane failures in an EVPN deployment over an MPLS network. These detect data plane failures in an EVPN deployment over an MPLS
mechanisms detect failures related to P2P and P2MP connectivity, for network. These mechanisms detect failures related to P2P and P2MP
multi-tenant unicast and multicast L2 traffic, between multi-tenant connectivity, for multi-tenant unicast and multicast L2 traffic,
access nodes connected to EVPN PE(s), and in a single-homed, single- between multi-tenant access nodes connected to EVPN PE(s), and in a
active or all-active redundancy model. single-homed, single- active or all-active redundancy model.
In general, EVPN OAM mechanisms defined for EVPN deployed in MPLS In general, EVPN OAM mechanisms defined for EVPN deployed in MPLS
networks are equally applicable for EVPN in NVO3 networks. networks are equally applicable for EVPN in NVO3 networks.
4.6. EVPN as the control plane for NVO3 security 4.6. EVPN as the control plane for NVO3 security
EVPN can be used to signal the security protection capabilities of a EVPN can be used to signal the security protection capabilities of a
sender NVE, as well as what portion of an NVO3 packet (taking a sender NVE, as well as what portion of an NVO3 packet (taking a
GENEVE packet as an example) can be protected by the sender NVE, to GENEVE packet as an example) can be protected by the sender NVE, to
ensure the privacy and integrity of tenant traffic carried over the ensure the privacy and integrity of tenant traffic carried over the
NVO3 tunnels. NVO3 tunnels.
4.7. Advanced EVPN Features For NVO3 Networks 4.7. Advanced EVPN Features For NVO3 Networks
This section describes how EVPN can be used to deliver advanced This section describes how EVPN can be used to deliver advanced
capabilities in NVO3 networks. capabilities in NVO3 networks.
4.7.1. Virtual Machine (VM) Mobility 4.7.1. Virtual Machine (VM) Mobility
[RFC7432] replaces the traditional Ethernet Flood-and-Learn behavior [RFC7432] replaces the traditional Ethernet Flood-and-Learn behavior
among NVEs with BGP-based MAC learning, which in return provides more among NVEs with BGP-based MAC learning, which in return provides more
control over the location of MAC addresses in the BD and consequently control over the location of MAC addresses in the BD and consequently
advanced features, such as MAC Mobility. If we assume that VM advanced features, such as MAC Mobility. If we assume that VM
Mobility means the VM's MAC and IP addresses move with the VM, EVPN's Mobility means the VM's MAC and IP addresses move with the VM, EVPN's
MAC Mobility is the required procedure that facilitates VM Mobility. MAC Mobility is the required procedure that facilitates VM Mobility.
According to [RFC7432] section 15, when a MAC is advertised for the According to [RFC7432] section 15, when a MAC is advertised for the
first time in a BD, all the NVEs attached to the BD will store first time in a BD, all the NVEs attached to the BD will store
Sequence Number zero for that MAC. When the MAC "moves" within the Sequence Number zero for that MAC. When the MAC "moves" within the
same BD but to a remote NVE, the NVE that just learned locally the same BD but to a remote NVE, the NVE that just learned locally the
MAC, increases the Sequence Number in the RT-2's MAC Mobility MAC, increases the Sequence Number in the RT-2's MAC Mobility
extended community to indicate that it owns the MAC now. That makes extended community to indicate that it owns the MAC now. That makes
all the NVE in the BD change their tables immediately with no need to all the NVE in the BD change their tables immediately with no need to
wait for any aging timer. EVPN guarantees a fast MAC Mobility without wait for any aging timer. EVPN guarantees a fast MAC Mobility
flooding or black-holes in the BD. without flooding or black-holes in the BD.
4.7.2. MAC Protection, Duplication Detection and Loop Protection 4.7.2. MAC Protection, Duplication Detection and Loop Protection
The advertisement of MACs in the control plane, allows advanced The advertisement of MACs in the control plane, allows advanced
features such as MAC protection, Duplication Detection and Loop features such as MAC protection, Duplication Detection and Loop
Protection. Protection.
[RFC7432] MAC Protection refers to EVPN's ability to indicate - in an [RFC7432] MAC Protection refers to EVPN's ability to indicate - in an
RT-2 - that a MAC must be protected by the NVE receiving the route. RT-2 - that a MAC must be protected by the NVE receiving the route.
The Protection is indicated in the "Sticky bit" of the MAC Mobility The Protection is indicated in the "Sticky bit" of the MAC Mobility
extended community sent along the RT-2 for a MAC. NVEs' ACs that are extended community sent along the RT-2 for a MAC. NVEs' ACs that are
connected to subject-to-be-protected servers or VMs may set the connected to subject-to-be-protected servers or VMs may set the
Sticky bit on the RT-2s sent for the MACs associated to the ACs. Also Sticky bit on the RT-2s sent for the MACs associated to the ACs.
statically configured MAC addresses should be advertised as Protected Also statically configured MAC addresses should be advertised as
MAC addresses, since they are not subject to MAC Mobility procedures. Protected MAC addresses, since they are not subject to MAC Mobility
procedures.
[RFC7432] MAC Duplication Detection refers to EVPN's ability to [RFC7432] MAC Duplication Detection refers to EVPN's ability to
detect duplicate MAC addresses. A "MAC move" is a relearn event that detect duplicate MAC addresses. A "MAC move" is a relearn event that
happens at an access AC or through an RT-2 with a Sequence Number happens at an access AC or through an RT-2 with a Sequence Number
that is higher than the stored one for the MAC. When a MAC moves a that is higher than the stored one for the MAC. When a MAC moves a
number of times N within an M-second window between two NVEs, the MAC number of times N within an M-second window between two NVEs, the MAC
is declared as Duplicate and the detecting NVE does not re-advertise is declared as Duplicate and the detecting NVE does not re-advertise
the MAC anymore. the MAC anymore.
While [RFC7432] provides MAC Duplication Detection, it does not [RFC7432] provides MAC Duplication Detection, and with an extension
protect the BD against loops created by backdoor links between NVEs. it can protect the BD against loops created by backdoor links between
However, the same principle (based on the Sequence Number) may be NVEs. The same principle (based on the Sequence Number) may be
extended to protect the BD against loops. When a MAC is detected as extended to protect the BD against loops. When a MAC is detected as
duplicate, the NVE may install it as a black-hole MAC and drop duplicate, the NVE may install it as a black-hole MAC and drop
received frames with MAC SA and MAC DA matching that duplicate MAC. received frames with MAC SA and MAC DA matching that duplicate MAC.
Loop Protection is described in [LOOP].
4.7.3. Reduction/Optimization of BUM Traffic In Layer-2 Services 4.7.3. Reduction/Optimization of BUM Traffic In Layer-2 Services
In BDs with a significant amount of flooding due to Unknown unicast In BDs with a significant amount of flooding due to Unknown unicast
and Broadcast frames, EVPN may help reduce and sometimes even and Broadcast frames, EVPN may help reduce and sometimes even
suppress the flooding. suppress the flooding.
In BDs where most of the Broadcast traffic is caused by ARP (Address In BDs where most of the Broadcast traffic is caused by ARP (Address
Resolution Protocol) and ND (Neighbor Discovery) protocols on the Resolution Protocol) and ND (Neighbor Discovery) protocols on the
TSes, EVPN's Proxy-ARP and Proxy-ND capabilities may reduce the TSes, EVPN's Proxy-ARP and Proxy-ND capabilities may reduce the
flooding drastically. The use of Proxy-ARP/ND is specified in [PROXY- flooding drastically. The use of Proxy-ARP/ND is specified in
ARP-ND]. [I-D.ietf-bess-evpn-proxy-arp-nd].
Proxy-ARP/ND procedures along with the assumption that TSes always Proxy-ARP/ND procedures along with the assumption that TSes always
issue a GARP (Gratuitous ARP) or an unsolicited Neighbor issue a GARP (Gratuitous ARP) or an unsolicited Neighbor
Advertisement message when they come up in the BD, may drastically Advertisement message when they come up in the BD, may drastically
reduce the unknown unicast flooding in the BD. reduce the unknown unicast flooding in the BD.
The flooding caused by TSes' IGMP/MLD or PIM messages in the BD may The flooding caused by TSes' IGMP/MLD or PIM messages in the BD may
also be suppressed by the use of IGMP/MLD and PIM Proxy functions, as also be suppressed by the use of IGMP/MLD and PIM Proxy functions, as
specified in [IGMP-MLD-PROXY] and [PIM-PROXY]. These two documents specified in [I-D.ietf-bess-evpn-igmp-mld-proxy] and
also specify how to forward IP multicast traffic efficiently within [I-D.skr-bess-evpn-pim-proxy]. These two documents also specify how
the same BD, translate soft state IGMP/MLD/PIM messages into hard to forward IP multicast traffic efficiently within the same BD,
state BGP routes and provide fast-convergence redundancy for IP translate soft state IGMP/MLD/PIM messages into hard state BGP routes
Multicast on multi-homed Ethernet Segments (ESes). and provide fast-convergence redundancy for IP Multicast on multi-
homed Ethernet Segments (ESes).
4.7.4. Ingress Replication (IR) Optimization For BUM Traffic 4.7.4. Ingress Replication (IR) Optimization For BUM Traffic
When an NVE attached to a given BD needs to send BUM traffic for the When an NVE attached to a given BD needs to send BUM traffic for the
BD to the remote NVEs attached to the same BD, IR is a very common BD to the remote NVEs attached to the same BD, IR is a very common
option in NVO3 networks, since it is completely independent of the option in NVO3 networks, since it is completely independent of the
multicast capabilities of the underlay network. Also, if the multicast capabilities of the underlay network. Also, if the
optimization procedures to reduce/suppress the flooding in the BD are optimization procedures to reduce/suppress the flooding in the BD are
enabled (section 4.7.3), in spite of creating multiple copies of the enabled (Section 4.7.3), in spite of creating multiple copies of the
same frame at the ingress NVE, IR may be good enough. However, in BDs same frame at the ingress NVE, IR may be good enough. However, in
where Multicast (or Broadcast) traffic is significant, IR may be very BDs where Multicast (or Broadcast) traffic is significant, IR may be
inefficient and cause performance issues on virtual-switch-based very inefficient and cause performance issues on virtual-switch-based
NVEs. NVEs.
[OPT-IR] specifies the use of AR (Assisted Replication) NVO3 tunnels [I-D.ietf-bess-evpn-optimized-ir] specifies the use of AR (Assisted
in EVPN BDs. AR retains the independence of the underlay network Replication) NVO3 tunnels in EVPN BDs. AR retains the independence
while providing a way to forward Broadcast and Multicast traffic of the underlay network while providing a way to forward Broadcast
efficiently. AR uses AR-REPLICATORs that can replicate the and Multicast traffic efficiently. AR uses AR-REPLICATORs that can
Broadcast/Multicast traffic on behalf of the AR-LEAF NVEs. The AR- replicate the Broadcast/Multicast traffic on behalf of the AR-LEAF
LEAF NVEs are typically virtual-switches or NVEs with limited NVEs. The AR-LEAF NVEs are typically virtual-switches or NVEs with
replication capabilities. AR can work in a single-stage replication limited replication capabilities. AR can work in a single-stage
mode (Non-Selective Mode) or in a dual-stage replication mode replication mode (Non-Selective Mode) or in a dual-stage replication
(Selective Mode). Both modes are detailed in [OPT-IR]. mode (Selective Mode). Both modes are detailed in
[I-D.ietf-bess-evpn-optimized-ir].
In addition, [OPT-IR] also describes a procedure to avoid sending In addition, [I-D.ietf-bess-evpn-optimized-ir] also describes a
Broadcast, Multicast or Unknown unicast to certain NVEs that don't procedure to avoid sending Broadcast, Multicast or Unknown unicast to
need that type of traffic. This is done by enabling PFL (Pruned Flood certain NVEs that don't need that type of traffic. This is done by
Lists) on a given BD. For instance, an virtual-switch NVE that learns enabling PFL (Pruned Flood Lists) on a given BD. For instance, an
all its local MAC addresses for a BD via Cloud Management System, virtual-switch NVE that learns all its local MAC addresses for a BD
does not need to receive the BD's Unknown unicast traffic. PFLs help via Cloud Management System, does not need to receive the BD's
optimize the BUM flooding in the BD. Unknown unicast traffic. PFLs help optimize the BUM flooding in the
BD.
4.7.5. EVPN Multi-homing 4.7.5. EVPN Multi-homing
Another fundamental concept in EVPN is multi-homing. A given TS can Another fundamental concept in EVPN is multi-homing. A given TS can
be multi-homed to two or more NVEs for a given BD, and the set of be multi-homed to two or more NVEs for a given BD, and the set of
links connected to the same TS is defined as Ethernet Segment (ES). links connected to the same TS is defined as Ethernet Segment (ES).
EVPN supports single-active and all-active multi-homing. In single- EVPN supports single-active and all-active multi-homing. In single-
active multi-homing only one link in the ES is active. In all-active active multi-homing only one link in the ES is active. In all-active
multi-homing all the links in the ES are active for unicast traffic. multi-homing all the links in the ES are active for unicast traffic.
Both modes support load-balancing: Both modes support load-balancing:
o Single-active multi-homing means per-service load-balancing o Single-active multi-homing means per-service load-balancing to/
to/from the TS, for example, in Figure 1, for BD1 only one of the from the TS, for example, in Figure 1, for BD1 only one of the
NVEs can forward traffic from/to TS2. For a different BD, the NVEs can forward traffic from/to TS2. For a different BD, the
other NVE may forward traffic. other NVE may forward traffic.
o All-active multi-homing means per-flow load-balanding for unicast o All-active multi-homing means per-flow load-balanding for unicast
frames to/from the TS. That is, in Figure 1 and for BD1, both frames to/from the TS. That is, in Figure 1 and for BD1, both
NVE4 and NVE5 can forward known unicast traffic to/from TS3. For NVE4 and NVE5 can forward known unicast traffic to/from TS3. For
BUM traffic only one of the two NVEs can forward traffic to TS3, BUM traffic only one of the two NVEs can forward traffic to TS3,
and both can forward traffic from TS3. and both can forward traffic from TS3.
There are two key aspects of EVPN multi-homing: There are two key aspects in the EVPN multi-homing procedures:
o DF (Designated Forwarder) election: the DF is the NVE that o DF (Designated Forwarder) election: the DF is the NVE that
forwards the traffic to the ES in single-active mode. In case of forwards the traffic to the ES in single-active mode. In case of
all-active, the DF is the NVE that forwards the BUM traffic to all-active, the DF is the NVE that forwards the BUM traffic to the
the ES. ES.
o Split-horizon function: prevents the TS from receiving echoed BUM o Split-horizon function: prevents the TS from receiving echoed BUM
frames that the TS itself sent to the ES. This is especially frames that the TS itself sent to the ES. This is especially
relevant in all-active ESes, where the TS may forward BUM frames relevant in all-active ESes, where the TS may forward BUM frames
to a non-DF NVE that can flood the BUM frames back to the DF NVE to a non-DF NVE that can flood the BUM frames back to the DF NVE
and then the TS. As an example, in Figure 1, assuming NVE4 is the and then the TS. As an example, in Figure 1, assuming NVE4 is the
DF for ES-2 in BD1, BUM frames sent from TS3 to NVE5 will be DF for ES-2 in BD1, BUM frames sent from TS3 to NVE5 will be
received at NVE4 and, since NVE4 is the DF for DB1, it will received at NVE4 and, since NVE4 is the DF for DB1, it will
forward them back to TS3. Split-horizon allows NVE4 (and any forward them back to TS3. Split-horizon allows NVE4 (and any
multi-homed NVE for that matter) to identify if an EVPN BUM frame multi-homed NVE for that matter) to identify if an EVPN BUM frame
is coming from the same ES or different, and if the frame belongs is coming from the same ES or different, and if the frame belongs
to the same ES2, NVE4 will not forward the BUM frame to TS3, in to the same ES2, NVE4 will not forward the BUM frame to TS3, in
spite of being the DF. spite of being the DF.
While [RFC7432] describes the default algorithm for the DF Election, While [RFC7432] describes the default algorithm for the DF Election,
[RFC8584] and [PREF-DF] specify other algorithms and procedures that [RFC8584] and [I-D.ietf-bess-evpn-pref-df] specify other algorithms
optimize the DF Election. and procedures that optimize the DF Election.
The Split-horizon function is specified in [RFC7432] and it is The Split-horizon function is specified in [RFC7432] and it is
carried out by using a special ESI-label that it identifies in the carried out by using a special ESI-label that it identifies in the
data path, all the BUM frames being originated from a given NVE and data path, all the BUM frames being originated from a given NVE and
ES. Since the ESI-label is an MPLS label, it cannot be used in all ES. Since the ESI-label is an MPLS label, it cannot be used in all
the non-MPLS NVO3 encapsulations, therefore [RFC8365] defines a the non-MPLS NVO3 encapsulations, therefore [RFC8365] defines a
modified Split-horizon procedure that is based on the IP SA of the modified Split-horizon procedure that is based on the IP SA of the
NVO3 tunnel, known as "Local-Bias". It is worth noting that Local- NVO3 tunnel, known as "Local-Bias". It is worth noting that Local-
Bias only works for all-active multi-homing, and not for single- Bias only works for all-active multi-homing, and not for single-
active multi-homing. active multi-homing.
4.7.6. EVPN Recursive Resolution for Inter-Subnet Unicast Forwarding 4.7.6. EVPN Recursive Resolution for Inter-Subnet Unicast Forwarding
Section 4.3. describes how EVPN can be used for Inter Subnet Section 4.3. describes how EVPN can be used for Inter Subnet
Forwarding among subnets of the same tenant. RT-2s and RT-5s allow Forwarding among subnets of the same tenant. RT-2s and RT-5s allow
the advertisement of host routes and IP Prefixes (RT-5) of any the advertisement of host routes and IP Prefixes (RT-5) of any
length. The procedures outlined by section 4.3. are similar to the length. The procedures outlined by Section 4.3 are similar to the
ones in [RFC4364], only for NVO3 tunnels. However, [EVPN-PREFIX] also ones in [RFC4364], only for NVO3 tunnels. However,
defines advanced Inter Subnet Forwarding procedures that allow the [I-D.ietf-bess-evpn-prefix-advertisement] also defines advanced Inter
resolution of RT-5s to not only BGP next-hops but also "overlay Subnet Forwarding procedures that allow the resolution of RT-5s to
indexes" that can be a MAC, a GW IP or an ESI, all of them in the not only BGP next-hops but also "overlay indexes" that can be a MAC,
tenant space. a GW IP or an ESI, all of them in the tenant space.
Figure 4 illustrates an example that uses Recursive Resolution to a Figure 4 illustrates an example that uses Recursive Resolution to a
GWIP as per [IP-PREFIX] section 4.4.2. In this example, IP-VRFs in GW-IP as per [I-D.ietf-bess-evpn-prefix-advertisement] section 4.4.2.
NVE1 and NVE2 are connected by a SBD (Supplementary BD). An SBD is a In this example, IP-VRFs in NVE1 and NVE2 are connected by a SBD
BD that connects all the IP-VRFs of the same tenant, via IRB, and has (Supplementary BD). An SBD is a BD that connects all the IP-VRFs of
no ACs. NVE1 advertises the host route TS2-IP/L (IP address and the same tenant, via IRB, and has no ACs. NVE1 advertises the host
Prefix Length of TS2) in an RT-5 with overlay index GWIP=IP1. Also, route TS2-IP/L (IP address and Prefix Length of TS2) in an RT-5 with
IP1 is advertised in an RT-2 associated to M1, VNI-S and BGP next-hop overlay index GWIP=IP1. Also, IP1 is advertised in an RT-2
NVE1. Upon importing the two routes, NVE2 installs TS2-IP/L in the associated to M1, VNI-S and BGP next-hop NVE1. Upon importing the
IP-VRF with a next-hop that is the GWIP IP1. NVE2 also installs M1 in two routes, NVE2 installs TS2-IP/L in the IP-VRF with a next-hop that
the SBD, with VNI-S and NVE1 as next-hop. If TS3 sends a packet with is the GWIP IP1. NVE2 also installs M1 in the SBD, with VNI-S and
IP DA=TS2, NVE2 will perform a Recursive Resolution of the RT-5 NVE1 as next-hop. If TS3 sends a packet with IP DA=TS2, NVE2 will
prefix information to the forwarding information of the correlated perform a Recursive Resolution of the RT-5 prefix information to the
RT-2. The RT-5's Recursive Resolution has several advantages such as forwarding information of the correlated RT-2. The RT-5's Recursive
better convergence in scaled networks (since multiple RT-5s can be Resolution has several advantages such as better convergence in
invalidated with a single withdrawal of the overlay index route) or scaled networks (since multiple RT-5s can be invalidated with a
the ability to advertise multiple RT-5s from an overlay index that single withdrawal of the overlay index route) or the ability to
can move or change dynamically. [EVPN-PREFIX] describes a few use- advertise multiple RT-5s from an overlay index that can move or
cases. change dynamically. [EVPN-PREFIX] describes a few use- cases.
+-------------------------------------+ +-------------------------------------+
| EVPN NVO3 | | EVPN NVO3 |
| + | +
NVE1 NVE2 NVE1 NVE2
+--------------------+ +--------------------+ +--------------------+ +--------------------+
| +---+IRB +------+ | | +------+IRB +---+ | | +---+IRB +------+ | | +------+IRB +---+ |
TS1-----|BD1|----|IP-VRF| | | |IP-VRF|----|BD3|-----TS3 TS1-----|BD1|----|IP-VRF| | | |IP-VRF|----|BD3|-----TS3
| +---+ | |-(SBD)------(SBD)-| | +---+ | | +---+ | |-(SBD)------(SBD)-| | +---+ |
| +---+IRB | |IRB(IP1/M1) IRB+------+ | | +---+IRB | |IRB(IP1/M1) IRB+------+ |
TS2-----|BD2|----| | | +-----------+--------+ TS2-----|BD2|----| | | +-----------+--------+
| +---+ +------+ | | | +---+ +------+ | |
+--------------------+ | +--------------------+ |
| RT-2(M1,IP1,VNI-S,NVE1)--> | | RT-2(M1,IP1,VNI-S,NVE1)--> |
| RT-5(TS2-IP/L,GWIP=IP1)--> | | RT-5(TS2-IP/L,GWIP=IP1)--> |
+-------------------------------------+ +-------------------------------------+
Figure 4 EVPN for L3 - Recursive Resolution example Figure 4: EVPN for L3 - Recursive Resolution example
4.7.7. EVPN Optimized Inter-Subnet Multicast Forwarding 4.7.7. EVPN Optimized Inter-Subnet Multicast Forwarding
The concept of the SBD described in section 4.7.6 is also used in The concept of the SBD described in Section 4.7.6 is also used in
[OISM] for the procedures related to Inter Subnet Multicast [I-D.ietf-bess-evpn-irb-mcast] for the procedures related to Inter
Forwarding across BDs of the same tenant. For instance, [OISM] allows Subnet Multicast Forwarding across BDs of the same tenant. For
the efficient forwarding of IP multicast traffic from any BD to any instance, [I-D.ietf-bess-evpn-irb-mcast] allows the efficient
other BD (or even to the same BD where the Source resides). The forwarding of IP multicast traffic from any BD to any other BD (or
[OISM] procedures are supported along with EVPN multi-homing, and for even to the same BD where the Source resides). The
any tree allowed on NVO3 networks, including IR or AR. [OISM] also [I-D.ietf-bess-evpn-irb-mcast] procedures are supported along with
describes the interoperability between EVPN and other multicast EVPN multi-homing, and for any tree allowed on NVO3 networks,
technologies such as MVPN (Multicast VPN) and PIM for inter-subnet including IR or AR. [I-D.ietf-bess-evpn-irb-mcast] also describes
multicast. the interoperability between EVPN and other multicast technologies
such as MVPN (Multicast VPN) and PIM for inter-subnet multicast.
[EVPN-MVPN] describes another potential solution to support EVPN to [I-D.sajassi-bess-evpn-mvpn-seamless-interop] describes another
MVPN interoperability. potential solution to support EVPN to MVPN interoperability.
4.7.8. Data Center Interconnect (DCI) 4.7.8. Data Center Interconnect (DCI)
Tenant Layer-2 and Layer-3 services deployed on NVO3 networks must be Tenant Layer-2 and Layer-3 services deployed on NVO3 networks must be
extended to remote NVO3 networks that are connected via non-NOV3 WAN extended to remote NVO3 networks that are connected via non-NOV3 WAN
networks (mostly MPLS based WAN networks). [EVPN-DCI] defines some networks (mostly MPLS based WAN networks).
architectural models that can be used to interconnect NVO3 networks [I-D.ietf-bess-dci-evpn-overlay] defines some architectural models
via MPLS WAN networks. that can be used to interconnect NVO3 networks via MPLS WAN networks.
When NVO3 networks are connected by MPLS WAN networks, [EVPN-DCI] When NVO3 networks are connected by MPLS WAN networks,
specifies how EVPN can be used end-to-end, in spite of using a [I-D.ietf-bess-dci-evpn-overlay] specifies how EVPN can be used end-
different encapsulation in the WAN. to-end, in spite of using a different encapsulation in the WAN.
Even if EVPN can also be used in the WAN for Layer-2 and Layer-3 Even if EVPN can also be used in the WAN for Layer-2 and Layer-3
services, there may be a need to provide a Gateway function between services, there may be a need to provide a Gateway function between
EVPN for NVO3 encapsulations and IPVPN for MPLS tunnels. [EVPN-IPVPN] EVPN for NVO3 encapsulations and IPVPN for MPLS tunnels.
specifics the interworking function between EVPN and IPVPN for [I-D.ietf-bess-evpn-ipvpn-interworking] specifics the interworking
unicast Inter Subnet Forwarding. If Inter Subnet Multicast Forwarding function between EVPN and IPVPN for unicast Inter Subnet Forwarding.
is also needed across an IPVPN WAN, [OISM] describes the required If Inter Subnet Multicast Forwarding is also needed across an IPVPN
WAN, [I-D.ietf-bess-evpn-irb-mcast] describes the required
interworking between EVPN and MVPN. interworking between EVPN and MVPN.
5. Conclusion 5. Conclusion
EVPN provides a unified control-plane that solves the NVE auto- EVPN provides a unified control-plane that solves the NVE auto-
discovery, tenant MAP/IP dissemination and advanced features required discovery, tenant MAP/IP dissemination and advanced features required
by NVO3 networks, in a scalable way and keeping the independence of by NVO3 networks, in a scalable way and keeping the independence of
the underlay IP Fabric, i.e. there is no need to enable PIM in the the underlay IP Fabric, i.e. there is no need to enable PIM in the
underlay network and maintain multicast states for tenant BDs. underlay network and maintain multicast states for tenant BDs.
This document justifies the use of EVPN for NVO3 networks, discusses This document justifies the use of EVPN for NVO3 networks, discusses
its applicability to basic Layer-2 and Layer-3 connectivity its applicability to basic Layer-2 and Layer-3 connectivity
requirements, as well as advanced features such as MAC-mobility, MAC requirements, as well as advanced features such as MAC-mobility, MAC
Protection and Loop Protection, multi-homing, DCI and much more. Protection and Loop Protection, multi-homing, DCI and much more.
6. Conventions used in this document 6. Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP "OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all 14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here. capitals, as shown here.
7. Security Considerations 7. Security Considerations
This document does not introduce any new procedure or additional This document does not introduce any new procedure or additional
signaling in EVPN, and relies on the security considerations of the signaling in EVPN, and relies on the security considerations of the
individual specifications used as a reference throughout the individual specifications used as a reference throughout the
document. In particular, and as mentioned in [RFC7432], control plane document. In particular, and as mentioned in [RFC7432], control
and forwarding path protection are aspects to secure in any EVPN plane and forwarding path protection are aspects to secure in any
domain, when applied to NVO3 networks. EVPN domain, when applied to NVO3 networks.
[RFC7432] mentions security techniques such as those discussed in [RFC7432] mentions security techniques such as those discussed in
[RFC5925] to authenticate BGP messages, and those included in [RFC5925] to authenticate BGP messages, and those included in
[RFC4271], [RFC4272] and [RFC6952] to secure BGP are relevant for [RFC4271], [RFC4272] and [RFC6952] to secure BGP are relevant for
EVPN in NVO3 networks as well. EVPN in NVO3 networks as well.
8. IANA Considerations 8. IANA Considerations
None. None.
9. References 9. References
9.1 Normative References 9.1. Normative References
[RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A., [RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A.,
Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based Ethernet Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based
VPN", RFC 7432, DOI 10.17487/RFC7432, February 2015, <http://www.rfc- Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February
editor.org/info/rfc7432>. 2015, <https://www.rfc-editor.org/info/rfc7432>.
[RFC7365] Lasserre, M., Balus, F., Morin, T., Bitar, N., and Y. [RFC7365] Lasserre, M., Balus, F., Morin, T., Bitar, N., and Y.
Rekhter, "Framework for Data Center (DC) Network Virtualization", Rekhter, "Framework for Data Center (DC) Network
RFC 7365, DOI 10.17487/RFC7365, October 2014, <http://www.rfc- Virtualization", RFC 7365, DOI 10.17487/RFC7365, October
editor.org/info/rfc7365>. 2014, <https://www.rfc-editor.org/info/rfc7365>.
[RFC7364] Narten, T., Ed., Gray, E., Ed., Black, D., Fang, L., [RFC7364] Narten, T., Ed., Gray, E., Ed., Black, D., Fang, L.,
Kreeger, L., and M. Napierala, "Problem Statement: Overlays for Kreeger, L., and M. Napierala, "Problem Statement:
Network Virtualization", RFC 7364, DOI 10.17487/RFC7364, October Overlays for Network Virtualization", RFC 7364,
2014, <http://www.rfc-editor.org/info/rfc7364>. DOI 10.17487/RFC7364, October 2014,
<https://www.rfc-editor.org/info/rfc7364>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March Requirement Levels", BCP 14, RFC 2119,
1997, <https://www.rfc-editor.org/info/rfc2119>. DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in
RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May
2017, <https://www.rfc-editor.org/info/rfc8174>.
9.2 Informative References
[IP-PREFIX] Rabadan et al., "IP Prefix Advertisement in EVPN", [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
draft-ietf-bess-evpn-prefix-advertisement-11, work in progress, May, 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
2018. May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[INTER-SUBNET] Sajassi et al., "IP Inter-Subnet Forwarding in EVPN", 9.2. Informative References
draft-ietf-bess-evpn-inter-subnet-forwarding-08, work in progress,
March, 2019.
[RFC8365] Sajassi-Drake et al., "A Network Virtualization Overlay [I-D.ietf-bess-evpn-prefix-advertisement]
Solution using EVPN", RFC 8365, March 2017, <http://www.rfc- Rabadan, J., Henderickx, W., Drake, J., Lin, W., and A.
editor.org/info/rfc8365>. Sajassi, "IP Prefix Advertisement in EVPN", draft-ietf-
bess-evpn-prefix-advertisement-11 (work in progress), May
2018.
[GENEVE] Gross et al., "Geneve: Generic Network Virtualization [I-D.ietf-bess-evpn-inter-subnet-forwarding]
Encapsulation", draft-ietf-nvo3-geneve-13, work in progress, March Sajassi, A., Salam, S., Thoria, S., Drake, J., and J.
2019. Rabadan, "Integrated Routing and Bridging in EVPN", draft-
ietf-bess-evpn-inter-subnet-forwarding-11 (work in
progress), October 2020.
[NVO3-ENCAP] Boutros et al., "NVO3 Encapsulation Considerations", [RFC8365] Sajassi, A., Ed., Drake, J., Ed., Bitar, N., Shekhar, R.,
draft-ietf-nvo3-encap-03, work in progress, July 2019. Uttaro, J., and W. Henderickx, "A Network Virtualization
Overlay Solution Using Ethernet VPN (EVPN)", RFC 8365,
DOI 10.17487/RFC8365, March 2018,
<https://www.rfc-editor.org/info/rfc8365>.
[TUNNEL-ENCAP] Rosen et al., "The BGP Tunnel Encapsulation [I-D.ietf-nvo3-geneve]
Attribute", draft-ietf-idr-tunnel-encaps-12, work in progress, May Gross, J., Ganga, I., and T. Sridhar, "Geneve: Generic
2019. Network Virtualization Encapsulation", draft-ietf-
nvo3-geneve-16 (work in progress), March 2020.
[EVPN-LSP-PING] Jain et al., "LSP-Ping Mechanisms for EVPN and PBB- [I-D.ietf-nvo3-encap]
EVPN", draft-ietf-bess-evpn-lsp-ping-00, work in progress, May 2019. Boutros, S., "NVO3 Encapsulation Considerations", draft-
ietf-nvo3-encap-05 (work in progress), February 2020.
[LOOP] Rabadan et al., "Loop Protection in EVPN networks", draft- [I-D.ietf-idr-tunnel-encaps]
snr-bess-evpn-loop-protect-02, work in progress, August 2018. Patel, K., Velde, G., Sangli, S., and J. Scudder, "The BGP
Tunnel Encapsulation Attribute", draft-ietf-idr-tunnel-
encaps-19 (work in progress), September 2020.
[PROXY-ARP-ND] Rabadan et al., "Operational Aspects of Proxy-ARP/ND [I-D.ietf-bess-evpn-lsp-ping]
in EVPN Networks", draft-ietf-bess-evpn-proxy-arp-nd-07, work in Jain, P., Salam, S., Sajassi, A., Boutros, S., and G.
progress, July 2019. Mirsky, "LSP-Ping Mechanisms for EVPN and PBB-EVPN",
draft-ietf-bess-evpn-lsp-ping-03 (work in progress),
August 2020.
[IGMP-MLD-PROXY] Sajassi et al., "IGMP and MLD Proxy for EVPN", [I-D.ietf-bess-evpn-proxy-arp-nd]
draft-ietf-bess-evpn-igmp-mld-proxy-03, work in progress, June 2019. Rabadan, J., Sathappan, S., Nagaraj, K., Hankins, G., and
T. King, "Operational Aspects of Proxy-ARP/ND in EVPN
Networks", draft-ietf-bess-evpn-proxy-arp-nd-09 (work in
progress), October 2020.
[PIM-PROXY] Rabadan et al., "PIM Proxy in EVPN Networks", draft-skr- [I-D.ietf-bess-evpn-igmp-mld-proxy]
bess-evpn-pim-proxy-01, work in progress, October 2017. Sajassi, A., Thoria, S., Patel, K., Drake, J., and W. Lin,
"IGMP and MLD Proxy for EVPN", draft-ietf-bess-evpn-igmp-
mld-proxy-05 (work in progress), April 2020.
[OPT-IR] Rabadan et al., "Optimized Ingress Replication solution for [I-D.skr-bess-evpn-pim-proxy]
EVPN", draft-ietf-bess-evpn-optimized-ir-06, work in progress, Rabadan, J., Kotalwar, J., Sathappan, S., Zhang, Z., and
October 2018. A. Sajassi, "PIM Proxy in EVPN Networks", draft-skr-bess-
evpn-pim-proxy-01 (work in progress), October 2017.
[RFC8584] Rabadan-Mohanty et al., "Framework for EVPN Designated [I-D.ietf-bess-evpn-optimized-ir]
Forwarder Election Extensibility", <https://rfc- Rabadan, J., Sathappan, S., Lin, W., Katiyar, M., and A.
editor.org/rfc/rfc8584.txt>, April 2019. Sajassi, "Optimized Ingress Replication solution for
EVPN", draft-ietf-bess-evpn-optimized-ir-07 (work in
progress), July 2020.
[PREF-DF] Rabadan et al., "Preference-based EVPN DF Election", [RFC8584] Rabadan, J., Ed., Mohanty, S., Ed., Sajassi, A., Drake,
draft-ietf-bess-evpn-pref-df-04, work in progress, June 2019. J., Nagaraj, K., and S. Sathappan, "Framework for Ethernet
VPN Designated Forwarder Election Extensibility",
RFC 8584, DOI 10.17487/RFC8584, April 2019,
<https://www.rfc-editor.org/info/rfc8584>.
[OISM] Lin at al., "EVPN Optimized Inter-Subnet Multicast (OISM) [I-D.ietf-bess-evpn-pref-df]
Forwarding", draft-ietf-bess-evpn-irb-mcast-02, work in progress, Rabadan, J., Sathappan, S., Przygienda, T., Lin, W.,
January 2019. Drake, J., Sajassi, A., and s. satyamoh@cisco.com,
"Preference-based EVPN DF Election", draft-ietf-bess-evpn-
pref-df-06 (work in progress), June 2020.
[EVPN-DCI] Rabadan et al., "Interconnect Solution for EVPN Overlay [I-D.ietf-bess-evpn-irb-mcast]
networks", draft-ietf-bess-dci-evpn-overlay-10, work in progress, Lin, W., Zhang, Z., Drake, J., Rosen, E., Rabadan, J., and
March 2018. A. Sajassi, "EVPN Optimized Inter-Subnet Multicast (OISM)
Forwarding", draft-ietf-bess-evpn-irb-mcast-05 (work in
progress), October 2020.
[BUM-UPDATE] Zhang et al., "Updates on EVPN BUM Procedures", draft- [I-D.ietf-bess-dci-evpn-overlay]
ietf-bess-evpn-bum-procedure-updates-06, work in progress, June 2019. Rabadan, J., Sathappan, S., Henderickx, W., Sajassi, A.,
and J. Drake, "Interconnect Solution for EVPN Overlay
networks", draft-ietf-bess-dci-evpn-overlay-10 (work in
progress), March 2018.
[EVPN-IPVPN] Rabadan-Sajassi et al., "EVPN Interworking with IPVPN", [I-D.ietf-bess-evpn-ipvpn-interworking]
draft-ietf-sajassi-bess-evpn-ipvpn-interworking-01, work in progress, Rabadan, J., Sajassi, A., Rosen, E., Drake, J., Lin, W.,
March 2019. Uttaro, J., and A. Simpson, "EVPN Interworking with
IPVPN", draft-ietf-bess-evpn-ipvpn-interworking-03 (work
in progress), May 2020.
[RFC7348] Mahalingam, M., Dutt, D., Duda, K., Agarwal, P., Kreeger, [RFC7348] Mahalingam, M., Dutt, D., Duda, K., Agarwal, P., Kreeger,
L., Sridhar, T., Bursell, M., and C. Wright, "Virtual eXtensible L., Sridhar, T., Bursell, M., and C. Wright, "Virtual
Local Area Network (VXLAN): A Framework for Overlaying Virtualized eXtensible Local Area Network (VXLAN): A Framework for
Layer 2 Networks over Layer 3 Networks", RFC 7348, DOI Overlaying Virtualized Layer 2 Networks over Layer 3
10.17487/RFC7348, August 2014, <http://www.rfc- Networks", RFC 7348, DOI 10.17487/RFC7348, August 2014,
editor.org/info/rfc7348>. <https://www.rfc-editor.org/info/rfc7348>.
[RFC7510] Xu, X., Sheth, N., Yong, L., Callon, R., and D. Black, [RFC7510] Xu, X., Sheth, N., Yong, L., Callon, R., and D. Black,
"Encapsulating MPLS in UDP", RFC 7510, DOI 10.17487/RFC7510, April "Encapsulating MPLS in UDP", RFC 7510,
2015, <http://www.rfc-editor.org/info/rfc7510>. DOI 10.17487/RFC7510, April 2015,
<https://www.rfc-editor.org/info/rfc7510>.
[RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private [RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private
Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, February 2006, Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, February
<http://www.rfc-editor.org/info/rfc4364>. 2006, <https://www.rfc-editor.org/info/rfc4364>.
[CLOS1953] Clos, C., "A Study of Non-Blocking Switching Networks", [CLOS1953]
The Bell System Technical Journal, Vol. 32(2), DOI 10.1002/j.1538- Clos, C., "A Study of Non-Blocking Switching Networks",
7305.1953.tb01433.x, March 1953. March 1953.
[EVPN-GENEVE] Boutros et al., "EVPN control plane for Geneve", [I-D.boutros-bess-evpn-geneve]
draft-boutros-bess-evpn-geneve-04, work in progress, March 2019. Boutros, S., Sajassi, A., Drake, J., Rabadan, J., and S.
Aldrin, "EVPN control plane for Geneve", draft-boutros-
bess-evpn-geneve-04 (work in progress), March 2019.
[EVPN-MVPN] Sajassi et al., "Seamless Multicast Interoperability [I-D.sajassi-bess-evpn-mvpn-seamless-interop]
between EVPN and MVPN PEs", draft-sajassi-bess-evpn-mvpn-seamless- Sajassi, A., Thiruvenkatasamy, K., Thoria, S., Gupta, A.,
interop-04, work in progress, July 2019. and L. Jalil, "Seamless Multicast Interoperability between
EVPN and MVPN PEs", draft-sajassi-bess-evpn-mvpn-seamless-
interop-04 (work in progress), July 2019.
[RFC5925] Touch, J., Mankin, A., and R. Bonica, "The TCP [RFC5925] Touch, J., Mankin, A., and R. Bonica, "The TCP
Authentication Option", RFC 5925, DOI 10.17487/RFC5925, June 2010, Authentication Option", RFC 5925, DOI 10.17487/RFC5925,
<https://www.rfc-editor.org/info/rfc5925>. June 2010, <https://www.rfc-editor.org/info/rfc5925>.
[RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A [RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A
Border Gateway Protocol 4 (BGP-4)", RFC 4271, DOI 10.17487/RFC4271, Border Gateway Protocol 4 (BGP-4)", RFC 4271,
January 2006, <https://www.rfc-editor.org/info/rfc4271>. DOI 10.17487/RFC4271, January 2006,
<https://www.rfc-editor.org/info/rfc4271>.
[RFC4272] Murphy, S., "BGP Security Vulnerabilities Analysis", RFC [RFC4272] Murphy, S., "BGP Security Vulnerabilities Analysis",
4272, DOI 10.17487/RFC4272, January 2006, <https://www.rfc- RFC 4272, DOI 10.17487/RFC4272, January 2006,
editor.org/info/rfc4272>. <https://www.rfc-editor.org/info/rfc4272>.
[RFC6952] Jethanandani, M., Patel, K., and L. Zheng, "Analysis of [RFC6952] Jethanandani, M., Patel, K., and L. Zheng, "Analysis of
BGP, LDP, PCEP, and MSDP Issues According to the Keying and BGP, LDP, PCEP, and MSDP Issues According to the Keying
Authentication for Routing Protocols (KARP) Design Guide", RFC 6952, and Authentication for Routing Protocols (KARP) Design
DOI 10.17487/RFC6952, May 2013, <https://www.rfc- Guide", RFC 6952, DOI 10.17487/RFC6952, May 2013,
editor.org/info/rfc6952>. <https://www.rfc-editor.org/info/rfc6952>.
10. Acknowledgments Appendix A. Acknowledgments
The authors want to thank Aldrin Isaac for his comments. The authors want to thank Aldrin Isaac for his comments.
11. Contributors Appendix B. Contributors
12. Authors' Addresses Appendix C. Authors' Addresses
Jorge Rabadan (Editor) Authors' Addresses
Jorge Rabadan (editor)
Nokia Nokia
777 E. Middlefield Road 777 Middlefield Road
Mountain View, CA 94043 USA Mountain View, CA 94043
Email: jorge.rabadan@nokia.com USA
Sami Boutros Email: jorge.rabadan@nokia.com
VMware
Email: boutross@vmware.com
Matthew Bocci Matthew Bocci
Nokia Nokia
Email: matthew.bocci@nokia.com Email: matthew.bocci@nokia.com
Sami Boutros
Ciena
Email: sboutros@ciena.com
Ali Sajassi Ali Sajassi
Cisco Cisco Systems, Inc.
Email: sajassi@cisco.com Email: sajassi@cisco.com
 End of changes. 205 change blocks. 
686 lines changed or deleted 735 lines changed or added

This html diff was produced by rfcdiff 1.48. The latest version is available from http://tools.ietf.org/tools/rfcdiff/