EVPN IOS-XR Deep Dive for

Service Providers and Data Center

Jiri Chaloupka, Principal Technical Marketing Engineer

BRKSPG-2835
Cisco Webex App

Questions?
Use Cisco Webex App to chat
with the speaker after the session

How
1 Find this session in the Cisco Live Mobile App
2 Click “Join the Discussion”
3 Install the Webex App or go directly to the Webex space Enter your personal notes here

4 Enter messages/questions in the Webex space

Webex spaces will be moderated

until February 24, 2023.

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 2
 • EVPN Basic Principles
• EVPN L2 All-Active Multihomed Service
• EVPN Distributed L3 Anycast Gateway
• EVPN Centralized Gateway
• EVPN L3 Interconnect Options
• EVPN & VPNv4/6 Interconnect
• EVPN Single-Active / Port-Active
• EVPN Routes - Summary
EVPN-VPWS Multihomed Service
Agenda
•
• EVPN L2 Interconnect & Seamless
Integration/Migration (L2 Services)
• EVPN ETREE
• EVPN Fast Re-Route (FRR)
• EVPN Multicast
• EVPN Head End
• EVPN Transport Integration
• Summary
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
From Mac Bridging to Mac Routing
Common BGP Control Plane
EVPN, VPNv4/6 Overlay
Evolution:
Underlay
Segment Routing (SR: MPLS, SRv6) SR, VXLAN SR, VXLAN

Data Center Network

Service Provider Network overlap

Leaf
VM

PE1 DCI1

Spine Spine

Leaf
VM
A1 Access WAN/Core

Leaf
PE2 DCI2 VM

BGP: VPNv4/6 VPLS Overlay

Existing Solution: Fabric-Path (Trill)
LDP: VPLS, PW Fabric-Path

MPLS: LDP, RSVP-TE MPLS, L2 L2, IP Underlay

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
Service Provider Network - Simplification Journey

Provisioning NETCONF NETCONF

YANG YANG

Programmability

L2/L3VPN Services LDP BGP LDP BGP BGP

Inter-Domain CP BGP-LU BGP-LU

FRR or TE RSVP IGP with
SR-MPLS or
LDP IGP with SR SRv6
Intra-Domain CP
IGP

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
Next-Generation Solutions for L2VPN
Solving VPLS challenges for per-flow Redundancy
M1 M2
CE1 PE1 PE3 CE2
• Existing VPLS solutions do not offer an All- Echo !
Active per-flow redundancy PE2 PE4

• Looping of Traffic Flooded from PE

M1 Duplicate !
M2
• Duplicate Frames from Floods from the
CE1 PE1 PE3 CE2
Core
• MAC Flip-Flopping over Pseudowire PE2 PE4

• E.g. Port-Channel Load-Balancing does not

produce a consistent hash-value for a M1 M2
frame with the same source MAC (e.g. non CE1 PE1
MAC
Flip-Flop PE3 CE2
MAC based
Hash-Schemes)
PE2 PE4

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
 MPLS Transport & BGP Service
BGP L3VPN/ L3 EVPN BGP L2VPN EVPN
BGP Signaling BGP Signaling BGP Signaling BGP Signaling

PE2 PE4 PE2 PE4

CE1 MPLS CE2 CE1 MPLS CE2

PE1 PE3 PE1 PE3

Data Plane Data Plane

IP Packet Transport IP Packet L2 Frame Transport L2 Frame

MPLS Label MPLS Label
Service Service
BGP Label BGP Label
IP Packet L2 Frame

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
EVPN – Basic
Principles
EVPN Advantages:
Integrated • Integrated Layer 2 and Layer 3 VPN services

Services • L3VPN-like principles and operational experience for scalability and control
• All-active Multi-homing & PE load-balancing (ECMP)

• Fast convergence (link, node, MAC moves)

Network
• Control-Place (BGP) learning. PWs are no longer used.
Efficiency
• Optimized Broadcast, Unknown-unicast, Multicast traffic delivery

• Choice of MPLS, VxLAN or SRv6 data plane encapsulation

Service • Support existing and new services types (E-LAN, E-Line, E-TREE)
Flexibility • Peer PE auto-discovery. Redundancy group auto-sensing

Fully support IPv4 and IPv6 in the data plane and control plane
Investment •

Protection • Open-Standard and Multi-vendor support

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
Concepts
EVPN Instance (EVI) Ethernet Segment BGP Routes BGP Route Attributes

SHD Route Types Extended Communities

CE1
BD EVI ESI1 PE1 [1] Ethernet Auto-Discovery (AD) Route ESI MPLS Label

[2] MAC/IP Advertisement Route ES-Import

MHD CE2
[3] Inclusive Multicast Route MAC Mobility
BD
EVI

PE2
ESI2 [4] Ethernet Segment Route Default Gateway
PE
[5] IP Prefix Advertisement Route Encapsulation

• EVI identifies a VPN in the • Represents a ‘site’ • New SAFI [70] • New BGP extended
network connected to one or more • Routes serve control communities defined
• Encompass one or more PEs plane purposes, • Expand information
bridge-domains, • Uniquely identified by a 10- including: carried in BGP routes,
depending on service byte global Ethernet MAC address reachability including:
interface type Segment Identifier (ESI) MAC mass withdrawal MAC address moves
Port-based • Could be a single device Split-Horizon label adv. Redundancy mode
VLAN-based (shown above) or an entire network Aliasing MAC / IP bindings of a GW
VLAN-bundling Single-Homed Device (SHD) Multicast endpoint discovery Split-horizon label encoding
Multi-Homed Device (MHD) Redundancy group discovery Data plane Encapsulation
Single-Homed Network (SHN) Designated forwarder election
Multi-Homed Network (MHN) IP address reachability
L2/L3 Integration

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
 EVPN - Load-Balancing Modes

All-Active Single-Active Port-Active

(per flow) (per VLAN) (per port)

PE1 PE2 PE1 PE2 PE1 PE2

V1 V1 V1 V2 V1, V2

CE1 CE2 CE3

Single LAG at the CE Multiple LAGs at the CE Single LAGs at the CE

VLAN goes to both PE VLAN active on single PE Port active on single PE
Traffic hashed per flow Traffic hashed per VLAN Traffic hashed per port
Benefits: Bandwidth, Convergence Benefits: Billing, Policing Benefits: Protocol Simplification

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
EVPN - Ethernet VPN
• Concepts are same!!! Pick your side!

Pick your side!

SP1 SP2
PE2 PE4
CE1
PE1 PE3
L1 L2 L3 L4

C1 C2
VM VM VM VM

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
 EVPN - Ethernet-Segment for Multi-Homing
L1 and L2 (L3 and L4) have to know if they multi-home same broadcast domain

SP1 SP2
The bundle on the Leafs
connecting to a node should
have Identical ES identifier (ESI)

L1 L2 L3 L4

Unique 10-byte global identifier

per Ethernet Segment Ethernet Segment represents a
C1 C2 node connected multiple Leaves

VM VM VM VM

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
 EVPN - Ethernet VPN
MAC address advertisement and MAC address table synchronization
Leaves run Multi-Protocol BGP to advertise & learn MAC addresses over the Network
MAC addresses are advertised to rest of Leaves
L3/4 – Learn MAC address advertised by L1
L2 – uses MAC address advertised by L1 to synchronize MAC address table
-> L2 forwards MAC via local ETH interface represented by same Ethernet Segment between L1 and L2

SP1 SP2

MAC advertisement &

learning/synchronization via BGP EVPN
NLRI

L1 L2 L3 L4
Data Plane learning
from the hosts
All Active multi-homing
C1 C2 Ethernet Segment
VM VM VM VM

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
EVPN L2 All-Active
Multihomed Service
EVPN - Testbed

RR103 RR104

LACP R39

H2
R38 R35

LACP R37 R34

H1
R36

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
EVPN Configuration CE has to receive same lacp system
MAC

lacp system mac 3637.3637.3637 l2vpn

bridge group 100
interface Bundle-Ether100 bridge-domain 100
l2transport interface Bundle-Ether100
! !
! evi 100
!
evpn RT-2 MAC advertise !
evi 100 !
advertise-mac !
!
interface Bundle-Ether100
ethernet-segment
identifier type 0 36.37.00.00.00.00.00.11.00
!
!

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
EVPN Configuration - BGP
router bgp 1
bgp router-id 3.3.3.36
address-family l2vpn evpn
!
neighbor-group rr
remote-as 1
update-source Loopback0 BGP EVPN CP
address-family l2vpn evpn
!
neighbor 3.3.3.103
use neighbor-group rr
!
neighbor 3.3.3.104
use neighbor-group rr
!
!

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
EVPN – Designated Forwarder (DF)
Challenge:
How to prevent duplicate copies of flooded traffic from being delivered to a multi-homed Ethernet Segment?
If (L3 and L4) Multi-Homing access via same Ethernet Segment -> only one of them can forward traffic to access
Same for (L1 and L2)

Why extra BUM Label?

What if Unicast Traffic is sent to L3 or L4 (not flooded)? -> DF Election applies only to BUM (from Core to Access)
DF, Redirect, Fast Re-Route (FRR), etc.
Service Label informs egress Leaf if traffic is BUM or Unicast

SP1 SP2

L1 L2 L3 L4
NDF DF
C1 Duplicate C2
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 19
DF Election per EVI/ESI - Algorithm
Service Carving

Nodes Position EVIs

R36

R37
0

1
+ 100

EVI-ID modulo Number of Nodes = Position

100 modulo 2 = 0

R36 is DF for EVI-100

Who will be DF for EVI-101?

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 20
Ethernet Segment - DF Election
R36#show evpn ethernet-segment esi 0036.3700.0000.0000.1100 carving detail
……
Ethernet Segment Id Interface Nexthops
------------------------ ---------------------------------- --------------------
0036.3700.0000.0000.1100 BE100 3.3.3.36
3.3.3.37
ES to BGP Gates : Ready
ES to L2FIB Gates : Ready
Main port :
Interface name : Bundle-Ether100
Interface MAC : 008a.9644.d8dd
IfHandle : 0x0800001c
State : Up
Redundancy : Not Defined
ESI type : 0
Value : 36.3700.0000.0000.1100
ES Import RT : 3637.0000.0000 (from ESI)
Source MAC : 0000.0000.0000 (N/A)
Topology :
Operational : MH, All-active
Configured : All-active (AApF) (default)
Service Carving : Auto-selection
Peering Details : 3.3.3.36[MOD:P:00] 3.3.3.37[MOD:P:00]
Service Carving Results:
Forwarders : 1
Permanent : 0
Elected : 1
EVI E : 100
Not Elected : 0
MAC Flushing mode : STP-TCN
Peering timer : 3 sec [not running]
Recovery timer : 30 sec [not running]
Carving timer : 0 sec [not running]
Local SHG label : 64005
Remote SHG labels : 1
64005 : nexthop 3.3.3.37
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
R36: RT-4 Ethernet Segment Router
R36#show bgp l2vpn evpn rd 3.3.3.36:0 [4][0036.3700.0000.0000.1100][32][3.3.3.36]/128
Mon Oct 15 03:24:50.736 UTC
BGP routing table entry for [4][0036.3700.0000.0000.1100][32][3.3.3.36]/128, Route Distinguisher: 3.3.3.36:0
Versions:
Process bRIB/RIB SendTblVer
RT-4 Ethernet Segment Identifier (ESI)
Speaker 82835 82835
Last Modified: Oct 14 21:32:13.399 for 05:52:37
Paths: (1 available, best #1)
Advertised to update-groups (with more than one peer):
0.2
Path #1: Received by speaker 0
Advertised to update-groups (with more than one peer):
0.2
Local
0.0.0.0 from 0.0.0.0 (3.3.3.36)
Origin IGP, localpref 100, valid, redistributed, best, group-best, import-candidate, rib-install
Received Path ID 0, Local Path ID 1, version 82835
Extended community: EVPN ES Import:3637.0000.0000 DF Election:00:0:00

Nodes which share same ESI import this route

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
EVPN – BUM Ingress Replication
Two service labels per EVPN instance
BUM Label – to forward Broadcast, Unknown Unicast and Multicast
Unicast Label – to forward Unicast

SP1 SP2

L1 L2 L3 L4

C1 C2
VM VM VM VM

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
R36: RT-3 Inclusive Multicast
R36#show bgp l2vpn evpn rd 3.3.3.36:100 [3][0][32][3.3.3.36]/80
Mon Oct 15 13:10:17.010 UTC
BGP routing table entry for [3][0][32][3.3.3.36]/80, Route Distinguisher: 3.3.3.36:100
Versions:
Process bRIB/RIB SendTblVer RT-3
Speaker 39774 39774
Last Modified: Aug 31 01:37:02.399 for 6w3d
Paths: (1 available, best #1)
Advertised to update-groups (with more than one peer):
0.2
Path #1: Received by speaker 0
Advertised to update-groups (with more than one peer):
0.2
Local
0.0.0.0 from 0.0.0.0 (3.3.3.36)
Origin IGP, localpref 100, valid, redistributed, best, group-best, import-candidate
Received Path ID 0, Local Path ID 1, version 39774
Extended community: RT:1:100 EVI 100 Route-Target
PMSI: flags 0x00, type 6, label 64120, ID 0x03030324

Ingress Replication Multicast (BUM) Label

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
EVPN – Split Horizon
Challenge:
How to prevent flooded traffic from echoing back to a multi-homed Ethernet Segment?

Transport
BUM Label Label

SP1 SP2
SH Label

L1 L2

Echo !
C1
VM VM

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
EVPN – Split Horizon
Challenge:
How to prevent flooded traffic from echoing back to a multi-homed Ethernet Segment?

Transport
BUM Label Label

SP1 SP2
SH Label

L1 L2

Echo !
C1 C11
VM VM VM VM

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
EVPN – MAC Mass-Withdraw
Challenge:
How to inform other Leafs of a failure affecting many MAC addresses quickly while the
control-plane re-converges?

SP2 MAC1 → ESI1 → Leaf1 + Leaf2

MAC1 can be SP1
reached via ESI1

L1 L2 L3 L4
MAC1 can NOT be
reached via ESI1
C1 C2
VM VM VM VM

ESI1 MAC1
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
R36: RT-1 Per ESI Ethernet Auto-Discovery
R36#show bgp l2vpn evpn rd 3.3.3.36:0 [1][3.3.3.36:1][0036.3700.0000.0000.1100][4294967295]/184
Sun Oct 14 20:56:59.687 UTC
BGP routing table entry for [1][3.3.3.36:1][0036.3700.0000.0000.1100][4294967295]/184, Route Distinguisher: 3.3.3.36:0
Versions:
Process bRIB/RIB SendTblVer RD - unique per advertising Ethernet Segment Identifier (ESI)
RT-1
Speaker 76372 76372 node (R36 unique)
Local Label: 0
Last Modified: Sep 18 23:02:40.399 for 3w4d
Paths: (1 available, best #1)
Advertised to update-groups (with more than one peer):
0.2
Path #1: Received by speaker 0
Advertised to update-groups (with more than one peer):
0.2
Local
0.0.0.0 from 0.0.0.0 (3.3.3.36)
Origin IGP, localpref 100, valid, redistributed, best, group-best, import-candidate, rib-install
Received Path ID 0, Local Path ID 1, version 76372
Extended community: EVPN ESI Label:0x00:64005 RT:1:100 EVI(s) Route-Target
All EVI(s) which use this ESI

Redundancy mode Split-Horizon Label

All-Active: 0x00
Single-Active: 0x01
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
R36, R37, R38, R39 - EVPN Startup
R36 - Example
1. RT4: DF Election & Multi-Homed Ethernet
Segment Auto-Discovery
LACP R39
Service Carving: 100 modulo 2 = 0 H2
R36 is DF for EVI-100
R38 R35

RT-4 - DF Election

LACP R37 R34

RD: 1.1.1.36:1

H1 ESI: 0036.3700.0000.0000.1100

R36 Ext-Com: 3637.0000.0000 (RT)

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 29
R36, R37, R38, R39 - EVPN Startup
R36 - Example
1. RT4: DF Election & Multi-Homed Ethernet
Segment Auto-Discovery
LACP R39
2. RT1: Per ESI Ethernet Auto-Discovery
(Split-Horizon, Mass-Withdraw) H2
RT-1 - Per ESI Ethernet AD
R38 R35
RD: 1.1.1.36:1

ESI: 0036.3700.0000.0000.1100

LACP R37 R34 Flag:0x00 All-Active

Ext-Com:
Split-Horizon Label: 64005
H1 Ext-Com: 1:100 (RT)

R36

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 30
R36, R37, R38, R39 - EVPN Startup
R36 - Example
1. RT4: DF Election & Multi-Homed Ethernet
Segment Auto-Discovery
LACP R39
2. RT1: Per ESI Ethernet Auto-Discovery
(Split-Horizon, Mass-Withdraw) H2
3. RT3: Inclusive Multicast RT-3 - Inclusive Multicast
R38 R35
RD: 1.1.1.36:100
Ext-Com: Type 6 Ingress-Replication
Multicast(BUM) Label: 64120
LACP R37 R34
Ext-Com: 1:100 (RT)

H1
R36

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 31
BUM Forwarding

1. RT4: DF Election & Multi-Homed Ethernet

Segment Auto-Discovery
LACP R39
2. RT1: Per ESI Ethernet Auto-Discovery
(Split-Horizon, Mass-Withdraw) H2
3. RT3: Inclusive Multicast
R38 R35

LACP R37 R34

H1 Transport Label R38-9

R36 BUM Label R38-9/EVI100

BUM - Traffic
IR BUM - Traffic

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 32
BUM Forwarding

1. RT4: DF Election & Multi-Homed Ethernet

Segment Auto-Discovery
LACP R39
2. RT1: Per ESI Ethernet Auto-Discovery
(Split-Horizon, Mass-Withdraw) H2
3. RT3: Inclusive Multicast
R38 R35

X R37 R34
LACP
Transport Label R37
H1 BUM Label R37/EVI100

R36 SH Label R37/ESIx

BUM - Traffic
IR BUM - Traffic

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 33
R36: RT-2 MAC Advertisement
R36#show bgp l2vpn evpn rd 3.3.3.36:100 [2][0][48][0062.ec71.fbd7][0]/104
Mon Oct 15 04:33:39.527 UTC
BGP routing table entry for [2][0][48][0062.ec71.fbd7][0]/104, Route Distinguisher: 3.3.3.36:100
Versions:
Process bRIB/RIB SendTblVer
Speaker 83317 83317 RT-2 Advertised MAC
Local Label: 64004
3.3.3.37 (metric 30) from 3.3.3.103 (3.3.3.37)
Received Label 64004
Origin IGP, localpref 100, valid, internal, import-candidate, imported, rib-install
Received Path ID 0, Local Path ID 0, version 0
Extended community: SoO:3.3.3.37:100 RT:1:100 R37 MAC DP Learned and
Originator: 3.3.3.37, Cluster list: 3.3.3.103 Advertised
EVPN ESI: 0036.3700.0000.0000.1100
Source AFI: L2VPN EVPN, Source VRF: default, Source Route Distinguisher: 3.3.3.37:100

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 34
R36: RT-2 MAC Advertisement
R36#show evpn evi mac
Mon Oct 15 20:57:14.505 UTC

VPN-ID Encap MAC address IP address Nexthop Label

---------- ------ -------------- ---------------------------------------- --------------------------------------- --------
100 MPLS 0062.ec71.1000 :: 3.3.3.38 64006
100 MPLS 0062.ec71.1000 :: 3.3.3.39 64006
100 MPLS 0062.ec71.fbd7 :: 3.3.3.37 64004
100 MPLS 0062.ec71.fbd8 :: Bundle-Ether100 64004
100 MPLS 0062.ec71.fbd9 :: 3.3.3.37 64004
100 MPLS 0062.ec71.fbe0 :: 3.3.3.38 64006
100 MPLS 0062.ec71.fbe0 :: Learned and Advertised 3.3.3.39 64006
100 MPLS 0062.ec71.fbe1 :: MAC 3.3.3.38 64006
100 MPLS 0062.ec71.fbe1 :: 3.3.3.39 64006

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
R36, R37, R38, R39 - EVPN Startup
R36 - Example
1. RT4: DF Election & Multi-Homed Ethernet
Segment Auto-Discovery
LACP R39
2. RT1: Per ESI Ethernet Auto-Discovery
(Split-Horizon, Mass-Withdraw) H2
3. RT3: Inclusive Multicast RT-2 - MAC Advertisement

4. RT2: MAC Advertisement R38 R35

RD: 1.1.1.36:100

ESI: 0036.3700.0000.0000.1100

LACP R37 R34

MAC: 0062.ec71.fbd7

H1 Label: 64004

R36 Ext-Com: 1:100 (RT)

L2 Frame SMAC:
0062.ec71.fbd7

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 36
Unicast Forwarding
L2 Frame Flow1
Transport Label R36
1. RT4: DF Election & Multi-Homed Ethernet DMAC: H1

Segment Auto-Discovery RT-2 MAC Label/EVI

LACP R39
2. RT1: Per ESI Ethernet Auto-Discovery L2 Frame Flow1
DMAC: H1
(Split-Horizon, Mass-Withdraw) H2
3. RT3: Inclusive Multicast
4. RT2: MAC Advertisement R38 R35

LACP R37 R34

H1
R36
L2 Frame Flow1
DMAC: H1

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 37
EVPN – Aliasing
Challenge:
How to load-balance traffic towards a multi-homed device across multiple Leafs when
MAC addresses are learnt by only a single Leaf?

MAC1 can also be

SP2 MAC1 → ESI1 → Leaf1 + Leaf2
reached via ESI1 SP1

L1 L2 L3 L4
MAC1 can be
reached via ESI1
C1 C2
VM VM VM VM

ESI1 MAC1
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
R36: RT-1 Per EVI Ethernet Auto-Discovery
RP/0/RP0/CPU0:R36#show bgp l2vpn evpn rd 3.3.3.36:100 [1][0036.3700.0000.0000.1100][0]/120
Mon Oct 15 03:35:13.604 UTC
BGP routing table entry for [1][0036.3700.0000.0000.1100][0]/120, Route Distinguisher: 3.3.3.36:100
Versions:
Process bRIB/RIB SendTblVer Ethernet Segment Identifier (ESI)
Speaker 79640 7964 RT-1
Last Modified: Oct 12 17:40:06.399 for 2d09h
Paths: (2 available, best #1)
Advertised to update-groups (with more than one peer):
0.2
Path #1: Received by speaker 0
Advertised to update-groups (with more than one peer):
0.2
Local
0.0.0.0 from 0.0.0.0 (3.3.3.36)
Origin IGP, localpref 100, valid, redistributed, best, group-best, import-candidate, rib-install
Received Path ID 0, Local Path ID 1, version 39769
Path #2: Received by speaker 0
Not advertised to any peer
Local
3.3.3.37 (metric 30) from 3.3.3.103 (3.3.3.37) Aliasing Label allocated by R37 for EVI 100
Received Label 64004
Origin IGP, localpref 100, valid, internal, import-candidate, imported, rib-install
Received Path ID 0, Local Path ID 0, version 0
Extended community: RT:1:100 EVI 100 Route-Target
Originator: 3.3.3.37, Cluster list: 3.3.3.103
Source AFI: L2VPN EVPN, Source VRF: default, Source Route Distinguisher: 3.3.3.37:100

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 39
R36, R37, R38, R39 - EVPN Startup
R36 - Example
1. RT4: DF Election & Multi-Homed Ethernet
Segment Auto-Discovery
LACP R39
2. RT1: Per ESI Ethernet Auto-Discovery
(Split-Horizon, Mass-Withdraw) H2
3. RT3: Inclusive Multicast RT-1 - Per EVI Ethernet AD

4. RT2: MAC Advertisement R38 R35

RD: 1.1.1.36:100
5. RT1: Per EVI Ethernet Auto-Discovery
ESI: 0036.3700.0000.0000.1100

LACP R37 R34

Aliasing-Label: 64004

H1 Ext-Com: 1:100 (RT)

R36

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 40
Unicast Forwarding
L2 Frame Flow1
Transport Label R36
1. RT4: DF Election & Multi-Homed Ethernet DMAC: H1

Segment Auto-Discovery RT-2 MAC Label/EVI100

LACP R39
2. RT1: Per ESI Ethernet Auto-Discovery L2 Frame Flow1
DMAC: H1
(Split-Horizon, Mass-Withdraw) H2
3. RT3: Inclusive Multicast
4. RT2: MAC Advertisement R38 R35
5. RT1: Per EVI Ethernet Auto-Discovery

LACP R37 R34

H1
R36
L2 Frame Flow1
DMAC: H1

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 41
Unicast Forwarding
L2 Frame Flow2
DMAC: H1

L2 Frame Flow1
Transport Label R36
1. RT4: DF Election & Multi-Homed Ethernet DMAC: H1

Segment Auto-Discovery RT-2 MAC Label/EVI100

LACP R39
2. RT1: Per ESI Ethernet Auto-Discovery L2 Frame Flow1
DMAC: H1
(Split-Horizon, Mass-Withdraw) H2
3. RT3: Inclusive Multicast Transport Label R37
4. RT2: MAC Advertisement R38 R35
5. RT1: Per EVI Ethernet Auto-Discovery L2 Frame Flow2
RT1 Label/EVI100

DMAC: H1 L2 Frame Flow2

DMAC: H1

LACP R37 R34

H1
R36
L2 Frame Flow1
DMAC: H1

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 42
Unicast Forwarding
L2 Frame Flow2
DMAC: H1

L2 Frame Flow1
Transport Label R36
1. RT4: DF Election & Multi-Homed Ethernet DMAC: H1

Segment Auto-Discovery RT-2 MAC Label/EVI100

LACP R39
2. RT1: Per ESI Ethernet Auto-Discovery L2 Frame Flow1
DMAC: H1
(Split-Horizon, Mass-Withdraw) H2
3. RT3: Inclusive Multicast Transport Label R37
4. RT2: MAC Advertisement R38 R35
5. RT1: Per EVI Ethernet Auto-Discovery L2 Frame Flow2
RT1 Label/EVI100

DMAC: H1 L2 Frame Flow2

Per Flow Balancing via R36 and R37 - Aliasing DMAC: H1

LACP R37 R34

H1
Per Flow Balancing via R36 and R37 - Aliasing
R36
L2 Frame Flow1
DMAC: H1

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 43
 EVPN – MAC Mobility
Challenge:
How to detect the correct location of MAC after the movement of host from one Ethernet
Segment to another also called “MAC move”?

Sequence number and Next-Hop MAC IP ESI Seq. Next-

value will be changed after the host Hop
SP1 SP
move MAC-1 IP-1 0 1 Leaf-3

MAC IP ESI Seq. Next-

Hop
L1 L2 L3 L4 Sequence number is incremented
MAC-1 IP-1 0 0 Leaf-1 and Next-hop is changed to Leaf-3

C1 C2
VM
Host move

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 44
EVPN L2 & L3 Integration
 Distributed vs Centralized Routing
Layer2 Bridging mandatory between Leaves only Layer2 Bridging mandatory between Leaves and DCI

IRB
L4 X.X.X.H2/24 L4 X.X.X.H2/24

H2 H2
IRB

DCI2 SP2 L3 IRB

DCI2 SP2 L3

CO IRB
CO
IRB
L2 L2
DCI1 SP1 DCI1 SP1
H1 H1

X.X.X.H1/24 X.X.X.H1/24
L1 IRB
L1

• Optimized forwarding of east-west traffic • All east<->west routed traffic traverses to centralized gateways
• ARP/MAC state localized to Leafs • Centralized gateways have full ARP/MAC state in the DCI
• Helps with horizontal scaling of DC • Scale challenge

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
EVPN Distributed
L3 Anycast Gateway
 Symmetric vs Asymmetric - Integrated Routing and
Symmetric Bridging (IRB) Asymmetric
IRB
IRB IRB
L4 Y.Y.Y.H2/24 L4 Y.Y.Y.H2/24

H2 H2
IRB

SP2 L3 SP2 L3 IRB

IRB

CO CO IRB
IRB IRB
L2 L2
SP1 SP1
H1 H1
IRB

X.X.X.H1/24 X.X.X.H1/24
L1 L1 IRB
IRB

• Ingress and Egress Leaf – Routing and Bridging • Ingress Leaf – Routing and Bridging
• ARP/MAC Entries optimization • Egress Leaf – Bridging Only!
• L1/L2 MAC/ARP of Hosts from X.X.X.0/24 only • ARP/MAC Entries optimization
• L3/L4 MAC/ARP of Hosts from Y.Y.Y.0/24 only • L1/L2 MAC/ARP of Hosts from X.X.X.0/24 and Y.Y.Y.0/24
• Horizontally scalable solution • L3/L4 MAC/ARP of Hosts from Y.Y.Y.0/24 and X.X.X.0/24
• Limited Scale
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 48
EVPN – Distributed Symmetric Anycast Gateway
Leaves run Multi-Protocol BGP to advertise & learn MAC + HOST IP addresses over the Network
MAC + IP addresses are advertised to rest of Leaves
L3/4 – Learn MAC + IP HOST address advertised by L1
-> L2/L3 update MAC address table + IP Forwarding table
L2 – uses MAC address advertised by L1 to synchronize MAC address table
-> L2 forwards MAC via local ETH interface represented by same Ethernet Segment between L1 and L2
L2 – uses MAC + IP HOST address advertised by L1 to synchronize ARP/ND information
-> L2 forwards IP via local ETH interface
Identical Anycast Gateway Virtual IP
Distributed Anycast Gateway serves and MAC address are configured on
as the gateway for connected hosts SP1 SP2 all the Leafs

BVI BVI BVI BVI

GW GW GW GW

L1 L2 L3 L4
All the BVIs perform active forwarding
in contrast to active/standby like First-
hop routing protocol
C1 C2
VM VM VM VM

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 49
EVPN – IRB in Network Fabric
Intra-subnet
Forwarding

Inter-subnet
Forwarding
SP1 SP2

BVI BVI BVI BVI

GW GW GW GW

L1 L2 L3 L4

C1 C2 C3 C4
VM VM VM VM

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 50
EVPN Distributed L3 Anycast GW - Symmetric IRB
Anycast IRB 192.168.2.1/24

RR103 RR104
IRB

LACP R39

H2: 192.168.2.20/24 H2 IRB

R38 R35

IRB
LACP R37 R34

H1: 192.168.1.10/24 H1
IRB
R36

Anycast IRB 192.168.1.1/24

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 51
EVPN Configuration - IRB
evpn
no evi 100
no advertise-mac
!

vrf a Not needed! We need MAC/IP RT-2

address-family ipv4 unicast
import route-target
100:100
!
export route-target
100:100
! VRF configuration
!
!

interface BVI100
host-routing MAC/IP RT2
vrf a
ipv4 address 192.168.1.1 255.255.255.0
mac-address 3637.3637.3637 Anycast Distributed IRB: Same IP and MAC
! R36,R37

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 52
EVPN Configuration - BGP VRF
router bgp 1
bgp router-id 3.3.3.36
address-family vpnv4 unicast
!
address-family l2vpn evpn
!
neighbor-group rr
remote-as 1
update-source Loopback0
address-family l2vpn evpn
!
neighbor 3.3.3.103
use neighbor-group rr
!
neighbor 3.3.3.104
use neighbor-group rr
!
vrf a
rd auto
address-family ipv4 unicast
additional-paths receive
maximum-paths ibgp 2 BGP Multi-Path for Inter-subnet forwarding
redistribute connected
!
!

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 53
R36: RT-2 MAC/IP Advertisement
R36#show bgp l2vpn evpn rd 3.3.3.36:100 [2][0][48][0062.ec71.fbd7][32][19$
Tue Oct 16 02:47:45.576 UTC
BGP routing table entry for [2][0][48][0062.ec71.fbd7][32][192.168.1.10]/136, Route Distinguisher: 3.3.3.36:100
Versions:
Process bRIB/RIB SendTblVer
Speaker 84847 84847 RT-2 Advertised MAC IP
Last Modified: Oct 15 23:14:52.399 for 03:32:53
Paths: (2 available, best #1)
Advertised to update-groups (with more than one peer):
0.2
Path #1: Received by speaker 0
Advertised to update-groups (with more than one peer):
0.2
Local
0.0.0.0 from 0.0.0.0 (3.3.3.36)
Second Label 64008
Origin IGP, localpref 100, valid, redistributed, best, group-best, import-candidate, rib-install
Received Path ID 0, Local Path ID 1, version 84838
Extended community: SoO:3.3.3.37:100 RT:1:100 RT:100:100
EVPN ESI: 0036.3700.0000.0000.1100
RT EVI 100 and RT VRF A
Path #2: Received by speaker 0
RT-2 per-BD label
Not advertised to any peer
Local
3.3.3.37 (metric 30) from 3.3.3.103 (3.3.3.37) VRF Agg label
Received Label 64004, Second Label 64008
Origin IGP, localpref 100, valid, internal, import-candidate, imported, rib-install
Received Path ID 0, Local Path ID 0, version 0
Extended community: SoO:3.3.3.37:100 RT:1:100 RT:100:100 RT EVI 100 and RT VRF A
Originator: 3.3.3.37, Cluster list: 3.3.3.103
EVPN ESI: 0036.3700.0000.0000.1100
Source AFI: L2VPN EVPN, Source VRF: default, Source Route Distinguisher: 3.3.3.37:100
RP/0/RP0/CPU0:R36#
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 54
R36: RT-2 MAC/IP
R36#show evpn evi mac
Tue Oct 16 02:52:22.437 UTC

VPN-ID Encap MAC address IP address Nexthop Label

---------- ------ -------------- ---------------------------------------- --------------------------------------- --------
100 MPLS 0062.ec71.fbd7 192.168.1.10 3.3.3.37 64004
65535 N/A 008a.9644.d8d8 :: Local 0

Learned and Advertised RT-2 per-BD label

MAC and IP

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 55
R36: VRF Routes
R36#show route vrf a
Tue Oct 16 02:46:34.463 UTC

Codes: C - connected, S - static, R - RIP, B - BGP, (>) - Diversion path

D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP
i - ISIS, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, su - IS-IS summary null, * - candidate default
U - per-user static route, o - ODR, L - local, G - DAGR, l - LISP
A - access/subscriber, a - Application route
M - mobile route, r - RPL, t - Traffic Engineering, (!) - FRR Backup path

Gateway of last resort is not set

C 192.168.1.0/24 is directly connected, 03:37:59, BVI100

L 192.168.1.1/32 is directly connected, 03:37:59, BVI100
B 192.168.1.10/32 [200/0] via 3.3.3.37 (nexthop in vrf default)
B 192.168.2.20/32 [200/0] via 3.3.3.38 (nexthop in vrf default), 03:28:28
[200/0] via 3.3.3.39 (nexthop in vrf default), 03:28:28

EVPN Learned Route BGP Multi Path to H2 connected to R38 and R39

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 56
R36, R37, R38, R39 - EVPN Startup
R36 - Example Anycast IRB 192.168.2.1/24

1. RT4: DF Election & Multi-Homed Ethernet IRB

Segment Auto-Discovery
LACP R39
2. RT1: Per ESI Ethernet Auto-Discovery
(Split-Horizon, Mass-Withdraw) H2 IRB
3. RT3: Inclusive Multicast RT-2 - MAC Advertisement

4. RT2: MAC/IP Advertisement R38 R35

RD: 1.1.1.36:100

IRB ESI: 0036.3700.0000.0000.1100

LACP R37 R34

MAC: 0062.ec71.fbd7

H1 Label: 64004(BD) + 64008(VRF)

R36 IP: 192.168.1.10

L2 Frame SMAC: Ext-Com: 1:100 (RT) + VRF RT

0062.ec71.fbd7
IP Header SurceIP:
192.168.1.10 Anycast IRB 192.168.1.1/24

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 57
EVPN
Centralized GW
CGW
EVPN Centralized Gateway (CGW)
CGW - Configuration
evpn
virtual access-evi
ethernet-segment
A1 identifier type 0 77.77.77.77.77.77.77.77.77

l2vpn
bridge group test
CE1 bridge-domain test
access-evi 300
routed interface BVI300

A2 CGW1

L2 EVPN L3 VPN
Core

A3 CGW2 Access - Configuration

evpn
evi 300
CE2
advertise-mac

l2vpn
A4 bridge group test
bridge-domain test
interface Bundle-Ether100
!
evi 300

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 59
EVPN Centralized Gateway (CGW)
R28#show evpn ethernet-segment

Ethernet Segment Id Interface Nexthops

------------------------ ---------------------------------- --------------------
0077.7777.7777.7777.7777 Access-EVI:all 1.1.1.26
A1 1.1.1.28

RP/0/RSP0/CPU0:R28#show arp vrf a

CE1 -------------------------------------------------------------------------------
0/0/CPU0
-------------------------------------------------------------------------------
Address Age Hardware Addr State Type Interface
A2 CGW1 192.168.250.1 - a011.1111.1111 Interface ARPA BVI300
192.168.250.10 - 28ac.9ea7.d41b EVPN_SYNC ARPA BVI300
L2 EVPN L3 VPN
Core
CGW in Single-Active mode from Access-to-CGW (South->North)
Based on Access-EVI DF election NDF CGW BVI is added to Core SHG
A3 CGW2
prevents traffic from access-EVI go to BVI
allows traffic from BVI to Access-EVI
CE2
Single-Active South->North
All-Active North->South
A4

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 60
Distributed vs Centralized Gateway
• Distributed Anycast Gateway is our priority!
• Best Scalable solution
• Optimal L2/L3 forwarding

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 61
BGP Layer3
Interconnect
 BGP Layer3 Interconnect
Principles
• DCI/BL provides Layer3 Interconnect
• DCI/BL participates in L3 Routing, but not in Layer2 Bridging
• DCI/BL summarization is required/recommended Layer2 Bridging Required over Leaves

IRB
L4 X.X.X.H2/24

H2

Z.Z.Z.CE1/24 DCI2 SP2 L3 IRB

CE1
PE/DCI
3 CORE CO
IRB
L2
DCI1 SP1
H1

X.X.X.H1/24
L1 IRB

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 63
 BGP Layer3 Interconnect
DCI/BL Summarization
Host-Routes are not required outside CO/DC
L3/4 VRF FIB:
PE/DCI3 VRF FIB: DCI1/2 VRF FIB: X.X.X.H1 -> L1, L2
X.X.X.H1 -> DC1, DCI2 X.X.X.H1 -> L1, L2 X.X.X.H2 -> IRB(local)
X.X.X.H2 -> DCI1, DCI2 X.X.X.H2 -> L3, L4 X.X.X.0/24 -> IRB(local)
X.X.X.0/24 -> DC1, DCI2 X.X.X.0/24 -> L1, L2, L3, L4 Z.Z.Z.0/24 -> DCI1, DCI2
Z.Z.Z.0/24 -> CE1 Z.Z.Z.0/24 -> PE3
IRB
L4 X.X.X.H2/24

H2

Z.Z.Z.CE1/24 DCI2 SP2 L3 IRB L1/2 VRF FIB:

X.X.X.H1 -> IRB(local)
CE1
PE/DCI
3 CORE CO X.X.X.H2 -> L3, L4
X.X.X.0/24 -> IRB(local)
IRB Z.Z.Z.0/24 -> DCI1, DCI2
L2
DCI1 SP1
H1

X.X.X.H1/24
L1 IRB

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 64
BGP Layer3 Interconnect
Control Plane
BGP - L3VPN VPNv4/6 BGP - L3VPN VPNv4/6 Option #1 – VPNv4/6 & VPNv4/6
BGP – EVPN L3 BGP – EVPN L3
Option #2 – EVPN & EVPN
BGP - L3VPN VPNv4/6 BGP – EVPN L3
Option #3 – VPNv4/6 & EVPN

IRB
L4 X.X.X.H2/24

H2

Z.Z.Z.CE1/24 DCI2 SP2 L3 IRB

CE1
PE/DCI
3 CORE CO
IRB
L2
DCI1 SP1
H1

X.X.X.H1/24
L1 IRB

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 65
BGP Layer3 Interconnect
Option #1 – VPNv4/6 & VPNv4/6
BGP - L3VPN VPNv4/6 BGP - L3VPN VPNv4/6
VPNv4: Z.Z.Z.0/24 VPNv4: Z.Z.Z.0/24

VPNv4: X.X.X.0/24 VPNv4: X.X.X.0/24

VPNv4: X.X.X.H1, X.X.X.H2

X
IRB
L4 X.X.X.H2/24

H2

Z.Z.Z.CE1/24 DCI2 SP2 L3 IRB

CE1
PE/DCI
3 CORE CO
IRB
L2
DCI1 SP1
H1

X.X.X.H1/24
L1 IRB
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 66
BGP Layer3 Interconnect
Option #2 – EVPN & EVPN
BGP – EVPN L3 BGP – EVPN L3
RT5: Z.Z.Z.0/24 RT5 Prefix: Z.Z.Z.0/24

RT5: X.X.X.0/24 RT5 Prefix: X.X.X.0/24

RT2 MAC/IP: X.X.X.H1, X.X.X.H2

X
IRB
L4 X.X.X.H2/24

H2

Z.Z.Z.CE1/24 DCI2 SP2 L3 IRB

CE1
PE/DCI
3 CORE CO
IRB
L2
DCI1 SP1
H1

X.X.X.H1/24
L1 IRB
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 67
BGP Layer3 Interconnect
Option #3 – VPNv4/6 & EVPN
BGP - L3VPN VPNv4/6 BGP – EVPN L3
VPNv4: Z.Z.Z.0/24 RT5 Prefix: Z.Z.Z.0/24

VPNv4: X.X.X.0/24 RT5 Prefix: X.X.X.0/24

RT2 MAC/IP: X.X.X.H1, X.X.X.H2

X
IRB
L4 X.X.X.H2/24

H2

Z.Z.Z.CE1/24 DCI2 SP2 L3 IRB

CE1
PE/DCI
3 CORE CO
IRB
L2
DCI1 SP1
H1

X.X.X.H1/24
L1 IRB
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 68
BGP Layer3 Interconnect
Control Plane Options Highlight
• Option #1 – VPNv4/6 & VPNv4/6
+ VPNv4/6 Industry proved solution for Layer3 VPN
+ DCI doesn’t need to understand BGP EVPN AF
- Leaf has to peer with Route-Reflector via both BGP EVPN and VPNv4/6 AF
EVPN AF to support L2 stretch (MAC advertisement) across DC/CO between Leaves
EVPN AF to sync ARP/ND for Multi-Homed All-Active
- DC/CO Route-Reflector has to support both BGP EVPN and VPNv4/6 AF
- Leaf has to advertise VM Host-Routes via VPNv4/6

• Option #2 – EVPN & EVPN

+ Single BGP Address Family End-To-End in Network
- Existing L3 VPNv4/6 services has to to migrated to L3 EVPN
No technical benefit to migrate existing L3 VPNv4/6 to L3 EVPN

• Option #3 – VPNv4/6 & EVPN

+ Recommended solution which benefits from both Options #1 and #2
+ New DC/CO - Leaf, Route-Reflector use single BGP AF EVPN
+ Existing L3 VPNv4/6 services stay untouched

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 69
R36: BGP Configuration - RT-5
router bgp 1
bgp router-id 3.3.3.36
address-family vpnv4 unicast
!
address-family l2vpn evpn
!
neighbor-group rr
remote-as 1
update-source Loopback0
address-family l2vpn evpn
advertise vpnv4 unicast
!
vrf a
rd auto
RT-5
address-family ipv4 unicast
additional-paths receive
maximum-paths ibgp 2
!

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 70
R36: RT-5 Route
R36#show bgp l2vpn evpn rd 3.3.3.37:0 [5][0][24][192.168.1.0]/80
Tue Oct 16 03:35:06.480 UTC
BGP routing table entry for [5][0][24][192.168.1.0]/80, Route Distinguisher: 3.3.3.37:0
Versions:
Process bRIB/RIB SendTblVer
Speaker 84912 84912
Last Modified: Oct 16 03:23:18.399 for 00:11:48
Paths: (2 available, best #1) RT-5 VRF A R37 RD
Not advertised to any peer prefix
Path #1: Received by speaker 0
Not advertised to any peer
Local
3.3.3.37 (metric 30) from 3.3.3.103 (3.3.3.37) VRF Agg label
Received Label 64008
Origin incomplete, metric 0, localpref 100, valid, internal, best, group-best, import-candidate, not-in-vrf
Received Path ID 0, Local Path ID 1, version 84912
Extended community: Flags 0x6: RT:100:100
Originator: 3.3.3.37, Cluster list: 3.3.3.103
EVPN ESI: 0000.0000.0000.0000.0000, Gateway Address : 0.0.0.0
Path #2: Received by speaker 0
VRF A Route-Target
Not advertised to any peer
Local
3.3.3.37 (metric 30) from 3.3.3.104 (3.3.3.37) VRF Agg label
Received Label 64008
Origin incomplete, metric 0, localpref 100, valid, internal, not-in-vrf
Received Path ID 0, Local Path ID 0, version 0
Extended community: Flags 0x6: RT:100:100
Originator: 3.3.3.37, Cluster list: 3.3.3.104 VRF A Route-Target
EVPN ESI: 0000.0000.0000.0000.0000, Gateway Address : 0.0.0.0
RP/0/RP0/CPU0:R36#

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 71
R36: VRF A - Routing Table
R36#show route vrf a

C 192.168.1.0/24 is directly connected, 04:55:09, BVI100

L 192.168.1.1/32 is directly connected, 04:55:09, BVI100
B 192.168.1.10/32 [200/0] via 3.3.3.37 (nexthop in vrf default)
B 192.168.2.0/24 [200/0] via 3.3.3.38 (nexthop in vrf default), 00:40:26
[200/0] via 3.3.3.39 (nexthop in vrf default), 00:40:26
B 192.168.2.20/32 [200/0] via 3.3.3.38 (nexthop in vrf default), 00:40:26
[200/0] via 3.3.3.39 (nexthop in vrf default), 00:40:26
RP/0/RP0/CPU0:R36

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 72
EVPN Routes - Summary
EVPN Routes – Cheat Sheet (Unicast)
BGP Signaling PE1 – Advertises:
PE2 PE4
PE1 RT-4 Ethernet Segment Route
CE1 MPLS CE2 • I have ESI1 in case when someone needs this information for
EVI1-L Designated Forwarder(DF) Election
PE1 PE3
BE1-SHL
Data Plane
EVI1-BUML RT-1 Per ESI Ethernet Auto-Discovery (AD) Route
L2 Frame Transport L2 Frame
MPLS Label • I have ESI1
Service
• ESI1 is All-Active
BGP Label
L2 Frame
BD1 EVI1 • AC with ESI1 is connected to EVI1 and EVI2
BD1 MAC • My Split Horizon Label for ESI1 is BE1-SHL
MAC-A

BVI1
MAC-A -> BE1.1
IP-A VRF1 ARP RT-1 Per EVI Ethernet Auto-Discovery (AD) Route(s)
BE1 - ESI1 • EVI1 per-EVI (Aliasing) Label is EVI1-L
Vlan1 IP-A MAC-A -> BVI1
VRF1 IP-B MAC-B -> BVI2 • EVI2 per-EVI (Aliasing) Label is EVI2-L
Vlan2 VRF1-AGGL

BVI2 RT-3 Inclusive Multicast Route(s)

MAC-B BD2 MAC • EVI1 Label for BUM traffic is EVI1-BUML
IP-B MAC-B -> BE1.2 • EVI2 Label for BUM traffic is EVI2-BUML
BD2 EVI2
RT-2 MAC/IP Advertisement Route(s)
• MAC-A in EVI1 via label EVI1-L and IP-A in VRF1 via label VRF1-AGGL
EVI2-L
• MAC-B in EVI2 via label EVI2-L and IP-B in VRF1 via label VRF1-AGGL
BE1-SHL
EVI2-BUML RT-5 Prefix Advertisement Route(s)
• IPv4/6 prefix of BVI1 in VRF1 via label VRF1-AGGL

• IPv4/6 prefix of BVI2 in VRF1 via label VRF1-AGGL

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 74
EVPN Single-Active
 EVPN - Load-Balancing Modes

All-Active Single-Active Port-Active

(per flow) (per VLAN) (per port)

PE1 PE2 PE1 PE2 PE1 PE2

V1 V1 V1 V2 V1, V2

CE1 CE2 CE3

Single LAG at the CE Multiple LAGs at the CE Single LAGs at the CE

VLAN goes to both PE VLAN active on single PE Port active on single PE
Traffic hashed per flow Traffic hashed per VLAN Traffic hashed per port
Benefits: Bandwidth, Convergence Benefits: Billing, Policing Benefits: Protocol Simplification

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 76
EVPN - Testbed

RR103 RR104

Single-Active
R39

H2
R38 R35

LACP R37 R34

H1
R36

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 77
All-Active - Example
R36#show evpn internal-label

VPN-ID Encap Ethernet Segment Id EtherTag Label

---------- ------ --------------------------- -------- --------
100 MPLS 0038.3900.0000.0000.1100 0 68103
Summary pathlist:
0x02000001 3.3.3.38 68096
0x02000002 3.3.3.39 68096

R36#show mpls forwarding labels 68103 detail

Local Outgoing Prefix Outgoing Next Hop Bytes

Label Label or ID Interface Switched
------ ----------- ------------------ ------------ --------------- ------------
68103 68096 EVPN:100 3.3.3.38 0
Updated: Jan 27 07:50:05.582
Version: 42, Priority: 3
Label Stack (Top -> Bottom): { 68096 }
NHID: 0x0, Encap-ID: 0x1386f00000002, Path idx: 0, Backup path idx: 0, Weight: 0
MAC/Encaps: 0/4, MTU: 0
Packets Switched: 0

68096 EVPN:100 3.3.3.39 0

Updated: Jan 27 07:50:05.582
Version: 42, Priority: 3
Label Stack (Top -> Bottom): { 68096 }
NHID: 0x0, Encap-ID: 0x1387100000002, Path idx: 1, Backup path idx: 0, Weight: 0
MAC/Encaps: 0/4, MTU: 0
Packets Switched: 0

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 78
Single-Active – Configuration and Verification
Remote R38/R39
R36#show evpn internal-label

VPN-ID Encap Ethernet Segment Id EtherTag Label

---------- ------ --------------------------- -------- -------- evpn
100 MPLS 0038.3900.0000.0000.1100 0 68103 interface Bundle-Ether100
Summary pathlist: ethernet-segment
0x02000001 3.3.3.38 68096 load-balancing-mode single-active
0x00000000 3.3.3.39 (B) 68096 !
!
R36#show mpls forwarding labels 68103 detail
Sun Jan 27 07:52:03.877 UTC
Local Outgoing Prefix Outgoing Next Hop Bytes
Label Label or ID Interface Switched
------ ----------- ------------------ ------------ --------------- ------------
68103 68096 EVPN:100 3.3.3.38 0
Updated: Jan 27 07:51:14.370
Path Flags: 0x400 [ BKUP-IDX:1 (0x0) ]
Version: 47, Priority: 3
Label Stack (Top -> Bottom): { 68096 }
NHID: 0x0, Encap-ID: 0x1386f00000002, Path idx: 0, Backup path idx: 1, Weight: 0
MAC/Encaps: 0/4, MTU: 0
Packets Switched: 0

68096 EVPN:100 3.3.3.39 0 (!)

Updated: Jan 27 07:51:14.370
Path Flags: 0x300 [ IDX:1 BKUP, NoFwd ]
Version: 47, Priority: 3
Label Stack (Top -> Bottom): { 68096 }
NHID: 0x0, Encap-ID: 0x1387100000002, Path idx: 1, Backup path idx: 0, Weight: 0
MAC/Encaps: 0/4, MTU: 0
Packets Switched: 0
(!): FRR pure backup
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 79
Single-Active ethernet-segment carving detail
R38#show evpn ethernet-segment esi 0038.3900.0000.0000.1100 carving detail

Ethernet Segment Id Interface Nexthops

------------------------ ---------------------------------- --------------------
0038.3900.0000.0000.1100 BE100 3.3.3.38
3.3.3.39
ES to BGP Gates : Ready
ES to L2FIB Gates : Ready
Main port :
Interface name : Bundle-Ether100
Interface MAC : 008a.967f.30dd
IfHandle : 0x0800002c
State : Up
Redundancy : Not Defined
ESI type : 0
Value : 38.3900.0000.0000.1100
ES Import RT : 3839.0000.0000 (from ESI)
Source MAC : 0000.0000.0000 (N/A)
Topology :
Operational : MH, Single-active
Configured : Single-active (AApS)
Service Carving : Auto-selection
Peering Details : 3.3.3.38[MOD:P:00] 3.3.3.39[MOD:P:00]
Service Carving Results:
Forwarders : 1
Permanent : 0
Elected : 1
EVI E : 100
Not Elected : 0
MAC Flushing mode : STP-TCN
Peering timer : 3 sec [not running]
Recovery timer : 30 sec [not running]
Carving timer : 0 sec [not running]
Local SHG label : 68098
Remote SHG labels : 1
68098 : nexthop 3.3.3.39
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 80
EVPN Port-Active
 EVPN - Load-Balancing Modes

All-Active Single-Active Port-Active

(per flow) (per VLAN) (per port)

PE1 PE2 PE1 PE2 PE1 PE2

V1 V1 V1 V2 V1, V2

CE1 CE2 CE3

Single LAG at the CE Multiple LAGs at the CE Single LAGs at the CE

VLAN goes to both PE VLAN active on single PE Port active on single PE
Traffic hashed per flow Traffic hashed per VLAN Traffic hashed per port
Benefits: Bandwidth, Convergence Benefits: Billing, Policing Benefits: Protocol Simplification

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 82
EVPN - Testbed

RR103 RR104

LACP R39

H2
R38 R35

R36/R37
LACP R37 R34
evpn
interface Bundle-Ether100
ethernet-segment H1
load-balancing-mode port-active
!
!
R36

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 83
 Port-Active –Verification
R36#show bundle R37#show bundle
Bundle-Ether100 Bundle-Ether100
Status: Up Status: LACP OOS (out of service)
Local links <active/standby/configured>: 1 / 0 / 1 Local links <active/standby/configured>: 0 / 1 / 1
Local bandwidth <effective/available>: 10000000 (10000000) kbps Local bandwidth <effective/available>: 0 (0) kbps
MAC address (source): 008a.9644.d8de (Chassis pool) MAC address (source): 008a.9644.08de (Chassis pool)
Inter-chassis link: No Inter-chassis link: No
Minimum active links / bandwidth: 1 / 1 kbps Minimum active links / bandwidth: 1 / 1 kbps
Maximum active links: 64 Maximum active links: 64
Wait while timer: 2000 ms Wait while timer: 2000 ms
Load balancing: Load balancing:
Link order signaling: Not configured Link order signaling: Not configured
Hash type: Default Hash type: Default
Locality threshold: None Locality threshold: None
LACP: Operational LACP: Operational
Flap suppression timer: Off Flap suppression timer: Off
Cisco extensions: Disabled Cisco extensions: Disabled
Non-revertive: Disabled Non-revertive: Disabled
mLACP: Not configured mLACP: Not configured
IPv4 BFD: Not configured IPv4 BFD: Not configured
IPv6 BFD: Not configured IPv6 BFD: Not configured

Port Device State Port ID B/W, kbps

Port Device State Port ID B/W, kbps -------------------- --------------- ----------- -------------- ----------
-------------------- --------------- ----------- -------------- ---------- Te0/0/0/0 Local Standby 0x8000, 0x0001 10000000
Te0/0/0/0 Local Active 0x8000, 0x0001 10000000 Link is in standby due to bundle out of service state
Link is Active

R37#show int bundle-ether 100

Bundle-Ether100 is down, line protocol is down

R37#show int tenGigE 0/0/0/0

TenGigE0/0/0/0 is up, line protocol is up

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 84
EVPN Single-Flow-Active
(SFA)
EVPN Load-Balancing Modes
Single-Flow-Active (SFA)

X
Single-Homed Single-Flow-Active (SFA)
STP/REP/G.8032 “break” L2 loop MST-AG/REP-AG/G.8032 “break” L2 loop

A3 A3
PE1 PE1

STP/REP/ MPLS MST/REP/ MPLS

A1 G.8032…. Core A1 G.8032…. Core

PE2 PE2
A2 A2
EVPN-MPLS EVPN-MPLS

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 86
EVPN Single-Flow-Active (SFA) - Configuration
PE1/PE2
evpn
interface Bundle-Ether100
ethernet-segment
identifier type 0 36.37.36.37.36.37.36.37.01
load-balancing-mode single-flow-active
convergence
mac-mobility

A3 PE37
P2
A2
MST/REP/ MPLS PE38
G8032
P1
A1 PE36

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 87
RT-1 Per ESI Ethernet Auto-Discovery
Single-Flow-Active (SFA)

R36#show bgp l2vpn evpn rd 3.3.3.36:0 [1][3.3.3.36:1][0036.3700.0000.0000.1100][4294967295]/184

Sun Oct 14 20:56:59.687 UTC
BGP routing table entry for [1][3.3.3.36:1][0036.3700.0000.0000.1100][4294967295]/184, Route Distinguisher: 3.3.3.36:0
Versions:
Process bRIB/RIB SendTblVer RD - unique per advertising Ethernet Segment Identifier (ESI)
RT-1
Speaker 76372 76372 node (R36 unique)
Local Label: 0
Last Modified: Sep 18 23:02:40.399 for 3w4d
Paths: (1 available, best #1)
Advertised to update-groups (with more than one peer):
0.2
Path #1: Received by speaker 0
Advertised to update-groups (with more than one peer):
0.2
Local
0.0.0.0 from 0.0.0.0 (3.3.3.36)
Origin IGP, localpref 100, valid, redistributed, best, group-best, import-candidate, rib-install
Received Path ID 0, Local Path ID 1, version 76372
Extended community: EVPN ESI Label:0x02:64005 RT:1:100 EVI(s) Route-Target
All EVI(s) which use this ESI
Redundancy mode Split-Horizon Label
All-Active: 0x00
Single-Active: 0x01
Single-Flow-Active: 0x02 NEW! draft-brissette-bess-evpn-l2gw-proto 88
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public
EVPN Single-Flow-Active (SFA)
• PE36/PE37 are both DF (L2 legacy protocol must break a loop)
• PE36 advertise A2 MAC+IP EVPN RT2 with BGP Local-Preference 100
• PE37 synchronize A2 ARP/ND (EVPN RT2 MAC+IP advertised by PE36)
• FIB Next-Hop -> PE36

37#show arp vrf a

192.168.100.100 - a0aa.cccc.cccc EVPN_SYNC ARPA BVI100

37#show cef vrf a 192.168.100.100

A3 PE37
Prefix Len 32, traffic index 0, precedence n/a, priority 3
via 3.3.3.36/32, 5 dependencies, recursive [flags 0x6000] P2
path-idx 0 NHID 0x0 [0x89dc1908 0x0] MST/REP/ PE38
A2
recursion-via-/32 G8032
next hop VRF - 'default', table - 0xe0000000
next hop 3.3.3.36/32 via 16036/0/21 P1
next hop 35.37.1.35/32 Te0/0/0/39 labels imposed {16036 28103} A1 PE36
next hop 34.37.1.34/32 Te0/0/0/38 labels imposed {16036 28103}

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 89
EVPN Single-Flow-Active (SFA)
• PE36/PE37 are both DF (L2 legacy protocol must break a loop)
• PE36 advertise A2 MAC+IP EVPN RT2 with BGP Local-Preference 100
• PE37 synchronize A2 ARP/ND (EVPN RT2 MAC+IP advertised by PE36)
• FIB Next-Hop -> PE36
• PE37 Re-advertise A2 MAC+IP RT2 with BGP Local-Preference 80
• PE38 prefers A2 via PE36 (BGP LP 100)

EVPN Re-originated RT2 A2-MAC+IP LocalPref 80

A3 PE37
P2
MST/REP/ PE38
A2
G8032
P1
A1 PE36

EVPN RT2 A2-MAC+IP LocalPref 100

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 90
EVPN-VPWS
Multihomed Service
EVPN vs EVPN-VPWS - Balancing Mode
Single-Active
• Both EVPN and EVPN-VPWS advertise RT1(per-ESI) PE2 PE4
• Signal All-Active or Single-Active CE1 MPLS CE2
PE1 PE3

• Remote node performs per-flow load-balancing -> All-Active mode

• How remote node knows who is Active in Single-Active mode?

• EVPN
• Remote node follows MAC (RT2) advertisement -> node advertising MAC is active
• EVPN-VPWS
• Additional signaling per-service is required to inform remote node who is Active

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 92
EVPN-VPWS Layer 2 Attributes Extended Community
RFC8214 IOS-XR 7.1.1
+-------------------------------------------+

| Type (0x06) / Sub-type (0x04) (2 octets) | 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5

+-------------------------------------------+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

| Control Flags (2 octets) | | MBZ |C|P|B| (MBZ = MUST Be Zero)

+-------------------------------------------+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

| L2 MTU (2 octets) |

+-------------------------------------------+ Control-Word(C) = 4
| Reserved (2 octets) | Primary(P) = 2
Backup(B) = 1
+-------------------------------------------+

L2 MTU is a 2-octet value indicating the MTU in bytes

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 93
EVPN-VPWS
All-Active
EVPN-VPWS - Testbed
Startup Sequence is almost identical with EVPN except:
RT3 and RT2 are not required
RR103 RR104

LACP R39

H2
R38 R35

R37 R34

H1 R36

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 95
 Config: EVPN-VPWS
R36 R38/R39
l2vpn l2vpn
xconnect group 500 xconnect group 500
p2p 500 p2p 500
interface Bundle-Ether100 interface Bundle-Ether100
neighbor evpn evi 500 target 333 source 333 neighbor evpn evi 500 target 333 source 333
! !
! !
! !

From IOS-XR 7.1.1 Simplified configuration option is available

if ”target id” and “source id” has same value => “service id” can be used

R36 R38/R39
l2vpn l2vpn
xconnect group 500 xconnect group 500
p2p 500 p2p 500
interface Bundle-Ether100 interface Bundle-Ether100
neighbor evpn evi 500 service 333 neighbor evpn evi 500 service 333
! !
! !
! !

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 96
R36: L2vpn xconnect status & Data Plane
verification
R36#show l2vpn xconnect
Legend: ST = State, UP = Up, DN = Down, AD = Admin Down, UR = Unresolved,
SB = Standby, SR = Standby Ready, (PP) = Partially Programmed

XConnect Segment 1 Segment 2

Group Name ST Description ST Description ST
------------------------ ----------------------------- -----------------------------
500 500 UP BE100 UP EVPN 500,3839,68106 UP
----------------------------------------------------------------------------------------

R36#show mpls forwarding labels 68106

Local Outgoing Prefix Outgoing Next Hop Bytes
Label Label or ID Interface Switched
------ ----------- ------------------ ------------ --------------- ------------
68106 68107 EVPN:500 3.3.3.38 0
68107 EVPN:500 3.3.3.39 0

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 97
R36: RT-1 Per EVI Ethernet Auto-Discovery
R36#show bgp l2vpn evpn rd 3.3.3.36:500 [1][0038.3900.0000.0000.1100][3839]/120
Control-Word(C) = 4
BGP routing table entry for [1][0038.3900.0000.0000.1100][3839]/120, Route Distinguisher: 3.3.3.36:500 Primary(P) = 2
Versions:
Process bRIB/RIB SendTblVer RT-1
Backup(B) = 1
Speaker 316 316 ESI R38/R39 AC-ID
Last Modified: Jan 27 08:24:37.527 for 00:01:42
Paths: (2 available, best #1)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer Control-Word + Primary
Local
3.3.3.38 (metric 30) from 3.3.3.103 (3.3.3.38) MTU 1500B
Received Label 68107
Origin IGP, localpref 100, valid, internal, best, group-best, import-candidate, imported, rib-install
Received Path ID 0, Local Path ID 1, version 314
Extended community: EVPN L2 ATTRS:0x06:1500 RT:1:500
Originator: 3.3.3.38, Cluster list: 3.3.3.103
Source AFI: L2VPN EVPN, Source VRF: default, Source Route Distinguisher: 3.3.3.38:500
Path #2: Received by speaker 0
Not advertised to any peer Control-Word + Primary
Local
3.3.3.39 (metric 30) from 3.3.3.103 (3.3.3.39)
MTU 1500B
Received Label 68107
Origin IGP, localpref 100, valid, internal, import-candidate, imported, rib-install
Received Path ID 0, Local Path ID 0, version 0
Extended community: EVPN L2 ATTRS:0x06:1500 RT:1:500
Originator: 3.3.3.39, Cluster list: 3.3.3.103
Source AFI: L2VPN EVPN, Source VRF: default, Source Route Distinguisher: 3.3.3.39:500

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 98
R36: EVPN-VPWS Instance View
R36#show evpn evi vpn-id 500 detail

VPN-ID Encap Bridge Domain Type

---------- ------ ---------------------------- -------------------
500 MPLS VPWS:500 VPWS (vlan-unaware)
Stitching: Regular
Unicast Label : 0
Multicast Label: 0
Flow Label: N EVPN-VPWS
Control-Word: Enabled
Forward-class: 0
• No RT2 – MAC
Advertise MACs: No • No RT3 - BUM
Advertise BVI MACs: No
Aliasing: Enabled
UUF: Enabled
Re-origination: Enabled
Multicast source connected: No

Statistics:
Packets Sent Received
Total : 0 0
Unicast : 0 0
BUM : 0 0
Bytes Sent Received
Total : 0 0
Unicast : 0 0
BUM : 0 0
RD Config: none
RD Auto : (auto) 3.3.3.36:500
RT Auto : 1:500
Route Targets in Use Type
------------------------------ ---------------------
1:500 Import
1:500 Export

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 99
EVPN-VPWS
Single-Active
EVPN-VPWS - Testbed
Startup Sequence is almost identical with EVPN except:
RT3 and RT2 are not required
RR103 RR104

Single-Active
R39

H2
R38 R35

LACP R37 R34

H1
R36

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 101
Config: EVPN-VPWS
R36 R38/R39
l2vpn l2vpn
xconnect group 500 xconnect group 500
p2p 500 p2p 500
interface Bundle-Ether100 interface Bundle-Ether100
neighbor evpn evi 500 target 3839 source 3637 neighbor evpn evi 500 target 3637 source 3839
! !
! !
! !

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 102
R36: L2vpn xconnect status & Data Plane
verification
R36#show l2vpn xconnect

XConnect Segment 1 Segment 2

Group Name ST Description ST Description ST
------------------------ ----------------------------- -----------------------------
500 500 UP BE100 UP EVPN 500,3839,24004 UP
----------------------------------------------------------------------------------------

R36#show mpls forwarding labels 24004

Local Outgoing Prefix Outgoing Next Hop Bytes

Label Label or ID Interface Switched
------ ----------- ------------------ ------------ --------------- ------------ Active
24004 28127 EVPN:500 3.3.3.39 0
28127 EVPN:500 3.3.3.38 0 (!)
Standby

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 103
R36: RT-1 Per EVI Ethernet Auto-Discovery
R36#show bgp l2vpn evpn rd 3.3.3.36:500 [1][0038.3900.0000.0000.1100][3839]/120
Tue Apr 14 07:47:20.033 UTC Control-Word(C) = 4
BGP routing table entry for [1][0038.3900.0000.0000.1100][3839]/120, Route Distinguisher: 3.3.3.36:500 Primary(P) = 2
Versions:
Process bRIB/RIB SendTblVer RT-1
Backup(B) = 1
Speaker 430 430 ESI R38/R39 AC-ID
Last Modified: Apr 14 07:47:09.651 for 00:00:10
Paths: (2 available, best #1)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer Control-Word + Backup
Local
3.3.3.38 (metric 30) from 3.3.3.103 (3.3.3.38) MTU 1500B
Received Label 28127
Origin IGP, localpref 100, valid, internal, best, group-best, import-candidate, imported, rib-install
Received Path ID 0, Local Path ID 1, version 428
Extended community: EVPN L2 ATTRS:0x05:1500 RT:1:500
Originator: 3.3.3.38, Cluster list: 3.3.3.103
Source AFI: L2VPN EVPN, Source VRF: default, Source Route Distinguisher: 3.3.3.38:500
Path #2: Received by speaker 0
Not advertised to any peer Control-Word + Primary
Local
3.3.3.39 (metric 30) from 3.3.3.103 (3.3.3.39)
MTU 1500B
Received Label 28127
Origin IGP, localpref 100, valid, internal, import-candidate, imported, rib-install
Received Path ID 0, Local Path ID 0, version 0
Extended community: EVPN L2 ATTRS:0x06:1500 RT:1:500
Originator: 3.3.3.39, Cluster list: 3.3.3.103
Source AFI: L2VPN EVPN, Source VRF: default, Source Route Distinguisher: 3.3.3.39:500

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 104
EVPN Interconnect/Migration
(L2 Services)
EVPN L2 Interconnect – Let’s connect everything together
Everything in one Bridge Domain
• Legacy L2: REP, G8032, STP, etc.
LACP
• VPLS VPWS
• EVPN-VXLAN/EVPN-MPLS CE A3
• EoMPLS(PW)
• Ethernet – MultiHomed, SingleHomed

Leaf
VM
A3

Spine Spine
DCI/PE DCI/PE
PE1

Leaf
VM
STP/REP/ MPLS MPLS Core
A1 G.8032…. Core

Leaf
VM DCI/PE DCI/PE
PE2
A2 EVPN - VXLAN
EVPN-MPLS
EVPN - MPLS

A1 A2
VPLS
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 106
EVPN & VPLS
Seamless Integration - Migration
 VPLS & EVPN Seamless Integration - Migration
VFI1 is by default in Split Horizon Group 1
CE2 R37 R39 CE4
• SHG1 protects loops in MPLS Core
• Full Mesh of pseudowires(PW) is required
for Any-to-Any forwarding
R36 MPLS
VFI1
PW_R37 UP
BD1 PW_R38 UP
PW_R39 UP
CE1
R38 CE3

l2vpn
bridge group 100
bridge-domain 100
vfi 1
neighbor x.x.x.37 pw-id 37
!
neighbor x.x.x.38 pw-id 38
!
neighbor x.x.x.39 pw-id 39
!
!

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 108
 VPLS & EVPN Seamless Integration - Migration
VFI1 is by default in Split Horizon Group 1
CE2 R37 R39 CE4
• SHG1 protects loops in MPLS Core
• Full Mesh of pseudowires(PW) is required
for Any-to-Any forwarding
R36 MPLS
VFI1
PW_R37 UP
EVI100 is also by default in Split Horizon Group 1
BD1 PW_R38 UP • R36 doesn’t forward data between VFI1 and EVI100
CE1 X PW_R39 UP

EVI100 R38 CE3

l2vpn
bridge group 100
bridge-domain 100
vfi 1
neighbor x.x.x.37 pw-id 37
!
neighbor x.x.x.38 pw-id 38
!
neighbor x.x.x.39 pw-id 39
!
evi 100
!

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 109
 VPLS & EVPN Seamless Integration - Migration
VFI1 is by default in Split Horizon Group 1
CE2 R37 R39 CE4
• SHG1 protects loops in MPLS Core
• Full Mesh of pseudowires(PW) is required
for Any-to-Any forwarding
R36 MPLS
VFI1
PW_R37 UP
EVI1 is also by default in Split Horizon Group 1
BD1 PW_R38 DOWN • R36 doesn’t forward data between VFI1 and EVI100
CE1 X PW_R39 UP
BGP EVPN
EVI100 R38 CE3 R36&R38 run BGP EVPN
• PW_R38 goes DOWN
• Data Forwarding between R36 and R38 via EVI100
l2vpn
bridge group 100
bridge-domain 100
vfi 1
neighbor x.x.x.37 pw-id 37
!
neighbor x.x.x.38 pw-id 38
!
neighbor x.x.x.39 pw-id 39
!
evi 100
!

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 110
PW & EVPN-VPWS
Seamless Migration
 EVPN-VPWS/Legacy-PW Seamless Migration
Supported Modes
CE1 PE38 MPLS PE39 CE2 Discovery: Static/BGP-AD
Signaling: LDP, BGP
LDP based PW
R38 Configuration
l2vpn
xconnect group test
p2p test
interface TenGigE0/0/0/0
neighbor ipv4 3.3.3.39 pw-id 10

R38#show l2vpn xconnect

XConnect Segment 1 Segment 2
Group Name ST Description ST Description ST
------------------------ ----------------------------- -----------------------------
test test UP Te0/0/0/0 UP 3.3.3.39 10 UP
----------------------------------------------------------------------------------------

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 112
EVPN-VPWS/Legacy-PW Seamless Migration
CE1 PE38 MPLS PE39 CE2

LDP based PW
R38 Configuration
l2vpn
xconnect group test Allows Tengig0/0/0/0 to be migrated
p2p test
vpws-seamless-integration
interface TenGigE0/0/0/0
neighbor ipv4 3.3.3.39 pw-id 10 Existing LDP based PW is UP and forwarding data
New EVPN-VPWS service is ready and is signaled via BGP EVPN AF
p2p test-new
interface TenGigE0/0/0/0
neighbor evpn evi 1000 service 10

R38#show l2vpn xconnect

XConnect Segment 1 Segment 2

Group Name ST Description ST Description ST
------------------------ ----------------------------- -----------------------------
test test UP Te0/0/0/0 UP 3.3.3.39 10 UP
----------------------------------------------------------------------------------------
test test-new DN Te0/0/0/0 UP EVPN 1000,10,None DN
----------------------------------------------------------------------------------------

R38#show bgp l2vpn evpn rd 3.3.3.38:1000

Route Distinguisher: 3.3.3.38:1000 (default for vrf VPWS:1000)
*> [1][0000.0000.0000.0000.0000][10]/120
0.0.0.0 0 i
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 113
 EVPN-VPWS/Legacy-PW Seamless Migration
CE1 PE38 MPLS PE39 CE2

LDP based PW - DOWN

EVPN-VPWS - UP
R38 Configuration R39 Configuration
EVPN-VPWS is UP
l2vpn
xconnect group test
l2vpn LDP PW is Down and service is in “Seamless Inactive” mode
p2p test
xconnect group test p2p test can be removed
p2p test
vpws-seamless-integration
vpws-seamless-integration
interface TenGigE0/0/0/0
interface TenGigE0/0/0/0
neighbor ipv4 3.3.3.39 pw-id 10
neighbor ipv4 3.3.3.38 pw-id 10
p2p test-new
p2p test-new
interface TenGigE0/0/0/0
interface TenGigE0/0/0/0
neighbor evpn evi 1000 service 10
neighbor evpn evi 1000 service 10

R38#show l2vpn xconnect

XConnect Segment 1 Segment 2
Group Name ST Description ST Description ST
------------------------ ----------------------------- -----------------------------
test test DN Te0/0/0/0 SB(SI) 3.3.3.39 10 UP
----------------------------------------------------------------------------------------
test test-new UP Te0/0/0/0 UP EVPN 1000,10,3.3.3.39 UP
----------------------------------------------------------------------------------------

R38#show bgp l2vpn evpn rd 3.3.3.38:1000

Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 3.3.3.38:1000 (default for vrf VPWS:1000)
*> [1][0000.0000.0000.0000.0000][10]/120
0.0.0.0 0 i
* i 3.3.3.39 100 0 i
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 114
EVPN-VPWS/Legacy-PW Seamless Migration
Usecases

New Node (PE40) insertion/replacement

LDP PW #1 PE38 EVPN-VPWS Seamless Migration configuration

CE1 PE38 PE39 CE2
#2 PE40 EVPN-VPWS Configuration
PE40 -> CE2 AC is down (not-connected/down)
PE38 <-> PE39 LDP PW is UP
PE40
#3 CE2 -> PE39 link remove and connect to PE40
PE38 <-> PE39 PW DOWN
PE40 -> Signal EVPN-VPWS
# PE38 <-> PE40 EVPN-VPWS UP
CE1 PE38 LDP PW PE39 CE2

PE40

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 115
EVPN-VPWS/Legacy-PW Seamless Migration
Usecases

Active/Backup PW – Multi-Homed CE
• CE Ethernet Bundle to PE39/40 with maximum link = 1
LDP PW • Link to PE40 is not active
CE1 PE38 PE39 CE2
#1 PE38 EVPN-VPWS Seamless Migration configuration
#2 PE40 EVPN-VPWS Configuration
PE40 PE40 -> CE2 AC is down (not active)
PE38 <-> PE39 LDP PW is UP
#3 CE2 changes ethernet bundle link priorities
PE38 <-> PE39 PW DOWN
PE40 -> Signal EVPN-VPWS
CE1 PE38 LDP PW PE39 CE2 # PE38 <-> PE40 EVPN-VPWS UP

PE40

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 116
EVPN &
VPLS/VPWS
Interconnect
 EVPN & VPLS Interconnect

CE2 A2 R37 PE2

LACP

MPLS Core/Access MPLS Core CE3

R36 PE1
CE1 A1
VPLS EVPN

R36/R37 Configuration R36 Configuration R37 Configuration

evpn l2vpn l2vpn
evi 100 bridge group 100 bridge group 100
advertise-mac bridge-domain 100 bridge-domain 100
! access-vfi 1 access-vfi 1
virtual vfi 1 neighbor x.x.x.A1 pw-id 1 neighbor x.x.x.A1 pw-id 10
ethernet-segment ! !
identifier type 0 11.11.11.11.11.11.11.11.11 neighbor x.x.x.A2 pw-id 2 neighbor x.x.x.A2 pw-id 20
! !
! !
Virtual Ethernet Segment (vES) evi 100 evi 100

• VPLS is Single-Active Access to EVPN

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 118
Virtual Ethernet-Segment (vES)
R36#show evpn ethernet-segment detail

Ethernet Segment Id Interface Nexthops

------------------------ ---------------------------------- --------------------
0011.1111.1111.1111.1111 VFI:1 3.3.3.36
3.3.3.37
ES to BGP Gates : Ready
ES to L2FIB Gates : Ready
Virtual Access :
Name : VFI_1
State : Up
Num PW Up : 1
ESI type : 0
Value : 11.1111.1111.1111.1111
ES Import RT : 1111.1111.1111 (from ESI)
Source MAC : 0000.0000.0000 (N/A)
Topology :
Operational : MH, Single-active
Configured : Single-active (AApS) (default)
Service Carving : Auto-selection
Peering Details : 3.3.3.36[MOD:P:00] 3.3.3.37[MOD:P:00]
Service Carving Results:
Forwarders : 2
Permanent : 0
Elected : 2
Not Elected : 0
MAC Flushing mode : Invalid
Peering timer : 3 sec [not running]
Recovery timer : 30 sec [not running]
Carving timer : 0 sec [not running]
Local SHG label : 64006
Remote SHG labels : 1
64009 : nexthop 3.3.3.37

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 119
EVPN & VPWS (Active/Backup) Interconnect

R37 PE2
Backup-PW LACP

CE1 A1 MPLS Core/Access MPLS Core CE3

Active-PW
R36 PE1

EVPN

R36 Configuration R37 Configuration

l2vpn l2vpn
bridge group 100 bridge group 100
bridge-domain 100 bridge-domain 100
neighbor x.x.x.A1 pw-id 1 neighbor x.x.x.A1 pw-id 10
! !
evi 100 evi 100

• VPWS Active/Backup is Single-Homed from EVPN point of view => VPWS ESI = 0
• A1 Configuration without modification

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 120
 EVPN & VPWS (Static-Anycast) Interconnect
Anycast Pseudonode R3637

R37 PE2
MPLS Core/Access LACP

CE1 A1 MPLS Core CE3

Active-PW
R36 PE1

EVPN

A1 Configuration R36/R37 Configuration R36/R37 Configuration

l2vpn evpn l2vpn
xconnect group 100 evi 100 bridge group 100
p2p 100 advertise-mac bridge-domain 100
interface TenGigE0/0/0/0 ! neighbor x.x.x.A1 pw-id 1
neighbor ipv4 x.x.36.37 pw-id 1 virtual neighbor x.x.x.A1 pw-id 1 mpls static label local 3637 remote 100
mpls static label local 100 remote 3637 ethernet-segment !
identifier type 0 11.11.11.11.11.11.11.11.11 evi 100

Virtual Ethernet Segment (vES)

• VPWS is All-Active Access to EVPN

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 121
EVPN ETREE
 EVPN ETREE – RT Constrains (Scenario 1a)
• Host connected to Leaf can talk ONLY to device connected to Root

• H1, H2, H3 can talk to H4

• H1, H2, H3 CANNOT talk to each other Leaf Additional Configuration

Root Configuration Prevents H1 and H2 to talk locally
evpn l2vpn
evi 100 bridge group evpn
Leaf4 bgp
route-target export 1:1000
bridge-domain evpn100
interface TenGigE0/0/0/0
route-target import 1:1000 split-horizon group
route-target import 1:100 !
H3 ! interface Bundle-Ether100
split-horizon group
!

Leaf3
MPLS Root1 H4
H2
Leaf2
Leaf Configuration
evpn
H1 evi 100
bgp
route-target export 1:100
Leaf1 route-target import 1:1000
!
etree
rt-leaf <- MAC Synchronization
!
© 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 123
 EVPN ETREE Leaf Label (Scenario 1b)
Root Configuration
No specific Root Configuration
l2vpn
bridge group test • ASR9k/NCS add Leaf ACs to SHG2 automatically
Leaf4 bridge-domain test => Prevents local Leaf to Leaf AC forwarding
interface Bundle-Ether100
!
evi 300
H3

Leaf3
MPLS Root1 H4
H2
Leaf2
Leaf Configuration
l2vpn
H1 bridge group test
bridge-domain test
etree
leaf
Leaf1 !
interface Bundle-Ether100
!
evi 300

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 124
 EVPN ETREE Leaf Label (Scenario 1b) - BUM
Leaf Configuration
l2vpn Each Leaf (device with at least one Leaf AC) advertises RT1 per-ESI
bridge group test with ESI 0 with ETREE extended community to distribute ETREE Label
bridge-domain test
etree R28#show bgp l2vpn evpn rd 1.1.1.28:0 [1][1.1.1.28:1][0000.0000.0000.0000.0000][4294967295]/184
leaf Wed Mar 23 03:41:36.734 UTC
! BGP routing table entry for [1][1.1.1.28:1][0000.0000.0000.0000.0000][4294967295]/184, Route Distinguisher: 1.1.1.28:0
Versions:
interface Bundle-Ether100 Process bRIB/RIB SendTblVer
! Speaker 1481327 1481327
evi 300 Local Label: 0
Last Modified: Mar 23 03:21:20.580 for 00:20:17
Paths: (1 available, best #1)
Advertised to update-groups (with more than one peer):
0.2
Path #1: Received by speaker 0
Leaf4 Advertised to update-groups (with more than one peer):
0.2
Local
0.0.0.0 from 0.0.0.0 (1.1.1.28)
Origin IGP, localpref 100, valid, redistributed, best, group-best, import-candidate, rib-install
H3 Received Path ID 0, Local Path ID 1, version 1481327
Extended community: EVPN E-TREE:0x00:24010 RT:1:3000

Leaf3 ETREE Label works same as Split-Horizon Label (SHL)

SHL prevents BUM forwarding between two ACs with the same ESI
H2 ETREE Label prevents forwarding between Leaves ACs

Leaf2 Leaf to Leaf BUM traffic has ETREE Label

If Traffic with ETREE label is received cannot be forwarded to Leaf AC
H1 Root to Leaf or Leaf to Root BUM traffic doesn’t have ETREE label
BUM between Root <-> Leaf is allowed

Leaf1
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 125
 EVPN ETREE Leaf Label (Scenario 1b) - Unicast
Leaf Configuration
l2vpn Leaf Advertises local MAC with ETREE extended community
bridge group test Same extended community was used to distribute ETREE Label
bridge-domain test
etree RP/0/RSP0/CPU0:R28#show bgp l2vpn evpn bridge-domain test [2][0][48][682c.7b24.c63d][0]/104
Wed Mar 23 04:13:10.244 UTC
leaf BGP routing table entry for [2][0][48][682c.7b24.c63d][0]/104, Route Distinguisher: 1.1.1.28:300
! Versions:
interface Bundle-Ether100 Process bRIB/RIB SendTblVer
Speaker 1481349 1481349
! Local Label: 24012
evi 300 Last Modified: Mar 23 03:21:48.580 for 00:51:22
Paths: (1 available, best #1)
Advertised to update-groups (with more than one peer):
0.2
Path #1: Received by speaker 0

Leaf4 Advertised to update-groups (with more than one peer):

0.2
Local
0.0.0.0 from 0.0.0.0 (1.1.1.28)
Origin IGP, localpref 100, valid, redistributed, best, group-best, import-candidate, rib-install

H3 Received Path ID 0, Local Path ID 1, version 1481349

Extended community: SoO:1.1.1.28:300 EVPN E-TREE:0x01:0 RT:1:300
EVPN ESI: 0026.2826.2826.2826.2802

Leaf3 ETREE Label is set to 0, but Leaf Flag is set to 1

H2 Unicast traffic is filtered by ingress node

If traffic is originated from Leaf AC and destination is local/remote Leaf AC frame is dropped
Leaf2

H1

Leaf1
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 126
 EVPN ETREE Leaf Label (Scenario 2) per-AC
Root/Leaf Configuration
l2vpn
bridge group test
bridge-domain test
interface Bundle-Ether100 <- interface to H4
Leaf4 interface Bundle-Ether200 <- interface to H5
etree
leaf
!
H3 !
evi 300
H5
Leaf3
MPLS
H2 Root
Leaf2 Leaf H4 Leaf Configuration
Same as Scenario 1b
H1
l2vpn
bridge group test
bridge-domain test
Leaf1 etree
leaf
!
interface Bundle-Ether100
!
evi 300

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 127
EVPN ETREE Summary
Scenario 1a: RT Constrains is simple and HW “friendly”
Unicast/BUM filtering by ingress node => scale benefit

Scenario 1b: Simple configuration, but additional ETREE label must be imposed for BUM
BUM filtered by egress node
Support IRB

Scenario 2: Same principle as Scenario 1b also compatible with Scenario 1b

ASR9k allows to combine Root/Leaf ACs in the same Bridge-Domain
Support IRB

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 128
EVPN FRR
Fast Convergence (FRR Data Plane) - Core
Core Failure (Link/Node) – PIC Core
Technology: RSVP-TE/LFA/rLFA/TI-LFA
Transport: IGP -> MPLS, SRv6
Overlay Service: Service Independent
Device: P-Router, Spine

PE2 PE4 L2
P2 S2

X
P1 X
S1
PE1 PE3 L1

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 130
Fast Convergence (FRR Control Plane) – DC Leaf/TOR
MAC Mobility
VM/MAC Move
Technology: EVPN Mac Mobility (EVPN RT-2)
Transport: Transport Independent
Overlay Service: EVPN MAC IP ESI Seq. Next-
Device: Leaf/TOR Hop
MAC-1 IP-1 0 1 Leaf-3/4
Sequence number is incremented and
L4 Next-hop is changed to Leaf-3/4

VM1

Sequence number and Next-Hop L3 S2

Move
value will be changed after the host
move
L2 S1

VM1
MAC IP ESI Seq. Next-
Hop
L1
MAC-1 IP-1 0 0 Leaf-1/2
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 131
Fast Convergence (FRR CP/DP) – Edge/Leaf/TOR
Leaf/TOR Failure (Link) – EVPN Mass Withdraw
Technology: EVPN RT1 Mass Withdraw
Transport: Transport Independent
Overlay Service: EVPN
Device: Leaf/TOR/Access/Edge

PE2
MAC-CE1 -> ESI1 -> PE1
PE4 X
-> PE2
P2
CE1
P1
PE1 PE3
RT1 ESI1 Mass Withdraw

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 132
Fast Convergence (FRR Data Plane) – Edge L3VPN
Edge Failure (Link) – BGP PIC Edge
Technology: BGP PIC Edge
Transport: MPLS, SRv6 (Transport Independent)
Overlay Service: L3VPN
Device: Access/PE
BGP CE-PE is mandatory!!!

PE2
P2

CE1 L3VPN
P1
PE1

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 133
Fast Convergence (FRR Data Plane) – Edge L2VPN
Edge Failure (Link) – EVPN FRR
Technology: EVPN FRR
Transport: Transport Independent
Overlay Service: EVPN
Device: Access/PE/Leaf/TOR

All-Active Single-Active

PE2 PE2
P2 P2

CE1 EVPN CE1 EVPN

P1 P1
PE1 PE1

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 134
Fast Convergence (EVPN FRR Data Plane) – Edge

• Single-Active NDF filter traffic in both directions

• Re-Directed traffic will be re-directed back to PE1 (L3 Loop) or dropped
• Solution is to bypass NDF => Only redirected packet can bypass NDF!
• Extra FRR label is used to bypass NDF
• FRR Label is used for both All-Active and Single-Active access

All-Active Single-Active

PE2 PE2
P2 P2

CE1 EVPN CE1 EVPN

P1 P1
PE1 PE1

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 135
EVPN FRR - Configuration
All-Active Single-Active
evpn evpn
interface Bundle-Ether100 interface Bundle-Ether100
ethernet-segment ethernet-segment
identifier type 0 36.37.36.37.36.37.36.37.01 identifier type 0 36.37.36.37.36.37.36.37.01
convergence load-balancing-mode single-active
reroute convergence
reroute

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 136
DF Election Convergence Improvements
evpn
interface Bundle-Ether100
ethernet-segment
identifier type 0 36.37.36.37.36.37.36.37.01 BGP Next-Hop Tracking for RT4
load-balancing-mode single-active
convergence
Node Failure Convergence
nexthop-tracking Improvement
reroute
NTP Timestamping for RT4

R37#show evpn ethernet-segment carving detail

Service Carving Synchronization:
Mode : NTP_SCT
Peer Updates :
3.3.3.36 [SCT: 2020-10-28 12:57:47:456146]
3.3.3.37 [SCT: 2020-10-28 12:57:47:451599] NTP Timestamping for RT4

R37#show ntp status

Clock is synchronized, stratum 3, reference is 10.255.11.1

R37#show bgp l2vpn evpn rd 3.3.3.36:0 [4][0036.3736.3736.3736.3701][32]

3.3.3.36 (metric 30) from 3.3.3.103 (3.3.3.36)
Origin IGP, localpref 100, valid, internal, best, group-best, import-candidate, not-in-vrf
Received Path ID 0, Local Path ID 1, version 1359
Extended community: EVPN ES Import:3637.3637.3637 DF Election:0:0x0008:0 EVPN NTP: 3812880149.4488
Originator: 3.3.3.36, Cluster list: 3.3.3.103

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 137
EVPN Selective
Multicast
RT6/7/8
EVPN ELAN L2 Selective Multicast – Route-Type 6
RT6 to IGMP Join (Proxy) not supported
Receiver1
Multicast must be received by PE5/6 from source without IGMP join

PE5/PE6 selectively ingress-replicate multicast to PE1 and PE2

IGMP Join
PE1
EVPN RT6
Selective Multicast
EVPN RT7
Receiver2
IGMP Join sync

PE2 PE5

Receiver3
Source1

PE3 PE6

Receiver7

PE4

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 139
EVPN – Native Multicast in the Network Fabric
Source

BL BL
PIM State sync
in EVPN
SP SP

L3
---
L2 L L L L
EVI-x
IGMP Join / Leave
mcast EVI-y
C C
evpn IRB
Receiver Receiver

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 140
EVPN Headend
L3 EVPN Head End (EVPN-PWHE)
HE Modes (PE):
1. Single-Active/Port-Active from Access and All-Active from Core (default)
2. All-Active

A2 PE2
PE1/PE2 Configuration
HE evpn

EVPN VPNv4/6 interface PW-Ether 1

ethernet-segment
CE1 identifier type 0 9.8.7.6.5.4.3.2.1
HE
A1 PE1 l2vpn
xconnect group xc100
p2p evpn-headend
Access Modes (A): interface PW-Ether1
1. All-Active EVPN-VPWS neighbor evpn evi 1 target 1 source 1

2. Port-Active EVPN-VPWS
3. Single-Active

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 142
Transport
Integration
EVPN & EVPN-VPWS On-Demand Next Hops (ODN)
DNX Platform
• RT1 and RT3 are advertised with color (color specifies SLA)

R37 R39

R36 Configuration CE1 MPLS CE2

segment-routing
traffic-eng
on-demand color 100
R36 R38
dynamic
metric
type igp

R36 Verification R38 Configuration

R36#show bgp l2vpn evpn rd 3.3.3.36:100 route-policy C100
Route Distinguisher: 3.3.3.36:100 (default for vrf evpn100) if evpn-route-type is 1 or evpn-route-type is 3 then
*>i[1][0038.3938.3938.3938.3901][0]/120 set extcommunity color c100
3.3.3.38 C:100 100 0 i endif
*>i[3][0][32][3.3.3.38]/80 end-policy
3.3.3.38 C:100 100 0 i
extcommunity-set opaque c100
100
R36#show segment-routing traffic-eng policy end-set

Color: 100, End-point: 3.3.3.38 evpn

Name: srte_c_100_ep_3.3.3.38 evi 100
Status: bgp
Admin: up Operational: up for 00:03:45 route-policy export C100
!
!
BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 144
EVPN Per-Flow Traffic Steering

R37 R39

CE1 MPLS CE2

R36 R38
R36/37 Configuration
class-map match-any test
match cos 5
end-class-map segment-routing
traffic-eng
on-demand color 100
policy-map per-flow dynamic
class test metric
set forward-class 5 type igp
!
!
interface Bundle-Ether999 !
l2transport on-demand color 1000
service-policy input per-flow per-flow
forward-class 5 color 100

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 145
Summary
Conclusion
• EVPN is an very important complement to BGP based services

• BGP is Unified Services Control Plane across SP Network

• EVPN All-Active Multihomed Service with Distributed Anycast Gateway & Integration to
L3VPN simplifies SPDC/NextGen-CO/WAN Integration

NETCONF
Provisioning YANG

Programmability

L2/L3VPN Services LDP BGP LDP BGP BGP

Inter-Domain CP BGP-LU BGP-LU

FRR or TE RSVP
IGP with SR
LDP IGP with SR
Intra-Domain CP
IGP

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 147
EVPN - Stay Up-To-Date

• https://e-vpn.io/

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 148
Complete your Session Survey
• Please complete your session survey
after each session. Your feedback
is important.
• Complete a minimum of 4 session
surveys and the Overall Conference
survey (open from Thursday) to
receive your Cisco Live t-shirt.
• All surveys can be taken in the Cisco Events Mobile App or
by logging in to the Session Catalog and clicking the
"Attendee Dashboard” at
https://www.ciscolive.com/emea/learn/sessions/session-catalog.html

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 149
Continue
Agenda Your Education

Visit the Cisco Showcase for related demos.

Book your one-on-one Meet the Engineer meeting.

Attend any of the related sessions at the DevNet,

Capture the Flag, and Walk-in Labs zones.

Visit the On-Demand Library for more sessions

at ciscolive.com/on-demand.

BRKSPG-2835 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 150
Thank you