Implementing VXLAN

In a Data Center
Lilian Quan, Principle Engineer, INSBU
Erum Frahim, Technical Leader, Services
Kevin Cook, Solution Architecture
LTRDCT-2223
Agenda

 VxLAN Overview
 Flood-&-Learn VXLAN
 VXLAN with MP-BGP EVPN Control Plane
 VXLAN Design Options
 MP-BGP EVPN VXLAN Configuration
 Lab Introduction
Agenda

 VxLAN Overview
 Flood-&-Learn VXLAN
 VXLAN with MP-BGP EVPN Control Plane
 VXLAN Design Options
 MP-BGP EVPN VXLAN Configuration
 Lab Introduction
Recap – What is VXLAN ?
• VXLAN is point to multi-point tunneling mechanism to extend Layer 2 networks over an IP network

VXLAN Tunnel
Ethernet Frames
IP Network IP Addr
NETWORK
OVERLAY

Host1 Host4
2.2.2.2
IP Addr
Host2 Switch 1 1.1.1.1 Switch 2 Host5

Host3 Host6
IP/UDP Packets IP Addr
3.3.3.3
Host7

Switch 3 Host8

Host9

• VXLAN uses MAC in UDP encapsulation (UDP destination port 4789)

Outer Outer Outer Outer Outer Outer VXLAN Inner Inner Optiona Original CRC
PLANE
DATA

MAC MAC 802.1Q IP DA IP SA UDP ID MAC MAC l Inner Ethernet

CRC
DA SA (24 bits) DA SA 802.1Q Payload

VXLAN Encapsulation Original Ethernet Frame

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
Why VXLAN?
VXLAN provides a Network with Segmentation, IP Mobility, and Scale
• “Standards” based Overlay
• Leverages Layer-3 ECMP – all links forwarding
• Increased Name-Space to 16M identifier
• Segmentation and Multi-Tenancy
• Integration of Physical and Virtual
• It’s SDN 

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
VXLAN VTEP (Virtual Tunnel End Point)
VXLAN terminates its tunnels on VTEPs. Each VTEP has two interfaces:
• Local Lan - provide bridging function for local hosts,
• IP Interface to underlay – send/receive VXLAN encapsulated packets

Tunnel
Ethernet Frames Endpoints
Host Host
1 4
Switch

Switch
Local LAN

Local LAN
Interface

Interface
Host
IP Addr IP Network IP Addr
Host
IP

IP
1.1.1.1 2.2.2.2
2 5
Host Host
3 IP/UDP Packets 6
IP Addr
3.3.3.3

Host
IP Interface 7
Host
Switch 8
Local LAN Host
9

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
 VXLAN Underlay Network – IP Routing
• IP routed Network
• Flexible topologies
• Recommend a network with redundant paths using ECMP for load sharing

• Support any routing protocols --- OSFP, EIGRP, IS-IS, BGP, etc.
• All proven best practices for IP routing network apply

IP Transport Network

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 8
 VXLAN Underlay Network – Typical DC Topologies
Fabric Design
3-Tier Design
DC Spine
DC Core

DC Aggregation

DC Access
DC Leaf

DC Interconnect
Collapsed Core/Aggregation
2-Tier Design
DC-1 DC-2
DC Core/ Aggregation

DC Access
WAN

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
Agenda

 VxLAN Overview
 Flood-&-Learn VXLAN
 VXLAN with MP-BGP EVPN Control Plane
 VXLAN Design Options
 MP-BGP EVPN VXLAN Configuration
 VxLAN Capability on Nexus 9000 Series Switches
 Lab Introduction
Flood-&-Learn VxLAN
 No VXLAN control plane
 Learning through data-driven flood-&-learn

End System

End System
VTEP-3
IP-3
VTEP 3
Multicast/Unicast
Replication
VTEP-1 VTEP-2
IP Network
VTEP 1 VTEP 2 End System B
End System A
IP-1 IP-2 MAC-B
MAC-A
IP-A IP-B

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
Challenges with Flood-&-Learn VXLAN Deployments
Scale, Mobility and Security Limitations

VXLAN Overlay
VTEP VTEP VTEP VTEP VTEP
VTEP VTEP

LIMITED SCALE LIMITED WORKLOAD MOBILITY Security Risk

• Flood and learn (BUM)- • Centralized Gateways – Traffic • No authentication for VXLAN
Inefficient Bandwidth Utilization Hair-pining devices (VTEPs)
• Resource Intensive – Large • Sub-Optimal Traffic Flow
MAC Tables

Barrier for Scaling out Large Data Centers and Cloud Deployments
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Agenda

 VxLAN Overview
 Flood-&-Learn VXLAN
 VXLAN with MP-BGP EVPN Control Plane
 VXLAN Design Options
 MP-BGP EVPN VXLAN Configuration
 VxLAN Capability on Nexus 9000 Series Switches
 Lab Introduction
 What is VXLAN/EVPN?
• Standards based Overlay (VXLAN) with Standards based Control-Plane (BGP)
• Layer-2 MAC and Layer-3 IP information distribution by Control-Plane (BGP)
• Forwarding decision based on Control-Plane (minimizes flooding)
• Integrated Routing/Bridging (IRB) for Optimized Forwarding in the Overlay

Control- EVPN MP-BGP - RFC 7432

Plane (draft-ietf-l2vpn-evpn)

Provider Backbone Bridges

Data- Multi-Protocol Label Switching (MPLS) Network Virtualization Overlay (NVO)
(PBB)
Plane draft-ietf-l2vpn-evpn draft-sd-l2vpn-evpn-overlay
draft-ietf-l2vpn-pbb-evpn

 EVPN over NVO Tunnels (VXLAN, NVGRE, MPLSoE) for Data Center Fabric encapsulations
 Provides Layer-2 and Layer-3 Overlays over simple IP Networks

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 15
 FYI

Cisco Supported VXLAN IETF Drafts

ID Title Ver Category Comments

RFC 7348 Virtual eXtensible Local Area Data Plane Flood and Learn based implemented
Network
RFC 7432 BGP MPLS based Ethernet VPNs EVPN Control Plane Control Plane is based on this RFC

draft-ietf-bess-evpn-overlay A Network Virtualization Overlay 0 EVPN Control Plane Implemented

Solution using EVPN
draft-ietf-bess-evpn-inter- Integrated Routing and Bridging in 0 EVPN Control Plane Implemented
subnet-forwarding EVPN

draft-rabadan-l2vpn-evpn- IP Prefix Advertisement in EVPN 2 EVPN Control Plane Implemented

evpn-prefix-advertisement

Draft-tissa-nvo3-oam-fm NVO3 Fault Management 1 Mgmt Plane Working in progress

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
Functions of VXLAN/EVPN
Host/Network Advertise host/network reachability
Reachability information through control protocol (MP-
Advertisement BGP)
VTEP Security & Authenticate VTEPs through BGP peer
Authentication authentication
Distributed
Seamless and Optimal vm-mobility
Anycast Gateway
Early ARP termination
ARP Suppression Localize ARP learning process
Minimize network flooding
Dynamic Ingress
Replication Unicast Alternative to Multicast underlay
Dynamically discover remote peers for
Ingress Replication

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
EVPN Primer --- MP-BGP Review
Virtual Routing and Forwarding (VRF)
Layer-3 segmentation for tenants’ routing space
BGP advertisement:
VPN-IPv4 Addr = RD:16.1/16
Route Distinguisher (RD): BGP Next-Hop = PE1
8-byte field, VRF parameters; unique value to make eBGP:
Route Target = 100:1
Label=42 eBGP:
VPN IP routes unique: RD + VPN IP prefix 16.1/16 16.1/16
IP Subnet IP Subnet

P P
Selective distribute VPN routes: CE1
PE1 Blue VPN PE2
CE2

Route Target (RT): 8-byte field, VRF parameter,

unique value to define the import/export rules for
VRFvrfparameters:
VPNv4 routes ip
Name
blue-vpn
= blue-vpn
RD 1:100
RD = 1:100 export 1:100
route-target
Import Route-Target
route-target = 100:1
import 1:100
Export Route-Target = 100:1

VPN Address-Family:
Distribute the MP-BGP VPN routes

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
MP-BGP for VXLAN EVPN Control Plane
EVPN Control Plane – Reachability Distribution
• EVPN Control Plane -- Host and Subnet Route Distribution
BGP Update
Spine
• Host-MAC
• Host-IP
• Internal IP Subnet
• External Prefixes VTEP VTEP VTEP VTEP Leaf

 Use MP-BGP with EVPN Address Family on leaf nodes to distribute internal
host MAC/IP addresses, subnet routes and external reachability information
 MP-BGP enhancements to carry up to 100s of thousands of routes with
reduced convergence time

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 19
 EVPN Control Plane -- Host Advertisement
Install host info to RIB/FIB:
Install host info to RIB/FIB:
H-MAC-1  MAC table
BGP Update: H-MAC-1  MAC table
H-IP-1  VRF IP host table 4 BGP Update:
H-MAC-1 4 H-IP-1  VRF IP host table
H-MAC-1
Host VNI VTEP H-IP-1
IP
H-IP-1 3 Route VTEP-1 MAC Host VNI VTEP
VTEP-1 3
Reflector VNI-1 IP
C-1 H-IP-1 VNII-1 VTEP-1 VNI-1
H-MAC-1 H-IP-1 VNII-1 VTEP-1
2
VTEP-2
VTEP-3 BGP Update:
H-MAC-1
H-IP-1
VTEP-1
VNI-1

VTEP-1
1

MAC Host VNI VTEP

IP

H-MAC-1 H-IP-1 VNII-1 VTEP-1

Local learning of host info: H-MAC-1
H-IP-1
H-MAC-1 (MAC table)
VLAN-1 /VNI-1
H-IP-1 (VRF IP host table )

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 20
VXLAN BGP Control Plane
EVPN Control Plane --- Host Movement

NLRI: Spine
• Host H-MAC-1, H-IP-1
• NVE VTEP-1
• VNI 5000

Ext. Community:
• Encapsulation: VXLAN VTEP-1 VTEP-2 VTEP-3 VTEP-4 Leaf
• Cost
• Sequence number :0 Host 1
H-MAC-1
H-IP-1
VLAN 10
VXLAN 5000
1. Host 1 attaches to VTEP-1

2. VTEP-1 detects Host1 and advertises H1 with seq #0

MAC IP VNI Next-Hop Encap Seq#

3. Other VTEPs learn about the host route of Host 1
H-MAC-1 H-IP-1 5000 VTEP-1 VXLAN 0

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
VXLAN BGP Control Plane
EVPN Control Plane --- Host Movement

NLRI: Spine
• Host H-MAC-1, H-IP-1
• NVE VTEP-3
• VNI 5000

Ext. Community:
• Encapsulation: VXLAN VTEP-1 VTEP-2 VTEP-3 VTEP-4 Leaf
• Cost
• Sequence number: 1 Host 1
H-MAC-1
H-IP-1
VLAN 10
VXLAN 5000
1. Host 1 moves to VTEP-3 from VTEP-1

2. VTEP-3 detects Host 1, sends MP-BGP update for Host 1 with its own VTEP address and a new seq #1

3. Other VTEPs learn about the new route of Host 1 MAC IP VNI Next-Hop Encap Seq#

H-MAC-1 H-IP-1 5000 VTEP-3 VXLAN 1

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
 VXLAN Routing
VXLAN EVPN has two slightly different integrated Route/Bridge (IRB) semantics

Routing ?
IP Transport Network
SVI A SVI B
VTEP-1 VTEP-2 VTEP-3 VTEP-4

Host 1 Host 2
H-MAC-1 H-MAC-2
H-IP-1 H-IP-2
VNI-A VNI-B

Asymmetric Symmetric
• Routing on the ingress VTEP and bridging • Routing on both the ingress and the egress
on the egress VTEP VTEPs
• Requires each VTEP to have all VNIs – • A VTEP only needs to have VNIs in which
can result in forwarding table resource they have local hosts. Optimal utilization of
wastage. forwarding table resources

• Cisco follows Symmetric IRB

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
Symmetric Integrated Routing & Bridging (IRB) (1)
• Routing on both ingress and egress VTEPs
• Layer-3 VNI
Layer-3 VNI
• Tenant VPN indicator (VRF VNI)

• One per tenant VRF

• VTEP Router MAC Layer-2 VNI Layer-2 VNI
(Green) (Orane)
• Ingress VTEP routes packets onto the (Network VNI) (Network VNI)
Layer-3 VNI
• Egress VTEP routes packets to the VTEP VTEP VTEP VTEP

destination Layer-2 VNI

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
EVPN Control Plane --- ARP Suppression
Minimize flood-&-learn behavior for host learning

MAC IP VNI Next-Hop Encap Seq Spine

H-MAC-2 H-IP-2 5000 VTEP-3 VXLAN 0

2 VTEP VTEP VTEP VTEP

VTEP-1 receives and intercepts the ARP 1 2 3 4 Leaf
Request. Checks in its own host table.
• If it has an match for H-IP-2, it’ll send ARP Host 1 Host 2
response on behave of Host-2 H-MAC1 H-MAC-2
H-IP 1 H-IP-2
• If it doesn’t have a match for H-IP-2, it’ll VLAN 10 VLAN 10
forward the ARP request to remote VTEPs VXLAN 5000 VXLAN 5000
via multicast encap or head-end replication

1
Host-1 sends ARP
Request for H-IP-2

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
 Distributed Anycast Gateway in MP-BGP EVPN

# VLAN to VNI mapping

vlan 200
vn-segment 5200

# Anycast Gateway MAC, identically configured on all VTEPs

fabric forwarding anycast-gateway-mac 0002.0002.0002

# Distributed IP Anycast Gateway (SVI)

The same anycast gateway virtual IP # Gateway IP address needs to be identically configured on all
address and MAC address are VTEPs
configured on all VTEPs in the VNI. interface vlan 200
no shutdown
vrf member Tenant-A
ip address 20.0.0.1/24
fabric forwarding mode anycast-gateway

SVI SVI SVI SVI

GW IP GW IP GW IP GW IP
GW MAC GW MAC GW MAC GW MAC
VTEP VTEP VTEP VTEP

Host 1 Host 2 Host 3 Host 4

MAC1 MAC2 MAC3 MAC4
IP 1 IP 2 IP 3 IP 4
VLAN A VLAN A VLAN A VLAN A
VXLAN A VXLAN A VXLAN A VXLAN A

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
EVPN Control Plane -- Head-end Replication
Head-end Replication (aka. Ingress replication):
Eliminate the need for underlay multicast to transport overlay BUM traffic

Multicast-Free Spine
Underlay

VTEP VTEP VTEP VTEP

1 2

VTEP-1 receives the overlay BUM traffic,

2 3 4
Leaf
encapsulates the packets into unicast VXLAN
packets, sends one copy to each remote VTEP
peer in the same VXLAN VNI

1
Host-1 sends BUM
traffic into the
VXLAN VNI

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
 Next-Gen VXLAN Fabric with BGP-EVPN Control Plane
Multi-Tenancy, Seamless Host Mobility and Security at Cloud Scale
Route Route
Reflector Reflector

BGP-EVPN VXLAN Overlay

VTEP VTEP VTEP VTEP VTEP VTEP VTEP

BGP Peers

INTEROPERABLE INCREASED SCALE OPTIMIZED OPERATIONAL SECURITY

• Standards Based • Eliminates MOBILITY FLEXIBILITY • VTEP peer
• BGP-EVPN Flooding • Distributed • Layer 2 or Layer 3 authentication via
• Conversational Anycast Gwy • Controller Choice BGP
• VXLAN
Learning • Integrated • White-list for
• Policy-Based Routing /Bridging unauthenticated
Updates • vPC & ECMP VTEPs

Breaking the Traditional VXLAN Scale Barriers

© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
Agenda

 VxLAN Overview
 Flood-&-Learn VXLAN
 VXLAN with MP-BGP EVPN Control Plane
 VXLAN Design Options
 MP-BGP EVPN VXLAN Configuration
 VxLAN Capability on Nexus 9000 Series Switches
 Lab Introduction
VXLAN Inter-PoD Extension
L3 Core L2 Link
L3 Link

VTEP
VTEP

VXLAN Overlay
(VLAN Extension)

Pod 1 Pod 2
IP GW IP GW

Layer-2 VLAN Domain Layer-2 VLAN Domain

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 31
VXLAN Fabric Design with MP-iBGP EVPN

RR RR Spine
MP-iBGP Sessions

VXLAN Overlay
MP-iBGP EVPN

VTEP VTEP VTEP VTEP VTEP VTEP Leaf

• VTEP Functions are on leaf layer

• Spine nodes are iBGP route reflector
• Spine nodes don’t need to be VTEP

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 32
VXLAN Fabric Design with MP-eBGP EVPN
AS 65000

Spine

MP-eBGP Sessions

VTEP
VTEP VTEP VTEP VTEP VTEP
Leaf

AS 65001 AS 65002 AS 65003 AS 65004 AS 65005 AS 65006

• VTEP Functions are on leaf layer

• Spine nodes are MP-eBGP Peers to VTEP leafs
• Spine nodes don’t need to be VTEP
• VTEP leafs can be in the same or different BGP AS’s
LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 33
EVPN VXLAN Fabric External Routing

VXLAN Overlay
Spine EVPN VRF/VRFs Space

VXLAN Overlay
EVPN MP-BGP Border Leaf

VTEP
VTEP VTEP VTEP VTEP VTEP
Leaf

Routing
Protocol
of
Choice Global Default VRF
Or User Space VRFs

IP Routing

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 34
 FYI
EVPN VXLAN External Routing with BGP
Sample Configuration Router bgp 100
vrf evpn-tenant-1
address-family ipv4 unicast
network 20.0.0.0/24
neighbor 30.10.1.2 remote-as 200
RR RR address-family ipv4 unicast
Spine prefix-list outbound-no-hosts out

VXLAN Overlay
Border Leaf interface Ethernet2/9.10
EVPN VRF VTE
Instance VTE
Space mtu 9216
VTE VTE VTEP
P P P P encapsulation dot1q 10
vrf member evpn-tenant-1
ip address 30.10.1.1/30

interface Ethernet1/50.10
mtu 9216
encapsulation dot1q 10
ip address 30.10.1.2/30

IP Routing in the Default router bgp 200

VRF Instance address-family ipv4 unicast
network 100.0.0.0/24
network 100.0.1.0/24
neighbor 30.10.1.1 remote-as 100
address-family ipv4 unicast
LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
EVPN Design for Stretched Fabric
VXLAN Overlay
RR RR RR EVPN
RR VRF/VRFs Space

DC #1 DCI Border Leaf DCI Border Leaf

DC #2
EVPN BGP EVPN BGP

VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

Inter-DC
EVPN eBGP (multi-hop)

Global Default VRF

Or User Space VRFs

One EVPN Administrative Domain

Stretched Across Two Data Centers

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 36
EVPN Configuration for Stretched Fabric
VXLAN Overlay
RR RR RR EVPN
RR VRF/VRFs Space

DC #1 DC #2
iBGP/eBGP
EVPN iBGP
DCI Border Leaf DCI Border Leaf
iBGP/eBGP
EVPN iBGP

VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

Multi-hop
Inter-DC
eBGP
EVPN eBGP (multi-hop)

Global Default VRF

Or User Space VRFs

One EVPN Administrative Domain

Stretched Across Two Data Centers

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 37
VXLAN EVPN Stretched Fabric (Alternative Option)
VXLAN Overlay
RR RR RR EVPN
RR VRF/VRFs Space

DC #1 DC #2
EVPN iBGP Border Leaf Border Leaf EVPN iBGP

VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

EVPN Domain #2
EVPN Domain #1

VTEP VTEP
VLAN hand-off
VLAN hand-off
Flood-&-Learn
Flood-&-Learn Inter-DC EVPN
OVT/VPLS

Inter-DC EVPN Domain

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
Agenda

 VxLAN Overview
 Flood-&-Learn VXLAN
 VXLAN with MP-BGP EVPN Control Plane
 VXLAN Design Options
 MP-BGP EVPN VXLAN Configuration
 Lab Introduction
 FYI

Building your VTEP (VXLAN Tunnel End-Point)

# Features & Globals Enables VTEP (only required on Leaf or Border)

feature bgp
feature nv overlay
nv overlay evpn
Enables EVPN Control-Plane in BGP
# Spine (S1)

# Leaf (V1) Configure the VTEP interface

interface nve1
RR RR RR RR
source-interface loopback0
host-reachability protocol bgp Use a Loopback for Source Interface

iBGP

V1 V2
Enable BGP for Host reachability

V3
*Simplified BGP configuration; would have 4 BGP peers (RR)
IGP not shown
LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 40
 FYI

Building your EVPN MP-BGP Control-Plane

# Features & Globals
feature bgp
feature nv overlay
nv overlay evpn
Enables EVPN Control-Plane in BGP
# Spine (S1)
router bgp 65500
router-id 10.10.10.1
address-family ipv4 unicast
address-family l2vpn evpn Activate L2VPN EVPN under each BGP neighbor
RR RR RR RR

neighbor 10.10.10.0/24 remote-as 65500

update-source loopback0
address-family l2vpn evpn iBGP
send-community both V2
V1
route-reflector-client

# Leaf (V1)
router bgp 65500 Send Extended BGP Community
router-id 10.10.10.10 to distribute EVPN route attributes
address-family ipv4 unicast V3
neighbor 10.10.10.1 remote-as 65500
update-source loopback0
address-family l2vpn evpn
send-community both
*

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 41
 FYI

Extend your VLAN to VXLAN

• VLAN to VNI configuration on a per-Switch based
• VLAN becomes “Switch Local Identifier” # Features
feature vn-segment-vlan-based
• VNI becomes “Network Global Identifier”
# VLAN to VNI mapping (MT-Lite)
• 4k VLAN limitation per-Switch does still apply Vlan 10 VLAN to Layer-2 VNI mapping
vn-segment 5010
• 4k Network limitation has been removed
• VLAN can be port-significant. The same vlan on # Activate Layer-2 VNI for EVPN
evpn
different ports can be mapping to different VNIs. vni 5010 l2 Enables EVPN Control-
rd auto Plane for Layer-2
route-target import auto Services
route-target export auto
Alternative is to use
VLAN
# Activate Layer-2 VNI on VTEP “ingress-replication
interface nve1 protocol bgp”
ethernet VLAN VNI vxlan source-interface loopback0
host-reachability protocol bgp
Multi-Tenancy Lite (MT-Lite) member vni 5010
mcast-group 239.239.239.100
suppress-arp Enables Layer-2 VNI
on VTEP and suppress
LTRDCT-2223 ARP 42
© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public
 FYI

Distributed Anycast Gateway for Extended VLANs

• All VTEPs in a VXLAN are the distributed anycast
gateway for its IP subnet.
• All VTEPs in a VXLAN need to be configured with
# VLAN to VNI mapping
an identical anycast gateway virtual MAC address vlan 200
• All VTEPs in a VXLAN need to be configured with vn-segment 5200
an identical anycast gateway virtual IP address # Anycast Gateway MAC, identically configured on all
VTEPs
fabric forwarding anycast-gateway-mac 0002.0002.0002

# Distributed IP Anycast Gateway (SVI)

One gateway virtual MAC per VTEP # Gateway IP address needs to be identically
configured on all VTEPs
interface vlan 200
no shutdown
vrf member Tenant-A
ip address 20.0.0.1/24
One gateway virtual IP per VLAN/VXLAN fabric forwarding mode anycast-gateway

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 43
 FYI

Routing in VXLAN – Define the Resources

Configuration Example for VRF-A
# Define VLAN for VRF routing instance
Layer-3 VNI
Vlan 999
(VNI 9999 / VLAN 999) VLAN to Layer-3 VNI mapping
vn-segment 9999

# Define SVI for VRF routing instance

Layer-2 VNI interface Vlan999
Layer-2 VNI VLAN to Layer-3 VNI mapping
(Network VNI) no shutdown
(Network VNI) mtu 9216 - ip forward required for prefix-
vrf member VRF-A based routing
VTE VTE VTE VTE
P P P ip forward
P

# VRF configuration for “customer” VRF

vrf context VRF-A
vni 9999
VRF context definition
1:1 mapping between rd auto - VNI
address-family ipv4 unicast
L3 VNI and tenant VRF route-target both auto
- Route-Distinguisher
route-target both auto evpn - Route-Targets
- IPv4 and/or IPv6

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 44
 FYI

Routing in VXLAN – Configure the Routing

Enables Layer-3 VNI on VTEP Configuration Example for VRF-A
andVNIassociate it to VRF # Activate Layer-3 VNI on VTEP
Layer-3
interface nve1
(VNI 9999 / VLAN 999)
source-interface loopback0
host-reachability protocol bgp
member vni 5010
Layer-2 VNI mcast-group 239.239.239.100
Layer-2 VNI suppress-arp
(Network VNI)
(Network VNI) member vni 9999 associate-vrf
VTE VTE VTE VTE
P P P # Route-Map for Redistribute Subnet
P
route-map REDIST-SUBNET permit 10
match tag 12345

1:1 mapping between # Control-Plane configuration for VRF (Tenant)

L3 VNI and tenant VRF router bgp 65500
…
vrf VRF-A
address-family ipv4 unicast
VRF/Tenant definition advertise l2vpn evpn
within Overlay Control-Plane redistribute direct route-map REDIST-SUBNET
maximum-paths ibgp 2

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 45
 FYI

VXLAN Hardware Gateway Redundancy (vPC)

• Redundant connectivity for classic
Ethernet hosts
• Extend the IP Interface (Loopback)
configuration for the vPC VTEP
V5
• Secondary IP address (anycast) is
used as the anycast VTEP address V4
• Both vPC VTEP switches need to
have the identical secondary IP
address configured under the
loopback interface
Host D
VNI 30000

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 46
 FYI

VXLAN Hardware Gateway Redundancy (vPC)

vPC VTEP Configuration Example
# VLAN to VNI mapping (MT-Lite)
vlan 55 interface loopback0
vn-segment 30000 ip address 10.10.10.5/32
ip address 10.10.10.99/32 secondary
# VTEP IP Interface; Source/Destination for all
VXLAN Encapsulated Traffic.
 Primary IP address is used for Orphan Hosts
 Secondary IP is for vPC Hosts (same IP on both
vPC Peers)
interface loopback0
Add Secondary IP to VTEP V5
ip address 10.10.10.5/32
ip address 10.10.10.99/32 secondary Loopback.
VXLAN automatically picks up
V4
# VTEP configuration using Loopback as source. the secondary IP address as
interface nve1 the VTEP address
source-interface loopback0
host-reachability protocol bgp
interface loopback0
member vni 5010 ip address 10.10.10.4/32
mcast-group 239.239.239.100 ip address 10.10.10.99/32 secondary
suppress-arp Host D
member vni 9999 associate-vrf VNI 30000

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 47
 FYI

VXLAN Hardware Gateway Redundancy (vPC)

vPC VTEP Configuration Example
# VPC Domain Configuration
vpc domain 99 interface loopback0
peer-gateway needs to be ip address 10.10.10.5/32
peer-switch
enabled so that vPC VTEP
peer-keepalive destination V4-mgmt source v5-mgmt ip address 10.10.10.99/32 secondary

peer-gateway
switches can forward traffic
ip arp synchronize for each other’s router MAC
address
# VPC Peer-Link
interface port-channelXX V5
switchport mode trunk
vpc peer-link
V4
# VPC Domain Routing Adjacency Routed Interface (SVI) for routing
interface Vlan3999
no shutdown
adjacency across VPC Peer-Link
ip address 10.254.254.1/30
interface loopback0
ip router ospf 1 area 0.0.0.0 ip address 10.10.10.4/32
ip ospf network point-to-point ip address 10.10.10.99/32 secondary
ip pim sparse-mode Host D
VNI 30000

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 48
 FYI
eBGP EVPN Configuration (1)
Next-hop Unchange
• BGP next-hop is used as the tunnel tail
end address. It shall be the advertising
eBGP configuration on a spine switch
VTEP’s address.
• Ensure the next-hop in the BGP route route-map permit-all permit 10

isn’t changed during the route distribution route-map nh-unchange permit 10

set ip next-hop unchanged
• eBGP changes next-hop to by default. router bgp 65000
Need to change the policy to next-hop router-id 10.1.1.1
address-family ipv4 unicast
unchanged
address-family l2vpn evpn
nexthop route-map nh-unchange
retain route-target all
neighbor 192.167.11.2 remote-as 65001
Set next-hop policy not to change address-family ipv4 unicast
the next-hop attribute address-family l2vpn evpn
send-community extended
route-map permit-all out

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 49
 FYI
eBGP EVPN Configuration(2)
Manually configure import/export route-target

• With eBPG, VTEPs will have different

route-targets if using auto RT generation vrf context evpn-tenant-1
• Need to manually configure RTs on vni 9999
rd auto
eBGP peers so that they have the same address-family ipv4 unicast
RTs route-target import 100:9999
route-target import 100:9999 evpn
route-target export 100:9999
route-target export 100:9999evpn
evpn
Manually configure route-target for vni 5010 l2
VRF rd auto
route-target import 100:5010
route-target export 100:5010

Manually configure route-target for

L2 VNI under EVPN

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 50
Agenda

 VxLAN Overview
 Flood-&-Learn VXLAN
 VXLAN with MP-BGP EVPN Control Plane
 VXLAN Design Options
 MP-BGP EVPN VXLAN Configuration
 Lab Introduction
 Lab Overview
Module Details
• Module 1 – Network Based Overlay DC1: In this module, students will
configure a network based overlay with Nexus 9000 switches and use them
as a VTEP. Students will also learn how to extend the Anycast gateway
using BGP EVPN.
• Module 2 – FW-Security Zone : In this module, students will create the
secure zone via placing the FW ( in transparent mode) between the Fabric
and External Connectivity.
• Module 3- MultiPOD: In this module, students will be able to extend the
VLANs from the first DC/POD to the second DC/POD and able to stretch the
fabric.
• Module 4- (Bonus): InterOp with Host based and Network based VXLAN
clients
LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 52
Lab Topology

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 53
Network Overlay DC1
Using BGP EVPN for Control Plane
• Using OSPF for underlay
• IBGP-for overlay-VXLAN EVPN
control plane- Route Reflectors are
on Spines S11 and S12
• Ingress Replication for the BUM
Traffic
• Each student has their own VRF
that will be representative of multi-
tenancy.

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 54
 Security Module
Creating the Security Zone via VXLAN
• The Hosts in the Secured Zone are in VXLAN X53.
• Transparent FW is attached to Edge E11-( No redundancy in this lab).

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 55
Firewall Usecase Flow

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 56
 DC2- Stretch Fabric
Using EBGP EVPN for Underlay/Overlay
• Using EBGP for Underlay/Overlay
• Ingress Replication for the BUM Traffic.
• One of the VLAN X51 will be stretch from DC1 to DC2.
• Each student has their own VRF that will be representative of multi-tenancy.

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 57
Manual Overview (1)
• Manual available at
https://cisco.box.com/LTRDCT-2223-Manual
• Password will be provided by proctor
• Any time you see text highlighted in cyan, change the numbers to your pod
number – Pod2 is always used as a sample
• Change XX to your two digit pod number (Pod1 = 01, Pod10 = 10)
• Change X to your single/two digit pod number (Pod1 = 1, Pod10 = 10)
• Only type commands that are shown in a box. Commands shown under
“Configuration Sample” are not meant to be typed into the devices.

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 58
Manual Overview (2)
• Manual available at
https://cisco.box.com/LTRDCT-2223-Manual
• RDP Server: vxlanlab.ciscolive.com:3390
• Username: vxlan\PODxuser
• Password:

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 59
Complete Your Online Session Evaluation
• Give us your feedback to be
entered into a Daily Survey
Drawing. A daily winner will
receive a $750 Amazon gift card.
• Complete your session surveys
through the Cisco Live mobile
app or from the Session Catalog
on CiscoLive.com/us.

Don’t forget: Cisco Live sessions will be available

for viewing on-demand after the event at
CiscoLive.com/Online

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 60
Continue Your Education
• Demos in the Cisco campus
• Walk-in Self-Paced Labs
• Lunch & Learn
• Meet the Engineer 1:1 meetings
• Related sessions

LTRDCT-2223 © 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 61
Please join us for the Service Provider Innovation Talk featuring:
Yvette Kanouff | Senior Vice President and General Manager, SP Business
Joe Cozzolino | Senior Vice President, Cisco Services

Thursday, July 14th, 2016

11:30 am - 12:30pm, In the Oceanside A room

What to expect from this innovation talk

• Insights on market trends and forecasts
• Preview of key technologies and capabilities
• Innovative demonstrations of the latest and greatest products
• Better understanding of how Cisco can help you succeed

Register to attend the session live now or

watch the broadcast on cisco.com
Thank you