0% found this document useful (0 votes)

81 views194 pages

BRKDCN 2304

Uploaded by

NabsNabs

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

81 views194 pages

BRKDCN 2304

Uploaded by

NabsNabs

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 194

L4-L7 Service Integration in Multi-

Tenant VXLAN EVPN

Data Center Fabrics

Matthias Wessendorf, Technical Marketing Engineer

BRKDCN-2304
Cisco Webex Teams

Questions?
Use Cisco Webex Teams to chat
with the speaker after the session

How
1 Find this session in the Cisco Events Mobile App
2 Click “Join the Discussion”
3 Install Webex Teams or go directly to the team space
4 Enter messages/questions in the team space

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
Session Goals and Non-Goals

• Learn important requirements for designing enterprise-level Multi-Tenant DCs

• Learn technologies and building blocks needed to create Multi-Tenant networks
• Learn technologies and requirements for service attachment in Multi-Tenant DCs
• Not an ACI session
• No deep dive on Service Configuration itself

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
Related Sessions

• BRKDCN-2450-VXLAN EVPN Day-2 operation

• BRKDCT-3378-Building Data Center Networks with VXLAN BGP-EVPN
• BRKDCT-2404-VXLAN Deployment Models - A practical perspective

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
Quick Break
Before we Start
With Some Help of my Friends

I would like to thank all the people, who

contributed to it.

• Max Ardica, Principal Engineer

• Lukas Krattiger, Principal Engineer

• Shyam Kapadia, Principal Engineer

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
Agenda

• Introduction
• Multi-Tenancy Functionality in Enterprise Data Centers
• Data Plane and Control Plane Considerations with VXLAN EVPN
• Layer 4-7 Services Integration
• Other useful Things?
• Fabric Provisioning and Management
• Conclusion

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 8
What is Multi-Tenancy?

• Multi-tenancy is an architecture in which a single instance of a software application

serves multiple customers. Each customer is called a tenant.

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
What Does this Mean for Data Centers?

Service Orchestration

Compute

Separated
Shared
Storage Resources Network

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
Agenda

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
Introduction
What is Multi-Tenancy for the Data Center
Infrastructure?

• Process of creating an environment where resources are split and combined, based
on consumption, demand, supply and policies

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
Multi-Tenancy ”Layers”

- Push-Pull network orchestration

- L4-L7 Services network integration
Fabric
- Orchestrator integration via APIs
Control

- Layer-2 Network segmentation

Functionality - Layer-3 domain/Tenant separation

Mechanism MP-BGP, MPLS, Distributed GW…

Identifier VLAN ID, VNID, SGT, Label…

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 15
Rules and Policies

Tenant-2
• Applications, network services, and
tenant identification
• Enforcement of separation between
segments
Multi-Tenant
• Providing network policy
Network
• Controlled shared access to select networks and
resources

Tenant-1
Tenant-3

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
Agenda

• Introduction
• Multi-Tenancy Functionality in Enterprise Data Centers
• Data Plane and Control Plane Considerations with VXLAN EVPN
• Layer 4-7 Services Integration
• Where to Head Next?
• Fabric Provisioning and Management
• Conclusion

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
Multi-Tenancy
Functionality in
Enterprise Data
Centers
Multi-Tenancy Functionality

• Layer 2 Network segmentation

• Micro-segmentation
Fabric
• Layer 3 domain/Tenant separation Control

Functionality

Mechanism

Identifier

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 19
Layer-2 Network
Segmentation
Network Segmentation

• Provides clear separation of Layer-2 segments in the network

• Leverages identifier in the frame tag or encapsulation
• Number of Layer-2 segment identifiers depends on a chosen namespace.
• For example: Dot1Q – 4096 VLANs, VXLAN – 16M VNIs

• Identification of a given frame’s tenant membership

• For example: VRF-lite, Symmetric IRB, etc.

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
Micro-segmentation
Network Micro-Segmentation in Traditional
Networks
• Private VLAN (PVLAN) is a good example of such mechanism
• Restricts access within a segment
• Grants access to shared service or gateway

Shared
Host A resource

Host B Default
Network Fabric
Gateway

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
ACI Micro-segmentation

VLAN or VLAN or VLAN or

VLAN VLAN VLAN
VXLAN VXLAN VXLAN
vDS Cisco AVS IP/MAC EPG Hyper-V vSwitch Open vSwitch Open vSwitch

Microsegmentation Yes Yes Yes Yes Yes* Yes*

Intra-EPG Isolation Yes Yes* Yes Yes* Yes* Yes*

BRKDCT-3001: Leveraging Micro Segmentation to Build Comprehensive Data Center Security Architecture

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
Tenant
Segmentation
Layer-2 Segment Termination

• SVI – Layer-2 segment termination

mechanism
• SVI (Switch Virtual Interface) terminates a
VLAN 10 SVI 10 VLAN and is assigned an IP address
• Multiple VLANs can terminate on a single
VLAN 20 SVI 20 device
• FHRP is typically used to provide HA
VLAN 30 SVI 30 • SVI is a member of “Default VRF” by default
• Data traffic can be routed within a given VRF
VLAN 40 SVI 40 without restrictions

Default VRF

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
Restricting Forwarding between Segments with
ACL
• Access Control Lists (ACL) between VLANs
Destination VLAN 10 VLAN 20 VLAN 30 VLAN 40
Source
Tenant-A

VLAN 10 SVI 10
VLAN 10 ✔ ✔ ✘ ✘
VLAN 20 ✔ ✔ ✘ ✘
VLAN 20 SVI 20
VLAN 30 ✘ ✘ ✔ ✔

VLAN 30 SVI 30
✘ VLAN 40 ✘ ✘ ✔ ✔
• Number and complexity of ACLs becomes too
VLAN 40 SVI 40 high
Tenant-B • No overlapping IP subnets between tenants
Default VRF

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
Routing Domain – VRF

• Virtual Routing and Forwarding (VRF)

• Independent IPv4 and IPv6 address spaces

• Full unicast and multicast routing protocol

VRF-A support
VLAN 10 SVI 10
• Two VRFs by default: Mgmt VRF and
VLAN 20 SVI 20 Default VRF
• All IP-based features in NX-OS are VRF
aware
VRF-B • Non-default VRFs are locally-significant on
VLAN 30 SVI 30 a router

VLAN 40 SVI 40 • Data traffic is not routed across VRFs with

the default configuration

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 29
Question: How do we bring L2
and L3 separation together on a
device and within a fabric?

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 30
Agenda
• Introduction
• Multi-Tenancy Functionality in Enterprise Data Center
• Data Plane and Control Plane Considerations with VXLAN EVPN
• Data plane
• Control plane: Underlay and Overlay

• Layer 4-7 Services Integration

• Other useful Things?
• Fabric Provisioning and Management
• Conclusion

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 31
What is Data Plane and Control Plane?

• Data plane – everything related to forwarding of the actual data: MAC address
tables, routing tables, ARP/ND tables, port and fabric buffers, frame/packet/header
formatting etc.
• Control plane – everything related to populating and managing above mentioned
tables. For example:
• SpanningTree protocols build loop-free switched networks
• OSPF or MP-BGP protocols populate and distribute routing information

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 32
What is a Fabric?

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 33
What is a Fabric?
Spine

L3 Leaf
L2
VM VM Physical
Hosts OS OS

Virtual

*Clos, Charles (Mar 1953). "A study of non-blocking switching networks". Bell System Technical Journal.

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 34
Data Plane
Choice of Data Planes

• IEEE 802.1Q, or simply Dot1Q VLAN

• FabricPath + Dot1Q
Fabric
• FabricPath + Segment-ID Control

• VXLAN
• MPLS Functionality

Mechanism

Identifier

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 36
Classic Ethernet IEEE 802.1Q Format

Classic Ethernet
• 12 bit namespace provides 4096 Frame
DMAC SMAC 802.1Q Etype Payload CRC
unique VLAN IDs
• Data-Plane based learning, also
known as Flood & Learn Destination MAC (DMAC)

Source MAC (SMAC)

TPID TCI
4 bytes 802.1Q 0x8100 PCP CFI VID
(16 bits) (3 bits) (1 bits) (12 bits)

Ether Type (Etype)

Data (Payload)

CRC/FCS

VLAN ID
12 bits

TPID = Tag Protocol Identifier, TCI = Tag Control Information, PCP = Priority Code
Point,
CFI = Canonical Format Indicator, VID = VLAN Identifier
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 37
Dot1Q Use Cases

• VLAN ID identifies the Layer-2 segment

• VLAN ID maps to the SVI that typically provides Default GW functionality
• Dot1q on the Sub-Interface typically identifies the VRF on the link between two
routers

VLAN 10  VRF-A
VLAN 20  VRF-B
VLAN 30  VRF-C
VRF-A VRF-A

VRF-B VRF-lite VRF-B

VRF-C VRF-C

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
VXLAN Taxonomy (1)

Edge Device
Edge Device

Local LAN
Local LAN Segment
IP Interface
Segment

Physical
Host Physical
Edge Device
Local LAN Host
Segment

Virtual Switch

Virtual Hosts

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 40
VXLAN Taxonomy (2)

VTEP
VTEP

V V
Local LAN
Local LAN Segment
Segment Encapsulation

Physical
Host VTEP V Physical
Local LAN Host
Segment

Virtual Switch
VTEP – VXLAN Tunnel End-Point
VNI/VNID – VXLAN Network Identifier
Virtual Hosts

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 41
How is this different from
STP/802.1Q based
deployments?

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 42
Add a Control Plane as a Secret
Sauce!

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 43
Let’s Pick the
Control Plane
Standards Based Control-Plane for Fabrics

• VXLAN with MP-BGP EVPN

• Nexus 9300/9500 can be Leaf, Spine, and Border Leaf/Border Gateway.
• N5600 and N7000/7700 with F3/M3 for Spine, Leaf, and Border Leaf*
• N3000*

*Check with your account team/partner for support matrix

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 45
What Makes Control-Plane based Fabrics so
Special?
• Underlay Control plane
• Discover and learn the fabric topology, i.e. location of fabric
nodes
Fabric
• Overlay Control plane Control
• Learn and distribute the end-host reachability information

• ARP suppression and Distributed Anycast

Functionality
Gateway

Mechanism

Identifier

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 46
Underlay Control-
Plane
Underlay Control-Plane for VXLAN with EVPN

• IGP such as OSPF or Layer-3 IS-IS or eBGP can be used:

• Full fabric topology view
• Shortest path unicast forwarding between leafs
• VTEPs IP reachability (typically routed loopback interfaces)

• Recommended choice is an IGP (OSPF or L3-ISIS)

• PIM-ASM or PIM-BiDir* for multicast underlay or Ingress Replication*
• VTEPs use this mechanism to forward BUM** traffic

**BUM – Broadcast, Unknown Unicast, Multicast (*depends on hardware)

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 48
Overlay Control-
Plane
Overlay Control-Plane
End-Host Reachability Information Distribution

RR RR

Fabric Host/Subnet External Subnet

Route Injection Route Injection
MP-BGP Control Plane
N1KV/OVS

MAN/WAN

• Use MP-BGP on the leaf nodes to distribute the end-host reachability information
iBGP Adjacencies

Note: Route-Reflectors deployed for scaling purposes

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 50
Overlay Control-Plane for VXLAN with EVPN

• EVPN address family

• Layer-2 MAC and Layer-3 IP information distribution by Control-Plane (BGP)
• Forwarding is done in hardware, based on Control-Plane learnings (minimises flooding)

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 51
The Old Fashioned Way to Configure Default
Gateway
• SVIs for Layer-2 segments
configured on all Leaf nodes
• Full sync of ARP & MAC states of all
VLANs across the Network
• Flooding to ALL nodes in the network
• Source and Destination VLAN has
to exist on Switch where routing
happens
• Unnecessary waste of resources

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 55
The Scoped Configuration with Distributed
Gateway
• Logical Configuration only instantiated at
respective Leaf (scoped)
• ARP & MAC state only for local hosted
VLAN/Segment-ID and SVI
• Flooding only to respective Leafs (where
VLAN/Segment-ID is instantiated)
• Host demanded provisioning; two models
available
• top-down Orchestration, push to Leaf
• bottom-up Orchestration, pull by Leaf

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 56
Data is Routed via Transit Segment

• Configured on all switches hosting VRF

• Additional Segment for Routing traffic (per VRF)
• From Host A via VLAN-43 routed to Segment “pink”
reaching destination VLAN-55
• From Host Y via VLAN-55 routed to Segment “pink”
reaching destination VLAN-43
• Used in Cisco VXLAN/EVPN and FabricPath with MP-
BGP

Host Y
VLAN 55
Host A
VLAN 43
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 57
VNI and VLAN IDs

• VNIDs are utilised for providing isolation at

Layer-2 and Layer-3 across the Fabric
• 802.1Q tagged frames received at the Segment ID
Leaf nodes from edge devices must be
mapped to specific Segments
• The VLAN-to-Segment mapping is
performed on a leaf device level
• VLANs become locally significant on the
N1KV/OVS Virtual Switch
leaf node and 1:1 mapped to a Segment-
ID
• VNIDs are globally significant, VLAN IDs
are locally significant VLAN VLAN
VLAN

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 58
Let’s Sum It Up
Building Blocks for Multi-Tenancy in EVPN
Fabrics

L3- L2-
VRF VLAN

L3- L2-
VNI VNI

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 60
Layer-2 Multi-Tenancy

• Two different Interface “Mode VLAN

of Operation”
ethernet VLAN VNI vxlan
• Switch level multi-tenancy
• VLAN to Segment ID mapping (4K Multi-Tenancy (MT-Switch)
vlans per switch)
• With VLAN we can achieve per
port significance
Bridge-
• Port level multi-tenancy Domain
• Leverages Virtual Services
Instance (VSI) approach ethernet VLAN VNI vxlan

• Use of 4K VLANs per port Multi-Tenancy (MT-Port)

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 61
L3 Separation – VRF

• Independent IPv4 and IPv6 address spaces

• Tenant Networks (VNIs) are mapped to VRFs

VRF-A • Configuration is consistent across all Switches

VNI10000 SVI 10 • Data traffic is not routed across VRFs, so L3 and
L2 Separation is ensured
VNI 20000 SVI 20

VRF-B
VNI 30000 SVI 30

VNI 40000 SVI 40

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 62
Multi-Tenancy ”Layers”

Fabric
Control

- Layer-2 Network segmentation

Functionality - Layer-3 domain/Tenant separation

Mechanism MP-BGP, EVPN, Anycast GW

Identifier VLAN ID, VNID

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 63
Agenda

• Introduction
• Multi-Tenancy Functionality in Enterprise Data Center
• Data Plane and Control Plane Considerations with VXLAN EVPN
• Layer 4-7 Services Integration
• Types of Service Deployment
• How to attach Services Nodes?
• Service Node Deployment with VXLAN EVPN

• Other useful Things?

• Fabric Provisioning and Management
• Conclusion

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 64
Layer 4-7
Services
Integration
Types of Service
Deployment
Prerequisites for Connecting Services

• In DC environments, Services may typically work in one of two modes:

• Transparent, also called Layer 2 ( also known as GO THROUGH)
• Routed, also called Layer 3 (also known as GO TO)
• Subnet default gateway configured on the firewall (most popular option)
• Subnet default gateway configured in the network and firewall is the routed next hop

• This will affect what network configurations are deployed in the fabric
• Be sure to define upfront the role of the service node (policy
enforcement intra-tenant, inter-tenant, etc.)

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 67
Intra-Tenant Services

• Filtering/policy enforcement between and within Segments for a Tenants

 Intra-VRF, inter-subnets

Apply PBR
Tenant-A Tenant-B
FW as default FW as
gateway VLAN 10 VLAN 40 L3 hop

VLAN 20 VLAN 30

Apply PBR

Option 1 : FW as default GW
Option 2 : PBR with FW as L3 hop
Option 3 : FW in transparent (less common)

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 68
Inter-Tenants Services

• Filtering/policy enforcement between Tenants

 Inter-VRF
Per tenant physical Separate ‘fusion
FW as ‘fusion router’, FW or virtual context routing’ function
interface dedicated per
tenant

Tenant-A Tenant-B Tenant-A Tenant-B

VLAN 10 VLAN 40 VLAN 10 VLAN 40

VLAN 20 VLAN 30 VLAN 20 VLAN 30

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 69
Tenant Edge Services
Filtering for North-South Communication
• Filtering/policy enforcement between Tenants and the external
world
Internet/
Per tenant physical WAN
FW or virtual context

Tenant-A Tenant-B

VLAN 10 VLAN 40
Tenant as a security zone:
allows intra-tenant
communication
VLAN 20 VLAN 30

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 70
Let’s Translate

• Tenant Edge Services  Inter VRF

• Inter Tenant Services  Inter VRF
• Intra Tenant Services  Intra VRF/Inter-VLAN

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 71
How to Attach
Services Nodes?
Service Node Deployment Examples
Cluster Active/Stan
dby
M0/0

Management Network
Primary
M0/0

Inside Outside
M0/0

Secondary

M0/0

Cluster
Control Links

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 73
How to Physically Connect Service Nodes

Fabric Fabric
BGP AS#100 BGP AS#100

Cluster Active/Standby
For clustered systems vPC is OK For Active/Standby systems vPC
(Cluster nodes need to be attached to the same vPC is NOT a recommended choice
pair) (no Multicast routing via vPC, no IPv6, etc.)

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 74
Where and How to Connect Services Nodes
• Border Leaf
• Inter-VRF
SL Service Leaf
• External RR RR

L Leaf
Fabric
• Service Leaf BGP AS#100 BL Border Leaf
• Intra-VRF, Inter-VLAN

• Leaf BL1 L1 L2 SL1

• Not recommended

H1 H2
Internet/ 10.10.10.20
(VLAN 40)
20.10.10.20
(VLAN 111)
WAN VRF-A VRF-B

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 75
Logical
Connectivity to an
EVPN Fabric
Transparent Insertion of Active/Standby Services

BL BL

• IGP
• 2 SVIs per VRF for peering to
11 12

upstream Router 10 11 10 13 13 12

• BGP
• Needs only a single SVI per VRF Failover

State

• Backup links between the standby

service used in case of switchover
to keep routing adjacencies up

Backup link
Primary link

BL Border Leaf
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 77
Routed Insertion of Active/Standby Services

BL BL

• 1 SVI per VRF for peering to ASA 11 12

10 11 10 13 13 12

• If switchover from A->S, VMAC is used

(same IP/MAC and will be switched
over) Failover

State

• Backup links between the standby

service used in case of switchover to
keep routing adjacencies up

Backup link
Primary link

BL Border Leaf
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 79
Routed Insertion of Clustered Services

BL vPC Domain
BL

• 1 SVI per VRF for peering to Core

11 12

10 11 10

•
13

ClusterLink can be a direct connection or

13 12

can be transported across the fabric

• If dynamic routing protocols are used, ClusterLink

consider L3 via vPC best practices

Backup link
Primary link

BL Border Leaf
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 81
Transparent Insertion of Clustered Services

vPC Domain
BL BL

• 1 SVIs per VRF for peering to Core

11 12

10 11 10

•
13

ClusterLink can be a direct connection or

13 12

can be transported across the fabric

ClusterLink

Backup link
Primary link

BL Border Leaf
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 82
External Virtual
Router/Service
Attachment
External Virtual Router/Service Attachment to the
Fabric

• Virtual router or virtual service node deployed on

Fabric AS#100
an hypervisor may need to establish L3 peering
with the fabric over vPC
BL BL

L3 • IPv4/IPv6 Layer 3 peering between leaf nodes and

L2 virtual router/FW is supported with the following
considerations:
 Peering can be established with unique SVI addresses on the leaf
nodes only for non-VXLAN VLANs
 For VXLAN VLANs, direct peering from the virtual router to to
VTEPs’ anycast GW IP address is not supported
 The recommendation is to configure a loopback in tenant VRF on
Hypervisor each VTEP for establishing the BGP peering with the virtual node

vpc1 vpc2

vlan10 vlan10
vlan 10 vlan 10
vn-segment 30010 vn-segment 30010

interface Vlan10 interface Vlan10

no shutdown no shutdown
vrf member VRF-A vrf member VRF-A
ip address 192.168.10.1/24 tag 12345 ip address 192.168.10.1/24 tag 12345
fabric forwarding mode anycast-gateway fabric forwarding mode anycast-gateway

router bgp 65501 router bgp 65501

vrf VRF-A vrf VRF-A
address-family ipv4 unicast address-family ipv4 unicast
neighbor 192.168.10.0/24 neighbor 192.168.10.0/24
remote-as 65502 remote-as 65502
update-source VLAN 10 update-source VLAN 10
address-family ipv4 unicast address-family ipv4 unicast

interface bond0
ip address 192.168.10.100/24

router bgp 65502

address-family ipv4 unicast

Not Supported neighbor 192.168.10.1

remote-as 65501

© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 85
External Virtual Node Attachment to the Fabric
vlan 10 vlan 10
Lo10 Lo10
vn-segment 30010 vn-segment 30010
vlan 3967 10.10.10.11/32 10.10.10.12/32 vlan 3967

system nve infra-vlans 3967 vpc1 vpc2 system nve infra-vlans 3967

interface loopback10 interface loopback10

no shutdown no shutdown
vrf member VRF-A .1 .1 vrf member VRF-A
ip address 10.10.10.11/32 tag 12345 VLAN10 VLAN10 ip address 10.10.10.12/32 tag 12345
192.168.10.0/24 192.168.10.0/24
interface Vlan10 interface Vlan10
no shutdown no shutdown
vrf member VRF-A vrf member VRF-A
ip address 192.168.10.1/24 tag 12345 ip address 192.168.10.1/24 tag 12345
fabric forwarding mode anycast-gateway fabric forwarding mode anycast-gateway

interface vlan 3967 interface vlan 3967

no shutdown no shutdown
vrf member VRF-A vrf member VRF-A
ip address 10.10.0.1/30 tag 12345 bond0 ip address 10.10.0.2/30 tag 12345
.100
router bgp 65501 router bgp 65501
vrf VRF-A vrf VRF-A
address-family ipv4 unicast interface bond0 address-family ipv4 unicast
neighbor 192.168.10.0/24 ip address 192.168.10.100/24 neighbor 192.168.10.0/24
remote-as 65502 remote-as 65502
ebgp-multihop 5 ip route 10.10.10.11/24 192.168.10.1 ebgp-multihop 5
update-source loopback 10 ip route 10.10.10.12/24 192.168.10.1 update-source loopback 10
address-family ipv4 unicast address-family ipv4 unicast
neighbor 10.10.0.2 router bgp 65502 neighbor 10.10.0.1
remote-as 65501 address-family ipv4 unicast remote-as 65501
update-source VLAN 3967 neighbor 10.10.10.11 update-source VLAN 3967
address-family ipv4 unicast remote-as 65501 address-family ipv4 unicast
next-hop-self ebgp-multihop 5 next-hop-self
update-source loopback 10
address-family ipv4 unicast

Supported
neighbor 10.10.10.12
remote-as 65501
ebgp-multihop 5
update-source loopback 10 86
External Virtual Node Attachment to the Fabric
Virtual Nodes Connected to Separate Leaf Pairs

Spine Spine • Active/Standby virtual FW pair connected to

separate leaf node pairs
• For minimizing the traffic outage after a FW
failover event, the active virtual FW should
Lo0
VTEP VTEP
Lo0
VTEP
Lo0
VTEP VTEP
Lo0 peer with local and remote leaf nodes
• After failover, there is no need to re-establish
EBGP sessions between the virtual FW and
the fabric
Active Standby

© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 87
External Virtual Node Attachment to the Fabric
Virtual Nodes Connected to Separate Leaf Pairs

Spine Spine • Active/Standby virtual FW pair connected to

© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 88
EVPN External
Connectivity
How to Integrate Service
3 Ways to route to or via a
Service:
• Dynamic/Static Routing RR RR
SL Service Leaf
• Recursive Next Hop (RNH) L
Leaf
Fabric
• Host Mobility Manager Route
(HMM) Tracking
BGP AS#100 BL
Border Leaf

BL1 L1 L2 SL1

H1 H2
Internet/ 10.10.10.20
(VLAN 40)
20.10.10.20
(VLAN 111)
WAN VRF-A VRF-B

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 90
VXLAN/EVPN Fabric External Routing

• The Border Leaf/Spine provides

Layer-2 and Layer-3 connectivity
to external networks
• Flexible routing protocol options V2
for external routing
• Today, VRF-lite allows to extend V3
VRFs outside of the fabric VBL
V1

WAN

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 91
Dynamic/Static Routing
Firewall establishes routing adjacencies with both
the Border Leaf and the Edge Router

• Per-VRF routing adjacency or static routes RR RR

between Border Leaf and Firewall

• Per-VRF routing adjacency or static routes Fabric
BGP AS#100
between Firewall and Edge Router

Routes are summarised or only a default route is

injected into the fabric on a per-VRF basis BL1 L1 L2

OSPF VRF-A

Internet/ H1
10.10.10.20
H2
20.10.10.20
WAN OSPF VRF-B
(VLAN 40)
VRF-A
(VLAN 111)
VRF-B

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 92
VXLAN/EVPN Fabric External Routing

VRFs for External Routing

need to exist on Border Leaf
VBL
VRF VRF VRF
A B C V2
V3
Interface-Type Options:
• Physical Routed Ports V1
• Sub-Interfaces
• VLAN SVIs over Trunk Ports Peering Interface can
be in Global or Tenant VRF

WAN

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 93
VXLAN/EVPN Fabric External Routing (eBGP)

# Sub-Interface Configuration
interface Ethernet1/1
no switchport
interface Ethernet1/1.10
encapsulation dot1q 10
VBL
VRF VRF VRF V2
vrf member VRF-A A B C
ip address 10.254.254.1/30
V3
# eBGP Configuration
router bgp 100
… Advertise external learned routes
vrf VRF-A
into EVPN (Route-Type 5)
V1
address-family ipv4 unicast
advertise l2vpn evpn
aggregate-address 10.0.0.0/8 summary-only Advertise an aggregate of the internal
neighbor 10.254.254.2 remote-as 65599 prefixes
update-source Ethernet1/1.10
address-family ipv4 unicast
WAN
Ensure that non-necessary routes are not advertised
AS# 65599
towards the External Network

# Interface Configuration

VBL
interface Ethernet1/1.10
encapsulation dot1q 10
vrf member VRF-A VRF VRF VRF V2
ip address 10.254.254.2/30 A B C

# eBGP Configuration V3
router bgp 65599
…
vrf VRF-A
address-family ipv4 unicast
neighbor 10.254.254.1 remote-as 100
V1
update-source Ethernet1/1.10
address-family ipv4 unicast

WAN
AS# 65599

© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 95
VXLAN/EVPN Fabric External Routing (OSPF)
# Sub-Interface Configuration
interface Ethernet1/1
no switchport
interface Ethernet1/1.10
encapsulation dot1q 10
vrf member VRF-A
ip address 10.254.254.1/30 VBL
VRF VRF VRF V2
ip router ospf 1 area 0.0.0.0
A B C
ip ospf network point-to-point

# BGP Configuration V3
router bgp 100
… Advertise external learned routes
vrf VRF-A
address-family ipv4 unicast into EVPN (Route-Type 5) V1
advertise l2vpn evpn
redistribute bgp 100 route-map OSPF-BGP* Redistribute internal prefixes with route-map

WAN
*Ensure that non-necessary routes are not advertised
towards the External Network

© 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 96
What about Static
Routes
Check Availability of Static Routes Next Hop

• Problem with Redistributing Static Routes

• What happens if the Next Hop goes down?
• How to deploy this redundant?

• 2 Solutions
• Recursive Next Hop (RNH)
• Host Mobility Manager Tracking (HMM Tracking)

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 98
Recursive Next Hop (RNH)

BL1# Show ip route vrf VRF-B 20.20.10.20

20.20.10.20/32, ubest/mbest: 1/0, attached
*via 20.20.10.20, Vlan1020, [190/0], 08:40:59, hmm RR RR

Fabric
BGP AS#100

VTEP
10.10.10.21 BL1 L1 L2

99.99.99.0/2
99.99.99.0/24
Internet/ H1
10.10.10.20
H2
20.10.10.20
4
WAN VRF-B
20.20.10.20
(VLAN 40)
VRF-A
(VLAN 111)
VRF-B

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 99
Recursive Next Hop (RNH)

BL1# Show ip route vrf VRF-B 20.20.10.20

20.20.10.20/32, ubest/mbest: 1/0, attached
L2# sh ip route vrf VRF-B 20.20.10.20
*via 20.20.10.20, Vlan1020, [190/0], 08:40:59, hmm
IP Route Table for VRF ”VRF-B" RR RR

'*' denotes best ucast next-hop

'**' denotes best mcast next-hop Fabric
'[x/y]' denotes [preference/metric] BGP AS#100
'%<string>' in via output denotes VRF <string>

20.20.10.20/32, ubest/mbest: 1/0 VTEP

BL1 L1 L2
10.10.10.21
*via 10.10.10.21%default, [200/0], 08:39:50, bgp-100, internal, tag 100
(evpn) segid: 50001 tunnelid: 0x1afb00c9 encap: VXLAN

99.99.99.0/2
99.99.99.0/24
Internet/ H1
10.10.10.20
H2
20.10.10.20
4
WAN VRF-B
20.20.10.20
(VLAN 40)
VRF-A
(VLAN 111)
VRF-B

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 100
Recursive Next Hop (RNH)

BL1# Show ip route vrf VRF-B 20.20.10.20

20.20.10.20/32, ubest/mbest: 1/0, attached
L2# sh ip route vrf VRF-B 20.20.10.20
*via 20.20.10.20, Vlan1020, [190/0], 08:40:59, hmm
IP Route Table for VRF ”VRF-B" RR RR

'*' L2#sh
denotes ip best
routeucast
vrf VRF-B 99.99.99.0
next-hop
'**'IPdenotes
Route Table
best for VRFnext-hop
mcast ”VRF-B"
Fabric
'*' denotes best ucast next-hop
'[x/y]' denotes [preference/metric] BGP AS#100
'**' denotes
'%<string>' best
in via mcast
output next-hop
denotes VRF <string>
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
20.20.10.20/32, ubest/mbest: 1/0 VTEP
BL1 L1 L2
10.10.10.21
*via 10.10.10.21%default, [200/0], 08:39:50, bgp-100, internal, tag 100
99.99.99.0/24,
(evpn) segid: 50001 ubest/mbest: 1/0
tunnelid: 0x1afb00c9 encap: VXLAN
*via 20.20.10.20, [1/0], 00:00:11, static segid: 50001 tunnelid: 0x1afb00c9 e
ncap: VXLAN

99.99.99.0/2
99.99.99.0/24
Internet/ H1
10.10.10.20
H2
20.10.10.20
4
WAN VRF-B
20.20.10.20
(VLAN 40)
VRF-A
(VLAN 111)
VRF-B

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 101
HMM Tracking

BL1# Show ip route vrf VRF-B 20.20.10.20

20.20.10.20/32, ubest/mbest: 1/0, attached
*via 20.20.10.20, Vlan1020, [190/0], 08:40:59, hmm RR RR

Fabric
BGP AS#100

VTEP
10.10.10.21 BL1 L1 L2

99.99.99.0/2
99.99.99.0/24
Internet/ H1
10.10.10.20
H2
20.10.10.20
4
WAN VRF-B
20.20.10.20
(VLAN 40)
VRF-A
(VLAN 111)
VRF-B

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 102
HMM Tracking

BL1# Show ip route vrf VRF-B 20.20.10.20

BL1#20.20.10.20/32,
sh track ubest/mbest: 1/0, attached
*via 20.20.10.20,
Track 2 Vlan1020, [190/0], 08:40:59, hmm RR RR
IP Route 20.20.10.20 Reachability
Reachability is UP
3 changes, last change 08:40:33 Fabric
BGP AS#100
VPN Routing/Forwarding table ”VRF-B"

VTEP
10.10.10.21 BL1 L1 L2

99.99.99.0/2
99.99.99.0/24
Internet/ H1
10.10.10.20
H2
20.10.10.20
4
WAN VRF-B
20.20.10.20
(VLAN 40)
VRF-A
(VLAN 111)
VRF-B

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 103
HMM Tracking

BL1# Show ip route vrf VRF-B 20.20.10.20

BL1#20.20.10.20/32,
sh track ubest/mbest: 1/0, attached
*via 20.20.10.20,
Track Vlan1020, [190/0], 08:40:59, hmm
BL1#2 RR RR
IP Route 20.20.10.20 Reachability
Reachability is UP
version 7.0(3)I5(2)
3track
changes, Fabric
2 ip routechange
last 08:40:33
20.20.10.20 reachability hmm BGP AS#100
VPN Routing/Forwarding
vrf member VRF-B table ”VRF-B"

vrf context VRF-B

vni 50001 VTEP
BL1 L1 L2
10.10.10.21
ip route 99.99.99.0/0 20.20.10.20 track 2 tag 12345

Redistribute static route into BGP

99.99.99.0/2
99.99.99.0/24
Internet/ H1
10.10.10.20
H2
20.10.10.20
4
WAN VRF-B
20.20.10.20
(VLAN 40)
VRF-A
(VLAN 111)
VRF-B

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 104
Policy Based
Routing for EVPN
Policy-Based Routing with VXLAN

• Redirect Layer-3 Traffic based

on 5-tuple

BorderLeaf • Only applicable to routed Traffic

• Service Redirection to Load-
Spine Balancers and Firewalls
• PBR policy needs to be applied
to all leaves, to ensure symmetric
ComputeLeaf traffic flows

Tenant VMs / servers Tenant VMs / servers

(App group 1/subnet1) (App group 2/subnet2)

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 106
PBR Support for the VXLAN BGP EVPN Fabric
feature pbr

ipv6 access-list bummy

statistics per-entry
10 permit ipv6 2001:10:1:1::20/128 any
ip access-list dummy
statistics per-entry
PBR rules on Leaf – L3VNI 10 permit ip 10.1.1.20/32 any
Redirect to FW
route-map bummy permit 10
BLeaf match ipv6 address bummy
set ipv6 next-hop 2001::DB8:800:200C:417A <== next-hop host behind some
intermediate VTEP
route-map dummy permit 10
match ip address dummy
set ip next-hop 10.1.1.40 <== next-hop host behind some intermediate VTEP
Spine
interface Vlan10
ip policy route-map dummy
ipv6 policy route-map bummy
L3 VXLAN
Leaf

Tenant VMs / servers Tenant VMs / servers

(App group 1/subnet1) (App group 2/subnet2)

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 107
PBR Support for the VXLAN BGP EVPN Fabric
feature pbr

ipv6 access-list bummy

statistics per-entry
10 permit ipv6 2001:10:1:1::20/128 any
PBR rules on Leaf – L3VNI ip access-list dummy
Redirect to FW
statistics per-entry
BLeaf 10 permit ip 10.1.1.20/32 any

route-map bummy permit 10

match ipv6 address bummy
set ipv6 next-hop 2001::DB8:800:200C:417A <== next-hop host behind some
intermediate VTEP
Spine route-map dummy permit 10
match ip address dummy
set ip next-hop 10.1.1.40 <== next-hop host behind some intermediate VTEP

interface Vlan2500
ip L3 VXLANroute-map dummy
policy
Leaf ipv6 policy route-map bummy

Tenant VMs / servers Tenant VMs / servers

(App group 1/subnet1) (App group 2/subnet2)

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 108
Service Node
Deployment in
EVPN
Inter-VRF Scenario
Inter-VRF Firewall: Transparent Mode

• Transparent Firewall is inserted inline between the Border

Leaf and the Edge-Router. We use VRF-Lite between RR RR
Border Leaf and Edge-Router

• Inside link between the Border Leaf and Firewall is a Fabric

Dot1Q trunk. Each VLAN corresponds to a protected VRF BGP AS#100
• Outside link between the Edge Router and Firewall is also
a Dot1Q trunk. Each VLAN corresponding to a protected
VRF
BL1 L1 L2
• Things to keep in mind:
• Some Firewalls may use different VLAN ID# between VLAN 10: VRF-A
Inside and Outside interfaces, so configuration is required VLAN 20: VRF-B
…………………..
on Border Leaf or Edge Router to establish routing VLAN N: VRF-X
adjacency
Internet/ H1
10.10.10.20
H2
20.10.10.20
WAN (VLAN 40)
VRF-A
(VLAN 111)
VRF-B

= Spine = Leaf = BorderLeaf = Fabric RR = Route-Reflector = Edge-Router / DCI-

Interface Device
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 111
Inter-VRF Firewall: Transparent Mode

• In EVPN by default Border Leaf does not allow routing between RR RR

different VRFs. This needs to be done on the upstream Edge router.
For unprotected inter VRF communication this will be added in an
upcoming SW release
Fabric
• Network administrator can configure the Firewall to filter only a subset BGP AS#100
of VRFs. Border Leaf and Edge router use SVIs and per-VRF routing
adjacency (with eBGP)

• The bandwidth and performance of the inline deployed Firewall

defines the overall performance of the protected VRF BL1 L1 L2

• SVI is the recommended mechanism of peering, due to FW

HA/Clustering requirements. If SVIs are used, no Anycast Gateway Routing Adjacencies for
configuration for these SVIs Unprotected VRFs

• BGP “Local AS” configuration is needed to prevent BGP from

dropping traffic
H1 H2
• Data traffic for unprotected VRFs, will use direct links between the 10.10.10.20 20.10.10.20
Border Leaf and Edge Router bypassing the Firewall Internet/ (VLAN 40)
VRF-A
(VLAN 111)
VRF-B
• Edge Router performs inter-VRF routing WAN Routing Adjacencies for
Protected VRFs

= Spine = Leaf = BorderLeaf = Fabric RR = Route-Reflector = Edge-Router / DCI-

Interface Device
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 112
Inter-VRF Firewall: Transparent Mode

• H1 in VRF-A talks to H2 in VRF-B.

RR RR
• VRF-A is protected by the Tenant-Edge
Firewall, in Transparent mode
Fabric
BGP AS#100
• VRF-B is unprotected.
• Traffic filtering and Policies enforcement 4

occurs on step 4. Firewall acts as a

SIP: 10.10.10.20
DIP: 20.10.10.20 3
BL1 L1 L2
transparent Layer 2 bridge. VLAN 400
SMAC: BL1_MAC Outer-DIP: BL1
DMAC: CR1_MAC
Traffic from Edge router to VRF-B traverses
Outer-SIP: L1
• DMAC: BL1_MAC

through FW Policy SMAC: L1_MAC

VNI50001
a direct link between the Edge router and the Enforcement DIP: 20.10.10.20

Border Leaf
1
Internet/ eBGP for VRF-A
H1
SIP: 10.10.10.20
10.10.10.20
H2
20.10.10.20
WAN DMAC: G_MAC
SMAC: H1_MAC
(VLAN 40)
VRF-A
(VLAN 111)
VRF-B
VLAN 40

= Spine = Leaf = BorderLeaf = Fabric RR =DIP:

Route-Reflector
20.10.10.20 = Edge-Router / DCI-
Interface SIP: 10.10.10.20
Device / Core Router (CR)
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 113
Inter-VRF Firewall: Transparent Mode

• Depending on performance and scale

requirements, a Per-VRF Firewall RR RR

deployment model may be used.

These Firewalls don’t need to be Fabric
BGP AS#100
physical
• Each Firewall will filter traffic for a
specific set of VRFs, and will require a BL1 L1 L2

separate link between Border Leaf and

Edge-Router
FW for VRF-C

Internet/ FW for VRF-A

H1
10.10.10.20
H2
20.10.10.20
WAN (VLAN 40)
VRF-A
(VLAN 111)
VRF-B

= Spine = Leaf = BorderLeaf = Fabric RR = Route-Reflector = Edge-Router / DCI-

Interface Device
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 114
Inter-VRF Firewall: Transparent Mode
Redundant Deployment (Active Standby)
• Peering with upstream router is done
on a VRF basis via an SVI on each RR RR

Border Leaf traversing the Firewall

Fabric
• Each Border Leaf needs to have a link BGP AS#100

to Active and Standby FW to ensure

peering will stay up when FWs are
failing over BL1 BL2

FW Active

Internet/ FW Standby

WAN
= Spine = Leaf = BorderLeaf = Fabric RR = Route-Reflector = Edge-Router / DCI-
Interface Device
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 115
Inter-VRF Firewall: Transparent Mode
Redundant Deployment (Cluster)
• Peering is done on a VRF basis via an
SVI on each Border Leaf to FW RR RR

• Each Border Leaf needs to have a link Fabric

to Cluster member to ensure peering BGP AS#100

will stay up when FWs are failing over VPC Domain

• If dynamic routing protocols are used BL1 BL2

consider L3 via vPC best practices
ç

• When operating a Firewall in Routed mode,

additional configuration is required: RR RR

• Per-VRF IGP routing adjacency between Border

Leaf and Firewall Fabric
• Per-VRF IGP routing adjacency between Firewall BGP AS#100
and Edge Router

• Firewall establishes routing adjacencies with

BL1
both the Border Leaf and the Edge Router, L1 L2

which in comparison to Transparent mode of

operation requires twice as many subnets OSPF VRF-A

Internet/ H1
10.10.10.20
H2
20.10.10.20
WAN OSPF VRF-B
(VLAN 40)
VRF-A
(VLAN 111)
VRF-B

= Spine = Leaf = BorderLeaf = Fabric RR = Route-Reflector = Edge-Router / DCI-

Interface Device
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 117
Inter-VRF Firewall: Routed Mode
Redundant Deployment (Active Standby)
• Peering is done on a VRF basis via an
SVI on each Border Leaf to FW RR RR

• Each Border Leaf needs to have a link Fabric

to Active and Standby FW to ensure BGP AS#100

peering will stay up when FWs are

failing over
BL1 BL2

FW Active

Internet/ FW Standby

WAN
= Spine = Leaf = BorderLeaf = Fabric RR = Route-Reflector = Edge-Router / DCI-
Interface Device
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 118
Inter-VRF Firewall: Routed Mode
Redundant Deployment (Cluster)
• Peering is done on a VRF basis via an
SVI on each Border Leaf to FW RR RR

• Each Border Leaf needs to have a link Fabric

to Cluster member to ensure peering BGP AS#100

will stay up when FWs are failing over. VPC Domain

• If dynamic routing protocols are used BL1 BL2

consider L3 via vPC best practices

Internet/
WAN
= Spine = Leaf = BorderLeaf = Fabric RR = Route-Reflector = Edge-Router / DCI-
Interface Device
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 119
Intra-VRF Scenario
Intra-VRF, Inter-VLAN Firewall: Transparent
Mode
• Inter-VLAN traffic flows for a single VRF can RR RR
also be filtered by the Firewall
• These secured VLANs are deployed in L2 Fabric
forwarding mode, i.e. Layer 2 Profile is BGP AS#100
applied to these VLANs on Leaf nodes
• With Firewall in transparent mode, Border
BL1
Leaf or Service Leaf becomes the VLAN L1 L2

termination point
• It is important to ensure that Firewall does
not propagate STP BPDUs, as the ports on
Leaf nodes should be configured with BPDU Internet/ H1 H2
10.10.10.20 20.10.10.20
Guard
WAN (VLAN 40)
VRF-A
(VLAN 111)
VRF-A

= Spine = Leaf = BorderLeaf = Fabric RR = Route-Reflector = Edge-Router / DCI-

Interface Device
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 121
Intra-VRF, Inter-VLAN Firewall: Transparent
Mode
• Inside link between the Services Leaf and RR RR
Firewall is a Dot1Q trunk. Each VLAN
corresponds to a protected subnet
Fabric
• Outside link between the Services Leaf and BGP AS#100
Firewall is also a Dot1Q trunk. Firewall in
Transparent mode acts as a Layer 2 switch,
bridging respective VLANs SL1 L1 L2

• On Services Leaf these VLANs are

terminated with SVIs, which are assigned to
respective VRFs. These SVIs are the default
gateways for the hosts
H1 H2
10.10.10.20 20.10.10.20
• Subsequent subnet prefixes are advertised (VLAN 40) (VLAN 111)
via BGP into the Fabric VRF-A VRF-A

= Spine = Leaf = BorderLeaf = Fabric RR = Route-Reflector = Edge-Router / DCI-

Interface Device
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 122
Intra-VRF, Inter-VLAN Firewall: Transparent
Mode
• In our example VNIs corresponding to VLAN 40 and VLAN 111 RR
are protected by Firewall and are deployed in Layer-2 mode
across the fabric.
Layer 2 trunk
• SVIs on SL1 act as Default Gateways for VLAN40 and
VLAN111

• Traffic is flowing from H1 to H3

L1 SL1
• First, Ethernet frame is switched by EVPN from L1 to SL1 and 1
then sent to Firewall via Inside DOT1Q trunk (using say vlan 41) 32
DIP: SL1
• After traffic passes through Firewall policies and filters, it is SIP: L1 SIP: DMAC:
10.10.10.20
DIP: L1
V40_G_MAC
received on a SVI for VLAN 40 on SL1 DMAC: V40_G_MAC DIP:SIP:
SMAC:30.10.10.20
H1_MAC
SL1
SMAC: H1_MAC VLAN
VLAN
DMAC: 222
41/40
L1_MAC
• SL1 routes traffic to H3 and forwards it to L1 using the VRF-A VNI 30001
H1 H2
SMAC:
DIP:
SMAC: L1_MAC
30.10.10.20
SL1_MAC
H3 H3_MAC
DMAC:
VNI, where L1 locally forwards it to H3 in VLAN222 DIP: 30.10.10.20
10.10.10.20 20.10.10.20
SIP: 10.10.10.20
VNI 50001
30.10.10.20
SIP: 10.10.10.20 DIP: 30.10.10.20
(VLAN 40) (VLAN 111) (VLAN 222)
• Depending on Transparent Firewall capabilities, inside and VRF-A VRF-A SIP: 10.10.10.20
VRF-A
outside logical interfaces may be bound to the same or
different VLAN

= Spine = Leaf = BorderLeaf = Fabric RR = Route-Reflector = Edge-Router / DCI-

Interface Device
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 123
Intra-VRF, Inter-VLAN Firewall: Routed Mode
VLAN 40 and VLAN 111
are protected VLANs

Default Gateway for

• A Firewall in Routed mode can be connected to either a Border Leaf VLAN40 and VLAN111 serve
RR as a
or a Service Leaf
Default Gateways and
corresponding subnets are
• Inside link between the Service Leaf and Firewall is a .1Q trunk, where Layer 2 trunk advertised in IGP
each VLAN corresponds to a protected subnet (VLAN)

• Protected VLANs are deployed in L2 forwarding mode on a Service

Leaf and across the fabric

• Outside link is a layer 3 point-to-point link L1 SL1

• Firewall establishes an routing protocol ( OSPF or EIGRP or eBGP)

routing adjacency with the Service Leaf over an Outside link

• On Firewall, protected VLANs are terminated with a BVI (bridged

virtual interface) or its equivalent and are then advertised into IGP.
These BVIs serve as the Default Gateway for protected VLANs
H1 H2 H3
10.10.10.20 20.10.10.20 30.10.10.20
• On Service Leaf, prefixes received from Firewall via routing protocol (VLAN 40) (VLAN 111) (VLAN 222)
are re-distributed into BGP VRF-A VRF-A VRF-A

• Service Leaf advertises a 0.0.0.0/0 default route or specific routes

reachability to Firewall over an IGP

= Spine = Leaf = BorderLeaf = Fabric RR = Route-Reflector = Edge-Router / DCI-

Interface Device
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 124
Load Balancer Integration using One-Arm Mode

Load-Balancer
• Load Balancer can connect to any Leaf, Services Leaf or RR
a Border Leaf with Layer 3 point-to-point link

• Load Balancer establishes routing adjacency with the

Leaf node via IGP (OSPF/EIGRP) over this link and also
receives a default route from the Leaf node. VIP-X1: 50.10.10.100
mapped to
• Every configured VIP on Load Balancer is advertised into OSPF=>iBGP Server 1: 20.10.10.11
L1 SL1 Server 2: 20.10.10.12
IGP as a /32 prefix

• These /32 prefixes are learned and redistributed into

BGP on the Leaf node.

• On a sample diagram, H1 host is trying to retrieve HTTP

web page from web-server at virtual IP VIP-X1: H1 Server2
Server1
50.10.10.100 which is configured on a Load-balancer 10.10.10.20 20.10.10.11 20.10.10.12
(VLAN 40) (VLAN 111) (VLAN 111)
• Load Balancer retrieves necessary data from Server1 or VRF-A VRF-A VRF-A
Server 2

• Data is then returned to the H1 host

= Spine = Leaf = BorderLeaf = Fabric RR = Route-Reflector = Edge-Router / DCI-
Interface Device
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 125
Load Balancer Integration using Two-Arm Mode

• It differs from the One-Arm mode in RR Load-Balancer

the following:
• “Back-end” communication with the applications
servers happens over a dedicated “Services
Segment” link.
OSPF=>iBGP
• Load balancer is statically configured with the L1 SL1
default route through the second arm (Services
Segment) VIP-X1: 50.10.10.100
mapped to
Server 1: 20.10.10.11
Server 2: 20.10.10.12

H1 Server1 Server2
10.10.10.20 20.10.10.11 20.10.10.12
(VLAN 40) (VLAN 111) (VLAN 111)
VRF-A VRF-A VRF-A

= Spine = Leaf = BorderLeaf = Fabric RR = Route-Reflector = Edge-Router / DCI-

Interface Device
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 126
How to Attach
Services Nodes in
Multi-Pod or Multi-
Site Deployments?
Multi-Pod
iBGP eBGP iBGP
Spine Spine VXLAN Overlay

RR
EVPN
RR
VRF/VRFs Space
RR RR

DC #1 DC #2
EVPN iBGP Border Leaf Border Leaf EVPN iBGP

VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

Leaf Leaf

Inter-DC
EVPN eBGP

Global Default VRF

Or User Space VRFs

Separation of Control plane but End-to-End VxLAN Dataplane

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 128
Multi-Pod with Services
iBGP eBGP iBGP
Spine Spine VXLAN Overlay

RR
EVPN
RR
VRF/VRFs Space
RR RR

DC #1 DC #2
EVPN iBGP Border Leaf Border Leaf EVPN iBGP

VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

Leaf Leaf

Inter-DC
EVPN eBGP

Global Default VRF

Acitve FW Standby FW
Or User Space VRFs

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 129
Multi-Fabric

Spine
 Next hop Self Next hop Self 
Spine
VXLAN Overlay
RR
EVPN
RR
VRF/VRFs Space
RR RR
Inter-DC
eBGP
DC #1 DC #2
iBGP Border Leaf Border Leaf iBGP

VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

Leaf Leaf

VXLAN EVPN Administrative Domain #1 VLAN Hand-off

VXLAN EVPN Administrative Domain #2
OTV OTV OTV OTV

Failure DomainGlobal
Containment:
Default VRF

• Unknown Or User Space VRFs

Unicasts
• ARPs
• STP
OTV/VPLS Domain

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 130
Multi-Fabric with Services

Spine
 Next hop Self Next hop Self 
Spine
VXLAN Overlay
RR
EVPN
RR
VRF/VRFs Space
RR RR
Inter-DC
eBGP
DC #1 DC #2
iBGP Border Leaf Border Leaf iBGP

VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

Leaf Leaf

VLAN Hand-off
OTV OTV OTV OTV

Global Default VRF

Acitve FW Standby FW
Or User Space VRFs

OTV/VPLS Domain

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 131
Multi-Fabric with Services
Ensure Traffic symmetry going out of
and into fabric
Spine
 Next hop Self Next hop Self 
Spine
VXLAN Overlay
RR
EVPN
RR
VRF/VRFs Space
RR RR
Inter-DC
eBGP
DC #1 DC #2
iBGP Border Leaf Border Leaf iBGP

VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

Leaf Leaf

VLAN Hand-off
OTV OTV OTV OTV

Cluster with Cluster with

spanned split spanned split
Etherchannel Etherchannel

ARP and Cluster/IP

MAC needs to be OTV/VPLS Domain
filtered

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 132
Network Services
Integration Models
with VXLAN Multi-
Site
VXLAN Multi-Site and Network Services Integration
WAN

Active and Standby pair deployed across Sites,

Inter-Site
Active FW Network Standby FW 
VTEP VTEP VTEP VTEP

BGW BGW BGW BGW

enforcement for N-S and E-W flows
Spine VXLAN EVPNSpine Spine VXLAN EVPNSpine
Site1 Site2
 No issues with asymmetric flows
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

 Various options possible (FW as endpoints

gateway or fabric as endpoints gateway)
Bareme Baremet Bareme
tal al tal

Active FW Standby FW

WAN
 Independent Active/Standby pairs deployed in
Active/Standby Active/Standby separate Sites
Inter-Site
Network

FW FW
VTEP VTEP VTEP VTEP

BGW BGW BGW BGW

Spine VXLAN EVPN Spine Spine VXLAN EVPN Spine  Need to avoid the creation of asymmetric paths
VTEP
Site1
VTEP VTEP VTEP VTEP
Site2
VTEP VTEP VTEP
crossing different active FW nodes
Baremet Baremet
 Only possible for N-S flows with perimeter FWs and
host routes advertisement or with PBR
al al

Active/Standby FW Active/Standby FW

WAN

Inter-Site  Active/Active FW Cluster stretched across Sites

Network

Split spanned ether-channel mode: not supported,

VTEP VTEP VTEP VTEP

BGW BGW BGW BGW 

VXLAN EVPNSpine
Spine

Site1
Spine VXLAN EVPNSpine
Site2
 Individual mode: supported with Cisco ASA software
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP
and hardware for N-S and E-W flows

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 134
Active/Active FW Cluster Bareme
tal
Active/Standby
Pair across Sites
Active and Standby pair deployed across Sites

Inter-Site
DCI
Network

…. ….
VTEP Multi-Site VTEP VTEP Multi-Site VTEP
VIP1 VIP2
Fabric BGW BGW BGW BGW

VXLAN EVPN VXLAN EVPN

Spine Spine Spine Spine

Site1 Site2
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

Active Standby

• Active/Standby model can be applied per context (i.e. can be deemed as

‘active/active’ support across contexts)
• Different deployment models
• FW as default gateway for the endpoints peering with the fabric (via IGP or BGP)
• FW as default gateway for the endpoints using static routing
• FW as default gateway for the endpoints peering directly with the external routers (fabric as L2)
• Fabric as default gateway and use of a perimeter FW

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 137
FW as Default
Gateway Peering
with the Fabric
Active/Standby Pair across Sites 1
FW as Default Gateway Peering with the Fabric

Logical View
• FW allows to apply intra-tenant security
External L3 policies (east-west) and between an internal
Routing function Domain subnet and the external L3 domain (north-
of the VXLAN
EVPN Fabric south) or a subnet in a different tenant (inter-
tenant)
Site 1 Inter-Site Site 2
Network • FW inside network(s) deployed as L2-only can
be extended across sites to allow flexible
IGP/BGP
Peering deployment for endpoints
Active Standby • FW outside interface used to peer with the
fabric
• The active FW can only peer with the leaf node(s)
in the local fabric (on a L3 interface or regular SVI)
• No need to extend the FW outside BD across sites
BDs Extended
via Multi-Site
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 139
Active/Standby Pair across Sites 1
FW as Default Gateway Peering with the Fabric

Logical View

External L3
Routing function Domain • After an active FW failure, two sequential
of the VXLAN events must happen:
EVPN Fabric
1. The standby FW must detect the failure
Site 1 Inter-Site Site 2 event and take over the active role
Network
2. Routing adjacencies must be re-
IGP/BGP established with the fabric by the newly
Peering activated FW
X Active
• The overall recovery process could lead to
long traffic outage (15+ seconds)

• When using an IGP to peer with the FW,

Logical View
must ensure to tag routes distributed from
EVPN into the IGP to avoid redistributing
External L3 them back into the EVPN CP
Routing function Domain
route-map EVPN-to-OSPF permit 10
of the VXLAN match route-type internal
EVPN Fabric set tag 100
!
router ospf 1 EVPN-to-OSPF
Site 1 Inter-Site Site 2 vrf tenant-1
Network router-id 1.1.1.1
redistribute bgp 65501 route-map EVPN-to-OSPF

route-map OSPF-to-EVPN deny 5

EVPN-to-IGP Routes IGP-to-EVPN match tag 100
Redistribution Routes !
Redistribution Active router bgp 65501
router-id 10.1.0.3 OSPF-to-EVPN
vrf tenant-1
address-family ipv4 unicast
advertise l2vpn evpn
redistribute ospf 1 route-map OSPF-to-EVPN

• No need to do this when peering BGP with

the FW
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 141
FW as Default
Gateway Using
Static Routing with
the Fabric
Active/Standby Pair across Sites 2
FW as Default Gateway Using Static Routing with the Fabric

Logical View
• FW allows to apply intra-tenant security
External L3 policies (east-west) and between an internal
Routing function Domain subnet and the external L3 domain (north-
of the VXLAN
EVPN Fabric south) or a subnet in a different tenant (inter-
tenant)
Site 1 Inter-Site Site 2
Network • FW inside network(s) deployed as L2-only can
be extended across sites to allow flexible
Static Static
Routing Routing deployment for endpoints
Active Standby • Two deployment options:
1. Centralized static routing with HMM
tracking
2. Distributed static routing with recursive
next-hop
BDs Extended
via Multi-Site
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 143
FW Using Static Routing with the Fabric
Centralized Static Routing with HMM Tracking (Configuration)
Inter-Site
DCI Network
Multi-Site Multi-Site

…. ….
VTEP VTEP VTEP VTEP
VIP1 VIP2
BGW BGW BGW BGW
Fabric
VXLAN EVPN VXLAN EVPN
Spine Spine Spine Spine

Site1 Site2
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

FW-IP
vrf context VRF1
Active vni 50000 Standby
ip route <endpoint-subnet> <FW-IP> tag 12345 track 1
!
track 1 ip route <FW-IP> reachability hmm Config applied only
vrf member VRF1 only on the leaf nodes
!
connected to the
router bgp 65001
vrf customera Active and Standby
address-family ipv4 unicast FWs
advertise l2vpn evpn
redistribute static route-map fabric-rmap-redist-subnet

Inter-Site
DCI Network
Multi-Site Multi-Site

…. ….
VTEP VTEP VTEP VTEP
VIP1 VIP2
BGW BGW BGW BGW
Fabric
VXLAN EVPN VXLAN EVPN
Spine Spine Spine Spine

Site1 Site2
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

FW-IP External L3
Domain
Active Standby

Traffic destined to endpoints behind the FW is always encapsulated

toward the leaf node connected to the active FW

Inter-Site
DCI Network
Multi-Site Multi-Site

…. ….
VTEP VTEP VTEP VTEP
VIP1 VIP2
BGW BGW BGW BGW
Fabric
VXLAN EVPN VXLAN EVPN
Spine Spine Spine Spine

Site1 Site2
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

FW-IP

Active Standby
vrf context VRF1
vni 50000
ip route <endpoint-subnet> <FW-IP>

Config applied on all the leaf nodes and

also on the Border Gateways

Inter-Site
DCI Network
Multi-Site Multi-Site

…. ….
VTEP VTEP VTEP VTEP
VIP1 VIP2
BGW BGW BGW BGW
Fabric
VXLAN EVPN VXLAN EVPN
Spine Spine Spine Spine

Site1 Site2
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

FW-IP External L3
Domain
Active Standby

Traffic destined to endpoints behind the FW is always encapsulated

toward the leaf node connected to the active FW

Centralized Static Routing Distributed Static Routing

with HMM Tracking with Recursive Next-Hop

👍👍 Centralized configuration (few touch 👍👍 Simpler configuration

points)
👍👍 Recursive Next-Hop functionality natively
👎👎 Convergence depending on HMM tracking integrated into VXLAN EVPN
and static routing redistribution into EVPN
👍👍 Convergence only dependent on FW-IP
👎👎 Scalability dependent on the number of discovery
routes to redistribute
👎👎 Distributed configuration (many touch
points), can be simplified with a provisioning
tool (DCNM)

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 148
FW as Default
Gateway Peering
Directly with the
External Routers
Active/Standby Pair across Sites 3
FW as Default Gateway Peering with the Fabric

Logical View

External L3
Bridging function Domain • VXLAN EVPN Fabric only performs Layer 2
of the VXLAN forwarding
EVPN Fabric
• FW inside network can be extended across
Site 1 Inter-Site Site 2
Network sites to allow flexible deployment for
IGP/BGP
Peering
endpoints
• Inter-sites bridging to allow endpoints to
reach their default gateway
Active Standby
• FW outside network used to peer with the
external router

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 150
Active/Standby Pair across Sites 3
FW as Default Gateway Peering with the Fabric
FW Outside Network is Not Stretched FW Outside Network is Stretched
External L3 External L3
Domain Domain

Site 1 Site 2 Site 1 Site 2

IGP/BGP IGP/BGP
Peering Peerings
Inter-Site Inter-Site
Network Network
Active Standby Active Standby

• The active FW peers only with the external router(s) • The active FW peers with the external routers
connected to the local site connected to all the sites

• Longer convergence after a FW failover event • No need to re-establish peering adjacencies after a
(similar to the previous scenario) FW failover  traffic outage only dependent on FW
failure detection mechanism
• Optimal inbound/outbound traffic paths for the
endpoints part of the site with the active FW • Sub-optimal inbound/outbound traffic paths for the
endpoints part of the site with the active FW

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 151
Fabric as Default
Gateway and Use
of a Perimeter FW
Active/Standby Pair across Sites 4
Fabric as Default Gateway and Use of a Perimeter FW (Intra-Tenant)

• Endpoints subnets can be stretched across

Logical View sites or locally defined in each site
Intra-Tenant N-S
Communication
• All the subnets in the same VRF are
considered part of the same security zone, so
External L3 communication is allowed without traversing
Domain
the FW
Active Standby • FW deployed in L3 mode and connected
Routing function between the BL nodes and the external
of the VXLAN
IGP/BGP Site 2 routers
EVPN Fabric
Peering (or
Static Routing) • Applies security enforcement to intra-tenant
N-S flows
Inter-Site
Intra-Tenant E-W Network
• Same considerations as in the previous
Communication
model for the peering between the FW and
the external routers
• FW could also be deployed in L2 mode and
Site 1 have the BL nodes peering directly with the
external router
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 153
Active/Standby Pair across Sites
Fabric as Default Gateway and Use of a Perimeter FW (Inter-Tenants)

Logical View

External L3
Domain • Communication between subnets part of
separate VRFs (tenants) can happen through
Active
Routing function Inter-Tenants E-W
Standby
the FW front-ending each VRF
of the VXLAN Communication • Use a a single FW with multiple interfaces
EVPN Fabric Site 2
(one for each VRF)
VRF VRF VRF VRF
Tenant 1 Tenant 2 Tenant 2 Tenant 1 • Alternatively, use of a FW context dedicated
Inter-Site to each VRF. The external network performs
Network
the role of “fusion routing” between FW
contexts belonging to separate tenants

Site 1

BDs Extended
via Multi-Site
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 154
Independent
Services in each
Site
Independent Active/Standby pairs deployed in
separate Sites
Inter-Site
DCI Network

…. ….
VTEP Multi-Site VTEP VTEP Multi-Site VTEP
VIP1 VIP2
Fabric BGW BGW BGW BGW

VXLAN EVPN VXLAN EVPN

Spine Spine Spine Spine

Site1 Site2
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

Active/Standby Pair Active/Standby Pair

• Independent pairs of services deployed in each site can’t synchronize connection

state between them
• Must avoid creating asymmetric traffic paths across separate stateful services

• For supporting workload mobility, the FW cannot be deployed as default gateway

for the endpoints
• Deployment of perimeter FW and host route advertisement
• Intra-Tenant: FW enforcement for north-south flows

• Inter-Tenants: FW enforcement for east-west communication

Logical View
Symmetric Intra-Tenant Symmetric Intra-Tenant • Endpoints subnets can be stretched across
N-S Communication N-S Communication
sites or locally defined in each site
External L3
• All the subnets in the same VRF are
Domain
considered part of the same security zone, so
communication is allowed without traversing
Active/Standby Active/Standby
the FW
Site 1 Inter-Site
Network
Site 2 • FW deployed in L3 mode and connected
between the BL nodes and the external
Inter-Site Routing routers
Intra-Tenant E-W • Applies security enforcement to intra-tenant
Communication N-S flows
• Host-route advertisement on the BL nodes to avoid
creation of asymmetric path though separate
stateful FW services

Host-Route
Advertisement
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 158
Independent Active/Standby pairs deployed in separate Sites
Local Host-Route Advertisement
Best path to EP1 via Only path to EP1 via
Site 1 Site 1
External L3 External L3
EP1 advertisement Domain EP1 advertisement Domain
(AS-Path: 65001) (AS-Path: EP1 advertisement EP1 advertisement
Active/Standby Active/Standby 65001,65002) Active/Standby Active/Standby filtered out

Site 1 Site 2 Site 1 Site 2

AS 65001 BGP BGP AS 65002 AS 65001 IGP IGP
X AS 65002
EVPN Update for EP1 EVPN Update for EP1

EVPN EVPN

EP1 EP1 EP2

• When using BGP for peering between the fabric and the • When using an IGP for peering between the fabric and the
external router (FW or L3 device), by default local host external router (FW or L3 device), it is possible to
routes are advertised with a better metric (because of AS- redistribute only BGP internal (i.e. local) host-routes into the
Path length) IGP
route-map EVPN-to-OSPF permit 10
match route-type internal
Configuration on !
BL Nodes in router ospf 1
vrf VRF1
both sites router-id <RID>
redistribute bgp <ASN> route-map EVPN-to-OSPF

Logical View
Host-Route
Advertisement

External L3
Domain • Communication between subnets part of
separate VRFs (tenants) can happen through
Active/Standby Active/Standby
the FW front-ending each VRF
Inter-Tenants E-W
Site 1 Communication Site 2 • Host route advertisement for local hosts
VRF VRF VRF between the fabric and the local FW
VRF
Tenant 2 Tenant 1
Tenant 1 Tenant 2 • Inter-site communication between hosts
Inter-Site
Network
part of separate VRFs must traverse FWs
deployed in both sites
• Host routes injected in the external L3
domain to allow this communication

BDs Extended
via Multi-Site
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 160
Independent Active/Standby pairs deployed in separate Sites
Deployment Options for Host-Routes Advertisement
External L3 External L3
Domain Domain

Host-Route Host-Route
Advertisement Advertisement
IGP/BGP
Peering
Multi-Hop
BGP Peering
IGP/BGP
Site 1 Peering Site 1

• Separate IGP/BGP peering FW-fabric and • Multi-Hop BGP peering between the fabric
FW-external router and the external router
• FW must be capable of receiving and • Host-routes exchanged directly with the
forwarding host route information external router
• FW is not aware of host route advertisement
and can simply leverage static routes
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 161
Active/Active
Cluster across
Sites
Active/Active FW Cluster stretched across Sites
Split-Spanned Ether-Channel Mode
Requires anycast IP service Not supported
support across Sites
Inter-Site
DCI Network

…. ….
VTEP Multi-Site VTEP VTEP Multi-Site VTEP
VIP1 VIP2
Fabric BGW BGW BGW BGW

VXLAN EVPN VXLAN EVPN

Spine Spine Spine Spine

Site1 Site2
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

Cluster with
split-spanned
MAC1/IP1 MAC1/IP1 ether-channel MAC1/IP1 MAC1/IP1

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 163
Active/Active FW Cluster stretched across Sites
Individual Mode
Supported on Cisco ASA software No supported on Cisco FirePower
(and some 3rd party FWs) software and hardware(no current
Inter-Site plans to add such support)
DCI Network

…. ….
VTEP Multi-Site VTEP VTEP Multi-Site VTEP
VIP1 VIP2
Fabric BGW BGW BGW BGW

VXLAN EVPN VXLAN EVPN

Spine Spine Spine Spine

Site1 Site2
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

Cluster in
individual mode
MAC1/IP1 MAC2/IP2 MAC3/IP3 MAC4/IP4

• Each cluster FW node owns a separate identity (MAC/IP address)

• Communication on the Cluster Control Link (CCL) between FW nodes happens

via Multi-Site (using a dedicated and extended L2VNI)
• Recommended to deploy the FW nodes in “one-arm” mode connected to a
“Service BD” associated to an extended L2VNI
 Simplifies the routing configuration on the FW nodes

• Recommended use of Policy Based Routing to redirect traffic to the local service
node(s)
 Only supported with 2nd generation leaf HW (EX and newer)

 When specifying multiple next-hops in the same PBR statement, in VXLAN deployments
traffic is load-balanced per flow across all of them by default

• The PBR policy can be generically

Apply the policy map on endpoints
defined to redirect all traffic to the FW Define the policy map (compute
and service leaf nodes)
SVI(s) of compute leaf nodes
nodes or made more granular and ip access-list ANY interface Vlan2101
specific !
10 permit ip any any no shutdown
vrf member tenant-1
route-map FW-PBR permit 10 no ip redirects

• The policy must then be applied to:

match ip address ANY ip address 192.168.11.254/24 tag 12345
set ip next-hop <FW-IP1, FW-IP2, …> no ipv6 redirects
fabric forwarding mode anycast-gateway
ip policy route-map FW-PBR
 SVIs of the endpoints subnets

 L3 interfaces of the BL nodes

Apply the policy map on L3 Apply the policy map on the “Core
connecting to the external network interfaces of the BL nodes SVI” of the service leaf nodes

 “Core SVI” of the service leaf

interface Ethernet1/23.10 interface Vlan2000
encapsulation dot1q 10 no shutdown
vrf member tenant-1 mtu 9216
ip address 172.16.2.1/30 vrf member VRF1
• The policy should NOT be applied to ip policy route-map FW-PBR no ip redirects
ip forward
the “Core SVI” of the BGW to avoid ipv6 forward
no ipv6 redirects
routing loops ip policy route-map FW-PBR

Inter-Site
DCI Network

…. ….
VTEP Multi-Site VTEP VTEP Multi-Site VTEP
VIP1 VIP2
Fabric BGW BGW BGW BGW

VXLAN EVPN VXLAN EVPN

Spine Spine Spine Spine
PBR on
Core SVI
Site1 Site2
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

PBR on Cluster in
endpoint SVI
individual mode
MAC1/IP1 MAC2/IP2 MAC3/IP3 MAC4/IP4

Inter-Site
DCI Network

…. ….
VTEP Multi-Site VTEP VTEP Multi-Site VTEP
VIP1 VIP2
Fabric BGW BGW BGW BGW

VXLAN EVPN VXLAN EVPN

Spine Spine Spine Spine
PBR on
Core SVI
Site1 Site2
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

PBR on
endpoint SVI

Cluster in
individual mode
MAC1/IP1 MAC2/IP2 MAC3/IP3 MAC4/IP4

Inter-Site
DCI Network

…. ….
VTEP Multi-Site VTEP VTEP Multi-Site VTEP
VIP1 VIP2
Fabric BGW BGW BGW BGW

VXLAN EVPN VXLAN EVPN

Spine Spine Spine Spine
PBR on
Core SVI
Site1 Site2
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

PBR on Cluster in
endpoint SVI
individual mode
MAC1/IP1 MAC2/IP2 MAC3/IP3 MAC4/IP4

Inter-Site
DCI Network

…. ….
VTEP Multi-Site VTEP VTEP Multi-Site VTEP
VIP1 VIP2
Fabric BGW BGW BGW BGW

VXLAN EVPN VXLAN EVPN

Spine Spine Spine Spine
PBR on PBR on
Core SVI Core SVI
Site1 Site2
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

Redirection
over CCL link

Cluster in PBR on
endpoint SVI
individual mode
MAC1/IP1 MAC2/IP2 MAC3/IP3 MAC4/IP4

Inter-Site
DCI Network

…. ….
VTEP Multi-Site VTEP VTEP Multi-Site VTEP
VIP1 VIP2
Fabric BGW BGW BGW BGW

VXLAN EVPN VXLAN EVPN

Spine Spine Spine Spine
PBR on

Site1 Core SVI

Site2
PBR on L3
VTEP VTEP VTEP VTEP interface VTEP VTEP VTEP VTEP

External L3
Domain
Cluster in
individual mode
MAC1/IP1 MAC2/IP2 MAC3/IP3 MAC4/IP4

Inter-Site
DCI Network

…. ….
VTEP Multi-Site VTEP VTEP Multi-Site VTEP
VIP1 VIP2
Fabric BGW BGW BGW BGW

VXLAN EVPN VXLAN EVPN

Spine Spine Spine Spine
PBR on

Site1 Core SVI

Site2
VTEP VTEP VTEP VTEP VTEP VTEP VTEP VTEP

External L3
Domain
PBR on Cluster in
endpoint SVI
individual mode
MAC1/IP1 MAC2/IP2 MAC3/IP3 MAC4/IP4

• Introduction
• Multi-Tenancy Functionality in Enterprise Data Center
• Data Plane and Control Plane Considerations with VXLAN EVPN
• Layer 4-7 Services Integration
• Other useful Things?
• Centralized Route Leaking in EVPN
• Fabric Provisioning and Management
• Conclusion

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 173
Other Useful
Things?
Centralized Route
Leaking in EVPN
Centralized Route Leaking
Extranet and Shared Services Support

Solution
External • Use Cases – Shared Services,
Network
External Connectivity
Border Border

VXLAN EVPN • VRF to VRF or VRF to Default

VRF VTEP VTEP VTEP VTEP VRF • Centralize Location for leaking
Tenant1 Tenant2
routes
Baremetal Baremetal Baremetal

Host A Host B Host C

10.10.10.101 20.20.20.102 30.30.30.103

Guidelines
External • We do NOT export already
Network
Border Border
imported routes

• Routes need to advertised

VXLAN EVPN
explicitly
VRF VTEP VTEP VTEP VTEP VRF
Tenant1 Tenant2 • Routes need to be LESS specific

Baremetal Baremetal Baremetal

Host A Host B Host C

10.10.10.101 20.20.20.102 30.30.30.103

Guidelines
External • We do NOT export already
Network
Border Border vrf context Tenant1 imported routes
vni 50001
rd auto
address-family ipv4 unicast
route-target both auto • Routes need to advertised
VXLAN EVPN route-target both auto evpn
route-target import 65501:50002
vrf context Tenant2
explicitly
vni 50002
VRF VTEP VTEP VTEP rd auto
VTEP VRF
Tenant1 address-family Tenant2
ipv4 unicast • Routes need to be LESS specific
route-target both auto
route-target both auto evpn
Baremetal Baremetal route-target import 65501:50001
Baremetal

Host A Host B Host C

10.10.10.101 20.20.20.102 30.30.30.103

Guidelines
External • We do NOT export already
Network
Border Border
imported routes
router bgp 65501
vrf Tenant1
• Routes need to advertised
address-family ipv4 unicast
VXLAN EVPN advertise l2vpn evpn
explicitly
network 52.52.52.0/23 (subnet to reach in VRF Tenant2)
redistribute direct route-map FABRIC-RMAP-REDIST-SUBNET
vrf Tenant2
VRF VTEP VTEP VTEP VTEP
address-familyVRFipv4 unicast
Tenant1 Tenant2
advertise l2vpn evpn • Routes need to be LESS specific
network 51.51.51.0/23 (subnet to reach in VRF Tenant1)
redistribute direct route-map FABRIC-RMAP-REDIST-SUBNET
Baremetal Baremetal Baremetal

Host A Host B Host C

10.10.10.101 20.20.20.102 30.30.30.103

Guidelines
External interface Vlan10
• We do NOT export already
Network no shutdown
Border Border vrf
vrfcontext
member Tenant1
Tenant1 imported routes
vniaddress
ip 50001 10.10.10.1/24 tag 12345
rd auto
fabric
router forwarding
bgp 65501 mode anycast-gateway
address-family
vrf Tenant1 ipv4 unicast
route-target
interface Vlan20
address-family both auto
ipv4 • Routes need to advertised
unicast
VXLAN EVPN noroute-target both evpn
shutdownl2vpn
advertise auto evpn

vrf
route-target
vrf member52.52.52.0/23
network
ip context
address
Tenant1
Tenant2
20.20.20.1/24
redistribute
explicitly
import 65501:50002
(subnet to reach in VRF Tenant2)
tag 12345
direct route-map FABRIC-RMAP-REDIST-SUBNET
vniTenant2
50002
fabric
vrf forwarding mode anycast-gateway
VRF VTEP VTEP VTEP rdaddress-family
VTEP auto VRF
ipv4 unicast
Tenant1 address-family
interface Vlan30
advertise Tenant2
ipv4evpn
l2vpn • Routes need to be LESS specific
unicast
noroute-target
shutdown
network both auto (subnet to reach in VRF Tenant1)
51.51.51.0/23
route-target
vrf both
member Tenant2
redistribute auto
direct evpn FABRIC-RMAP-REDIST-SUBNET
route-map
Baremetal Baremetal iproute-target
Baremetal import 65501:50001
address 30.30.30.1/24 tag 12345
fabric forwarding mode anycast-gateway
Host A Host B Host C
10.10.10.101 20.20.20.102 30.30.30.103

EVPN based Fabric Open NXOS Programmability

VXLAN
EVPN

Open NXOS

Scalable
Modular OS with Open NX-
Layer 2 & Layer 3 Multi-Tenancy
APIs or YANG
Host Mobility with Optimal
Routing
Automation Ecosystem with
Standards-Based Puppet, Ansible, etc..
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 185
DCNM Fabric Management

Programmable Fabric
On Demand Provisioning

Comprehensive Management in today’s

data Center requires managing both the
Physical Infrastructure (Underlay) and the Physical Network Underlay
Virtualised Networks existing on the fabric VXLAN Overlay
(Overlay).

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 186
Day 1+ Operations: Manage, Monitor Visualize, Search
Challenge: Manage & Grow Underlay with minimal overhead & keep consistent intent

Deployed Fabric Manage Monitor / Visualize /

Search / Update
Underlay:
• SDN Networks [VTEPs]
• Image Update [ISSU]
• View Fabric Topology
• Monitor Health, Events,
Performance
[cpu/mem/iface/syslog]
• Add Devices/Expand

Cisco Advantage:
• Turnkey Management
• Integrated Views
• Comprehensive Fabric Views

Overlay Tasks: Monitor / Visualize / Search

• Visualize Overlays [VXLAN, VLAN, etc..]
• Add, Manage SDN Networks
• Shows VM Networking Path [vCenter]
• Find, Track VMs, Workloads [EPL]
• Find VN’s and VNI’s [VXLAN]
• View VXLAN E2E Connectivity [OAM]
• Identify Errors
• Validate Compliance

Cisco Advantage:
• Seamless Overlay/Underlay Correlation
• Easy to find workloads, VN’s, VNI’s on vast
fabric
• Easy to See Host-Network chain

Compliance Tasks: Detect and Fix

• Monitor Fabric
• Compare device configuration
against Fabric policy
• Remediate [revert or change Policy]
On-Demand
remediation

Cisco Advantage:
• Constant Monitoring
• Compliance engine brings fabric back to
intended configuration
• No un-anticipated behavior
Compliance engine remediates to intended configuration
BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 189
Multi Tenancy and
Service Integration
with DCNM
Deploying The Network

1) Select Network

Staged Deployment is Blue

Yellow is “Deploying” 2 ) Choose which Switches to Deploy to
Green indicates Success
Red indicates failure
3) Deploy

Double-Click Deploy to this switch Ready to Deploy

[De-Select to remove Network]

Select Ports if desired

(Not necessary if default is ‘trunk’)

In Progress Deploye
d

• Double Click the Switches where you want the network

• Select “Apply to Switch”
• Select Deploy
• Green indicates success

Preview

Add Switches to
Fabric
Refresh

Auto-Refresh
on/off

These templates are customizable

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 193
External Fabric Connectivity Provisioning
Border Node Deployments
• Setting up base and setup
configuration
• Deploying VRFs
• Deploying using sub-
interfaces with pool
management of dot1q IDs
• IPv4 & IPv6 support
• VPC Support
• Deploying Networks for
vanilla VLAN hand-off

Virtual & Physical Form Factor

Static & Dynamic Peering
vPC/Non-vPC Attachments

Intra-tenant/Inter-tenant One-armed/Two-armed

• PBR Use-cases • PBR Use-case (No SNAT)

• Tenant-Edge Firewall

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 199
L4-L7 Service Attachment GUI
• The anchor screen lists the defined service nodes and associated route peering and service
policies for a selected easy fabric.
• Enabling/disabling the route peering and service policy will cause the corresponding network
and VRF configuration to be updated. User can preview the generated configurations on
involved switches and deploy them on one shot. Select
Fabric
• User can export/import route peering and service policies.

enable/attach
service policy

• The redirected flow section is added to the switch info overlay screen when user double-clicks the icon of the
switch, which has service configured network attached, on the topology.

Show more flows

BRKDCN-2304 © 2020 Cisco and/or its affiliates. All rights reserved. Cisco Public 201
Complete your
online session
survey • Please complete your session survey
after each session. Your feedback
is very important.
• Complete a minimum of 4 session
surveys and the Overall Conference
survey (starting on Thursday) to
receive your Cisco Live t-shirt.
• All surveys can be taken in the Cisco Events
Mobile App or by logging in to the Content
Catalog on ciscolive.com/emea.

Cisco Live sessions will be available for viewing on

demand after the event at ciscolive.com.

Demos in the
Walk-in labs
Cisco campus

Meet the engineer

Related sessions
1:1 meetings

E-Sabong API REST Integration-Draft
No ratings yet
E-Sabong API REST Integration-Draft
15 pages
Cisco Live Multisite Evpn Vxlan BRKDCN-2035
100% (1)
Cisco Live Multisite Evpn Vxlan BRKDCN-2035
124 pages
Konica Minolta Bizhub 223 Bizhub 423363283223 Network Administrator User Guide 63b4ae9
No ratings yet
Konica Minolta Bizhub 223 Bizhub 423363283223 Network Administrator User Guide 63b4ae9
371 pages
BRKDCN 2218
No ratings yet
BRKDCN 2218
128 pages
BRKDCT-2404 - VXLAN Fundamentals
No ratings yet
BRKDCT-2404 - VXLAN Fundamentals
153 pages
BRKDCT 3378
No ratings yet
BRKDCT 3378
132 pages
BRKDCT 2083
No ratings yet
BRKDCT 2083
166 pages
D3 T1 S3 VXLAN EVPN Multi-Site With Service Insertion PDF
No ratings yet
D3 T1 S3 VXLAN EVPN Multi-Site With Service Insertion PDF
61 pages
Brkaci 3101 PDF
No ratings yet
Brkaci 3101 PDF
176 pages
BRKDCN-2958 DC Operation and Maintenance Best Practice
100% (1)
BRKDCN-2958 DC Operation and Maintenance Best Practice
97 pages
Ltraci 2226 PDF
No ratings yet
Ltraci 2226 PDF
43 pages
Untitled
No ratings yet
Untitled
112 pages
ACI Multisite Deployment - BRKACI-3502
100% (1)
ACI Multisite Deployment - BRKACI-3502
76 pages
BRKDCN 3040
No ratings yet
BRKDCN 3040
109 pages
BRKDCN 2035 PDF
No ratings yet
BRKDCN 2035 PDF
117 pages
5.1.2.6 Packet Tracer - Configure Wireless Security
No ratings yet
5.1.2.6 Packet Tracer - Configure Wireless Security
3 pages
F5 LTM and Asm Lab Setup
100% (1)
F5 LTM and Asm Lab Setup
12 pages
B DCNM 114 Vxlan Evpn v1
No ratings yet
B DCNM 114 Vxlan Evpn v1
66 pages
MPLS
No ratings yet
MPLS
111 pages
BRKDCT-2081 Cisco FabricPath Technology and Design (2011 London)
No ratings yet
BRKDCT-2081 Cisco FabricPath Technology and Design (2011 London)
98 pages
Cyber Security Workshop Lab File
No ratings yet
Cyber Security Workshop Lab File
23 pages
Vendor: Juniper Exam Code: JN0-103 Exam Name: Junos, Associate (JNCIA-Junos) Version: 20.061
No ratings yet
Vendor: Juniper Exam Code: JN0-103 Exam Name: Junos, Associate (JNCIA-Junos) Version: 20.061
12 pages
VXLAN Deployment - Use Cases and Best Practices
No ratings yet
VXLAN Deployment - Use Cases and Best Practices
76 pages
09 VLAN Principles PDF
No ratings yet
09 VLAN Principles PDF
29 pages
BRKDCN 1645
No ratings yet
BRKDCN 1645
62 pages
BRKDCN 1645
No ratings yet
BRKDCN 1645
62 pages
BRKDCN 2988
No ratings yet
BRKDCN 2988
235 pages
Fdocuments - in Cisco Live Data Center Design Guide
No ratings yet
Fdocuments - in Cisco Live Data Center Design Guide
71 pages
BRKDCT-2334 Real World Data Centre Deployments and Best Practice Session
No ratings yet
BRKDCT-2334 Real World Data Centre Deployments and Best Practice Session
126 pages
Cisco NX-OS Software Architecture
100% (1)
Cisco NX-OS Software Architecture
69 pages
BRKDCN-1621 Introduction To VXLAN The Future Path of Your Datacenter
0% (1)
BRKDCN-1621 Introduction To VXLAN The Future Path of Your Datacenter
65 pages
Introduction To ACI - BRKDCN-1001
No ratings yet
Introduction To ACI - BRKDCN-1001
41 pages
BRKSDN 2040
No ratings yet
BRKSDN 2040
83 pages
BRKDCT 2218 PDF
No ratings yet
BRKDCT 2218 PDF
97 pages
How To Setup NEC IP Phone On SL1000
No ratings yet
How To Setup NEC IP Phone On SL1000
10 pages
VPC Configuration Best Practices
100% (1)
VPC Configuration Best Practices
113 pages
Overlay Management With VXLAN
No ratings yet
Overlay Management With VXLAN
98 pages
CN Unit 2
No ratings yet
CN Unit 2
25 pages
Configuring Advanced BGP: BSCI Module 6
No ratings yet
Configuring Advanced BGP: BSCI Module 6
34 pages
Blcen Ethernet/Ip: Configuration Guide
No ratings yet
Blcen Ethernet/Ip: Configuration Guide
31 pages
BRKDCN 2267
No ratings yet
BRKDCN 2267
113 pages
Unit V Application Layer Traditional Applications
No ratings yet
Unit V Application Layer Traditional Applications
29 pages
2006 Conference On IEEE 1588 - Tutorial
No ratings yet
2006 Conference On IEEE 1588 - Tutorial
57 pages
Internet Applications and Network Programming
No ratings yet
Internet Applications and Network Programming
28 pages
PAN Command Line Hierarchy
No ratings yet
PAN Command Line Hierarchy
156 pages
USB HUB Controller
No ratings yet
USB HUB Controller
9 pages
Bangur Hospital Case Study
No ratings yet
Bangur Hospital Case Study
1 page
BRKDCN 2984
No ratings yet
BRKDCN 2984
220 pages
BRKDCN 2980
No ratings yet
BRKDCN 2980
83 pages
SOAP (Simple Object Access Protocol)
No ratings yet
SOAP (Simple Object Access Protocol)
10 pages
ACI Multi-Site Architecture and Dep
No ratings yet
ACI Multi-Site Architecture and Dep
83 pages
BRKDCN 1119
No ratings yet
BRKDCN 1119
109 pages
4.5.1-Packet-Tracer - Inter-Vlan-Routing-Challenge
No ratings yet
4.5.1-Packet-Tracer - Inter-Vlan-Routing-Challenge
7 pages
Lksn2019 Itnetwork Modul A Actual
No ratings yet
Lksn2019 Itnetwork Modul A Actual
11 pages
CCIE Data Center v3 Exam Topics
No ratings yet
CCIE Data Center v3 Exam Topics
6 pages
BRKDCN 1621
No ratings yet
BRKDCN 1621
124 pages
Building DC With VxLAN EVPN Overlay
No ratings yet
Building DC With VxLAN EVPN Overlay
160 pages
BRKDCN 2218
No ratings yet
BRKDCN 2218
83 pages
STT Tên TR M Đ T Long Lat 144 144 144 144 144 144 144 144 144 Mã TR M 4G Mã TR M 4G E/// VNPT Asp Config Existing Config New VNPT (Boq)
No ratings yet
STT Tên TR M Đ T Long Lat 144 144 144 144 144 144 144 144 144 Mã TR M 4G Mã TR M 4G E/// VNPT Asp Config Existing Config New VNPT (Boq)
12 pages
Manual - Layer-3 MPLS VPN Example
No ratings yet
Manual - Layer-3 MPLS VPN Example
7 pages
Fabricpath Dci
No ratings yet
Fabricpath Dci
24 pages
VxLAN BGP Evpn
No ratings yet
VxLAN BGP Evpn
490 pages
BRKDCT 3378
No ratings yet
BRKDCT 3378
140 pages
Cisco BRKDCN-2218
No ratings yet
Cisco BRKDCN-2218
89 pages
Sretna N Godina
No ratings yet
Sretna N Godina
3 pages
Interview Question Asked To Me ..
No ratings yet
Interview Question Asked To Me ..
4 pages
02-Legacy Network Design To ACI Migration (Bas - Steve Sharman)
No ratings yet
02-Legacy Network Design To ACI Migration (Bas - Steve Sharman)
33 pages
BRKDCN 2035multi X 171227064025
No ratings yet
BRKDCN 2035multi X 171227064025
141 pages
Brkens-2092 2023
No ratings yet
Brkens-2092 2023
71 pages
BRKDCN 2913
No ratings yet
BRKDCN 2913
70 pages
Install Cluster With GNS
No ratings yet
Install Cluster With GNS
6 pages
MP BGP Vxlan Aci Demo
No ratings yet
MP BGP Vxlan Aci Demo
30 pages
CN 2023 Jan
No ratings yet
CN 2023 Jan
2 pages
BRKDCN 1959
No ratings yet
BRKDCN 1959
128 pages
DC Modernization
No ratings yet
DC Modernization
2 pages
3.set 2 Questions
No ratings yet
3.set 2 Questions
4 pages
BRKDCN 2974
No ratings yet
BRKDCN 2974
77 pages
BRKDCN 2913
No ratings yet
BRKDCN 2913
68 pages
Introduction To ACI
No ratings yet
Introduction To ACI
38 pages
#ITNE350 20190981 Lab2
No ratings yet
#ITNE350 20190981 Lab2
12 pages
CL - AM24 - BRKDCN-2667 - Simple Leaf-Spine With A Touch of ToR-Network Designs For The Modern Data Center
No ratings yet
CL - AM24 - BRKDCN-2667 - Simple Leaf-Spine With A Touch of ToR-Network Designs For The Modern Data Center
68 pages
BRKDCN 1621
No ratings yet
BRKDCN 1621
115 pages
Troubleshooting
No ratings yet
Troubleshooting
6 pages
Vxlan Evpn
No ratings yet
Vxlan Evpn
121 pages
K SCHEME ACN CH 1 Internet Architecture and Network Layer 12M
No ratings yet
K SCHEME ACN CH 1 Internet Architecture and Network Layer 12M
11 pages

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.