Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release

7.x
First Published: 2015-01-27
Last Modified: 2020-03-04

Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 527-0883
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS REFERENCED IN THIS DOCUMENTATION ARE SUBJECT TO CHANGE WITHOUT NOTICE.
EXCEPT AS MAY OTHERWISE BE AGREED BY CISCO IN WRITING, ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS DOCUMENTATION ARE
PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED.

The Cisco End User License Agreement and any supplemental license terms govern your use of any Cisco software, including this product documentation, and are located at:
http://www.cisco.com/go/softwareterms.Cisco product warranty information is available at http://www.cisco.com/go/warranty. US Federal Communications Commission Notices are found
here http://www.cisco.com/c/en/us/products/us-fcc-notice.html.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT
LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS
HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Any products and features described herein as in development or available at a future date remain in varying stages of development and will be offered on a when-and if-available basis. Any
such product or feature roadmaps are subject to change at the sole discretion of Cisco and Cisco will have no liability for delay in the delivery or failure to deliver any products or feature
roadmap items that may be set forth in this document.

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network
topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional
and coincidental.

The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based
on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language
that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com
go trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any
other company. (1721R)
© 2015–2020 Cisco Systems, Inc. All rights reserved.
 CONTENTS

PREFACE Preface xvii

Audience xvii
Document Conventions xvii
Related Documentation for Cisco Nexus 9000 Series Switches xviii
Documentation Feedback xviii
Communications, Services, and Additional Information xviii

CHAPTER 1 New and Changed Information 1

New and Changed Information 1

CHAPTER 2 Overview 9

Licensing Requirements 9
About Interfaces 9
Ethernet Interfaces 10
Access Ports 10
Routed Ports 10
Management Interface 10
Port-Channel Interfaces 10
Subinterfaces 10
Loopback Interfaces 11
Breakout Interfaces 11
Module Level Breakout 11
Dynamic Breakout (Per-port Level Breakout) 11
About the Lane Selector 12
Notes About Breakout Interfaces 13
Virtual Device Contexts 17

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
iii
 Contents

High Availability for Interfaces 17

CHAPTER 3 Configuring Basic Interface Parameters 19

About the Basic Interface Parameters 19

Description 19
Beacon 19
Error Disabled 19
Interface Status Error Policy 20
Modifying Interface MTU Size 20
Bandwidth 21
Throughput Delay 21
Administrative Status 22
Unidirectional Link Detection Parameter 22
UDLD Overview 22

Default UDLD Configuration 23

UDLD Aggressive and Nonaggressive Modes 23
Port-Channel Parameters 24
Port Profiles 24
Cisco QSFP+ to SFP+ Adapter Module Support 26
Cisco SFP+ Adapter Module Support 27
25G Autonegotiation Overview 27

Guidelines and Limitations 27

Default Settings 29
Configuring the Basic Interface Parameters 30
Specifying the Interfaces to Configure 30
Configuring the Description 32
Configuring the Beacon Mode 33
Configuring the Error-Disabled State 34
Enabling the Error-Disable Detection 34
Enabling the Error-Disabled Recovery 35
Configuring the Error-Disabled Recovery Interval 36
Configuring the MTU Size 37
Configuring the Interface MTU Size 37
Configuring the System Jumbo MTU Size 39

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
iv
 Contents

Configuring the Bandwidth 40

Configuring the Throughput Delay 41
Shutting Down and Activating the Interface 43
Configuring the UDLD Mode 44
Configuring Debounce Timers 46
Configuring Port Profiles 48
Creating a Port Profile 48
Entering Port-Profile Configuration Mode and Modifying a Port Profile 49
Assigning a Port Profile to a Range of Interfaces 50
Enabling a Specific Port Profile 51
Inheriting a Port Profile 51
Removing a Port Profile from a Range of Interfaces 52
Removing an Inherited Port Profile 53
Verifying the Basic Interface Parameters 53
Monitoring the Interface Counters 54
Displaying Interface Statistics 54
Clearing Interface Counters 55
Configuration Example for QSA 56

CHAPTER 4 Configuring Layer 2 Interfaces 57

Information About Access and Trunk Interfaces 58

About Access and Trunk Interfaces 58

IEEE 802.1Q Encapsulation 59

Access VLANs 60
Native VLAN IDs for Trunk Ports 60
Tagging Native VLAN Traffic 61
Allowed VLANs 61
Switchport Isolated for up to 3976 VLANs on Trunk Interfaces 61
Default Interfaces 62
Switch Virtual Interface and Autostate Behavior 62
SVI Autostate Exclude 62
SVI Autostate Disable 63
High Availability 63
Virtualization Support 63

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
v
 Contents

Counter Values 63
Prerequisites for Layer 2 Interfaces 64
Guidelines and Limitations for Layer 2 Interfaces 65
Default Settings for Layer 2 Interfaces 67
Configuring Access and Trunk Interfaces 68

Guidelines for Configuring Access and Trunk Interfaces 68

Configuring a VLAN Interface as a Layer 2 Access Port 68
Configuring Access Host Ports 70
Configuring Trunk Ports 71
Configuring the Native VLAN for 802.1Q Trunking Ports 73
Configuring the Allowed VLANs for Trunking Ports 74
Configuring MAC addresses Limitation on a VLAN 76
Configuring switchport isolated 77
Configuring a Default Interface 78

Configuring SVI Autostate Exclude 79

Configuring SVI Autostate Disable for the System 80
Configuring SVI Autostate Disable Per SVI 81
Configuring the Device to Tag Native VLAN Traffic 83

Configuring Interface Breakout Profile for 50-G Interfaces in a 16-Slot Chassis 84

Changing the System Default Port Mode to Layer 2 85

Verifying the Interface Configuration 86

Monitoring the Layer 2 Interfaces 87
Configuration Examples for Access and Trunk Ports 87
Related Documents 88

CHAPTER 5 Configuring Layer 3 Interfaces 89

About Layer 3 Interfaces 89
Routed Interfaces 89
Subinterfaces 90
Limitations for Subinterfaces 90
VLAN Interfaces 91
Changing VRF Membership for an Interface 91
Notes About Changing VRF Membership for an Interface 92
Loopback Interfaces 92

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
vi
 Contents

IP Unnumbered 92
MAC-Embedded IPv6 Address 93
High Availability 93
Virtualization Support 93
DHCP Client 93
Limitations for Using DHCP Client on Interfaces 94
Prerequisites for Layer 3 Interfaces 94
Guidelines and Limitations 94
Default Settings 95
Configuring Layer 3 Interfaces 96
Configuring a Routed Interface 96
Configuring a Subinterface on a Routed Interface 98
Configuring a Subinterface on a Port-Channel Interface 99
Configuring a VLAN Interface 100
Enabling Layer 3 Retention During VRF Membership Change 102
Configuring a Loopback Interface 102
Configuring IP Unnumbered on an Ethernet Interface 103
Configuring OSPF for an IP Unnumbered Interface 104
Configuring ISIS for an IP Unnumbered Interface 106
Configuring PBR on SVI on the Gateway 107
Configuring IP Unnumbered on SVI Secondary VLAN on the Gateway 109
Assigning an Interface to a VRF 110
Configuring a MAC-Embedded IPv6 Address 111
Configuring a DHCP Client on an Interface 114
Verifying the Layer 3 Interfaces Configuration 115
Monitoring the Layer 3 Interfaces 116
Configuration Examples for Layer 3 Interfaces 117
Example of Changing VRF Membership for an Interface 117
Related Documents 119

CHAPTER 6 Configuring Bidirectional Forwarding Detection 121

About BFD 121

Asynchronous Mode 121
BFD Detection of Failures 122

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
vii
Contents

Distributed Operation 123

BFD Echo Function 123
Security 123
High Availability 123
Virtualization Support 123
Prerequisites for BFD 123
Guidelines and Limitations 124
Default Settings 126
Configuring BFD 127
Configuration Hierarchy 127
Task Flow for Configuring BFD 127
Enabling the BFD Feature 127
Configuring Global BFD Parameters 128
Configuring BFD on an Interface 129
Configuring BFD on a Port Channel 131
Configuring the BFD Echo Function 132
Configuring Per-Member Link BFD Sessions 133
BFD Enhancement to Address Per-link Efficiency 133
Limitations of the IETF Bidirectional Forwarding Detection 134
Configuring Port Channel Interface 135
Configuring BFD Start Timer 136
Enabling IETF Per-link BFD 136
Configuring BFD Destination IP Address 136
Verifying Micro BFD Session Configurations 137

Examples: Configuring Micro BFD Sessions 137

Configuring BFD Support for Routing Protocols 140
Configuring BFD on BGP 140
Configuring BFD on EIGRP 141
Configuring BFD on OSPF 142
Configuring BFD on IS-IS 144
Configuring BFD on HSRP 145
Configuring BFD on VRRP 146
Configuring BFD on PIM 147
Configuring BFD on Static Routes 148

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
viii
 Contents

Disabling BFD on an Interface 149

Configuring BFD Interoperability 150
Configuring BFD Interoperability in Cisco NX-OS Devices in a Point-to-Point Link 150
Configuring BFD Interoperability in Cisco NX-OS Devices in a Switch Virtual Interface 151
Configuring BFD Interoperability in Cisco NX-OS Devices in Logical Mode 152
Verifying BFD Interoperability in a Cisco Nexus 9000 Series Device 153
Verifying the BFD Configuration 153
Monitoring BFD 154
Configuration Examples for BFD 154
Show Example for BFD 155
Related Documents 155
RFCs 155

CHAPTER 7 Configuring Port Channels 157

About Port Channels 157

Port Channels 158
Port-Channel Interfaces 159
Basic Settings 159
Compatibility Requirements 160
Load Balancing Using Port Channels 162
Symmetric Hashing 163
Guidelines and Limitations for ECMP 163
Resilient Hashing 164
GTP Tunnel Load Balancing 164
LACP 165
LACP Overview 165
Port-Channel Modes 166
LACP ID Parameters 168
LACP System Priority 168
LACP Port Priority 168
LACP Administrative Key 168
LACP Marker Responders 168
LACP-Enabled and Static Port Channels Differences 169
LACP Compatibility Enhancements 169

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
ix
Contents

LACP Port-Channel Minimum Links and MaxBundle 170

LACP Fast Timers 170
Virtualization Support 170
High Availability 171
Prerequisites for Port Channeling 171
Guidelines and Limitations 171
Default Settings 173
Configuring Port Channels 174
Creating a Port Channel 174
Adding a Layer 2 Port to a Port Channel 176
Adding a Layer 3 Port to a Port Channel 178
Applying Port Channel Configuration to all Configured Port Channels 180
Configuring the Bandwidth and Delay for Informational Purposes 181
Shutting Down and Restarting the Port-Channel Interface 182
Configuring a Port-Channel Description 183
Configuring the Speed and Duplex Settings for a Port-Channel Interface 184
Configuring Load Balancing Using Port Channels 185
Enabling LACP 186
Configuring LACP Port-Channel Port Modes 187
Configuring LACP Port-Channel Minimum Links 188
Configuring the LACP Port-Channel MaxBundle 190
Configuring the LACP Fast Timer Rate 191
Configuring the LACP System Priority 192
Configuring the LACP Port Priority 193
Configuring LACP System MAC and Role 194
Disabling LACP Graceful Convergence 195
Reenabling LACP Graceful Convergence 196
Disabling LACP Suspend Individual 197
Reenabling LACP Suspend Individual 198
Configuring Delayed LACP 199
Configuring Port Channel Hash Distribution 201
Configuring Port Channel Hash Distribution at the Global Level 201
Configuring Port Channel Hash Distribution at the Port Channel Level 202
Enabling ECMP Resilient Hashing 203

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
x
 Contents

Disabling ECMP Resilient Hashing 203

Configuring ECMP Load Balancing based on Inner NVGRE Header 204
Verifying the ECMP Resilient Hashing Configuration 205
Verifying the Port-Channel Configuration 206
Monitoring the Port-Channel Interface Configuration 206
Example Configurations for Port Channels 207
Related Documents 208

CHAPTER 8 Configuring vPCs 209

Information About vPCs 209
vPC Overview 209
Hitless vPC Role Change 211
vPC Terminology 212
vPC Peer Link Overview 213
Features That You Must Manually Configure on the Primary and Secondary Devices 215
Configuring Layer 3 Backup Routes on a vPC Peer Link 216
Peer-Keepalive Link and Messages 216
vPC Peer-Gateway 217
vPC Domain 218
vPC Topology 218
Compatibility Parameters for vPC Interfaces 220
Configuration Parameters That Must Be Identical 220
Configuration Parameters That Should Be Identical 221
Consequences of Parameter Mismatches 222
vPC Number 222
Moving Other Port Channels into a vPC 223
vPC Object Tracking 223
vPC Interactions with Other Features 225
vPC and LACP 225
vPC Peer Links and STP 225
vPC Peer Switch 227
vPC and ARP or ND 228
vPC Multicast—PIM, IGMP, and IGMP Snooping 228
Multicast PIM Dual DR (Proxy DR ) 229

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
xi
Contents

IP PIM PRE-BUILD SPT 230

vPC Peer Links and Routing 230
CFSoE 231
vPC and Orphan Ports 232
Virtualization Support 232
vPC Recovery After an Outage 232
Autorecovery 232
vPC Peer Roles After a Recovery 232
High Availability 232
vPC Forklift Upgrade Scenario 233
Guidelines and Limitations 235
Best Practices for Layer 3 and vPC Configuration 238
Layer 3 and vPC Configuration Overview 238
Supported Topologies for Layer 3 and vPC 239
Peering with an External Router Using Layer 3 Links 239
Peering Between vPC Devices for a Backup Routing Path 240
Direct Layer 3 Peering Between Routers 241
Peering Between Two Routers with vPC Devices as Transit Switches 241
Peering with an External Router on Parallel Interconnected Routed Ports 242
Peering between vPC Switch Pairs on Parallel Interconnected Routed Ports 242
Peering Over a PC Interconnection and Dedicated Interswitch Link Using non-vPC VLAN 243
Peering Directly Over a vPC Connection 243
Configuring Layer 3 over vPC 245
Default Settings 247
Configuring vPCs 247
Enabling vPCs 247
Disabling vPCs 248
Creating a vPC Domain and Entering vpc-domain Mode 249
Configuring a vPC Keepalive Link and Messages 250
Creating a vPC Peer Link 252
Configuring a vPC Peer-Gateway 253
Configuring Fast Convergence 254
Configuring LACP vPC Convergence 255
Configuring a Graceful Consistency Check 256

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
xii
 Contents

Checking the Configuration Compatibility on a vPC Peer Link 257

Moving Other Port Channels into a vPC 258
Manually Configuring a vPC Domain MAC Address 259
Manually Configuring the System Priority 260
Manually Configuring the vPC Peer Device Role 262
Configuring vPC Object Tracking Tracking Feature on a Single-Module vPC 263
Configuring for Recovery After an Outage 264
Configuring Reload Restore 264
Configuring an Autorecovery 266
Configuring the Suspension of Orphan Ports 268
Configuring Delay Restore on an Orphan Port 269
Configuring the vPC Peer Switch 270
Configuring a Pure vPC Peer Switch Topology 270
Configuring a Hybrid vPC Peer Switch Topology 272
Configuring Hitless vPC Role Change 273
Use Case Scenario for vPC Role Change 274
Enabling STP to Use the Cisco MAC Address 275
Verifying the vPC Configuration 275
Monitoring vPCs 276
Configuration Examples for vPCs 276
Related Documents 278

CHAPTER 9 Configuring IP Tunnels 279

Information About IP Tunnels 279

IP Tunnel Overview 279
GRE Tunnels 280
Point-to-Point IP-in-IP Tunnel Encapsulation and Decapsulation 280
Multi-Point IP-in-IP Tunnel Decapsulation 280
Path MTU Discovery 280
High Availability 281
Prerequisites for IP Tunnels 281
Guidelines and Limitations 281
Default Settings 285
Configuring IP Tunnels 286

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
xiii
 Contents

Enabling Tunneling 286

Creating a Tunnel Interface 286
Creating an IP-in-IP Tunnel with a Netmask 288
Configuring a Tunnel Interface 290
Configuring a GRE Tunnel 293
Configuring a GRE Tunnel 294
Enabling Path MTU Discovery 296
Assigning VRF Membership to a Tunnel Interface 297
Verifying the IP Tunnel Configuration 298
Configuration Examples for IP Tunneling 298
Related Documents 299

CHAPTER 10 Configuring Q-in-Q VLAN Tunnels 301

Information About Q-in-Q Tunnels 301

Q-in-Q Tunneling 301
Native VLAN Hazard 303
Information About Layer 2 Protocol Tunneling 304
Guidelines and Limitations 305
Configuring Q-in-Q Tunnels and Layer 2 Protocol Tunneling 307
Creating a 802.1Q Tunnel Port 307
Configuring VLAN Mapping for Selective Q-in-Q on a 802.1Q Tunnel Port 308
Changing the EtherType for Q-in-Q 309
Enabling the Layer 2 Protocol Tunnel 311
Configuring Global CoS for L2 Protocol Tunnel Ports 312
Configuring Thresholds for Layer 2 Protocol Tunnel Ports 313
Configuring Q-in-Q Double Tagging 314
Verifying the Q-in-Q Configuration 315
Configuration Examples for Q-in-Q and Layer 2 Protocol Tunneling 316

CHAPTER 11 Configuring Smart-Channel 317

About Smart Channel 317
Smart Channel Features 317
Benefits of Smart Channel 318
Examples of the Deployment Use Cases 318

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
xiv
 Contents

Topology Examples for Smart Channel 318

Prerequisites for Smart Channel 320

Guidelines and Limitations for Smart Channel 321
Default Settings for Smart Channel 321
Configuring Smart Channel 321
Enabling Smart Channel 321
Configuring Port Groups 322
Configuring the Smart Channel Service 323
Verifying the Smart Channel Configuration 324
Configuration Examples for Smart Channel 325

CHAPTER 12 Configuring Static and Dynamic NAT Translation 329

Network Address Translation Overview 329

Information About Static NAT 330
Dynamic NAT Overview 331
Timeout Mechanisms 332
NAT Inside and Outside Addresses 333
Pool Support for Dynamic NAT 333
Static and Dynamic Twice NAT Overview 334
VRF Aware NAT 334
Guidelines and Limitations for Static NAT 336
Restrictions for Dynamic NAT 337
Guidelines and Limitations for Dynamic Twice NAT 338
Configuring Static NAT 339
Enabling Static NAT 339
Configuring Static NAT on an Interface 339
Enabling Static NAT for an Inside Source Address 340
Enabling Static NAT for an Outside Source Address 341
Configuring Static PAT for an Inside Source Address 341
Configuring Static PAT for an Outside Source Address 342
Configuring Static Twice NAT 342
Configuration Example for Static NAT and PAT 344
Example: Configuring Static Twice NAT 344
Verifying the Static NAT Configuration 345

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
xv
 Contents

Configuring Dynamic NAT 346

Configuring Dynamic Translation and Translation Timeouts 346
Configuring Dynamic NAT Pool 348
Configuring Source Lists 349
Configuring Dynamic Twice NAT for an Inside Source Address 350
Configuring Dynamic Twice NAT for an Outside Source Address 351
Clearing Dynamic NAT Translations 352
Verifying Dynamic NAT Configuration 352
Example: Configuring Dynamic Translation and Translation Timeouts 355

APPENDIX A Configuring Layer 2 Data Center Interconnect 357

Overview 357
Example of Layer 2 Data Center Interconnect 357

APPENDIX B IETF RFCs supported by Cisco NX-OS Interfaces 359

IPv6 RFCs 359

APPENDIX C Configuration Limits for Cisco NX-OS Interfaces 361

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
xvi
 Preface
This preface includes the following sections:
• Audience, on page xvii
• Document Conventions, on page xvii
• Related Documentation for Cisco Nexus 9000 Series Switches, on page xviii
• Documentation Feedback, on page xviii
• Communications, Services, and Additional Information, on page xviii

Audience
This publication is for network administrators who install, configure, and maintain Cisco Nexus switches.

Document Conventions
Command descriptions use the following conventions:

Convention Description
bold Bold text indicates the commands and keywords that you enter literally
as shown.

Italic Italic text indicates arguments for which you supply the values.

[x] Square brackets enclose an optional element (keyword or argument).

[x | y] Square brackets enclosing keywords or arguments that are separated by

a vertical bar indicate an optional choice.

{x | y} Braces enclosing keywords or arguments that are separated by a vertical

bar indicate a required choice.

[x {y | z}] Nested set of square brackets or braces indicate optional or required

choices within optional or required elements. Braces and a vertical bar
within square brackets indicate a required choice within an optional
element.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
xvii
 Preface
Related Documentation for Cisco Nexus 9000 Series Switches

Convention Description
variable Indicates a variable for which you supply values, in context where italics
cannot be used.

string A nonquoted set of characters. Do not use quotation marks around the
string or the string includes the quotation marks.

Examples use the following conventions:

Convention Description
screen font Terminal sessions and information the switch displays are in screen font.

boldface screen font Information that you must enter is in boldface screen font.

italic screen font Arguments for which you supply values are in italic screen font.

<> Nonprinting characters, such as passwords, are in angle brackets.

[] Default responses to system prompts are in square brackets.

!, # An exclamation point (!) or a pound sign (#) at the beginning of a line

of code indicates a comment line.

Related Documentation for Cisco Nexus 9000 Series Switches

The entire Cisco Nexus 9000 Series switch documentation set is available at the following URL:
http://www.cisco.com/en/US/products/ps13386/tsd_products_support_series_home.html

Documentation Feedback
To provide technical feedback on this document, or to report an error or omission, please send your comments
to nexus9k-docfeedback@cisco.com. We appreciate your feedback.

Communications, Services, and Additional Information

• To receive timely, relevant information from Cisco, sign up at Cisco Profile Manager.
• To get the business impact you’re looking for with the technologies that matter, visit Cisco Services.
• To submit a service request, visit Cisco Support.
• To discover and browse secure, validated enterprise-class apps, products, solutions and services, visit
Cisco Marketplace.
• To obtain general networking, training, and certification titles, visit Cisco Press.
• To find warranty information for a specific product or product family, access Cisco Warranty Finder.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
xviii
Preface
Preface

Cisco Bug Search Tool

Cisco Bug Search Tool (BST) is a web-based tool that acts as a gateway to the Cisco bug tracking system
that maintains a comprehensive list of defects and vulnerabilities in Cisco products and software. BST provides
you with detailed defect information about your products and software.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
xix
 Preface
Preface

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
xx
 CHAPTER 1
New and Changed Information
This chapter provides release-specific information for each new and changed feature in the Cisco Nexus 9000
Series NX-OS Interfaces Configuration Guide.
• New and Changed Information, on page 1

New and Changed Information

This table summarizes the new and changed features for the Cisco Nexus 9000 Series NX-OS Interfaces
Configuration Guide and where they are documented.

Table 1: New and Changed Features

Feature Description Changed Where Documented

in
Release

Configuring ECMP Load Support to enable ECMP load 7.0(3)I7(8) Configuring ECMP Load
Balancing based on Inner sharing based on IPv6 inner Balancing based on Inner
NVGRE Header NVGRE header for IPv6 NVGRE Header, on page 204
packets on Cisco Nexus
9500-FM-E platform switches.
Command interface Added support to select the 7.0(3)I7(8) Guidelines and Limitations, on
port-channel all configuration a port channel page 171
and then apply that
Applying Port Channel
configuration to the member
Configuration to all Configured
ports of all the configured port
Port Channels, on page 180
channels using interface
port-channel all command.
FEC Information. Added support to display FEC 7.0(3)I7(7) Caveats, on page 13
information.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
1
 New and Changed Information
New and Changed Information

Feature Description Changed Where Documented

in
Release

Debounce timer for the link-up Added support to configure 7.0(3)I7(6) Configuring Debounce Timers,
events debounce timer for the link-up on page 46
events on Cisco Nexus
N9K-C9732C-FX,
N9K-C9364C and
N9K-X97160YC-EX switches.
QSFP-100G SRBD Added support for QSFP-100G 7.0(3)I7(5) Guidelines and Limitations, on
SRBD on Cisco Nexus page 124
N9K-C93180YC-FX and
N9K-C93180TC-FX switches.
Debounce timer for the link-up Added support to configure 7.0(3)I7(5) Configuring Debounce Timers,
events debounce timer for the link-up on page 46
events on Cisco Nexus
N9K-C9336C-FX2 and
N9K-C93240YC-FX2 switches.
Link Aggregation Control Added support for Link 7.0(3)I7(5) Configuring LACP vPC
Protocol (LACP) vPC Aggregation Control Protocol Convergence, on page 255
convergence (LACP) vPC convergence on
Cisco Nexus 9500 Series
Switches.
Configuring MAC addresses Added support to configure 7.0(3)F3(4) Configuring MAC addresses
Limitation on a VLAN MAC addresses Limitation on Limitation on a VLAN, on page
a VLAN 76

Configuring Custom EtherType Added support to Custom 7.0(3)F3(4) Changing the EtherType for
or Tag Protocol Identifier EtherType or Tag Protocol Q-in-Q, on page 309
(TPID) Identifier

QinQ Double Tagging Added support to configure the 7.0(3)I7(3) Information About Layer 2
switch to all multi-tagged Protocol Tunneling, on page 304
BPDUs on a tunnel port.
Configuring Q-in-Q Double
Tagging, on page 314

GPRS Tunneling Protocol Added support to deliver 7.0(3)I7(3) GTP Tunnel Load Balancing,
mobile data on wireless on page 164
networks via Cisco Nexus
9300-EX Series switches.

Tunnel Source Direct Support added for the tunnel 7.0(3)I7(3) Guidelines and Limitations, on
source direct command. page 281

1024 x 50-G Interfaces Added support to breakout 7.0(3)I7(3) Configuring Interface Breakout
100-G ports into two 50-G Profile for 50-G Interfaces in a
interfaces for slots 8 to 16 in the 16-Slot Chassis, on page 84
Cisco Nexus 9516 switch for
-EX line cards.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
2
New and Changed Information
New and Changed Information

Feature Description Changed Where Documented

in
Release

QSFP 40/100-G BiDi QSFP 40/100-G BiDi comes up 7.0(3)I7(3) Guidelines and Limitations, on
in the highest possible speed page 124
available on the port.

LACP System MAC and Role Added support to configure the 7.0(3)I7(3) Configuring LACP System
MAC address used by the MAC and Role, on page 194
LACP for protocol exchanges
and the option role.

mac-address bpdu source Triggers STP to use the new 7.0(3)I7(2) Enabling STP to Use the Cisco
version 2 command. Cisco MAC address MAC Address, on page 275
(00:26:0b:xx:xx:xx).

Breakout support for the Cisco Added breakout support for the 7.0(3)I7(1) Cisco Nexus 9000
Nexus C93180LC-EX switch Cisco Nexus C93180LC-EX C93180LC-EX Switch, on page
switch. (TOR switch) 15

ECMP Resilient Hashing Added support for the Cisco 7.0(3)I7(1) Resilient Hashing, on page 164
Nexus 9200 and Cisco Nexus
9300-EX platform switches.
Hitless vPC Role Change Added support for hitless vPC 7.0(3)I7(1) Hitless vPC Role Change, on
role change. page 211

IP Unnumbered on SVI Added support for IP 7.0(3)I7(1) Configuring IP Unnumbered on

Secondary VLAN on the unnumbered on the secondary SVI Secondary VLAN on the
Gateway SVI in the gateway. Gateway, on page 109

vPC Fast Convergence Introduced this feature. 7.0(3)I7(1) Configuring Fast Convergence,
Convergence on page 254
Configuring LACP vPC
Convergence, on page 255

mac-address bpdu source Triggers STP to use the new 7.0(3)I6(2) Enabling STP to Use the Cisco
version 2 command. Cisco MAC address MAC Address, on page 275
(00:26:0b:xx:xx:xx).

Configuring IP-in-IP tunnel Added support for configuring 7.0(3)I6(1) Guidelines and Limitations, on
decapsulation on directly IP-in-IP tunnel decapsulation page 281 and Configuring a
connected IP addresses using on any directly connected IP Tunnel Interface, on page 290
tunnel source direct and addresses.
tunnel mode ipv6ipv6
decapsulate-any

Configuring IP unnumbered Added a guideline for 7.0(3)I6(1) IP Unnumbered, on page 92

interfaces for port channels configuring IP unnumbered
interfaces for port channels on
all Cisco Nexus 9000 Series
switches.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
3
 New and Changed Information
New and Changed Information

Feature Description Changed Where Documented

in
Release

Smart channel Introduced this feature. 7.0(3)I6(1) Configuring Smart-Channel, on

page 317

Port profile support on Cisco Updated the guideline to 7.0(3)I5(2) Guidelines and Limitations, on
Nexus 9500 Series switches include port profile support on page 27
Cisco Nexus 9500 Series
switches.

Support for QSA adapter Updated the guideline to 7.0(3)I5(2) Cisco QSFP+ to SFP+ Adapter
include QSA adapter support Module Support, on page 26
on Cisco Nexus 9200 and
9300-EX Series switches and
Cisco Nexus 3232C and 3264Q
Series switches.

Configuring Layer 3 over vPC Added support for configuring 7.0(3)I5(1) Configuring Layer 3 over vPC,
Layer 3 over vPC. on page 245

Configuring selective Q-in-Q Added support for configuring 7.0(3)I5(1) Configuring VLAN Mapping
tunneling on a 802.1Q tunnel selective Q-in-Q tunneling on for Selective Q-in-Q on a
port a 802.1Q tunnel port. 802.1Q Tunnel Port, on page
308

Configuring BFD per-member Added per-link Bidirectional 7.0(3)I5(1) BFD Enhancement to Address
link support (RFC 7130) Forwarding support that enables Per-link Efficiency, on page 133
to configure individual BFD
sessions on every Link
Aggregation Group member
interfaces (as defined in RFC
7130).

SFP+ adapter module Added 25G optics support on 7.0(3)I4(2) Cisco SFP+ Adapter Module
the CVR-2QSFP28-8SFP Support, on page 27
adapter for 100G ports of the
Cisco Nexus 9236C switch.

Support for changing VRF Added support for changing 7.0(3)I4(1) Changing VRF Membership for
membership for an SVI VRF membership for an SVI. an Interface

Support for port profiles. Added support for port profiles. 7.0(3)I4(1) Port Profiles

Support for PIM SSM over Added support for PIM SSM 7.0(3)I4(1) vPC Multicast—PIM, IGMP,
vPCs. over vPCs. and IGMP Snooping

NAT support for In Service Added NAT support for In 7.0(3)I4(1) Guidelines and Limitations for
Software Upgrade (ISSU). Service Software Upgrade Static NAT
(ISSU).

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
4
New and Changed Information
New and Changed Information

Feature Description Changed Where Documented

in
Release

Breakout and FEX support for Added breakout and FEX 7.0(3)I3(1) Cisco Nexus C9332PQ Switch
the Cisco Nexus C9332PQ support for the Cisco Nexus
switch. C9332PQ switch. (TOR switch)

Breakout support for the Cisco Added breakout support for the 7.0(3)I3(1) Cisco Nexus C92160YC Switch
Nexus 92160YC switch. Cisco Nexus 92160YC switch.
(TOR switch)

Breakout support for the Cisco Added breakout support for the 7.0(3)I3(1) Cisco Nexus C9272Q Switch
Nexus 9272Q switch Cisco Nexus 9272Q switch.
(TOR switch)

ip unnumbered command Added support for IP 7.0(3)I3(1) IP Unnumbered

unnumbered command.

Layer 2 Data Center Added support for Layer 2 Data 7.0(3)I2(2) Configuring Layer 2 Data
Interconnect Center Interconnect. Center Interconnect

shut and no shut support for Added support for shut and no 7.0(3)I2(2) Configuring vPCs
vPC domain command shut for vPC domain
command.

DHCP Client Added support for Cisco Nexus 7.0(3)I2(2) DHCP Client
9500 Series switches.

switchport isolated support Added support for the 7.0(3)I2(1) Switchport Isolated for up to
switchport isolated command. 3976 VLANs on Trunk
Interfaces

DHCP Client Added support to configure the 7.0(3)I2(1) DHCP Client

IPv4 or IPv6 address of a
DHCP client on a management
interface or a physical Ethernet
interface.

GRE tunnel enhancements Added support for GRE v6 7.0(3)I2(1) Configuring a GRE Tunnel
payload over v4 tunnel and
GRE v4 payload over v6 tunnel.

Source interface support Added source-interface 7.0(3)I2(1) Guidelines and Limitations

command option support to
configure an IPv4 and/or IPv6
inband or outband source IP
address.

regex support Support for regular expression 7.0(3)I2(1) Guidelines and Limitations
to address a set of interfaces.

BFD startup timer Introduced this feature. 7.0(3)I2(1) Configuring Global BFD
Parameters

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
5
 New and Changed Information
New and Changed Information

Feature Description Changed Where Documented

in
Release

Static and Dynamic NAT Added support for static and 7.0(3)I2(1) Configuring Static and
Translation support dynamic NAT translation. Dynamic NAT Translation

Switch Virtual Interface and Added support for switch 7.0(3)I2(1) Switch Virtual Interface and
Autostate Behavior support virtual interface and autostate Autostate Behavior
behavior.

IP-in-IP tunnel mask support Added support for IP-in-IP 7.0(3)I2(1) Creating a Tunnel Interface
tunnel mask.

Q-in-Q VLAN Tunnel support Added support for Q-in-Q 7.0(3)I2(1) Configuring Q-in-Q VLAN
VLAN tunnels. Tunnels

MAC-embedded IPv6 (MEv6) Introduced this feature. 7.0(3)I2(1) MAC-Embedded IPv6 Address
address

vPC forklift upgrade support Support for upgrading from a 7.0(3)I1(2) vPC Forklift Upgrade Scenario
pair of Nexus 9000 switches in
a vPC topology to a different
pair of Nexus 9000 series
switches.

Delayed LACP support The delayed LACP feature 7.0(3)I1(2) LACP Compatibility
enables the delay of Enhancements, on page 169
port-channel members to come
up until LACP PDUs are
received.

negotiate auto command Enables configuration of speed, 7.0(3)I1(2) Guidelines and Limitations
support duplex, and automatic flow
control for an Ethernet
interface.

IP-in-IP tunnel support Enables encapsulation and 7.0(3)I1(2) Point-to-Point IP-in-IP Tunnel
decapsulation of packets to Encapsulation and
create a tunnel. Decapsulation

Subinterface support on Support for one or more 7.0(3)I1(2) Configuring a Subinterface on

port-channel interfaces subinterfaces on a port-channel a Port-Channel Interface
interface.

Dynamic Breakout support Enables high bandwidth ports 7.0(3)I1(1) Dynamic Breakout (Per-port
to be split into four breakout Level Breakout)
ports.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
6
New and Changed Information
New and Changed Information

Feature Description Changed Where Documented

in
Release

Symmetric Hashing support When enabled on a port 7.0(3)I1(1) Symmetric Hashing

channel, symmetric hashing
ensures bidirectional traffic
uses the same physical
interface.

Additional show interface Added support for statistics for 7.0(3)I1(1) Verifying the IP Tunnel
tunnel command support. interface counters. Configuration

BFDv6 support Added BFDv6 support. 7.0(3)I1(1) Configuring Bidirectional

Forwarding Detection, on page
121

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
7
 New and Changed Information
New and Changed Information

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
8
 CHAPTER 2
Overview
• Licensing Requirements, on page 9
• About Interfaces, on page 9
• Virtual Device Contexts, on page 17
• High Availability for Interfaces, on page 17

Licensing Requirements
For a complete explanation of Cisco NX-OS licensing recommendations and how to obtain and apply licenses,
see the Cisco NX-OS Licensing Guide.

About Interfaces
Cisco NX-OS supports multiple configuration parameters for each of the interface types supported. Most of
these parameters are covered in this guide but some are described in other documents.
The following table shows where to get further information on the parameters you can configure for an
interface.

Table 2: Interface Parameters

Feature Parameters Further Information

Basic parameters description, duplex, error disable, “Configuring Basic Interface

flow control, MTU, beacon Parameters”

Layer 3 medium, IPv4 and IPv6 addresses “Configuring Layer 3 Interfaces”

Layer 3 bandwidth, delay, IP routing, VRFs Cisco Nexus 9000 Series NX-OS
Unicast Routing Configuration
Guide
Cisco Nexus 9000 Series NX-OS
Multicast Routing Configuration
Guide

Port Channels channel group, LACP “Configuring Port Channels”

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
9
 Overview
Ethernet Interfaces

Feature Parameters Further Information

Security EOU Cisco Nexus 9000 Series NX-OS

Security Configuration Guide

Ethernet Interfaces
Ethernet interfaces include routed ports.

Access Ports
An access port carries traffic for one VLAN. This type of port is a Layer 2 interface only.
For more information on access ports, see the “Information About Access and Trunk Interfaces” section.

Routed Ports
A routed port is a physical port that can route IP traffic to another device. A routed port is a Layer 3 interface
only.
For more information on routed ports, see the “Routed Interfaces” section.

Management Interface
You can use the management Ethernet interface to connect the device to a network for remote management
using a Telnet client, the Simple Network Management Protocol (SNMP), or other management agents. The
management port (mgmt0) is autosensing and operates in full-duplex mode at a speed of 10/100/1000 Mb/s.
For more information on the management interface, see the Cisco Nexus 9000 Series NX-OS Fundamentals
Configuration Guide. You will also find information on configuring the IP address and default IP routing for
the management interface in this document.

Port-Channel Interfaces
A port channel is a logical interface that is an aggregation of multiple physical interfaces. You can bundle up
to 32 individual links to physical ports into a port channel to improve bandwidth and redundancy. You can
also use port channeling to load balance traffic across these channeled physical interfaces. For more information
about port-channel interfaces, see the “Configuring Port Channels” section.

Subinterfaces
You can create virtual subinterfaces on a parent interface configured as a Layer 3 interface. A parent interface
can be a physical port. Subinterfaces divide the parent interface into two or more virtual interfaces on which
you can assign unique Layer 3 parameters such as IP addresses and dynamic routing protocols.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
10
 Overview
Loopback Interfaces

Loopback Interfaces
A virtual loopback interface is a virtual interface with a single endpoint that is always up. Any packet that is
transmitted over a virtual loopback interface is immediately received by that interface. Loopback interfaces
emulate a physical interface. For more information about subinterfaces, see the “Loopback Interfaces” section.

Breakout Interfaces
Cisco NX-OS supports the breakout of high bandwidth 40G interfaces at the module level or at the per-port
level.

Module Level Breakout

For module level breakout, the interface breakout command splits the high bandwidth 40G interface of a
module into four 10G interfaces. The module is reloaded and the configuration for the interface is removed
when the command is executed.
The following is an example of the command:
switch# configure terminal
switch(config)# interface breakout module 1
Module will be reloaded. Are you sure you want to continue(yes/no)? yes

The no interface breakout module module_number command undoes the breakout configuration. It puts all
interfaces of the module in 40G mode and deletes the configuration for the previous 10G interfaces.

Dynamic Breakout (Per-port Level Breakout)

For dynamic breakout (per-port level breakout), the interface breakout command splits high bandwidth
40G ports into four 10G breakout ports and 100G ports into four 25G breakout ports. The breakout ports are
identified as Ethernet <slot>/<front-panel-port>/<breakout-port>. For example, per-port level breakout
ports can be identified as Ethernet 1/2/1, Ethernet 1/2/2, Ethernet 1/2/3, and Ethernet 1/2/4.
When one or more 40G interfaces of the module are broken-out at the per-port level, the configuration for
the interfaces are removed when the command is executed.

Note A per-port level breakout does not require the module to be reloaded.

The following is an example of configuring a breakout port:

switch(config)# interface breakout module 1 port 1 map 10g-4x

switch(config)#

The following is an example of configuring a multiple breakout ports:

switch(config)# interface breakout module 1 port 1-4 map 10g-4x

switch(config)#

The following is an example of configuring 40G and 10G interfaces mixed together.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
11
 Overview
About the Lane Selector

switch(config-if)# show int eth1/49 transceiver

Ethernet1/49
transceiver is present
type is QSFP-40G-SR-BD
name is CISCO-AVAGO
part number is AFBR-79EBPZ-CS2
revision is 01

switch(config-if)# show int eth1/52 transceiver

Ethernet1/52
transceiver is present
type is QSFP-Cazadero
name is CISCO-DNI
part number is CAZADERO-R
revision is 03
nominal bitrate is 10000 MBit/sec per channel

switch(config-if)# show int eth1/53 transceiver

Ethernet1/53
transceiver is present
type is QSFP-Cazadero
name is CISCO-DNI
part number is CAZADERO-R
revision is 03
nominal bitrate is 10000 MBit/sec per channel

switch(config)# interface breakout module 1 port 52-53 map 10g-4x

switch(config-if)# show int br | i up

mgmt0 -- up 10.122.160.192 100 1500
Eth1/49 -- eth routed up none 40G(D) — << Running 40G
Eth1/50 -- eth routed up none 40G(D) --
Eth1/52/1 -- eth routed up none 10G(D) — << Broken out to 10G
Eth1/53/1 -- eth routed up none 10G(D) -- << Broken out to 10G

You can undo the breakout ports with the no interface breakout command.
The following is an example to undo the breakout ports:

switch(config)# no interface breakout module 1 port 1 map 10g-4x

switch(config)#

About the Lane Selector

The lane selector is a push button switch and 4 LEDs located on the Cisco Nexus switch (left side of front
panel, labeled "LS"). The push button switch and LEDs are used to indicate the status of the ports. The lane
selector is supported on Cisco Nexus Series 9000 series switches and the Cisco Nexus 3164 and 3232 switches.
By default, the LEDs indicate the link/activity status of a 1 x 40G configuration. When the ports are configured
as 4 x 10G, you can access the link status of each individual 10G port with the lane selector.
By pressing the lane selector push button, the port LED shows the selected lane’s link/activity status. The 1st
time the push button is pressed, the first LED displays the status of the first port. Pressing the push button a
2nd time displays the status of the second port, and so on. You can display the status of each of the four ports
by pressing the push button in this manner.
For example, if port 60 is configured as 4 x 10G, pressing the lane selector push button once displays the link
status of 60/1/1. Pressing the push button a second time displays the link status of 60/1/2.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
12
 Overview
Notes About Breakout Interfaces

When you press the push button after displaying the status of the last port, all four of the LEDs should extinguish
to indicate that the lane selector has returned to display the status for the default 1 x 40G configuration.

Note A 10G breakout port's LED blinks when the beacon feature has been configured for it.

Note When a port is configured to be in 10G breakout mode and no lane is selected, the 40G port's LED illuminates
as green even though only one of the 10G breakout ports is up.

Notes About Breakout Interfaces

Beginning with Cisco NX-OS Release 7.0(3)F2(1), the 36-port 100-Gigabit Ethernet QSFP28 line cards
(N9K-X9636C-R) and 36-port 40-Gigabit Ethernet QSFP+ line cards (N9K-X9636Q-R) provide 4x10-Gigabit
support.

Caveats
• As of Cisco NX-OS Release 7.0(3)I7(2), manual breakout of QSA ports is not supported.
• When a break-out port is configured as a part of a port-channel, you need to apply the configuration
twice (after write-erase/reload), to ensure the effectiveness of the port-channel.
• When you upgrade a Cisco Nexus 9000 device to Cisco NX-OS Release 7.0(3)I7(2) or later, if a QSFP
port is configured with the manual breakout command and is using a QSA, the configuration of the
interface Ethernet 1/50/1 is no longer supported and will need to be removed. To restore the configuration,
you must manually configure the Ethernet 1/50 on the device.
• Cisco Nexus 9000 Series switches have 40G ports. When you breakout one of the 40G ports into 4x10G
ports with a QSFP breakout cable, not all of the subinterfaces can be added to a port channel. The following
error message is displayed:

switch# channel-group 99 mode active

command failed: port not compatible [Buffer boost]

Note As a workaround, configure no buffer-boost on all of the subinterfaces. This

enables the channel-group configuration to go through.

Using the force keyword on the port-channel does not enable all the interfaces to be added to the
port-channel even though the error message suggests using the keyword.
• Beginning with Cisco NX-OS Release 7.0(3)I7(3) you see two additional options to configure FEC such
as rs-cons16 and rs-ieee as per IEEE standards.
• Beginning with Cisco NX-OS Release 7.0(3)I7(7) you can display the admin and oper status of FEC
interface information with the show interface fec command.
Example:

switch# show interface fec

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
13
 Overview
High Bandwidth Interfaces

---------------------------------------------------------------------------------------------------
Name Ifindex Admin-fec Oper-fec Status Speed Type
---------------------------------------------------------------------------------------------------
Eth1/1 0x1a000000 auto auto connected 10G SFP-H10GB-AOC2M
Eth1/2 0x1a000200 Rs-fec Rs-fec notconneced auto QSFP-100G-AOC3M
Eth1/3/1 0x38014000 auto auto disabled auto QSFP-H40G-AOC3M
Eth1/3/2 0x38015000 auto auto disabled auto QSFP-H40G-AOC3M
Eth1/3/3 0x38016000 auto auto disabled auto QSFP-H40G-AOC3M
Eth1/3/4 0x38017000 auto auto disabled auto QSFP-H40G-AOC3M

Note Auto-FEC is not supported in Cisco NX-OS Release 7.0(3)I7(x)

High Bandwidth Interfaces

The breakout of high bandwidth interfaces (module level or per-port level) are supported only on:
• The X9636PQ, X9432PQ, X9536PQ, and X9732C-EX line cards on a Cisco Nexus 9500 Series switch.
• The Cisco Nexus 9332PQ switch.
• The Cisco Nexus 3164Q switch.

Cisco Nexus C92160YC Switch

For 7.0(3)I3(1) and later, the Cisco Nexus C92160YC switch provides two different modes of operation:
• Mode 1: 48 * 10G/25G + 4 * 40G + 2 * 100G (Default configuration)
• Hardware profile portmode 48x25G + 2x100G + 4x40G
• Breakout is supported in 2 * 100G ports

• Mode 2: 48 * 10G/25G + 4 * 100G

• Hardware profile portmode 48x25G + 4x100G
• Breakout is supported on 3 * 100G ports (Ports 50,51 & 52).

Use the show running-config | grep portmode command to display the current operation mode.
Example:

switch(config-if-range)# show running-config | grep portmode

hardware profile portmode 48x25G+2x100G+4x40G

See the installation guide for the Cisco Nexus C92160YC switches for more information. ( Install and Upgrade
Guides for Cisco Nexus 9000 Series Switches )
With the Cisco Nexus C92160YC switch , there are two breakout modes:
• 40G to 4x10G breakout ports
• Enables the breakout of 40G ports into 4 X 10G ports.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
14
 Overview
Cisco Nexus C9272Q Switch

• Use the interface breakout module 1 port x map 10g-4x command.

• 100G to 4x25G breakout ports

• Enables the breakout of 100G ports into 4 X 25G ports.
• Use the interface breakout module 1 port x map 25g-4x command.

Cisco Nexus C9272Q Switch

For 7.0(3)I3(1) and later, the Cisco Nexus C9272Q Switch provides 72 40G ports. Ports 37 - 71 support
breakout interfaces.
To configure a breakout interface, use the interface breakout module 1 port x map 10g-4x command.
Example:

switch(config)# interface breakout module 1 port 38 map 10g-4x

switch(config)# show interface ethernet 1/38 capabilities | grep -i break

Breakout capable: yes

Cisco Nexus C9332PQ Switch

For 7.0(3)I3(1) and later, the Cisco Nexus C9332PQ Switch provides 24 40G ports that support breakout
mode and can be connected to four 10G NIF ports on a FEX. Ports 1 - 12 and 15 - 26 ports are supported.
(Ports 13 and 14 ports are reserved and cannot be used for breakout mode.)

Note All FEXs are supported.

Note Only the Cisco Nexus 9332PQ switch provides interface breakout support for FEX fabric interfaces. (7.0(3)I3(1)
and later)

Cisco Nexus 9000 C93180LC-EX Switch

For 7.0(3)I7(1) and later, Cisco Nexus 9000 C93180LC-EX switch provides three different modes of operation:
• Mode 1: 28 x 40G + 4 x 40G/100G (Default configuration)
• Hardware profile portmode 4x100g + 28x40g.
• 10x4 breakout is supported on the top ports from 1 to 27 (ports 1,3,5, 7...27). If any of the top port
is broken out, the corresponding bottom port becomes non-operational. For example, if port 1 is
broken out port 2 becomes non-operational.
• 1 Gigabit and 10 Gigabit QSA is supported on ports 29, 30, 31, and 32. However, QSAs on the top
and bottom front panel ports must be of same speed.
• Ports 29, 30, 31, and 32 support 10x4, 25x4, and 50x2 breakout.

• Mode 2: 24 x 40G + 6 x 40G/100G

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
15
 Overview
Cisco Nexus 9000 C93180LC-EX Switch

• Hardware profile portmode 6x100g + 24x40g.

• 10x4 breakout is supported on the top ports from 1 to 23 (ports 1,3,5, 7...23). If any of the top port
is broken out the corresponding bottom port becomes non-operational.
• Ports 25, 27, 29, 30, 31, and 32 support 10x4, 25x4, and 50x2 breakout.
• 1 Gigabit and 10 Gigabit QSA is supported on ports 29, 30, 31, and 32. However, QSAs on the top
and bottom front panel ports must be of same speed.

• Mode 3: 18 x 40G/100G
• Hardware profile portmode 18x100g.
• 10x4, 25x4, and 50x2 breakout is supported on top ports from 1 to 27 (ports 1,3,5, 7...27) and on
ports 29,30,31,32.
• 1 Gigabit and 10 Gigabit QSA is supported on all the 18 ports.

Changing Mode 3 to any other mode or vice versa requires copy running-config startup-config command
followed by reload command to take effect. However, moving between Modes 1 and 2 is dynamic and
requires only copy running-config startup-config command.
Use the show running-config | grep portmode command to display the current operation mode.
Example:

switch(config-if-range)# show running-config | grep portmode

hardware profile portmode 4x100G+28x40G

With the Cisco Nexus C93180LC-EX switch, there are three breakout modes:
• 40G to 4x10G breakout ports
• Enables the breakout of 40G ports into 4 X 10G ports.
• Use the interface breakout module 1 port x map 10g-4x command.

• 100G to 4x25G breakout ports

• Enables the breakout of 100G ports into 4 X 25G ports.
• Use the interface breakout module 1 port x map 25g-4x command.

• 100G to 2x50G breakout ports

• Enables the breakout of 100G ports into 2 X 50G ports.
• Use the interface breakout module 1 port x map 50g-2x command.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
16
 Overview
Virtual Device Contexts

Virtual Device Contexts

Cisco NX-OS can segment operating system and hardware resources into virtual device contexts (VDCs) that
emulate virtual devices. The Cisco Nexus 9000 Series switch does not support multiple VDCs. All switch
resources are managed in the default VDC.

High Availability for Interfaces

Interfaces support stateful and stateless restarts. A stateful restart occurs on a supervisor switchover. After
the switchover, Cisco NX-OS applies the runtime configuration.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
17
 Overview
High Availability for Interfaces

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
18
 CHAPTER 3
Configuring Basic Interface Parameters
This chapter describes how to configure the basic interface parameters on Cisco NX-OS devices.
• About the Basic Interface Parameters, on page 19
• Guidelines and Limitations, on page 27
• Default Settings, on page 29
• Configuring the Basic Interface Parameters, on page 30
• Verifying the Basic Interface Parameters, on page 53
• Monitoring the Interface Counters, on page 54
• Configuration Example for QSA, on page 56

About the Basic Interface Parameters

Description
For the Ethernet and management interfaces, you can configure the description parameter to provide a
recognizable name for the interface. Using a unique name for each interface allows you to quickly identify
the interface when you are looking at a listing of multiple interfaces.
For information about setting the description parameter for port-channel interfaces, see the “Configuring a
Port-Channel Description” section. For information about configuring this parameter for other interfaces, see
the “Configuring the Description” section.

Beacon
The beacon mode allows you to identify a physical port by flashing its link state LED with a green light. By
default, this mode is disabled. To identify the physical port for an interface, you can activate the beacon
parameter for the interface.
For information about configuring the beacon parameter, see the “Configuring the Beacon Mode” section.

Error Disabled
A port is in the error-disabled (err-disabled) state when the port is enabled administratively (using the no
shutdown command) but disabled at runtime by any process. For example, if UDLD detects a unidirectional
link, the port is shut down at runtime. However, because the port is administratively enabled, the port status

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
19
 Configuring Basic Interface Parameters
Interface Status Error Policy

displays as err-disable. Once a port goes into the err-disable state, you must manually reenable it or you can
configure a timeout value that provides an automatic recovery. By default, the automatic recovery is not
configured, and by default, the err-disable detection is enabled for all causes.
When an interface is in the err-disabled state, use the errdisable detect cause command to find information
about the error.
You can configure the automatic error-disabled recovery timeout for a particular error-disabled cause and
configure the recovery period.
The errdisable recovery cause command provides an automatic recovery after 300 seconds.
You can use the errdisable recovery interval command to change the recovery period within a range of 30
to 65535 seconds. You can also configure the recovery timeout for a particular err-disable cause.
If you do not enable the error-disabled recovery for the cause, the interface stays in the error-disabled state
until you enter the shutdown and no shutdown commands. If the recovery is enabled for a cause, the interface
is brought out of the error-disabled state and allowed to retry operation once all the causes have timed out.
Use the show interface status err-disabled command to display the reason behind the error.

Interface Status Error Policy

Cisco NX-OS policy servers such as Access Control List (ACL) Manager and Quality of Service (QoS)
Manager, maintain a policy database. A policy is defined through the command-line interface.
Policies are pushed when you configure a policy on an interface to ensure that policies that are pushed are
consistent with the hardware policies. To clear the errors and to allow the policy programming to proceed
with the running configuration, enter the no shutdown command. If the policy programming succeeds, the
port is allowed to come up. If the policy programming fails, the configuration is inconsistent with the hardware
policies and the port is placed in an error-disabled policy state. The error-disabled policy state remains and
the information is stored to prevent the same port from being brought up in the future. This process helps to
avoid unnecessary disruption to the system.

Modifying Interface MTU Size

The maximum transmission unit (MTU) size specifies the maximum frame size that an Ethernet port can
process. For transmissions to occur between two ports, you must configure the same MTU size for both ports.
A port drops any frames that exceed its MTU size.
Cisco NX-OS allows you to configure MTU on an interface, with options to configure it on different level in
the protocol stack. By default, each interface has an MTU of 1500 bytes, which is the IEEE 802.3 standard
for Ethernet frames. Larger MTU sizes are possible for more efficient processing of data to allow different
application requirements. The larger frames, are also called jumbo frames, can be up to 9216 bytes in size.
MTU is configured per interface, where an interface can be a Layer 2 or a Layer 3 interface. For a Layer 2
interface, you can configure the MTU size with one of two values, the value system default MTU value or
the system jumbo MTU value. The system default MTU value is 1500 bytes. Every Layer 2 interface is
configured with this value by default. You can configure an interface with the default system jumbo MTU
value, that is 9216 bytes. To allow an MTU value from 1500 through 9216, you must adjust the system jumbo
MTU to an appropriate value where interface can be configured with the same value.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
20
 Configuring Basic Interface Parameters
Bandwidth

Note You can change the system jumbo MTU size. When the value is changed, the Layer 2 interfaces that use the
system jumbo MTU value, will automatically changes to the new system jumbo MTU value.

A Layer 3 interface, can be Layer 3 physical interface (configure with no switchport), switch virtual interface
(SVI), and sub-interface, you can configure an MTU size between 576 and 9216 bytes.
For the Cisco Nexus 9372 switch, the following applies:
• The 10-G interfaces are mapped to specific hardware ports where the default MTU is 1500.
• The 40-G interfaces are mapped as a HiGiG port where the default MTU is 3FFF and the MTU limit
check is disabled.
• In the case of 40-G interfaces, since the MTU limit check is disabled, it ignores the packet size and traffic
flows irrespective of its MTU.

For information about setting the MTU size, see the Configuring the MTU Size section.

Note On Cisco Nexus 9300-FX2 and 9300-GX devices, if ingress interface is configured with an MTU less than
9216, FTE does not capture input errors and does not display any events. However, if the ingress interface is
configured with an MTU of 9216, FTE displays all the events.

Bandwidth
Ethernet ports have a fixed bandwidth of 1,000,000 Kb at the physical layer. Layer 3 protocols use a bandwidth
value that you can set for calculating their internal metrics. The value that you set is used for informational
purposes only by the Layer 3 protocols—it does not change the fixed bandwidth at the physical layer. For
example, the Enhanced Interior Gateway Routing Protocol (EIGRP) uses the minimum path bandwidth to
determine a routing metric, but the bandwidth at the physical layer remains at 1,000,000 Kb.
For information about configuring the bandwidth parameter for port-channel interfaces, see the “Configuring
the Bandwidth and Delay for Informational Purposes” section. For information about configuring the bandwidth
parameter for other interfaces, see the “Configuring the Bandwidth” section.

Throughput Delay
Specifying a value for the throughput-delay parameter provides a value used by Layer 3 protocols; it does not
change the actual throughput delay of an interface. The Layer 3 protocols can use this value to make operating
decisions. For example, the Enhanced Interior Gateway Routing Protocol (EIGRP) can use the delay setting
to set a preference for one Ethernet link over another, if other parameters such as link speed are equal. The
delay value that you set is in the tens of microseconds.
For information about configuring the bandwidth parameter for port-channel interfaces, see the “Configuring
the Bandwidth and Delay for Informational Purposes” section. For information about configuring the
throughput-delay parameter for other interfaces, see the “Configuring the Throughput Delay” section.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
21
 Configuring Basic Interface Parameters
Administrative Status

Administrative Status
The administrative-status parameter determines whether an interface is up or down. When an interface is
administratively down, it is disabled and unable to transmit data. When an interface is administratively up, it
is enabled and able to transmit data.
For information about configuring the administrative status parameter for port-channel interfaces, see the
“Shutting Down and Restarting the Port-Channel Interface” section. For information about configuring the
administrative-status parameter for other interfaces, see the “Shutting Down and Activating the Interface”
section.

Unidirectional Link Detection Parameter

UDLD Overview
The Cisco-proprietary Unidirectional Link Detection (UDLD) protocol allows devices that are connected
through fiber-optic or copper (for example, Category 5 cabling) Ethernet cables to monitor the physical
configuration of the cables and detect when a unidirectional link exists. When a device detects a unidirectional
link, UDLD shuts down the affected LAN port and alerts the user. Unidirectional links can cause a variety of
problems.
UDLD performs tasks that autonegotiation cannot perform, such as detecting the identities of neighbors and
shutting down misconnected LAN ports. When you enable both autonegotiation and UDLD, Layer 1 detections
work to prevent physical and logical unidirectional connections and the malfunctioning of other protocols.
A unidirectional link occurs whenever traffic transmitted by the local device over a link is received by the
neighbor but traffic transmitted from the neighbor is not received by the local device. If one of the fiber strands
in a pair is disconnected, as long as autonegotiation is active, the link does not stay up. In this case, the logical
link is undetermined, and UDLD does not take any action. If both fibers are working normally at Layer 1,
UDLD determines whether those fibers are connected correctly and whether traffic is flowing bidirectionally
between the correct neighbors. This check cannot be performed by autonegotiation, because autonegotiation
operates at Layer 1.
The Cisco Nexus 9000 Series device periodically transmits UDLD frames to neighbor devices on LAN ports
with UDLD enabled. If the frames are echoed back within a specific time frame and they lack a specific
acknowledgment (echo), the link is flagged as unidirectional and the LAN port is shut down. Devices on both
ends of the link must support UDLD in order for the protocol to successfully identify and disable unidirectional
links. You can configure the transmission interval for the UDLD frames, either globally or for the specified
interfaces.

Note By default, UDLD is locally disabled on copper LAN ports to avoid sending unnecessary control traffic on
this type of media.

The figure shows an example of a unidirectional link condition. Device B successfully receives traffic from
device A on the port. However, device A does not receive traffic from device B on the same port. UDLD
detects the problem and disables the port.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
22
 Configuring Basic Interface Parameters
Default UDLD Configuration

Figure 1: Unidirectional Link

Default UDLD Configuration

The following table shows the default UDLD configuration.

Table 3: UDLD Default Configuration

Feature Default Value

UDLD global enable state Globally disabled

UDLD per-port enable state for fiber-optic media Enabled on all Ethernet fiber-optic LAN ports

UDLD per-port enable state for twisted-pair (copper) Disabled on all Ethernet 10/100 and 1000BASE-TX
media LAN ports

UDLD aggressive mode Disabled

UDLD message interval 15 seconds

For information about configuring the UDLD for the device and its port, see the “Configuring the UDLD
Mode” section.

UDLD Aggressive and Nonaggressive Modes

UDLD aggressive mode is disabled by default. You can configure UDLD aggressive mode only on
point-to-point links between network devices that support UDLD aggressive mode. If UDLD aggressive mode
is enabled, when a port on a bidirectional link that has a UDLD neighbor relationship established stops
receiving UDLD frame, UDLD tries to reestablish the connection with the neighbor. After eight failed retries,
the port is disabled.
When you enable the UDLD aggressive mode, the following occurs:
One side of a link has a port stuck (both transmission and receive)
One side of a link remains up while the other side of the link is down
In these cases, the UDLD aggressive mode disables one of the ports on the link, which prevents traffic from
being discarded.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
23
 Configuring Basic Interface Parameters
Port-Channel Parameters

Note You enable the UDLD aggressive mode globally to enable that mode on all the fiber ports. You must enable
the UDLD aggressive mode on copper ports on specified interfaces.

Tip When a line card upgrade is being performed during an in-service software upgrade (ISSU) and some of the
ports on the line card are members of a Layer 2 port channel and are configured with UDLD aggressive mode,
if you shut down one of the remote ports, UDLD puts the corresponding port on the local device into an
error-disabled state. This behavior is correct.

To restore service after the ISSU has completed, enter the shutdown command followed by the no shutdown
command on the local port.

Port-Channel Parameters
A port channel is an aggregation of physical interfaces that comprise a logical interface. You can bundle up
to 32 individual interfaces into a port channel to provide increased bandwidth and redundancy. Port channeling
also load balances traffic across these physical interfaces. The port channel stays operational if at least one
physical interface within the port channel is operational.
You can create Layer 3 port channels by bundling compatible Layer 3 interfaces.
Any configuration changes that you apply to the port channel are applied to each interface member of that
port channel.
For information about port channels and for information about configuring port channels, see Chapter 6,
“Configuring Port Channels.”

Port Profiles
Beginning with 7.0(3)I4(1) on Cisco Nexus 9300 Series switches, you can create a port profile that contains
many interface commands and apply that port profile to a range of interfaces. Each port profile can be applied
only to a specific type of interface; the choices are as follows:
• Ethernet
• VLAN network interface
• Port channel

When you choose Ethernet or port channel as the interface type, the port profile is in the default mode which
is Layer 3. Enter the switchport command to change the port profile to Layer 2 mode.
You inherit the port profile when you attach the port profile to an interface or range of interfaces. When you
attach, or inherit, a port profile to an interface or range of interfaces, the system applies all the commands in
that port profile to the interfaces. Additionally, you can have one port profile inherit the settings from another
port profile. Inheriting another port profile allows the initial port profile to assume all of the commands of
the second, inherited, port profile that do not conflict with the initial port profile. Four levels of inheritance
are supported. The same port profile can be inherited by any number of port profiles.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
24
Configuring Basic Interface Parameters
Port Profiles

The system applies the commands inherited by the interface or range of interfaces according to the following
guidelines:
• Commands that you enter under the interface mode take precedence over the port profile’s commands
if there is a conflict. However, the port profile retains that command in the port profile.
• The port profile’s commands take precedence over the default commands on the interface, unless the
port-profile command is explicitly overridden by the default command.
• When a range of interfaces inherits a second port profile, the commands of the initial port profile override
the commands of the second port profile if there is a conflict.
• After you inherit a port profile onto an interface or range of interfaces, you can override individual
configuration values by entering the new value at the interface configuration level. If you remove the
individual configuration values at the interface configuration level, the interface uses the values in the
port profile again.
• There are no default configurations associated with a port profile.

A subset of commands are available under the port-profile configuration mode, depending on which interface
type you specify.

Note You cannot use port profiles with Session Manager. See the Cisco Nexus 9000 Series NX-OS System
Management Configuration Guide for information about Session Manager.

To apply the port-profile configurations to the interfaces, you must enable the specific port profile. You can
configure and inherit a port profile onto a range of interfaces prior to enabling the port profile. You would
then enable that port profile for the configurations to take effect on the specified interfaces.
If you inherit one or more port profiles onto an original port profile, only the last inherited port profile must
be enabled; the system assumes that the underlying port profiles are enabled.
When you remove a port profile from a range of interfaces, the system undoes the configuration from the
interfaces first and then removes the port-profile link itself. Also, when you remove a port profile, the system
checks the interface configuration and either skips the port-profile commands that have been overridden by
directly entered interface commands or returns the command to the default value.
If you want to delete a port profile that has been inherited by other port profiles, you must remove the inheritance
before you can delete the port profile.
You can also choose a subset of interfaces from which to remove a port profile from among that group of
interfaces that you originally applied the profile. For example, if you configured a port profile and configured
ten interfaces to inherit that port profile, you can remove the port profile from just some of the specified ten
interfaces. The port profile continues to operate on the remaining interfaces to which it is applied.
If you delete a specific configuration for a specified range of interfaces using the interface configuration mode,
that configuration is also deleted from the port profile for that range of interfaces only. For example, if you
have a channel group inside a port profile and you are in the interface configuration mode and you delete that
port channel, the specified port channel is also deleted from the port profile as well.
Just as in the device, you can enter a configuration for an object in port profiles without that object being
applied to interfaces yet. For example, you can configure a virtual routing and forward (VRF) instance without
it being applied to the system. If you then delete that VRF and related configurations from the port profile,
the system is unaffected.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
25
 Configuring Basic Interface Parameters
Cisco QSFP+ to SFP+ Adapter Module Support

After you inherit a port profile on an interface or range of interfaces and you delete a specific configuration
value, that port-profile configuration is not operative on the specified interfaces.
If you attempt to apply a port profile to the wrong type of interface, the system returns an error.
When you attempt to enable, inherit, or modify a port profile, the system creates a checkpoint. If the port-profile
configuration fails, the system rolls back to the prior configuration and returns an error. A port profile is never
only partially applied.

Cisco QSFP+ to SFP+ Adapter Module Support

The Cisco QSFP+ to SFP+ Adapter (QSA) module provides 10G support for the 40G uplink ports that are a
part of the Cisco Nexus M6PQ and Cisco Nexus M12PQ uplink modules of specific Cisco Nexus 9300 devices.
A group of six consecutive ports in the M6PQ or M12PQ uplink module must be operating at the same speed
(40G or 10G) to use the QSA/QSFP modules.
• For Cisco Nexus 9396PX devices, 2/1-6 ports form the first port speed group and the remaining 2/7-12
ports form the second port speed group.
• For Cisco Nexus 93128PX/TX devices, 2/1-6 ports form the first port speed group and the remaining
2/7-8 ports form the second port speed group.
• For Cisco Nexus 937xPX/TX devices, 1/49-54 ports form the only port speed group.
• For Cisco Nexus 93120TX devices, 1/97-102 ports form the only port speed group.
• For Cisco Nexus 9332PQ devices, 1/27-32 ports form the only port speed group.

Use the speed-group 10000 command to configure the first port of a port speed group for the QSA. This
command specifies the administrator speed preference for the port group. (The default port speed is 40G.)
• The speed-group 10000 command specifies a speed of 10G.
• The no speed-group 10000 command specifies a speed of 40G.
• If you add, modify, or remove the speed-group command on any interface in the speed group, all interfaces
in the speed group will change to the newly configured speed. This may cause link state changes based
on the transceiver that is plugged in. This also includes the removal of an existing speed-group command
using the default interface command.
• Uplink modules should not be removed from a Cisco Nexus 9300 platform switch that is running Cisco
NX-OS Release 7.0(3)I7(5). The ports on uplink modules should be used only for uplinks.

After the speed has been configured, the compatible transceiver modules are enabled. The remaining transceiver
modules in the port group (incompatible transceiver modules) become error disabled with a reason of "check
speed-group config".

Note The Cisco QSFP+ to SFP+ Adapter (QSA) module does not provide 10G support for the 40G line cards for
Cisco Nexus 9500 devices.
Beginning with Cisco NX-OS Release 7.0(3)I5(2), you can use a QSFP-to-SFP adapter on Cisco Nexus 9200
and 9300-EX Series switches and Cisco Nexus 3232C and 3264Q Series switches.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
26
 Configuring Basic Interface Parameters
Cisco SFP+ Adapter Module Support

Cisco SFP+ Adapter Module Support

Beginning with Cisco NX-OS Release 7.0(3)I4(2), you can use the CVR-2QSFP28-8SFP adapter for 25-Gigabit
optics support on 100-Gigabit ports of the Cisco Nexus 9236C switch.
The interface breakout module command can be used to split this switch's 100G interfaces into four 25G
interfaces. After you enter this command, you must copy the running configuration to the startup configuration.

25G Autonegotiation Overview

Guidelines and Limitations

Basic interface parameters have the following configuration guidelines and limitations:
• MDIX is enabled by default on copper ports. It is not possible to disable it.
• show commands with the internal keyword are not supported.
• Fiber-optic Ethernet ports must use Cisco-supported transceivers. To verify that the ports are using
Cisco-supported transceivers, use the show interface transceivers command. Interfaces with
Cisco-supported transceivers are listed as functional interfaces.
• A port can be either a Layer 2 or a Layer 3 interface; it cannot be both simultaneously.
By default, each port is a Layer 3 interface.
You can change a Layer 3 interface into a Layer 2 interface by using the switchport command. You can
change a Layer 2 interface into a Layer 3 interface by using the no switchport command.
• You usually configure Ethernet port speed and duplex mode parameters to auto to allow the system to
negotiate the speed and duplex mode between ports. If you decide to configure the port speed and duplex
modes manually for these ports, consider the following:
• Before you configure the speed and duplex mode for an Ethernet or management interface, see the
Default Settings section for the combinations of speeds and duplex modes that can be configured
at the same time.
• If you set the Ethernet port speed to auto, the device automatically sets the duplex mode to auto.
• If you enter the no speed command, the device automatically sets both the speed and duplex
parameters to auto (the no speed command produces the same results as the speed auto command).
• If you configure an Ethernet port speed to a value other than auto (for example, 1G, 10G, or 40G),
you must configure the connecting port to match. Do not configure the connecting port to negotiate
the speed.
• To configure speed, duplex, and automatic flow control for an Ethernet interface, you can use the
negotiate auto command. To disable automatic negotiation, use the no negotiate auto command.

Note The device cannot automatically negotiate the Ethernet port speed and duplex
mode if the connecting port is configured to a value other than auto.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
27
 Configuring Basic Interface Parameters
Guidelines and Limitations

Caution Changing the Ethernet port speed and duplex mode configuration might shut
down and reenable the interface.

• When using a QSFP-40G-CR4 cable to connect between N9K-C9332PQ non-ALE ports and
N9K-C9372PX ALE ports, you need to manually set the speed to 40000.
• For BASE-T copper ports, auto-negotiation is enabled even when fixed speed is configured.
• For 7.0(3)I2(1) and later, using a regular expression to address a set of interfaces is supported with the
regex command option. The regex command option is an extension that is available for all interface
commands.
Example:

switch(config-if-range)# interface ethernet regex [2]/

switch(config-if-range)# where
conf; interface Ethernet2/1-8 admin@switch%default
switch(config-if-range)# interface ethernet regex [1]/2[2-4]
switch(config-if-range)# where
conf; interface Ethernet1/22-24 admin@switch%default

• For 7.0(3)I2(1) and later, the source-interface command option provides support for management
applications to configure an IPv4 and/or IPv6 inband or outband source IP address for the copy command
and other processes (such as tacacs, ntp, ping/ping6, icmp-error and traceroute).
• Configuration commands
ip services source-interface interface vrf vrf name
Examples:
• ip ftp source-interface ethernet 8/1 vrf management

• ip http source-interface loopback 1 vrf blue

• ip ssh source-interface ethernet ethernet 5/1

/*This command executes in the VRF context.*/

• ip ping source-interface ethernet 8/1 vrf blue

• ip traceroute source-interface ethernet 8/1 vrf red

• ip icmp-errors source-interface ethernet 8/1

/*This command executes in the VRF context.*/

• Show commands:
show ip copy services source-interface interface vrf vrf name
• show ip ftp source-interface ethernet 8/1 vrf management

• show ip http source-interface loopback 1 vrf blue

• show ip ssh source-interface ethernet ethernet 5/1

/*This command executes in the VRF context.*/

• show ip ping source-interface ethernet 8/1 vrf blue

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
28
 Configuring Basic Interface Parameters
Default Settings

• show ip traceroute source-interface ethernet 8/1 vrf red

• show ip icmp-errors source-interface ethernet 8/1

/*This command executes in the VRF context.*/

• Service commands:
copy service://username@hostname/path file source-interface interface name
Examples:
• copy ftp://username@hostname/usr/local/bin file source-interface ethernet 8/1

• copy scp://username@hostname/usr/local/bin file source-interface ethernet 8/1

• copy tftp://username@hostname/usr/local/bin file source-interface ethernet 8/1

• copy http://username@hostname/usr/local/bin file source-interface ethernet 8/1

• copy sftp://username@hostname/usr/local/bin file source-interface ethernet 8/1

• Beginning with Cisco NX-OS Release 7.0(3)I5(2), port profiles are supported on Cisco Nexus 9300
Series switches and Cisco Nexus 9500 Series switches.
• When using a QSFP-40G-CR4 on Cisco Nexus 9000 switches, you must configure the default speed as
40G in the auto-negotiation parameters. Otherwise, the interface may not be able to bring the link up.
• When you use the QSFP ports on N9K-X9788TC-FX line card, you must set the speed appropriately
and disable auto negotiation on ports 49 to 52. You must also set the speed and disable auto negotiation
on the peering device.

Default Settings
The following lists the default settings for the basic interface parameters.

Parameter Default

Description Blank

Beacon Disabled

Bandwidth Data rate of interface

Throughput delay 100 microseconds

Administrative status Shutdown

MTU 1500 bytes

UDLD global Globally disabled

UDLD per-port enable state for fiber-optic media Enabled on all Ethernet fiber-optic LAN ports

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
29
 Configuring Basic Interface Parameters
Configuring the Basic Interface Parameters

Parameter Default

UDLD per-port enable state for copper media Disabled on all Ethernet 1G, 10G, or 40G LAN ports

UDLD message interval Disabled

UDLD aggressive mode Disabled

Error disable Disabled

Error disable recovery Disabled

Error disable recovery interval 300 seconds

Buffer-boost Enabled
Note Feature available on N9K-X9564TX and
N9K-X9564PX line cards and Cisco Nexus
9300 series devices.

Configuring the Basic Interface Parameters

When you configure an interface, you must specify the interface before you can configure its parameters.

Specifying the Interfaces to Configure

Before you begin
Before you can configure the parameters for one or more interfaces of the same type, you must specify the
type and the identities of the interfaces.
The following table shows the interface types and identities that you should use for specifying the Ethernet
and management interfaces.

Table 4: Information Needed to Identify an Interface for Configurations

Interface Type Identity

Ethernet I/O module slot numbers and port numbers on the

module

Management 0 (for port 0)

The interface range configuration mode allows you to configure multiple interfaces with the same configuration
parameters. After you enter the interface range configuration mode, all command parameters you enter are
attributed to all interfaces within that range until you exit out of the interface range configuration mode.
You enter a range of interfaces using dashes (-) and commas (,). Dashes separate contiguous interfaces and
commas separate noncontiguous interfaces. When you enter noncontiguous interfaces, you must enter the
media type for each interface.
This example shows how to configure a contiguous interface range:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
30
Configuring Basic Interface Parameters
Specifying the Interfaces to Configure

switch(config)# interface ethernet 2/29-30

switch(config-if-range)#

This example shows how to configure a noncontiguous interface range:

switch(config)# interface ethernet 2/29, ethernet 2/33, ethernet 2/35
switch(config-if-range)#

You can specify subinterfaces in a range only when the subinterfaces are on the same port, for example,
2/29.1-2. But you cannot specify the subinterfaces in a range of ports, for example, you cannot enter
2/29.2-2/30.2. You can specify two of the subinterfaces discretely, for example, you can enter 2/29.2, 2/30.2.
This example shows how to configure a a breakout cable:
switch(config)# interface ethernet 1/2/1
switch(config-if-range)#

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface interface Specifies the interface that you are configuring.
You can specify the interface type and identity.
Example:
For an Ethernet port, use ethernet slot/port. For
switch(config)# interface ethernet 2/1 the management interface, use mgmt0.
switch(config-if)#
Examples:
Example:
switch(config)# interface mgmt0 • The 1st example shows how to specify the
switch(config-if)# slot 2, port 1 Ethernet interface.
• The 2nd example shows how to specify
the management interface.

Note You do not need to add a space

between the interface type and
identity (port or slot/port number)
For example, for the Ethernet slot 4,
port 5 interface, you can specify
either “ethernet 4/5” or
“ethernet4/5.” The management
interface is either “mgmt0” or
“mgmt 0.”

When you are in the interface configuration

mode, the commands that you enter configure
the interface that you specified for this mode.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
31
 Configuring Basic Interface Parameters
Configuring the Description

Configuring the Description

You can provide textual interface descriptions for the Ethernet and management interfaces.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface interface Specifies the interface that you are configuring.
You can specify the interface type and identity.
Example:
For an Ethernet port, use ethernet slot/port. For
switch(config)# interface ethernet 2/1 the management interface, use mgmt0.
switch(config-if)#
Examples:
Example:
switch(config)# interface mgmt0 • The 1st example shows how to specify the
switch(config-if)# slot 2, port 1 Ethernet interface.
• The 2nd example shows how to specify
the management interface.

Step 3 description text Specifies the description for the interface.

Example:
switch(config-if)# description Ethernet
port 3 on module 1
switch(config-if)#

Step 4 show interface interface (Optional) Displays the interface status, which
includes the description parameter.
Example:
switch(config)# show interface ethernet
2/1

Step 5 exit Exits the interface mode.

Example:
switch(config-if)# exit
switch(config)#

Step 6 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to set the interface description to Ethernet port 24 on module 3:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
32
 Configuring Basic Interface Parameters
Configuring the Beacon Mode

switch# configure terminal

switch(config)# interface ethernet 3/24
switch(config-if)# description server1
switch(config-if)#

The output of the show interface eth command is enhanced as shown in the following example:

Switch# show version

Software
BIOS: version 06.26
NXOS: version 6.1(2)I2(1) [build 6.1(2)I2.1]
BIOS compile time: 01/15/2014
NXOS image file is: bootflash:///n9000-dk9.6.1.2.I2.1.bin
NXOS compile time: 2/25/2014 2:00:00 [02/25/2014 10:39:03]

switch# show interface ethernet 6/36

Ethernet6/36 is up
admin state is up, Dedicated Interface
Hardware: 40000 Ethernet, address: 0022.bdf6.bf91 (bia 0022.bdf8.2bf3)
Internet Address is 192.168.100.1/24
MTU 9216 bytes, BW 40000000 Kbit, DLY 10 usec

Configuring the Beacon Mode

You can enable the beacon mode for an Ethernet port to flash its LED to confirm its physical location.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface ethernet slot/port Specifies an interface to configure, and enters

interface configuration mode.
Example:
switch(config)# interface ethernet 3/1
switch(config-if)#

Step 3 [no] beacon Enables the beacon mode or disables the beacon
mode. The default mode is disabled.
Example:
switch(config)# beacon
switch(config-if)#

Step 4 show interface ethernet slot/port (Optional) Displays the interface status, which
includes the beacon mode state.
Example:
switch(config)# show interface ethernet
2/1
switch(config-if)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
33
 Configuring Basic Interface Parameters
Configuring the Error-Disabled State

Command or Action Purpose

Step 5 exit Exits the interface mode.
Example:
switch(config-if)# exit
switch(config)#

Step 6 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to enable the beacon mode for the Ethernet port 3/1:
switch# configure terminal
switch(config)# interface ethernet 3/1
switch(config-if)# beacon
switch(config-if)#

This example shows how to disable the beacon mode for the Ethernet port 3/1:
switch# configure terminal
switch(config)# interface ethernet 3/1
switch(config-if)# no beacon
switch(config-if)#

This example shows how to configure the dedicated mode for Ethernet port 4/17 in the group that
includes ports 4/17, 4/19, 4/21, and 4/23:
switch# configure terminal
switch(config)# interface ethernet 4/17, ethernet 4/19, ethernet 4/21, ethernet 4/23
switch(config-if)# shutdown
switch(config-if)# interface ethernet 4/17
switch(config-if)# no shutdown
switch(config-if)#

Configuring the Error-Disabled State

You can view the reason that an interface moves to the error-disabled state and configure automatic recovery.

Enabling the Error-Disable Detection

You can enable error-disable detection in an application. As a result, when a cause is detected on an interface,
the interface is placed in an error-disabled state, which is an operational state that is similar to the link-down
state.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
34
 Configuring Basic Interface Parameters
Enabling the Error-Disabled Recovery

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 errdisable detect cause {acl-exception | all | Specifies a condition under which to place the
link-flap | loopback} interface in an error-disabled state. The default
is enabled.
Example:
switch(config)# errdisable detect cause
all
switch(config-if)#

Step 3 shutdown Brings the interface down administratively. To

manually recover the interface from the
Example:
error-disabled state, enter this command first.
switch(config-if)# shutdown
switch(config)#

Step 4 no shutdown Brings the interface up administratively and

enables the interface to recover manually from
Example:
the error-disabled state.
switch(config-if)# no shutdown
switch(config)#

Step 5 show interface status err-disabled (Optional) Displays information about

error-disabled interfaces.
Example:
switch(config)# show interface status
err-disabled

Step 6 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to enable the error-disabled detection in all cases:
switch(config)# errdisable detect cause all
switch(config)#

Enabling the Error-Disabled Recovery

You can specify the application to bring the interface out of the error-disabled state and retry coming up. It
retries after 300 seconds, unless you configure the recovery timer (see the errdisable recovery interval
command).

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
35
 Configuring Basic Interface Parameters
Configuring the Error-Disabled Recovery Interval

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 errdisable recovery cause {all | bpduguard | Specifies a condition under which the interface
failed-port-state | link-flap | loopback | automatically recovers from the error-disabled
miscabling | psecure-violation | state, and the device retries bringing the
security-violation | storm-control | udld | interface up. The device waits 300 seconds to
vpc-peerlink} retry. The default is disabled.
Example:
switch(config)# errdisable recovery cause
all
switch(config-if)#

Step 3 show interface status err-disabled (Optional) Displays information about

error-disabled interfaces.
Example:
switch(config)# show interface status
err-disabled
switch(config-if)#

Step 4 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to enable error-disabled recovery under all conditions:
switch(config)# errdisable recovery cause all
switch(config)#

Configuring the Error-Disabled Recovery Interval

You can configure the error-disabled recovery timer value.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
36
 Configuring Basic Interface Parameters
Configuring the MTU Size

Command or Action Purpose

Step 2 errdisable recovery interval interval Specifies the interval for the interface to recover
from the error-disabled state. The range is from
Example:
30 to 65535 seconds, and the default is 300
switch(config)# errdisable recovery seconds.
interval 32
switch(config-if)#

Step 3 show interface status err-disabled (Optional) Displays information about

error-disabled interfaces.
Example:
switch(config)# show interface status
err-disabled
switch(config-if)#

Step 4 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to configure the error-disabled recovery timer to set the interval for recovery
to 32 seconds:
switch(config)# errdisable recovery interval 32
switch(config)#

Configuring the MTU Size

MTU is configured per interface, where the interface can be a Layer 2 or a Layer 3 interface. Every interface
has default MTU of 1500 bytes. This value is called system default MTU. You can configure a Layer 2
interface, with a a value of 9216 bytes, which is the default value of the system jumbo MTU. To allow an
MTU value that is between 1500 and 9216, system jumbo MTU needs to be adjusted to appropriate value
where interface can be configured with the same value.

Note You can change the system jumbo MTU size. When the value is changed, the Layer 2 interfaces that use the
system jumbo MTU value, will automatically changes to the new system jumbo MTU value.

A Layer 3 interface, can be Layer 3 physical interface switch virtual interface (SVI), and subinterface, you
can configure an MTU size between 576–9216 bytes.

Configuring the Interface MTU Size

For Layer 3 interfaces, you can configure an MTU with keyword MTU and value in bytes where value is
between 576–9216 bytes.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
37
 Configuring Basic Interface Parameters
Configuring the Interface MTU Size

For Layer 2 interfaces, you can configure an interface using the keyword MTU with value in bytes. The value
can be a system default MTU size of 1500 bytes or the system jumbo MTU value that can be adjusted to the
default size of 9216 bytes.
If you need to use a different system jumbo MTU size for Layer 2 interfaces, see the Configuring the System
Jumbo MTU Size section.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface ethernet slot/port, vlan vlan-id mgmt Specifies an Ethernet interface to configure, and
0 enters interface configuration mode.
Example:
switch(config)# interface ethernet 3/1
switch(config-if)#
switch(config)# interface vlan 100
switch(config-if)#
switch(config)# interface mgmt 0
switch(config-if)#

Step 3 mtu size Configure the MTU value on an interface.

Example: For a Layer 3 interface, a physical Layer 3
switch(config-if)# mtu 9216 interface, an SVI or sub-interface, then the value
switch(config-if)# can be between 576-9216 bytes. If the interface
is a physical Layer 2 interface, then the value
can be 1500 or system jumbo MTU value.

Step 4 exit Exits the interface mode.

Example:
switch(config-if)# exit
switch(config)#

Example
This example shows how to configure the Layer 2 Ethernet port 3/1 with the default MTU size (1500):
switch# configure terminal
switch(config)# interface ethernet 3/1
switch(config-if)# switchport
switch(config-if)# mtu 1500
switch(config-if)#

This example displays the output of show running-config interface command:

switch# show run int mgmt0
!Command: show running-config interface mgmt0
!Running configuration last done at: Fri May 31 11:32:28 2019
!Time: Fri May 31 11:32:33 2019

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
38
 Configuring Basic Interface Parameters
Configuring the System Jumbo MTU Size

version 9.3(1) Bios:version 07.65

interface mgmt0
mtu 9216
vrf member management
ip address 168.51.170.73/82

Configuring the System Jumbo MTU Size

You can configure the system jumbo MTU size, which can be used to specify the MTU size for Layer 2
interfaces. You can specify an even number between 1500 and 9216. If you do not configure the system jumbo
MTU size, it defaults to 9216 bytes.

Note To configure jumbo frames for FEX modules, configure the FEX fabric port-channel interface with the required
MTU size for the FEX module.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 system jumbomtu size Specifies the system jumbo MTU size. Use an
even number between 1500 and 9216.
Example:
switch(config)# system jumbomtu 8000 Note In general accepted practice, a jumbo
switch(config)# frame is considered to have an MTU
size greater than 9000 bytes.

Step 3 show running-config all (Optional) Displays the current operating

configuration, which includes the system jumbo
Example:
MTU size.
switch(config)# show running-config all
| include jumbomtu

Step 4 interface type slot/port Specifies an interface to configure and enters

interface configuration mode.
Example:
switch(config)# interface ethernet 2/1
switch(config-if)#

Step 5 interface type Specifies the management interface to

configure.
Example:
switch(config if)# interface mgmt0
switch(config-if)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
39
 Configuring Basic Interface Parameters
Configuring the Bandwidth

Command or Action Purpose

Step 6 mtu size For a Layer 2 interface, specifies either the
default MTU size (1500) or the system jumbo
Example:
MTU size that you specified earlier.
switch(config-if)# mtu 1500
switch(config-if)# For a Layer 3 interface, specifies any even size
between 576 and 9216.

Step 7 exit Exits the interface mode.

Example:
switch(config-if)# exit
switch(config)#

Step 8 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to configure the system jumbo MTU as 8000 bytes and how to change the
MTU specification for an interface that was configured with the previous jumbo MTU size:
switch# configure terminal
switch(config)# system jumbomtu 8000
switch(config)# show running-config
switch(config)# interface ethernet 2/2
switch(config-if)# switchport
switch(config-if)# mtu 1500
switch(config-if)#

Configuring the Bandwidth

You can configure the bandwidth for Ethernet interfaces. The physical layer uses an unchangeable bandwidth
of 1G, 10G, or 40G, but you can configure a value of 1 to 100,000,000 KB for Level 3 protocols.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface ethernet slot/port Specifies an Ethernet interface to configure, and

enters interface configuration mode.
Example:
switch(config)# interface ethernet 3/1
switch(config-if)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
40
 Configuring Basic Interface Parameters
Configuring the Throughput Delay

Command or Action Purpose

Step 3 bandwidth kbps Specifies the bandwidth as an
informational-only value between 1 and
Example:
100,000,000.
switch(config-if)# bandwidth 1000000
switch(config-if)#

Step 4 show interface ethernet slot/port (Optional) Displays the interface status, which
includes the bandwidth value.
Example:
switch(config)# show interface ethernet
2/1

Step 5 exit Exits the interface mode.

Example:
switch(config-if)# exit
switch(config)#

Step 6 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to configure an informational value of 1,000,000 Kb for the Ethernet slot
3, port 1 interface bandwidth parameter:
switch# configure terminal
switch(config)# interface ethernet 3/1
switch(config-if)# bandwidth 1000000
switch(config-if)#

Configuring the Throughput Delay

You can configure the interface throughput delay for Ethernet interfaces. The actual delay time does not
change, but you can set an informational value between 1 and 16777215, where the value represents the
number of tens of microseconds.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface ethernet slot/port Specifies an Ethernet interface to configure, and

enters interface configuration mode.
Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
41
 Configuring Basic Interface Parameters
Configuring the Throughput Delay

Command or Action Purpose

switch(config)# interface ethernet 3/1
switch(config-if)#

Step 3 delay value Specifies the delay time in tens of

microseconds. You can set an informational
Example:
value range between 1 and 16777215 tens of
switch(config-if)# delay 10000 microseconds.
switch(config-if)#

Step 4 show interface ethernet slot/port (Optional) Displays the interface status, which
includes the throughput-delay time.
Example:
switch(config)# show interface ethernet
3/1
switch(config-if)#

Step 5 exit Exits the interface mode.

Example:
switch(config-if)# exit
switch(config)#

Step 6 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to configure the throughput-delay time so that one interface is preferred
over another. A lower delay value is preferred over a higher value. In this example, Ethernet 7/48 is
preferred over 7/47. The default delay for 7/48 is less than the configured value on 7/47, which is
set for the highest value (16777215):
switch# configure terminal
switch(config)# interface ethernet 7/47
switch(config-if)# delay 16777215
switch(config-if)# ip address 192.168.10.1/24
switch(config-if)# ip router eigrp 10
switch(config-if)# no shutdown
switch(config-if)# exit
switch(config)# interface ethernet 7/48
switch(config-if)# ip address 192.168.11.1/24
switch(config-if)# ip router eigrp 10
switch(config-if)# no shutdown
switch(config-if)#

Note You must first ensure the EIGRP feature is enabled by running the feature eigrp command.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
42
 Configuring Basic Interface Parameters
Shutting Down and Activating the Interface

Shutting Down and Activating the Interface

You can shut down and restart Ethernet or management interfaces. When you shut down interfaces, they
become disabled and all monitoring displays show them as being down. This information is communicated
to other network servers through all dynamic routing protocols. When the interfaces are shut down, the interface
is not included in any routing updates. To activate the interface, you must restart the device.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface interface Specifies the interface that you are configuring.
You can specify the interface type and identity.
Example:
For an Ethernet port, use ethernet slot/port. For
switch(config)# interface ethernet 2/1 the management interface, use mgmt0.
switch(config-if)#
Examples:
switch(config)# interface mgmt0
switch(config-if)# • The 1st example shows how to specify the
slot 2, port 1 Ethernet interface.
• The 2nd example shows how to specify
the management interface.

Step 3 shutdown Disables the interface.

Example:
switch(config-if)# shutdown
switch(config-if)#

Step 4 show interface interface (Optional) Displays the interface status, which
includes the administrative status.
Example:
switch(config-if)# show interface
ethernet 2/1
switch(config-if)#

Step 5 no shutdown Reenables the interface.

Example:
switch(config-if)# no shutdown
switch(config-if)#

Step 6 show interface interface (Optional) Displays the interface status, which
includes the administrative status.
Example:
switch(config-if)# show interface
ethernet 2/1
switch(config-if)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
43
 Configuring Basic Interface Parameters
Configuring the UDLD Mode

Command or Action Purpose

Step 7 exit Exits the interface mode.
Example:
switch(config-if)# exit
switch(config)#

Step 8 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to change the administrative status for Ethernet port 3/1 from disabled to
enabled:
switch# configure terminal
switch(config)# interface ethernet 3/1
switch(config-if)# shutdown
switch(config-if)# no shutdown
switch(config-if)#

Configuring the UDLD Mode

You can configure normal unidirectional link detection (UDLD) modes for Ethernet interfaces on devices
configured to run UDLD.
Before you can enable a UDLD mode for an interface, you must make sure that UDLD is already enabled on
the device that includes the interface. UDLD must also be enabled on the other linked interface and its device.

Note If the interface is a copper port, you must use the command enable UDLD to enable the UDLD. If the interface
is a fiber port you need not explicitly enable UDLD on the interface. However if you attempt to enable UDLD
on a fiber port using the enable UDLD command, you may get an error message indicating that is not a valid
command.

The following table lists CLI details to enable and disable UDLD on different interfaces

Table 5: CLI Details to Enable or Disable UDLD on Different Interfaces

Description Fiber port Copper or Nonfiber port

Default setting Enabled Disabled

Enable UDLD command no udld disable udld enable

Disable UDLD command udld disable no udld enable

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
44
Configuring Basic Interface Parameters
Configuring the UDLD Mode

Before you begin

You must enable UDLD for the other linked port and its device.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 [no] feature udld Enables/Disables UDLD for the device.

Example:
switch(config)# feature udld
switch(config)#
switch(config)# no feature udld
switch(config)#

Step 3 udld message-time seconds (Optional) Specifies the interval between

sending UDLD messages. The range is from 7
Example:
to 90 seconds, and the default is 15 seconds.
switch(config)# udld message-time 30
switch(config)#

Step 4 udld aggressive Optional) Specifies UDLD mode to be

aggressive.
Example:
switch(config)# udld aggressive Note For copper interfaces, you enter the
switch(config)# interface command mode for those
interfaces you want to configure for
UDLD aggressive mode and issue
this command in interface command
model.

Step 5 interface ethernet slot/port (Optional) Specifies an interface to configure,

and enters interface configuration mode.
Example:
switch(config)# interface ethernet 3/1
switch(config-if)#

Step 6 udld [enable | disable] (Optional) Enables UDLD on the specified

copper port or disables UDLD on the specified
Example:
fiber port.
switch(config-if)# udld enable
switch(config-if)# To enable UDLD on copper ports, enter the
udld enable command. To enable UDLD on
fiber ports, enter the no udld disable command.

Step 7 show udld [ethernet slot/port | global | (Optional) Displays the UDLD status.
neighbors]
Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
45
 Configuring Basic Interface Parameters
Configuring Debounce Timers

Command or Action Purpose

switch(config)# show udld
switch(config)#

Step 8 exit Exits the interface mode.

Example:
switch(config-if-range)# exit
switch(config)#

Step 9 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to enable the UDLD for the device:
switch# configure terminal
switch(config)# feature udld
switch(config)#

This example shows how to set the UDLD message interval to 30 seconds:
switch# configure terminal
switch(config)# feature udld
switch(config)# udld message-time 30
switch(config)#

This example shows how to disable UDLD for Ethernet port 3/1:
switch# configure terminal
switch(config)# interface ethernet 3/1
switch(config-if-range)# no udld enable
switch(config-if-range)# exit

This example shows how to disable UDLD for the device:

switch# configure terminal
switch(config)# no feature udld
switch(config)# exit

Configuring Debounce Timers

You can enable the debounce timer for Ethernet ports by specifying a debounce time (in milliseconds) or
disable the timer by specifying a debounce time of 0.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
46
Configuring Basic Interface Parameters
Configuring Debounce Timers

Note The link state of 10G and 100G ports may change repeatedly when connected to service provider network.
As a part of link reset or break-link functionality, it is expected that the Tx power light on the SFP to change
to N/A state, at an event of link state change.
However, to prevent this behavior during the link state change, you may increase the link debounce timer to
start from 500ms and increase it in 500ms intervals until the link stabilizes. On the DWDM, UVN, and WAN
network, it is recommended to disable automatic link suspension (ALS) whenever possible. ALS suspends
the link on the WAN when the Nexus turn off the link.

Note The link debounce time and link debounce link-up time commands can only be applied to a physical Ethernet
interface.
Use the show interface debounce command to display the debounce times for all Ethernet ports.
The link debounce time command is not supported on 10G and 40G ports on the Cisco Nexus 93300YC-FX
and Cisco Nexus 9336C-FX switches.
The link debounce time command is supported on 1G, 10G, 40G, 25G and 100G SFP/QSFP ports on the
Cisco Nexus 9000 series switches.
The link debounce time is supported on 1G, 10G, 25G, 40G and 100G ports on Cisco Nexus N9K-C9732C-FX,
N9K-C9364C, N9K-X97160YC-EX, N9K-C9336C-FX2, and N9K-C93240YC-FX2 platform switches.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface ethernet slot/port Specifies an Ethernet interface to configure, and

enters interface configuration mode.
Example:
switch(config)# interface ethernet 3/1
switch(config-if)#

Step 3 link debounce time time Enables the debounce timer for the specified
time (1 to 5000 milliseconds).
Example:
switch(config-if)# link debounce time If you specify 0 milliseconds, the debounce
1000 timer is disabled.
switch(config-if)#

Step 4 link debounce link-up time Enables the link-up timer for the specified time
(1000 to 10000 milliseconds). This command
Example:
applies only if the port speeds are 10G, 25G,
switch(config-if)# link debounce link-up 40G and 100G.
1000
switch(config-if)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
47
 Configuring Basic Interface Parameters
Configuring Port Profiles

Command or Action Purpose

The default value of the timer is 0. If the value
is set to 0 the interface will be up without any
delay.
Note The no link debounce link-up
command also resets the value to 0.

Note This command is supported only on

Cisco Nexus N9K-X9732C-FX ,
N9K-C93300YC-FX,
N9K-C9336C-FX2, N9K-C9364C
and N9K-X97160YC-EX switches.

Example
• The following example enables the debounce timer and sets the debounce time to 1000
milliseconds for an Ethernet interface:

switch# configure terminal

switch(config)# interface ethernet 1/4
switch(config-if)# link debounce time 1000

• The following example disables the debounce timer for an Ethernet interface:

switch# configure terminal

switch(config)# interface ethernet 1/4
switch(config-if)# link debounce time 0

• The following example sets the debounce link-up timer to 1000 milliseconds for an Ethernet
interface:

switch# configure terminal

switch(config)# interface ethernet 1/4
switch(config-if)# link debounce link-up time 1000

Configuring Port Profiles

Beginning with 7.0(3)I4(1), you can apply several configuration parameters to a range of interfaces
simultaneously. All the interfaces in the range must be the same type. You can also inherit the configurations
from one port profile into another port profile. The system supports four levels of inheritance.

Creating a Port Profile

You can create a port profile on the device. Each port profile must have a unique name across types and the
network.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
48
 Configuring Basic Interface Parameters
Entering Port-Profile Configuration Mode and Modifying a Port Profile

Note Port profile names can include only the following characters:
• a-z
• A-Z
• 0-9
• No special characters are allowed, except for the following:
•.
•-
•_

Procedure

Command or Action Purpose

Step 1 configure terminal Enters the global configuration mode.

Step 2 port-profile [type {ethernet | interface-vlan Creates and names a port profile for the
| port-channel}] name specified type of interface and enters the
port-profile configuration mode.

Step 3 exit Exits the port-profile configuration mode.

Step 4 (Optional) show port-profile Displays the port-profile configuration.

Step 5 (Optional) copy running-config startup-config Copies the running configuration to the startup
configuration.

Example
This example shows how to create a port profile named test for ethernet interfaces:

switch# configure terminal

switch(config)# port-profile type ethernet test
switch(config-ppm)#

Entering Port-Profile Configuration Mode and Modifying a Port Profile

You can enter the port-profile configuration mode and modify a port profile. To modify the port profile, you
must be in the port-profile configuration mode.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters the global configuration mode.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
49
 Configuring Basic Interface Parameters
Assigning a Port Profile to a Range of Interfaces

Command or Action Purpose

Step 2 port-profile [type {ethernet | interface-vlan Enters the port-profile configuration mode for
| port-channel}] name the specified port profile and allows you to add
or remove configurations to the profile.

Step 3 exit Exits the port-profile configuration mode.

Step 4 (Optional) show port-profile Displays the port-profile configuration.

Step 5 (Optional) copy running-config startup-config Copies the running configuration to the startup
configuration.

Example
This example shows how to enter the port-profile configuration mode for the specified port profile
and bring all the interfaces administratively up:

switch# configure terminal

switch(config)# port-profile type ethernet test
switch(config-ppm)# no shutdown
switch(config-ppm)#

Assigning a Port Profile to a Range of Interfaces

You can assign a port profile to an interface or to a range of interfaces. All the interfaces must be the same
type.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters the global configuration mode.

Step 2 interface [ethernet slot/port | interface-vlan Selects the range of interfaces.

vlan-id | port-channel number]
Step 3 inherit port-profile name Assigns the specified port profile to the selected
interfaces.

Step 4 exit Exits the port-profile configuration mode.

Step 5 (Optional) show port-profile Displays the port-profile configuration.

Step 6 (Optional) copy running-config startup-config Copies the running configuration to the startup
configuration.

Example
This example shows how to assign the port profile named adam to Ethernet interfaces 7/3 to 7/5,
10/2, and 11/20 to 11/25:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
50
 Configuring Basic Interface Parameters
Enabling a Specific Port Profile

switch# configure terminal

switch(config)# interface ethernet7/3-5, ethernet10/2, ethernet11/20-25
switch(config-if)# inherit port-profile adam
switch(config-if)#

Enabling a Specific Port Profile

To apply the port-profile configurations to the interfaces, you must enable the specific port profile. You can
configure and inherit a port profile onto a range of interfaces before you enable that port profile. You would
then enable that port profile for the configurations to take effect on the specified interfaces.
If you inherit one or more port profiles onto an original port profile, only the last inherited port profile must
be enabled; the system assumes that the underlying port profiles are enabled.
You must be in the port-profile configuration mode to enable or disable port profiles.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters the global configuration mode.

Step 2 port-profile [type {ethernet | interface-vlan Creates and names a port profile for the
| port-channel}] name specified type of interface and enters the
port-profile configuration mode.

Step 3 state enabled Enables that port profile.

Step 4 exit Exits the port-profile configuration mode.

Step 5 (Optional) show port-profile Displays the port-profile configuration.

Step 6 (Optional) copy running-config startup-config Copies the running configuration to the startup
configuration.

Example
This example shows how to enter the port-profile configuration mode and enable the port profile:

switch# configure terminal

switch(config)# port-profile type ethernet test
switch(config-ppm)# state enabled
switch(config-ppm)#

Inheriting a Port Profile

You can inherit a port profile onto an existing port profile. The system supports four levels of inheritance.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters the global configuration mode.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
51
 Configuring Basic Interface Parameters
Removing a Port Profile from a Range of Interfaces

Command or Action Purpose

Step 2 port-profile name Enters the port-profile configuration mode for

the specified port profile.

Step 3 inherit port-profile name Inherits another port profile onto the existing
one. The original port profile assumes all the
configurations of the inherited port profile.

Step 4 exit Exits the port-profile configuration mode.

Step 5 (Optional) show port-profile Displays the port-profile configuration.

Step 6 (Optional) copy running-config startup-config Copies the running configuration to the startup
configuration.

Example
This example shows how to inherit the port profile named adam onto the port profile named test:

switch# configure terminal

switch(config)# port-profile test
switch(config-ppm)# inherit port-profile adam
switch(config-ppm)#

Removing a Port Profile from a Range of Interfaces

You can remove a port profile from some or all of the interfaces to which you have applied the profile. You
do this configuration in the interfaces configuration mode.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters the global configuration mode.

Step 2 interface [ethernet slot/port | interface-vlan Selects the range of interfaces.

vlan-id | port-channel number]
Step 3 no inherit port-profile name Un-assigns the specified port profile to the
selected interfaces.

Step 4 exit Exits the port-profile configuration mode.

Step 5 (Optional) show port-profile Displays the port-profile configuration.

Step 6 (Optional) copy running-config startup-config Copies the running configuration to the startup
configuration.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
52
 Configuring Basic Interface Parameters
Removing an Inherited Port Profile

Example
This example shows how to unassign the port profile named adam to Ethernet interfaces 7/3 to 7/5,
10/2, and 11/20 to 11/25:

switch# configure terminal

switch(config)# interface ethernet 7/3-5, 10/2, 11/20-25
switch(config-if)# no inherit port-profile adam
switch(config-if)#

Removing an Inherited Port Profile

You can remove an inherited port profile. You do this configuration in the port-profile mode.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters the global configuration mode.

Step 2 port-profile name Enters the port-profile configuration mode for

the specified port profile.

Step 3 no inherit port-profile name Removes an inherited port profile from this port
profile.

Step 4 exit Exits the port-profile configuration mode.

Step 5 (Optional) show port-profile Displays the port-profile configuration.

Step 6 (Optional) copy running-config startup-config Copies the running configuration to the startup
configuration.

Example
This example shows how to remove the inherited port profile named adam from the port profile
named test:

switch# configure terminal

switch(config)# port-profile test
switch(config-ppm)# no inherit port-profile adam
switch(config-ppm)#

Verifying the Basic Interface Parameters

You can verify the basic interface parameters by displaying their values. You can also clear the counters listed
when you display the parameter values.
To display basic interface configuration information, perform one of the following tasks:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
53
 Configuring Basic Interface Parameters
Monitoring the Interface Counters

Command Purpose

show cdp all Displays the CDP status.

show interface interface Displays the configured states of one or all interfaces.

show interface brief Displays a table of interface states.

show interface status err-disabled Displays information about error-disabled interfaces.

show udld interface Displays the UDLD status for the current interface or
all interfaces.

show udld global Displays the UDLD status for the current device.

Monitoring the Interface Counters

You can display and clear interface counters using Cisco NX-OS.

Displaying Interface Statistics

You can set up to three sampling intervals for statistics collections on interfaces.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface ether slot/port Specifies interface.

Example:
switch(config)# interface ether 4/1
switch(config)#

Step 3 load-interval counters [1 | 2 | 3] seconds Sets up to three sampling intervals to collect

bit-rate and packet-rate statistics. The default
Example:
values for each counter is as follows:
switch(config)# load-interval counters
1 100 1—30 seconds (60 seconds for VLAN)
switch(config)#
2—300 seconds
3—not configured

Step 4 show interface interface (Optional) Displays the interface status, which
includes the counters.
Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
54
 Configuring Basic Interface Parameters
Clearing Interface Counters

Command or Action Purpose

switch(config)# show interface ethernet
2/2
switch#

Step 5 exit Exits the interface mode.

Example:
switch(config-if-range)# exit
switch(config)#

Step 6 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to set the three sample intervals for the Ethernet port 3/1:
switch# configure terminal
switch(config)# interface ethernet 3/1
switch(config-if)# load-interval counter 1 60
switch(config-if)# load-interval counter 2 135
switch(config-if)# load-interval counter 3 225
switch(config-if)#

Clearing Interface Counters

You can clear the Ethernet and management interface counters by using the clear counters interface command.
You can perform this task from the configuration mode or interface configuration mode.

Procedure

Command or Action Purpose

Step 1 clear counters interface [all | ethernet Clears the interface counters.
slot/port | loopback number | mgmt number |
port channel channel-number]
Example:
switch# clear counters ethernet 2/1
switch#

Step 2 show interface interface (Optional) Displays the interface status.

Example:
switch# show interface ethernet 2/1
switch#

Step 3 show interface [ethernet slot/port | port (Optional) Displays the interface counters.
channel channel-number] counters

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
55
 Configuring Basic Interface Parameters
Configuration Example for QSA

Command or Action Purpose

Example:
switch# show interface ethernet 2/1
counters
switch#

Example
This example shows how to clear the counters on Ethernet port 5/5:
switch# clear counters interface ethernet 5/5
switch#

Configuration Example for QSA

For a Cisco Nexus 9396PX:
• Using the default configuration on port 2/1, all the QSFPs in port group 2/1-6 are brought up with a speed
of 40G. If there are any QSA modules in port group 2/1-6, they are error disabled.
• Using the speed-group [ 10000 | 40000] command to configure port 2/7, all the QSAs in port group
2/7-12 are brought up with a speed of 10G or 40G. If there are any QSFP modules in port group 2/7-12,
they are error disabled.

This example shows how to configure QSA for the first port in the speed group for a Cisco Nexus 9396PX:

switch# conf t
switch(config)# interface ethernet 2/7
switch(config-if)# speed-group 10000

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
56
 CHAPTER 4
Configuring Layer 2 Interfaces
This chapter describes how to configure Layer 2 switching ports as access or trunk ports on Cisco NX-OS
devices.

Note A Layer 2 port can function as either one of the following:

• A trunk port
• An access port

Note See the Cisco Nexus 9000 Series NX-OS System Management Configuration Guide for information about
configuring a SPAN destination interface.

You can configure Layer 2 switching ports as access or trunk ports. Trunks carry the traffic of multiple VLANs
over a single link and allow you to extend VLANs across an entire network. All Layer 2 switching ports
maintain media access control (MAC) address tables.

Note See the Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration Guide for information about
VLANs, MAC address tables, private VLANs, and the Spanning Tree Protocol.

• Information About Access and Trunk Interfaces, on page 58

• Prerequisites for Layer 2 Interfaces, on page 64
• Guidelines and Limitations for Layer 2 Interfaces, on page 65
• Default Settings for Layer 2 Interfaces, on page 67
• Configuring Access and Trunk Interfaces , on page 68
• Verifying the Interface Configuration, on page 86
• Monitoring the Layer 2 Interfaces, on page 87
• Configuration Examples for Access and Trunk Ports, on page 87
• Related Documents, on page 88

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
57
 Configuring Layer 2 Interfaces
Information About Access and Trunk Interfaces

Information About Access and Trunk Interfaces

Note See the Cisco Nexus 9000 Series NX-OS High Availability and Redundancy Guide for complete information
on high-availability features.

Note The device supports only IEEE 802.1Q-type VLAN trunk encapsulation.

About Access and Trunk Interfaces

A Layer 2 port can be configured as an access or a trunk port as follows:
• An access port can have only one VLAN configured on that port; it can carry traffic for only one VLAN.
• A trunk port can have two or more VLANs configured on that port; it can carry traffic for several VLANs
simultaneously.

By default, all the ports on Cisco Nexus 9300-EX switches are Layer 3 ports and all the ports on Cisco Nexus
9300 switches are Layer 2 ports.
You can make all ports Layer 2 ports using the setup script or by entering the system default
switchportcommand. See the Cisco Nexus 9000 Series NX-OS Fundamentals Configuration Guide for
information about using the setup script. To configure the port as a Layer 2 port using the CLI, use the
switchport command.
All ports in the same trunk must be in the same VDC, and trunk ports cannot carry VLANs from different
VDCs.
The following figure shows how you can use trunk ports in the network. The trunk port carries traffic for two
or more VLANs.
Figure 2: Trunk and Access Ports and VLAN Traffic

Note See the Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration Guide for information about
VLANs.

In order to correctly deliver the traffic on a trunk port with several VLANs, the device uses the IEEE 802.1Q
encapsulation, or tagging, method (see the “IEEE 802.1Q Encapsulation” section for more information).

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
58
 Configuring Layer 2 Interfaces
IEEE 802.1Q Encapsulation

Note See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide for information about
subinterfaces on Layer 3 interfaces.

To optimize the performance on access ports, you can configure the port as a host port. Once the port is
configured as a host port, it is automatically set as an access port, and channel grouping is disabled. Use the
host designation to decrease the time that it takes the designated port to begin to forward packets.
Only an end station can be set as a host port; you will receive an error message if you attempt to configure
other ports as hosts.
If an access port receives a packet with an 802.1Q tag in the header other than the access VLAN value, that
port drops the packet without learning its MAC source address.
A Layer 2 interface can function as either an access port or a trunk port; it cannot function as both port types
simultaneously.
When you change a Layer 2 interface back to a Layer 3 interface, that interface loses all the Layer 2
configuration and resumes the default VLAN configurations.

IEEE 802.1Q Encapsulation

Note For information about VLANs, see the Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration
Guide.

A trunk is a point-to-point link between the switch and another networking device. Trunks carry the traffic
of multiple VLANs over a single link and allow you to extend VLANs across an entire network.
To correctly deliver the traffic on a trunk port with several VLANs, the device uses the IEEE 802.1Q
encapsulation, or tagging, method that uses a tag that is inserted into the frame header. This tag carries
information about the specific VLAN to which the frame and packet belong. This method allows packets that
are encapsulated for several different VLANs to traverse the same port and maintain traffic separation between
the VLANs. Also, the encapsulated VLAN tag allows the trunk to move traffic end-to-end through the network
on the same VLAN.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
59
 Configuring Layer 2 Interfaces
Access VLANs

Figure 3: Header Without and With 802.1Q Tag

Access VLANs
When you configure a port in access mode, you can specify which VLAN will carry the traffic for that interface.
If you do not configure the VLAN for a port in access mode, or an access port, the interface carries traffic for
the default VLAN (VLAN1).
You can change the access port membership in a VLAN by specifying the new VLAN. You must create the
VLAN before you can assign it as an access VLAN for an access port. If you change the access VLAN on an
access port to a VLAN that is not yet created, the system shuts that access port down.
If an access port receives a packet with an 802.1Q tag in the header other than the access VLAN value, that
port drops the packet without learning its MAC source address.

Native VLAN IDs for Trunk Ports

A trunk port can carry nontagged packets simultaneously with the 802.1Q tagged packets. When you assign
a default port VLAN ID to the trunk port, all untagged traffic travels on the default port VLAN ID for the
trunk port, and all untagged traffic is assumed to belong to this VLAN. This VLAN is referred to as the native
VLAN ID for a trunk port. That is, the native VLAN ID is the VLAN that carries untagged traffic on trunk
ports.

Note Native VLAN ID numbers must match on both ends of the trunk.

The trunk port sends an egressing packet with a VLAN that is equal to the default port VLAN ID as untagged;
all the other egressing packets are tagged by the trunk port. If you do not configure a native VLAN ID, the
trunk port uses the default VLAN.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
60
 Configuring Layer 2 Interfaces
Tagging Native VLAN Traffic

Note You cannot use a Fibre Channel over Ethernet (FCoE) VLAN as a native VLAN for an Ethernet trunk
switchport.

Tagging Native VLAN Traffic

The Cisco software supports the IEEE 802.1Q standard on trunk ports. In order to pass untagged traffic through
the trunk ports, you must create a VLAN that does not tag any packets (or you can use the default VLAN).
Untagged packets can pass through trunk ports and access ports.
However, all packets that enter the device with an 802.1Q tag that matches the value of the native VLAN on
the trunk are stripped of any tagging and egress the trunk port as untagged packets. This situation can cause
problems because you may want to retain the tagging on packets on the native VLAN for the trunk port.
You can configure the device to drop all untagged packets on the trunk ports and to retain the tagging of
packets entering the device with 802.1Q values that are equal to that of the native VLAN ID. All control traffic
still passes on the native VLAN. This configuration is global; trunk ports on the device either do or do not
retain the tagging for the native VLAN.

Allowed VLANs
By default, a trunk port sends traffic to and receives traffic from all VLANs. All VLAN IDs are allowed on
each trunk. However, you can remove VLANs from this inclusive list to prevent traffic from the specified
VLANs from passing over the trunk. Later, you can add any specific VLANs that you may want the trunk to
carry traffic for back to the list.
To partition the Spanning Tree Protocol (STP) topology for the default VLAN, you can remove VLAN1 from
the list of allowed VLANs. Otherwise, VLAN1, which is enabled on all ports by default, will have a very big
STP topology, which can result in problems during STP convergence. When you remove VLAN1, all data
traffic for VLAN1 on this port is blocked, but the control traffic continues to move on the port.

Note See the Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration Guide for more information about
STP.

Note You can change the block of VLANs reserved for internal use. See the Cisco Nexus 9000 Series NX-OS
Layer 2 Switching Configuration Guide for more information about changing the reserved VLANs.

Switchport Isolated for up to 3976 VLANs on Trunk Interfaces

Trunk interface can carry multiple VLANs, where a trunk interface that is configured in switchport isolated
mode allows you to configure multiple VLANs per interface. The Cisco Nexus 9300 and 9500 switches
support up to 512 VLANs per interface. Sometimes, you may require higher number VLANs per port. The
scale of logical ports of Per-VLAN Spanning Tree (PVST) and virtual ports of Multiple Spanning Tree (MST)

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
61
 Configuring Layer 2 Interfaces
Default Interfaces

can be limiting. By configuring switchport isolated on a trunk interface, you can configure up to 48 interfaces
with up to 3976 VLANs per port, on switches in Cisco Nexus 9000 portfolio.
When you change the member VLANs on the isolated interfaces, all VLANs on these interfaces get moved
to forwarding state. Switchport isolated feature is supported only on the host interfaces, because no spanning
tree is running on these ports (the switch doesn't send STP BPDUs), connecting other network devices may
create a loop in the network. Switchport isolated feature is supported on physical interfaces, port channels,
and vPC. Switchport isolated feature has the following restrictions:
• Switchport isolated is supported with MST. It allows some ports to be in isolated mode, and some ports
run MST with same VLANs.
• Is not supported on switch running PVRST any of the VLANs.
• Is not supported on FEX HIF, FEX fabric interface, on an interface when another network device is
connected.
• Supports up to 48 ports configured with up to 3967 VLANs.
• When used in vPC environment if configured inconsistently will trigger vPC type 1 constancy checks.
• Port channel members are required to have the same switchport isolated configuration.

Default Interfaces
You can use the default interface feature to clear the configured parameters for both physical and logical
interfaces such as the Ethernet, loopback, VLAN network, tunnel, and the port-channel interface.

Note A maximum of eight ports can be selected for the default interface. The default interfaces feature is not
supported for management interfaces because the device could go to an unreachable state.

Switch Virtual Interface and Autostate Behavior

In Cisco NX-OS, a switch virtual interface (SVI) represents a logical interface between the bridging function
and the routing function of a VLAN in the device.
The operational state of this interface is governed by the state of the various ports in its corresponding VLAN.
An SVI interface on a VLAN comes up when at least one port in that VLAN is in the Spanning Tree Protocol
(STP) forwarding state. Similarly, this interface goes down when the last STP forwarding port goes down or
goes to another STP state.

SVI Autostate Exclude

Typically, when a VLAN interface has multiple ports in the VLAN, the SVI goes to the down state when all
the ports in the VLAN go down. You can use the SVI autostate exclude feature to exclude specific ports and
port channels while defining the status of the SVI (up or down) even if it belongs to the same VLAN. For
example, even if the excluded port or port channel is in the up state and other ports are in the down state in
the VLAN, the SVI state is changed to down.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
62
 Configuring Layer 2 Interfaces
SVI Autostate Disable

Note You can use the SVI autostate exclude feature only for switched physical Ethernet ports and port channels.

SVI Autostate Disable

You can configure the autostate disable feature to keep an SVI up even if no interface is up in the corresponding
VLAN. You can configure this feature for the system (for all SVIs) or for an individual SVI.

High Availability
The software supports high availability for Layer 2 ports.

Note See the Cisco Nexus 9000 Series NX-OS High Availability and Redundancy Guide for complete information
about high availability features.

Virtualization Support
All ports in the same trunk must be in the same VDC, and trunk ports cannot carry VLANs from different
VDCs.

Counter Values
See the following information on the configuration, packet size, incremented counter values, and traffic.

Configuration Packet Size Incremented Counters Traffic

L2 port – without any 6400 and 10000 Jumbo, giant, and input Dropped
MTU configuration error

L2 port – with jumbo 6400 Jumbo Forwarded

MTU 9216 in
network-qos configuration

L2 port – with jumbo 10000 Jumbo, giant, and input Dropped

MTU 9216 in error
network-qos configuration

Layer 3 port with default 6400 Jumbo Packets are punted to the
Layer 3 MTU and jumbo CPU (subjected to CoPP
MTU 9216 in configs), get fragmented,
network-qos configuration and then they are
forwarded by the
software.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
63
 Configuring Layer 2 Interfaces
Prerequisites for Layer 2 Interfaces

Configuration Packet Size Incremented Counters Traffic

Layer 3 port with default 6400 Jumbo Packets are punted to the
Layer 3 MTU and jumbo CPU (subjected to CoPP
MTU 9216 in configs), get fragmented,
network-qos configuration and then they are
forwarded by the
software.

Layer 3 port with default 10000 Jumbo, giant, and input Dropped
Layer 3 MTU and jumbo error
MTU 9216 in
network-qos configuration

Layer 3 port with jumbo 6400 Jumbo Forwarded without any

Layer 3 MTU and jumbo fragmentation.
MTU 9216 in
network-qos configuration

Layer 3 port with jumbo 10000 Jumbo, giant, and input Dropped
Layer 3 MTU and jumbo error
MTU 9216 in
network-qos configuration

Layer 3 port with jumbo 6400 and 10000 Jumbo, giant, and input Dropped
Layer 3 MTU and default error
L2 MTU configuration

Note • Under 64 bytes packet with good CRC–The short frame counter increments.
• Under 64 bytes packet with bad CRC–The runts counter increments.
• Greater than 64 bytes packet with bad CRC–The CRC counter increments.

Prerequisites for Layer 2 Interfaces

Layer 2 interfaces have the following prerequisites:
• You are logged onto the device.
• By default, Cisco NX-OS configures Layer 3 parameters. If you want to configure Layer 2 parameters,
you need to switch the port mode to Layer 2. You can change the port mode by using the switchport
command.
• You must configure the port as a Layer 2 port before you can use the switchport modecommand. By
default, all ports on the device are Layer 3 ports. By default, all ports on the Cisco Nexus 9504 and Cisco
Nexus 9508 devices are Layer 2 ports.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
64
 Configuring Layer 2 Interfaces
Guidelines and Limitations for Layer 2 Interfaces

Guidelines and Limitations for Layer 2 Interfaces

VLAN trunking has the following configuration guidelines and limitations:
• Cisco Nexus 9000 Series switches have the vlan dot1q tag native command enabled by default. This
tags the native VLAN on the configured trunk ports. However, connected switches such as Catalyst 6500
or third-party switches, probably would not have a similar configuration enabled. This could result in
unexpected behaviors. Therefore, it is recommended to have the vlan dot1q tag native command
disabled in case the connected switch does not have it configured.
• Beginning with Release 7.0(3)F3(2), auto-negotiation is not supported on Cisco Nexus 9508 switches
with N9K-X9636C-R, N9K-X9636C-RX, and N9K-X9636Q-R lines cards.
•
• Auto-negotiation is not supported on 25-G Ethernet transceiver modules on Cisco Nexus 9200 and
9300-FX platform switches, and Cisco Nexus 9500 platform switches that use N9K-X9700-EX line
cards.
• On the Cisco Nexus 9364C switches, auto-negotiation might not work on ports 49-64 when bringing up
100G links using the QSFP-100G-CR4 cable. The workaround for this issue is that you must hard code
the speed on ports 49-64 and disable auto-negotiation.
• Autonegotiation (40 G/100 G) and 1 Gb with QSA is not supported on the following ports:
• Cisco Nexus 9336C-FX2 switch: ports 1-6 and 33-36
• Cisco Nexus 9364C switch: ports 49-66
• Cisco Nexus 93240YC-FX2 switch: ports 51-54
• Cisco Nexus 9788TC line card: ports 49-52

Note Peer speed must be set when using coper cables on these ports.

• show commands with the internal keyword are not supported.

• Starting with Release 7.0(3)I2(1) or later releases on Cisco Nexus 9300 platform switches, a unicast ARP
request to SVI is flooded to the other ports within the VLAN.
• ASE2 and ASE3 based Cisco Nexus 9000 Series switches acting as transit switches do not preserve the
inner tag for double-tagged packets.
The following CLI is mandatory only on LSE based Cisco Nexus 9000 Series switches. For seamless
packet forwarding and preservation of all VLAN tags on pure transit boxes in the SP cloud that have no
Q-in-Q encapsulation or decapsulation requirement, configure the CLI command, system dot1q-tunnel
transit . To remove the CLI, use no system dot1q-tunnel transit CLI command.
The caveats with the CLI that is executed on the switches are:
• L2 frames that egress out of the trunk ports are tagged even on the native VLAN on the port.
• Any other tunneling mechanism, for example, VXLAN and MPLS does not work with the CLI
configured.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
65
 Configuring Layer 2 Interfaces
Guidelines and Limitations for Layer 2 Interfaces

• A port can be either a Layer 2 or a Layer 3 interface; it cannot be both simultaneously.

• When you change a Layer 3 port to a Layer 2 port or a Layer 2 port to a Layer 3 port, all layer-dependent
configuration is lost. When you change an access or trunk port to a Layer 3 port, all information about
the access VLAN, native VLAN, allowed VLANs, and so forth, is lost.
• Do not connect devices with access links because access links may partition a VLAN.
• When connecting Cisco devices through an 802.1Q trunk, make sure that the native VLAN for an 802.1Q
trunk is the same on both ends of the trunk link. If the native VLAN on one end of the trunk is different
from the native VLAN on the other end, spanning tree loops might result.
• Disabling spanning tree on the native VLAN of an 802.1Q trunk without disabling spanning tree on every
VLAN in the network can cause spanning tree loops. You must leave spanning tree enabled on the native
VLAN of an 802.1Q trunk. If you cannot leave spanning tree enabled, you must disable spanning tree
on every VLAN in the network. Make sure that your network has no physical loops before you disable
spanning tree.
• When you connect two Cisco devices through 802.1Q trunks, the devices exchange spanning tree bridge
protocol data units (BPDUs) on each VLAN allowed on the trunks. The BPDUs on the native VLAN of
the trunk are sent untagged to the reserved IEEE 802.1D spanning tree multicast MAC address
(01-80-C2-00-00-00). The BPDUs on all other VLANs on the trunk are sent tagged to the reserved Cisco
Shared Spanning Tree (SSTP) multicast MAC address (01-00-0c-cc-cc-cd).
• Non-Cisco 802.1Q devices maintain only a single instance of spanning tree (the Mono Spanning Tree)
that defines the spanning tree topology for all VLANs. When you connect a Cisco switch to a non-Cisco
switch through an 802.1Q trunk, the Mono Spanning Tree of the non-Cisco switch and the native VLAN
spanning tree of the Cisco switch combine to form a single spanning tree topology known as the Common
Spanning Tree (CST).
• Because Cisco devices transmit BPDUs to the SSTP multicast MAC address on VLANs other than the
native VLAN of the trunk, non-Cisco devices do not recognize these frames as BPDUs and flood them
on all ports in the corresponding VLAN. Other Cisco devices connected to the non-Cisco 802.1Q cloud
receive these flooded BPDUs. This BPDU reception allows Cisco switches to maintain a per-VLAN
spanning tree topology across a cloud of non-Cisco 802.1Q devices. The non-Cisco 802.1Q cloud that
separates the Cisco devices is treated as a single broadcast segment between all devices connected to the
non-Cisco 802.1Q cloud through 802.1Q trunks.
• Make certain that the native VLAN is the same on all of the 802.1Q trunks that connect the Cisco devices
to the non-Cisco 802.1Q cloud.
• If you are connecting multiple Cisco devices to a non-Cisco 802.1Q cloud, all of the connections must
be through 802.1Q trunks. You cannot connect Cisco devices to a non-Cisco 802.1Q cloud through access
ports because doing so places the access port on the Cisco device into the spanning tree “port inconsistent”
state and no traffic will pass through the port.
• You can group trunk ports into port-channel groups, but all trunks in the group must have the same
configuration. When a group is first created, all ports follow the parameters set for the first port to be
added to the group. If you change the configuration of one of these parameters, the device propagates
that setting to all ports in the group, such as the allowed VLANs and the trunk status. For example, if
one port in a port group ceases to be a trunk, all ports cease to be trunks.
• If you try to enable 802.1X on a trunk port, an error message appears, and 802.1X is not enabled. If you
try to change the mode of an 802.1X-enabled port to trunk, the port mode is not changed.
• Only ingress unicast packet counters are supported for SVI counters.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
66
 Configuring Layer 2 Interfaces
Default Settings for Layer 2 Interfaces

• When MAC addresses are cleared on a VLAN with the clear mac address-table dynamic command, the
dynamic ARP (Address Resolution Protocol) entries on that VLAN are refreshed.
• If a static ARP entry exists on the VLAN and no MAC address to port mapping is present, the supervisor
may generate an ARP request to learn the MAC address. Upon learning the MAC address, the adjacency
entry points to the correct physical port.
• Cisco NX-OS does not support transparent bridging between two VLANs when one of the SVIs is on
the Cisco Nexus 9000 using the BIA MAC (burned-in MAC address). This occurs when the BIA MAC
is shared between SVIs/VLANs. A MAC, different from the BIA MAC, can be configured under the
SVI for transparent bridging to work properly.

Note This behavior is applicable to Cisco Nexus 9300 Switches (Network Forwarding
Engine) and Cisco Nexus 9500 Switches with 95xx,96xx,94xx line cards. This
behavior is not applicable to Cisco Nexus 9200 Switches, Cisco Nexus 9300-EX
and Cisco Nexus 9500 Switches with 9700-EX line cards.

• Port-local VLANs do not support Fabric Extenders (FEX).

• On Cisco Nexus 9364C switches, auto-negotiation may not work on ports 49-64 when bringing up 100G
links using QSFP-100G-CR4 cable. To workaround this issue, you must hard-code the speed on ports
49-64 and disable auto-negotiation.
• Cisco Nexus 9000 switch may fail to connect to a checkpoint with 1G SFP where an autonegotiation
enabled Cisco Nexus 93180YC-EX switch is connected. The workaround is to disable autonegotiation
and to manually enter the speed and duplex values.

Default Settings for Layer 2 Interfaces

The following table lists the default settings for device access and trunk port mode parameters.

Table 6: Default Access and Trunk Port Mode Parameters (7.0(3)I1(2) and earlier)

Parameters Default

Switchport mode Access

Allowed VLANs 1 to 3967, 4048 to 4094

Access VLAN ID VLAN1

Native VLAN ID VLAN1

Native VLAN ID tagging Disabled

Administrative state Shut

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
67
 Configuring Layer 2 Interfaces
Configuring Access and Trunk Interfaces

Table 7: Default Access and Trunk Port Mode Parameters (7.0(3)I2(1) and later)

Parameters Default

Switchport mode Access

Allowed VLANs 1 to 3967, 4048 to 4094

Access VLAN ID VLAN1

Native VLAN ID VLAN1

Native VLAN ID tagging Disabled

Administrative state Shut

SVI autostate Enabled

Configuring Access and Trunk Interfaces

Note If you are familiar with the Cisco IOS CLI, be aware that the Cisco NX-OS commands for this feature might
differ from the Cisco IOS commands that you would use.

Guidelines for Configuring Access and Trunk Interfaces

All VLANs on a trunk must be in the same VDC.

Configuring a VLAN Interface as a Layer 2 Access Port

You can configure a Layer 2 port as an access port. An access port transmits packets on only one, untagged
VLAN. You specify which VLAN traffic that the interface carries, which becomes the access VLAN. If you
do not specify a VLAN for an access port, that interface carries traffic only on the default VLAN. The default
VLAN is VLAN1.
The VLAN must exist before you can specify that VLAN as an access VLAN. The system shuts down an
access port that is assigned to an access VLAN that does not exist.

Before you begin

Ensure that you are configuring a Layer 2 interface.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
68
Configuring Layer 2 Interfaces
Configuring a VLAN Interface as a Layer 2 Access Port

Command or Action Purpose

switch# configure terminal
switch(config)#

Step 2 interface ethernet {{type slot/port} | Specifies an interface to configure, and enters
{port-channel number}} interface configuration mode.
Example:
switch(config)# interface ethernet 3/1
switch(config-if)#

Step 3 switchport mode [access | trunk] Sets the interface as a nontrunking nontagged,
single-VLAN Layer 2 interface. An access port
Example:
can carry traffic in one VLAN only. By default,
switch(config-if)# switchport mode access an access port carries traffic for VLAN1; to set
the access port to carry traffic for a different
VLAN, use the switchport access vlan
command.

Step 4 switchport access vlan vlan-id Specifies the VLAN for which this access port
will carry traffic. If you do not enter this
Example:
command, the access port carries traffic on
switch(config-if)# switchport access vlan VLAN1 only; use this command to change the
5
VLAN for which the access port carries traffic.

Step 5 exit Exits the interface configuration mode.

Example:
switch(config-if)# exit
switch(config)#

Step 6 show interface (Optional) Displays the interface status and

information.
Example:
switch# show interface

Step 7 no shutdown (Optional) Clears the errors on the interfaces

and VLANs where policies correspond with
Example:
hardware policies. This command allows policy
switch# configure terminal programming to continue and the port to come
switch(config)# int e3/1
switch(config-if)# no shutdown
up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Step 8 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to set Ethernet 3/1 as a Layer 2 access port that carries traffic for VLAN
5 only:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
69
 Configuring Layer 2 Interfaces
Configuring Access Host Ports

switch# configure terminal

switch(config)# interface ethernet 3/1
switch(config-if)# switchport mode access
switch(config-if)# switchport access vlan 5
switch(config-if)#

Configuring Access Host Ports

Note You should apply the switchport host command only to interfaces that are connected to an end station.

You can optimize the performance of access ports that are connected to end stations by simultaneously setting
that port as an access port. An access host port handles the STP like an edge port and immediately moves to
the forwarding state without passing through the blocking and learning states. Configuring an interface as an
access host port also disables port channeling on that interface.

Note See “Configuring Port Channels” section and the Cisco Nexus 9000 Series NX-OS Layer 2 Switching
Configuration Guide for information about port-channel interfaces

Before you begin

Ensure that you are configuring the correct interface to an interface that is an end station.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface ethernet type slot/port Specifies an interface to configure, and enters
interface configuration mode.
Example:
switch(config)# interface ethernet 3/1
switch(config-if)#

Step 3 switchport host Sets the interface to be an access host port,

which immediately moves to the spanning tree
Example:
forwarding state and disables port channeling
switch(config-if)# switchport host on this interface.
Note Apply this command only to end
stations.

Step 4 exit Exits the interface mode.

Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
70
 Configuring Layer 2 Interfaces
Configuring Trunk Ports

Command or Action Purpose

switch(config-if-range)# exit
switch(config)#

Step 5 show interface (Optional) Displays the interface status and

information.
Example:
switch# show interface

Step 6 no shutdown (Optional) Clears the errors on the interfaces

and VLANs where policies correspond with
Example:
hardware policies. This command allows policy
switch# configure terminal programming to continue and the port to come
switch(config)# int e3/1
switch(config-if)# no shutdown
up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Step 7 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to set Ethernet 3/1 as a Layer 2 access port with PortFast enabled and port
channel disabled:
switch# configure terminal
switch(config)# interface ethernet 3/1
switch(config-if)# switchport host
switch(config-if)#

Configuring Trunk Ports

You can configure a Layer 2 port as a trunk port. A trunk port transmits untagged packets for one VLAN plus
encapsulated, tagged, packets for multiple VLANs. (See the “IEEE 802.1Q Encapsulation” section for
information about encapsulation.)

Note The device supports 802.1Q encapsulation only.

Before you begin

Before you configure a trunk port, ensure that you are configuring a Layer 2 interface.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
71
 Configuring Layer 2 Interfaces
Configuring Trunk Ports

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface {type slot/port | port-channel Specifies an interface to configure, and enters
number} interface configuration mode.
Example:
switch(config)# interface ethernet 3/1
switch(config-if)#

Step 3 switchport mode [access | trunk] Sets the interface as a Layer 2 trunk port. A
trunk port can carry traffic in one or more
Example:
VLANs on the same physical link (VLANs are
switch(config-if)# switchport mode trunk based on the trunk-allowed VLANs list). By
default, a trunk interface can carry traffic for
all VLANs. To specify that only certain VLANs
are allowed on the specified trunk, use the
switchport trunk allowed vlan command.

Step 4 exit Exits the interface mode.

Example:
switch(config-if)# exit
switch(config)#

Step 5 show interface (Optional) Displays the interface status and

information.
Example:
switch# show interface

Step 6 no shutdown (Optional) Clears the errors on the interfaces

and VLANs where policies correspond with
Example:
hardware policies. This command allows policy
switch# configure terminal programming to continue and the port to come
switch(config)# int e3/1
switch(config-if)# no shutdown
up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Step 7 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to set Ethernet 3/1 as a Layer 2 trunk port:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
72
 Configuring Layer 2 Interfaces
Configuring the Native VLAN for 802.1Q Trunking Ports

switch# configure terminal

switch(config)# interface ethernet 3/1
switch(config-if)# switchport mode trunk
switch(config-if)#

Configuring the Native VLAN for 802.1Q Trunking Ports

You can configure the native VLAN for 802.1Q trunk ports. If you do not configure this parameter, the trunk
port uses the default VLAN as the native VLAN ID.

Note You cannot configure an FCoE VLAN as a native VLAN for an Ethernet interface.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface {{type slot/port} | {port-channel Specifies an interface to configure, and enters
number}} interface configuration mode.
Example:
switch(config)# interface ethernet 3/1
switch(config-if)#

Step 3 switchport trunk native vlan vlan-id Sets the native VLAN for the 802.1Q trunk.
Valid values are from 1 to 4094, except those
Example:
VLANs reserved for internal use. The default
switch(config-if)# switchport trunk value is VLAN1.
native vlan 5

Step 4 exit Exits interface configuration mode.

Example:
switch(config-if-range)# exit
switch(config)#

Step 5 show vlan (Optional) Displays the status and information

of VLANs.
Example:
switch# show vlan

Step 6 no shutdown (Optional) Clears the errors on the interfaces

and VLANs where policies correspond with
Example:
hardware policies. This command allows policy
switch# configure terminal programming to continue and the port to come
switch(config)# int e3/1
switch(config-if)# no shutdown
up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
73
 Configuring Layer 2 Interfaces
Configuring the Allowed VLANs for Trunking Ports

Command or Action Purpose

Step 7 copy running-config startup-config (Optional) Copies the running configuration to
the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to set the native VLAN for the Ethernet 3/1, Layer 2 trunk port to VLAN
5:
switch# configure terminal
switch(config)# interface ethernet 3/1
switch(config-if)# switchport trunk native vlan 5
switch(config-if)#

Configuring the Allowed VLANs for Trunking Ports

You can specify the IDs for the VLANs that are allowed on the specific trunk port.

Note The switchport trunk allowed vlan vlan-list command replaces the current VLAN list on the specified port
with the new list. You are prompted for confirmation before the new list is applied.
If you are doing a copy and paste of a large configuration, you might see some failures because the CLI is
waiting for a confirmation before accepting other commands. To avoid this problem, you can disable prompting
by using the terminal dont-ask command before you paste the configuration.

Before you begin

Before you configure the allowed VLANs for the specified trunk ports, ensure that you are configuring the
correct interfaces and that the interfaces are trunks.

Note You can change the block of VLANs reserved for internal use. See the Cisco Nexus 9000 Series NX-OS
Layer 2 Switching Configuration Guide for more information about changing the reserved VLANs.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
74
Configuring Layer 2 Interfaces
Configuring the Allowed VLANs for Trunking Ports

Command or Action Purpose

Step 2 interface {ethernet slot/port | port-channel Specifies an interface to configure, and enters
number} interface configuration mode.
Example:
switch(config)# interface ethernet 3/1

Step 3 switchport trunk allowed vlan {vlan-list add Sets the allowed VLANs for the trunk interface.
vlan-list | all | except vlan-list | none | remove The default is to allow all VLANs on the trunk
vlan-list} interface: 1 to 3967 and 4048 to 4094. VLANs
3968 to 4047 are the default VLANs reserved
Example:
for internal use by default. By default, all
switch(config-if)# switchport trunk VLANs are allowed on all trunk interfaces.
allowed vlan add 15-20#
The default reserved VLANs are 3968 to 4094,
and you can change the block of reserved
VLANs. See the Cisco Nexus 9000 Series
NX-OS Layer 2 Switching Configuration Guide
for more information.
Note You cannot add internally allocated
VLANs as allowed VLANs on trunk
ports. The system returns a message
if you attempt to list an internally
allocated VLAN as an allowed
VLAN.

Step 4 exit Exits the interface mode.

Example:
switch(config-if)# exit
switch(config)#

Step 5 show vlan (Optional) Displays the status and information

for VLANs.
Example:
switch# show vlan

Step 6 no shutdown (Optional) Clears the errors on the interfaces

and VLANs where policies correspond with
Example:
hardware policies. This command allows policy
switch# configure terminal programming to continue and the port to come
switch(config)# int e3/1
switch(config-if)# no shutdown
up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Step 7 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
75
 Configuring Layer 2 Interfaces
Configuring MAC addresses Limitation on a VLAN

Example
This example shows how to add VLANs 15 to 20 to the list of allowed VLANs on the Ethernet 3/1,
Layer 2 trunk port:
switch# configure terminal
switch(config)# interface ethernet 3/1
switch(config-if)# switchport trunk allowed vlan 15-20
switch(config-if)#

Configuring MAC addresses Limitation on a VLAN

Beginning with 7.0(3)F3(4), Cisco Nexus 9500 Series switches with N9K-X9636C-R, N9K-X9636Q-R,
N9K-X9636C-RX line cards provides the ability to set an upper limit for the number of MAC addresses that
can reside inside MAC address table of a Line-card Expansion-module (LEM). You can configure the limitations
at System, VLAN, port, trunk and tunnel levels. For instance if the specified VLAN limitation is 2000 MACs,
the Layer 2 Forwarding Manager (L2FM) accepts the first 2000 MACs it receives and reject the remaining
MACs. To configure MAC address limitation on VLAN, complete the following steps:

Note Though this method stops learning MAC address learning via VLAN or system after the limit is reached, the
traffic passes through the interface.

Procedure

Command or Action Purpose

Step 1 switch# configure terminal Enters global configuration mode.

Step 2 switch(config)# mac address-table limit Specifies an upper limit for MAC learning at
system value system level.

Step 3 switch(config)# mac address-table limit vlan Specifies an upper limit for MAC learning at
value VLAN level.

Step 4 switch(config)# exit Exits configuration mode.

Example
This example shows how to configure the upper limit for MAC learning at system and VLAN levels:

switch# configure terminal

switch(config)# mac address-table limit system 10000
Configuring Mac address limit will result in flushing existing Macs in the specified
VLAN/System.Proceed(yes/no)? [no] yes
switch(config)# mac address-table limit vlan 30 3000
Configuring Mac address limit will result in flushing existing Macs in the specified
VLAN/System.Proceed(yes/no)? [no] yes
switch(config)# exit

This example shows how to display the MAC address limitations:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
76
 Configuring Layer 2 Interfaces
Configuring switchport isolated

switch# configure terminal

switch(config)# sh mac address-table limit

System Limit: 10000

Vlan Learning Limit

---- ----------
1 196000
20 196000
30 3000
100 196000
switch(config)# exit

Configuring switchport isolated

Beginning with 7.0(3)I2(1), the switchport isolated feature is supported. Switchport isolated can be configured
on an interface to accommodate up to 3967 VLANs on an interface. The interfaces that are configured with
switchport isolated do not send STP BPDUs.

Note The switchport isolated mode is not supported on an interface that is connected to a FEX, a switch, router
or any other networking devices. Switchport Isolated is not supported on the FEX HIF ports.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface {{ethernet slot/port} | {port-channel Specifies an interface to configure, and enters
number}} interface configuration mode.
Example:
switch(config)# interface ethernet 3/1
switch(config-if)#

Step 3 switchport isolated Enables the switchport isolated feature.

Example:
switch(config-if)# switchport isolated

Step 4 show running-config interface port-channel (Optional) Displays the interface status and
port-channel-number information.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
77
 Configuring Layer 2 Interfaces
Configuring a Default Interface

Configuring a Default Interface

The default interface feature allows you to clear the existing configuration of multiple interfaces such as
Ethernet, loopback, VLAN network, port-channel, and tunnel interfaces. All user configuration under a
specified interface will be deleted. You can optionally create a checkpoint before clearing the interface
configuration so that you can later restore the deleted configuration.

Note The default interface feature is not supported for management interfaces because the device could go to an
unreachable state.
If the speed group is configured, the default interface command displays the following error:
Error: default interface is not supported as speed-group is configured

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 default interface int-if [checkpoint name] Deletes the configuration of the interface and
restores the default configuration. Use the ?
Example:
keyword to display the supported interfaces.
switch(config)# default interface
ethernet 3/1 checkpoint test8 Use the checkpoint keyword to store a copy of
the running configuration of the interface before
clearing the configuration.

Step 3 exit Exits global configuration mode.

Example:
switch(config)# exit
switch(config)#

Step 4 show interface (Optional) Displays the interface status and

information.
Example:
switch# show interface

Step 5 no shutdown (Optional) Clears the errors on the interfaces

and VLANs where policies correspond with
Example:
hardware policies. This command allows policy
switch# configure terminal programming to continue and the port to come
switch(config)# int e3/1
switch(config-if)# no shutdown
up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
78
 Configuring Layer 2 Interfaces
Configuring SVI Autostate Exclude

Example
This example shows how to delete the configuration of an Ethernet interface while saving a checkpoint
of the running configuration for rollback purposes:
switch# configure terminal
switch(config)# default interface ethernet 3/1 checkpoint test8
.......Done
switch(config)#

Configuring SVI Autostate Exclude

Beginning with7.0(3)I2(1), you can configure the SVI Autostate Exclude feature on an Ethernet interface or
a port channel. You can use the Autostate Exclude option to enable or disable the port from bringing up or
down the SVI calculation and applying it to all VLANs that are enabled on the selected port. You can also
use the SVI Autostate Exlude VLAN feature to exclude a VLAN from the autostate excluded interface.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface {{type slot/port} | {port-channel Specifies an interface to configure, and enters
number}} interface configuration mode.
Example:
switch(config)# interface ethernet 3/1
switch(config-if)#

Step 3 switchport Sets the interface as a Layer 2 interface.

Example:
switch(config-if)# switchport

Step 4 [no] switchport autostate exclude Excludes this port from the VLAN interface
link-up calculation when there are multiple ports
Example:
in the VLAN.
switch(config-if)# switchport autostate
exclude To revert to the default settings, use the no form
of this command.

Step 5 [no] switchport autostate exclude vlan {vlan (Optional) Excludes a vlan or a set of vlans
id | all | except} from the autostate-excluded interface. This will
help to minimize any disruption to the system.
Example:
switch(config-if)# switchport autostate To revert to the default settings, use the no form
exclude vlan 10 of this command.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
79
 Configuring Layer 2 Interfaces
Configuring SVI Autostate Disable for the System

Command or Action Purpose

Step 6 exit Exits the interface configuration mode.
Example:
switch(config-if)# exit
switch(config)#

Step 7 show running-config interface {{type (Optional) Displays configuration information

slot/port} | {port-channel number}} about the specified interface.
Example:
switch(config)# show running-config
interface ethernet 3/1

Step 8 no shutdown (Optional) Clears the errors on the interfaces

and VLANs where policies correspond with
Example:
hardware policies. This command allows policy
switch# configure terminal programming to continue and the port to come
switch(config)# int e3/1
switch(config-if)# no shutdown
up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Step 9 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to exclude a port from the VLAN interface link-up calculation on the Cisco
NX-OS device:
switch# configure terminal
switch(config)# interface ethernet 3/1
switch(config-if)# switchport
switch(config-if)# switchport autostate exclude

This example shows how to exclude a VLAN from the auto-excluded interface:
switch# configure terminal
switch(config)# interface ethernet 3/1
switch(config-if)# switchport
switch(config-if)# switchport autostate exclude
switch(config-if)# switchport autostate exclude vlan 10

Configuring SVI Autostate Disable for the System

You can manage an SVI with the SVI autostate feature. You can configure the SVI autostate disable feature
to keep an SVI up even if no interface is up in the corresponding VLAN. (Similarly, configure the SVI autostate
enable feature so an SVI goes down when no interface is up in the corresponding VLAN). Use this procedure
to configure this feature for the entire system.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
80
 Configuring Layer 2 Interfaces
Configuring SVI Autostate Disable Per SVI

Note The system default interface-vlan autostate command enables the SVI autostate feature.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 [no] system default interface-vlan autostate Disables the default autostate behavior for the
device.
Example:
switch(config)# no system default Note Use the system default
interface-vlan autostate interface-vlan autostate command
to enable the autostate behavior for
the device.

Step 3 no shutdown (Optional) Clears the errors on the interfaces

and VLANs where policies correspond with
Example:
hardware policies. This command allows policy
switch# configure terminal programming to continue and the port to come
switch(config)# int e3/1
switch(config-if)# no shutdown
up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Step 4 show running-config [all] (Optional) Displays the running configuration.

Example: To display the default and configured
switch(config)# show running-config information, use the all keyword.

Example
This example shows how to disable the default autostate behavior on the Cisco NX-OS device:
switch# configure terminal
switch(config)# no system default interface-vlan autostate
switch(config)# show running-config

Configuring SVI Autostate Disable Per SVI

You can configure SVI autostate enable or disable on individual SVIs. The SVI-level setting overrides the
system-level SVI autostate configuration for that particular SVI.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
81
 Configuring Layer 2 Interfaces
Configuring SVI Autostate Disable Per SVI

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 feature interface-vlan Enables VLAN interface mode.

Example:
switch(config)# feature interface-vlan

Step 3 interface vlan vlan-id Creates a VLAN interface and enters interface
configuration mode. The range is from 1 and
Example:
4094.
switch(config-if)# interface vlan10

switch(config)#

Step 4 [no] autostate By default, enables the SVI autostate feature

on specified interface.
Example:
switch(config-if)# no autostate To disable the default settings, use the no form
of this command.

Step 5 exit Exits the interface configuration mode.

Example:
switch(config-if)# exit
switch(config)#

Step 6 show running-config interface vlan vlan-id (Optional) Displays the running configuration
for the specified VLAN interface.
Example:
switch(config)# show running-config
interface vlan10

Step 7 no shutdown (Optional) Clears the errors on the interfaces

and VLANs where policies correspond with
Example:
hardware policies. This command allows policy
switch# configure terminal programming to continue and the port to come
switch(config)# int e3/1
switch(config-if)# no shutdown
up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Step 8 show startup-config interface vlan vlan-id (Optional) Displays the VLAN configuration
in the startup configuration.
Example:
switch(config)# show startup-config
interface vlan10

Example
This example shows how to disable the default autostate behavior on an individual SVI:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
82
 Configuring Layer 2 Interfaces
Configuring the Device to Tag Native VLAN Traffic

switch# configure terminal

switch(config)# feature interface-vlan
switch(config)# interface vlan10
witch(config-if)# no autostate

Configuring the Device to Tag Native VLAN Traffic

When you are working with 802.1Q trunked interfaces, you can maintain the tagging for all packets that enter
with a tag that matches the value of the native VLAN ID and drops all untagged traffic (you will still carry
control traffic on that interface). This feature applies to the entire device; you cannot apply it to selected
VLANs on a device.
The vlan dot1q tag native global command changes the behavior of all native VLAN ID interfaces on all
trunks on the device.

Note If you enable 802.1Q tagging on one device and disable it on another device, all traffic is dropped on the
device and this feature is disabled. You must configure this feature identically on each device.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 vlan dot1q tag native Modifies the behavior of a 802.1Q trunked
native VLAN ID interface. The interface
Example:
maintains the taggings for all packets that enter
switch(config)# vlan dot1q tag native with a tag that matches the value of the native
VLAN ID and drops all untagged traffic. The
control traffic is still carried on the native
VLAN. The default is disabled.

Step 3 exit Exits the interface configuration mode.

Example:
switch(config-if-range)# exit
switch(config)#

Step 4 show vlan (Optional) Displays the status and information

for VLANs.
Example:
switch# show vlan

Step 5 no shutdown (Optional) Clears the errors on the interfaces

and VLANs where policies correspond with
Example:
hardware policies. This command allows policy

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
83
 Configuring Layer 2 Interfaces
Configuring Interface Breakout Profile for 50-G Interfaces in a 16-Slot Chassis

Command or Action Purpose

switch# configure terminal programming to continue and the port to come
switch(config)# int e3/1 up. If policies do not correspond, the errors are
switch(config-if)# no shutdown
placed in an error-disabled policy state.

Step 6 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to change the behavior of the native VLAN on an 802.1Q trunked interface
to maintain the tagged packets and drop all untagged traffic (except control traffic):
switch# configure terminal
switch(config)# vlan dot1q tag native
switch#

Configuring Interface Breakout Profile for 50-G Interfaces in a 16-Slot Chassis

The interface breakout profile is needed to breakout high bandwidth 100-G ports into two 50-G interfaces for
slot 8 to 16 in the Cisco Nexus 9516 switch for -EX line cards.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal

Step 2 (Optional) interface breakout-profile This command is required to breakout slots 8

50g-2x-only to 16. It is not required for slots 1 to 7.
Example:
switch(config)# interface
breakout-profile 50g-2x-only
Warning: Please save config and reload
the switch for breakout-profile config
to take effect
Please save config and reload the switch
for the configuration to take effect

Step 3 copy running-config startup-config Copies the running configuration to the startup
configuration.
Example:
switch(config-inf)# copy running-config
startup-config
[########################################]
100%
Copy complete, now saving to disk (please

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
84
 Configuring Layer 2 Interfaces
Changing the System Default Port Mode to Layer 2

Command or Action Purpose

wait)...
Copy complete.

Step 4 reload Reboots the switch.

Example: Note After the switch reloads and the
switch(config-inf)# reload modules are up, enter the following
This command will reboot the system. CLI for any module or ports to
(y/n)? [n] y breakout.

Step 5 interface breakout module module-number Breaks out the 100-Gb port to 2 50-Gb ports.
port port-range map [10g-4x | 25g-4x | 50g-2x] The range of module-number is 1 to 30. The
range of port-range is 1 to 72.
Example:
switch(config)# interface breakout module
1 port 1-32 map 50g-2x

Changing the System Default Port Mode to Layer 2

You can set the system default port mode to Layer 2 access ports.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 system default switchport [shutdown] Sets the default port mode for all interfaces on
the system to Layer 2 access port mode and
Example:
enters interface configuration mode. By default,
switch(config-if)# system default all the interfaces are Layer 3.
switchport
Note When the system default
switchport shutdown command is
issued:
• Any FEX HIFs that are not
configured with no shutdown
are shutdown. To avoid the
shutdown, configure the FEX
HIFs with no shut
• Any Layer 2 port that is not
specifically configured with no
shutdown are shutdown. To
avoid the shutdown, configure
the Layer 2 port with no shut

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
85
 Configuring Layer 2 Interfaces
Verifying the Interface Configuration

Command or Action Purpose

Step 3 exit Exits the interface configuration mode.
Example:
switch(config-if)# exit
switch(config)#

Step 4 show interface brief (Optional) Displays the status and information
for interfaces.
Example:
switch# show interface brief

Step 5 no shutdown (Optional) Clears the errors on the interfaces

and VLANs where policies correspond with
Example:
hardware policies. This command allows policy
switch# configure terminal programming to continue and the port to come
switch(config)# int e3/1
switch(config-if)# no shutdown
up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Step 6 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to set the system ports to be Layer 2 access ports by default:
switch# configure terminal
switch(config-if)# system default switchport
switch(config-if)#

Verifying the Interface Configuration

To display access and trunk interface configuration information, perform one of the following tasks.

Command Purpose

show interface ethernet slot/port [brief | | counters Displays the interface configuration.
| debounce | description | flowcontrol | mac-address
| status | transceiver]

show interface brief Displays interface configuration information,

including the mode.

show interface switchport Displays information, including access and trunk

interface, information for all Layer 2 interfaces.

show interface trunk [module module-number | vlan Displays trunk configuration information.
vlan-id]

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
86
 Configuring Layer 2 Interfaces
Monitoring the Layer 2 Interfaces

Command Purpose

show interface capabilities Displays information about the capabilities of the

interfaces.

show running-config [all] Displays information about the current configuration.

The all command displays the default and current
configurations.

show running-config interface ethernet slot/port Displays configuration information about the specified
interface.

show running-config interface port-channel Displays configuration information about the specified
slot/port port-channel interface.

show running-config interface vlan vlan-id Displays configuration information about the specified
VLAN interface.

Monitoring the Layer 2 Interfaces

Use the following commands to display Layer 2 interfaces:

Command Purpose

clear counters interface [interface] Clears the counters.

load- interval {interval seconds {1 | 2 | 3}} Cisco Nexus 9000 Series devices set three different
sampling intervals to bit-rate and packet-rate statistics.

show interface counters [module module] Displays input and output octets unicast packets,
multicast packets, and broadcast packets.

show interface counters detailed [all] Displays input packets, bytes, and multicast as well
as output packets and bytes.

show interface counters errors [module module] Displays information on the number of error packets.

Configuration Examples for Access and Trunk Ports

This example shows how to configure a Layer 2 access interface and assign the access VLAN mode for that
interface:
switch# configure terminal
switch(config)# interface ethernet 2/30
switch(config-if)# switchport
switch(config-if)# switchport mode access
switch(config-if)# switchport access vlan 5
switch(config-if)#

This example shows how to configure a Layer 2 trunk interface, assign the native VLAN and the allowed
VLANs, and configure the device to tag the native VLAN traffic on the trunk interface:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
87
 Configuring Layer 2 Interfaces
Related Documents

switch# configure terminal

switch(config)# interface ethernet 2/35
switch(config-if)# switchport
switch(config-if)# switchport mode trunk
switch(config-if)# switchport trunk native vlan 10
switch(config-if)# switchport trunk allowed vlan 5, 10
switch(config-if)# exit
switch(config)# vlan dot1q tag native
switch(config)#

Related Documents
Related Documents Document Title

Configuring Layer 3 interfaces Configuring Layer 2 Interfaces section

Port Channels Configuring Port Channels section

VLANs, private VLANs, and STP Cisco Nexus 9000 Series NX-OS Layer 2 Switching
Configuration Guide

System management Cisco Nexus 9000 Series NX-OS System Management

Configuration Guide

High availability Cisco Nexus 9000 Series NX-OS High Availability

and Redundancy Guide

Licensing Cisco NX-OS Licensing Guide

Release Notes Cisco Nexus 9000 Series NX-OS Release Notes

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
88
 CHAPTER 5
Configuring Layer 3 Interfaces
• About Layer 3 Interfaces, on page 89
• Prerequisites for Layer 3 Interfaces, on page 94
• Guidelines and Limitations, on page 94
• Default Settings, on page 95
• Configuring Layer 3 Interfaces, on page 96
• Verifying the Layer 3 Interfaces Configuration, on page 115
• Monitoring the Layer 3 Interfaces, on page 116
• Configuration Examples for Layer 3 Interfaces, on page 117
• Related Documents, on page 119

About Layer 3 Interfaces

Layer 3 interfaces forward IPv4 and IPv6 packets to another device using static or dynamic routing protocols.
You can use Layer 3 interfaces for IP routing and inter-VLAN routing of Layer 2 traffic.

Routed Interfaces
You can configure a port as a Layer 2 interface or a Layer 3 interface. A routed interface is a physical port
that can route IP traffic to another device. A routed interface is a Layer 3 interface only and does not support
Layer 2 protocols, such as the Spanning Tree Protocol (STP).
All Ethernet ports are routed interfaces by default. You can change this default behavior with the CLI setup
script.

Note The default behavior varies based on the type of switch (Cisco Nexus 9300, Cisco Nexus 9500, or Cisco
Nexus 3164).

Note Cisco Nexus 9300 Series switches (except Cisco Nexus 9332 switch) have a Layer 2 default mode.

You can assign an IP address to the port, enable routing, and assign routing protocol characteristics to this
routed interface.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
89
 Configuring Layer 3 Interfaces
Subinterfaces

You can also create a Layer 3 port channel from routed interfaces. For more information about port channels,
see the “Configuring Port Channels” section.
Routed interfaces support exponentially decayed rate counters. Cisco NX-OS tracks the following statistics
with these averaging counters:
• Input packets/sec
• Output packets/sec
• Input bytes/sec
• Output bytes/sec

Subinterfaces
You can create virtual subinterfaces on a parent interface configured as a Layer 3 interface. A parent interface
can be a physical port.
Subinterfaces divide the parent interface into two or more virtual interfaces on which you can assign unique
Layer 3 parameters such as IP addresses and dynamic routing protocols. The IP address for each subinterface
should be in a different subnet from any other subinterface on the parent interface.
You create a subinterface with a name that consists of the parent interface name (for example, Ethernet 2/1)
followed by a period and then by a number that is unique for that subinterface. For example, you could create
a subinterface for Ethernet interface 2/1 named Ethernet 2/1.1 where .1 indicates the subinterface.
Cisco NX-OS enables subinterfaces when the parent interface is enabled. You can shut down a subinterface
independent of shutting down the parent interface. If you shut down the parent interface, Cisco NX-OS shuts
down all associated subinterfaces as well.
One use of subinterfaces is to provide unique Layer 3 interfaces to each virtual local area network (VLAN)
supported by the parent interface. In this scenario, the parent interface connects to a Layer 2 trunking port on
another device. You configure a subinterface and associate the subinterface to a VLAN ID using 802.1Q
trunking.
The following figure shows a trunking port from a switch that connects to router B on interface E 2/1. This
interface contains three subinterfaces that are associated with each of the three VLANs carried by the trunking
port.
Figure 4: Subinterfaces for VLANs

For more information about VLANs, see the Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration
Guide.

Limitations for Subinterfaces

The following are the limitations for subinterfaces:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
90
 Configuring Layer 3 Interfaces
VLAN Interfaces

• Statistics for subinterfaces are not supported.

VLAN Interfaces
A VLAN interface, or switch virtual interface (SVI), is a virtual routed interface that connects a VLAN on
the device to the Layer 3 router engine on the same device. Only one VLAN interface can be associated with
a VLAN, but you need to configure a VLAN interface for a VLAN only when you want to route between
VLANs or to provide IP host connectivity to the device through a virtual routing and forwarding (VRF)
instance that is not the management VRF. When you enable VLAN interface creation, Cisco NX-OS creates
a VLAN interface for the default VLAN (VLAN 1) to permit remote switch administration.
You must enable the VLAN network interface feature before you can see configure it. The system automatically
takes a checkpoint prior to disabling the feature, and you can roll back to this checkpoint. See the Cisco Nexus
9000 Series NX-OS System Management Configuration Guide for information on rollbacks and checkpoints.

Note You cannot delete the VLAN interface for VLAN 1.

You can route across VLAN interfaces to provide Layer 3 inter-VLAN routing by configuring a VLAN
interface for each VLAN that you want to route traffic to and assigning an IP address on the VLAN interface.
For more information about IP addresses and IP routing, see the Cisco Nexus 9000 Series NX-OS Unicast
Routing Configuration Guide.
The following figure shows two hosts connected to two VLANs on a device. You can configure VLAN
interfaces for each VLAN that allows Host 1 to communicate with Host 2 using IP routing between the VLANs.
VLAN 1 communicates at Layer 3 over VLAN interface 1 and VLAN 10 communicates at Layer 3 over
VLAN interface 10.
Figure 5: Connecting Two VLANs with VLAN interfaces

Changing VRF Membership for an Interface

When you enter the vrf member command under an interface, you receive an alert regarding the deletion of
interface configurations and to notify the clients/listeners (such as CLI-Server) to delete configurations with
respect to the interface.
Entering the system vrf-member-change retain-l3-config command (7.0(3)I4(1) and later) enables the
retention of the Layer 3 configuration when the VRF member changes on the interface. It does this by sending
notification to the clients/listeners to store (buffer) the existing configurations, delete the configurations from
the old vrf context, and reapply the stored configurations under the new VRF context.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
91
 Configuring Layer 3 Interfaces
Notes About Changing VRF Membership for an Interface

Note When the system vrf-member-change retain-l3-config command is enabled, the Layer 3 configuration is
not deleted and remains stored (buffered). When this command is not enabled (default mode), the Layer 3
configuration is not retained when the VRF member changes.

You can disable the retention of the Layer 3 configuration with the no system vrf-member-change
retain-l3-config command. In this mode, the Layer 3 configuration is not retained when the VRF member
changes.

Notes About Changing VRF Membership for an Interface

• Momentary traffic loss may occur when changing the VRF name.
• Only the configurations under the interface level are processed when the system vrf-member-change
retain-l3-config command is enabled. You must manually process any configurations at the router level
to accommodate routing protocols after a VRF change.
• The system vrf-member-change retain-l3-config command supports interface level configurations
with:
• Layer 3 configurations maintained by the CLI Server, such as ip address and ipv6 address
(secondary) and all OSPF/ISIS/EIGRP CLIs available under the interface configuration.
• HSRP
• DHCP Relay Agent CLIs, such as ip dhcp relay address [use-vrf] and ipv6 dhcp relay address
[use-vrf].

• For DHCP:
• As a best practice, the client and server interface VRF should be changed one at a time. Otherwise,
the DHCP packets cannot be exchanged on the relay agent.
• When the client and server are in different VRFs, use the ip dhcp relay address [use-vrf] command
to exchange the DHCP packets in the relay agent over the different VRFs.

Loopback Interfaces
A loopback interface is a virtual interface with a single endpoint that is always up. Any packet transmitted
over a loopback interface is immediately received by this interface. Loopback interfaces emulate a physical
interface. You can configure up to 1024 loopback interfaces, numbered 0 to 1023.
You can use loopback interfaces for performance analysis, testing, and local communications. Loopback
interfaces can act as a termination address for routing protocol sessions. This loopback configuration allows
routing protocol sessions to stay up even if some of the outbound interfaces are down.

IP Unnumbered
The IP unnumbered feature enables the processing of IP packets on a point to point (p2p) interface without
explicitly configuring a unique IP address on it. This approach borrows an IP address from another interface
and conserves address space on point to point links.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
92
 Configuring Layer 3 Interfaces
MAC-Embedded IPv6 Address

Any interface which conforms to the point to point mode can be used as an IP unnumbered interface. For
7.0(3)I3(1) and later, the IP unnumbered feature is supported only on Ethernet interfaces and sub-interfaces.
The borrowed interface can only be a loopback interface and is known as the numbered interface.
A loopback interface is ideal as a numbered interface in that it is always functionally up. However, because
loopback interfaces are local to a switch/router, the reachability of unnumbered interfaces first needs to be
established through static routes or by using an interior gateway protocol, such as OSPF or ISIS.
Beginning with Cisco NX-OS Release 7.0(3)I6(1), configuring IP unnumbered interfaces for port channels
is supported on all Cisco Nexus 9000 Series switches.

MAC-Embedded IPv6 Address

BGP allows an IPv4 prefix to be carried over an IPv6 next hop. The IPv6 next hop is leveraged to remove
neighbor discovery (ND)-related traffic from the network. To do this (7.0(3)I2(1) and later), the MAC address
is embedded in the IPv6 address. Such an address is called a MAC-embedded IPv6 (MEv6) address. The
router extracts the MAC address directly from the MEv6 address instead of going through ND. Local interface
and next-hop MAC addresses are extracted from the IPv6 addresses.
On MEv6-enabled IPv6 interfaces, the same MEv6-extracted MAC address is used for IPv4 traffic as well.
MEv6 is supported on all Layer 3-capable interfaces except switch virtual interfaces (SVIs).

Important When MEv6 is enabled on an interface, ping6 to the IPv6 link local address, OSPFv3, and BFDv6 are not
supported on that interface.

High Availability
Layer 3 interfaces support stateful and stateless restarts. After the switchover, Cisco NX-OS applies the runtime
configuration after the switchover.
See the Cisco Nexus 9000 Series NX-OS High Availability and Redundancy Guide for complete information
about high availability.

Virtualization Support
Layer 3 interfaces support Virtual Routing and Forwarding instances (VRFs). VRFs exist within virtual device
contexts (VDCs). By default, Cisco NX-OS places you in the default VDC and default VRF .

Note You must assign an interface to a VRF before you configure the IP address for that interface.

DHCP Client
Beginning with 7.0(3)I2(1), Cisco NX-OS supports DHCP client for IPv4 and IPv6 addresses on SVIs, physical
Ethernet, and management interfaces. You can configure the IP address of a DHCP client by using the ip
address dhcp or ipv6 address dhcp command. These commands send a request from the DHCP client to the
DHCP server soliciting an IPv4 or IPv6 address from the DHCP server. The DHCP client on the Cisco Nexus

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
93
 Configuring Layer 3 Interfaces
Limitations for Using DHCP Client on Interfaces

switch identifies itself to the DHCP server. The DHCP server uses this identifier to send the IP address back
to the DHCP client.
When a DHCP client is configured on the SVI with the DHCP server sending router and DNS options, the ip
route 0.0.0.0/0 router-ip and ip name-server dns-ip commands are configured on the switch automatically.

Limitations for Using DHCP Client on Interfaces

The following are the limitations for using DHCP client on interfaces:
• This feature is supported only on physical Ethernet interfaces, management interfaces, and SVIs.
• This feature is supported on non-default virtual routing and forwarding (VRF) instances.
• The DNS server and default router option-related configurations are saved in the startup configuration
when you enter the copy running-config startup-config command. When you reload the switch, if this
configuration is not applicable, you might have to remove it.
• You can configure a maximum of six DNS servers on the switch, which is a switch limitation. This
maximum number includes the DNS servers configured by the DHCP client and the DNS servers
configured manually.
If the number of DNS servers configured on the switch is more than six, and if you get a DHCP offer
for an SVI with DNS option set, the IP address is not assigned to the SVI.
• A Cisco Nexus 9000 Series switch supports a maximum of 10 IPv4 and 10 IPv6 DHCP clients (7.0(3)I4(1)
and later).
• DHCP relay and DHCP client configurations are incompatible and are not supported on the same switch.
You must remove the DHCP relay configuration before configuring the DHCP Client on an interface.
• When DHCP snooping is enabled on the VLAN whose SVI is configured with the DHCP client, the
DHCP snooping is not enforced on the SVI DHCP client.
• When configuring the IPv6 DHCP client, you must configure with the ipv6 address use-link-local-only
command before the ipv6 address dhcp command.

Prerequisites for Layer 3 Interfaces

Layer 3 interfaces have the following prerequisites:
• You are familiar with IP addressing and basic configuration. See the Cisco Nexus 9000 Series NX-OS
Unicast Routing Configuration Guide for more information about IP addressing.

Guidelines and Limitations

Layer 3 interfaces have the following configuration guidelines and limitations:
• show commands with the internal keyword are not supported.
• Configuring a subinterface on a physical interface that is configured to be a member of a port-channel
is not supported. One must configure the subinterface under the port-channel interface itself.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
94
 Configuring Layer 3 Interfaces
Default Settings

• If you change a Layer 3 interface to a Layer 2 interface, Cisco NX-OS shuts down the interface, reenables
the interface, and removes all configuration specific to Layer 3. (7.0(3)I1(2) and later)
• If you change a Layer 2 interface to a Layer 3 interface, Cisco NX-OS shuts down the interface, reenables
the interface, and deletes all configuration specific to Layer 2. (7.0(3)I1(2) and later)
• The Dynamic Host Configuration Protocol (DHCP) option is not supported when configuring a subinterface
on a port-channel interface.
• When an IP unnumbered interface is configured, a loopback interface should be in the same VRF as the
IP unnumbered interface. (7.0(3)I3(1) and later)
• An admin-shutdown command on a loopback interface that is a numbered interface does not bring down
the IP unnumbered interface. This means that the routing protocols running over the IP unnumbered
interface continue to be up. (7.0(3)I3(1) and later)
• Static routes running over the IP unnumbered interface should use pinned static routes. (7.0(3)I3(1) and
later)

Note The IP unnumbered interface through which the route is resolved needs to be
specified.

• An IP unnumbered interface is supported only on physical and sub-interfaces. (7.0(3)I3(1) and later)
• Only loopback interfaces can use unnumbered interfaces as numbered interfaces. (7.0(3)I3(1) and later)
• OSPF over an IP unnumbered interface is supported. (7.0(3)I3(1) and later)
• ISIS over an IP unnumbered interface is supported. (7.0(3)I3(1) and later)
• BGP over a loopback interface with an IP unnumbered interface as an overlay interface is supported.
(7.0(3)I5(1) and later)
• The default and non-default VRF is supported by IP unnumbered interfaces. (7.0(3)I3(1) and later)
• The switch has a limit of 16 user-defined MAC addresses (MEv6/static). Configuring beyond this limit
might result in issues documented in CSCux84428.
• Layer 3 routed traffic for missing Layer 2 adjacency information is not flooded back onto VLAN members
of ingress units when the source MAC address of routed traffic is a non-VDC (Virtual Device Context)
MAC address. This limitation is for hardware flood traffic and can occur when the SVI (Switched Virtual
Interface) has a user-configured MAC address.

Note If you are familiar with the Cisco IOS CLI, be aware that the Cisco NX-OS commands for this feature might
differ from the Cisco IOS commands that you would use.

Default Settings
The following table lists the default settings for Layer 3 interface parameters.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
95
 Configuring Layer 3 Interfaces
Configuring Layer 3 Interfaces

Table 8: Default Layer 3 Interface Parameters

Parameters Default

Admin state Shut

Configuring Layer 3 Interfaces

Configuring a Routed Interface
You can configure any Ethernet port as a routed interface.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface ethernet slot/port Enters interface configuration mode.

Example:
switch(config)# interface ethernet 2/1
switch(config-if)#

Step 3 no switchport Configures the interface as a Layer 3 interface.

Example:
switch(config-if)# no switchport

Step 4 [ip address ip-address/length | ipv6 address • Configures an IP address for this interface.
ipv6-address/length] See the Cisco Nexus 9000 Series NX-OS
Unicast Routing Configuration Guide for
Example:
more information about IP addresses.
switch(config-if)# ip address 192.0.2.1/8
• Configures an IPv6 address for this
Example:
interface. See the Cisco Nexus 9000 Series
switch(config-if)# ipv6 address NX-OS Unicast Routing Configuration
2001:0DB8::1/8
Guide for more information about IPv6
addresses.

Step 5 show interfaces (Optional) Displays the Layer 3 interface

statistics.
Example:
switch(config-if)# show interfaces
ethernet 2/1

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
96
Configuring Layer 3 Interfaces
Configuring a Routed Interface

Command or Action Purpose

Step 6 no shutdown (Optional) Clears the errors on the interfaces
where policies correspond with hardware
Example:
policies. This command allows policy
switch# programming to continue and the port to come
switch(config-if)# int e2/1
switch(config-if)# no shutdown up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Step 7 copy running-config startup-config (Optional) Saves the configuration change.

Example:
switch(config)# copy running-config
startup-config

Example
• Use the medium command to set the interface medium to either point to point or broadcast.

Command Purpose

medium {broadcast | p2p} Configures the interface medium as either point to

point or broadcast.
Example:
switch(config-if)# medium p2p medium p2p

Note The default setting is broadcast, and this setting does not appear in any of the show commands.
However, if you do change the setting to p2p, you will see this setting when you enter the show
running config command.

• Use the switchport command to convert a Layer 3 interface into a Layer 2 interface.

Command Purpose

switchport Configures the interface as a Layer 2 interface and

deletes any configuration specific to Layer 3 on this
Example:
interface.
switch(config-if)# switchportswitchport

• This example shows how to configure a routed interface:

switch# configure terminal
switch(config)# interface ethernet 2/1
switch(config-if)# no switchport
switch(config-if)# ip address 192.0.2.1/8
switch(config-if)# copy running-config startup-config

The default setting for interfaces is routed. If you want to configure an interface for Layer 2,
enter the switchport command. Then, if you change a Layer 2 interface to a routed interface,
enter the no switchport command.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
97
 Configuring Layer 3 Interfaces
Configuring a Subinterface on a Routed Interface

Configuring a Subinterface on a Routed Interface

You can configure one or more subinterfaces on a routed interface made from routed interfaces.

Before you begin

Configure the parent interface as a routed interface.
See the “Configuring a Routed Interface” section.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface ethernet slot/port.number Creates a subinterface and enters subinterface

configuration mode. The number range is from
Example:
1 to 4094.
switch(config)# interface ethernet 2/1.1
switch(config-subif)#

Step 3 [ip address ip-address/length | ipv6 address • Configures an IP address for this
ipv6-address/length] subinterface. See the Cisco Nexus 9000
Series NX-OS Unicast Routing
Example:
Configuration Guide for more information
switch(config-subif)# ip address on IP addresses.
192.0.2.1/8

Example: • Configures an IPv6 address for this

subinterface. See the Cisco Nexus 9000
switch(config-subif)# ipv6 address
2001:0DB8::1/8
Series NX-OS Unicast Routing
Configuration Guide for more information
on IPv6 addresses.

Step 4 encapsulation dot1Q vlan-id Configures IEEE 802.1Q VLAN encapsulation

on the subinterface. The range is from 2 to
Example:
4093.
switch(config-subif)# encapsulation dot1Q
33

Step 5 show interfaces (Optional) Displays the Layer 3 interface

statistics.
Example:
switch(config-subif)# show interfaces
ethernet 2/1.1

Step 6 copy running-config startup-config (Optional) Saves the configuration change.

Example:
switch(config)# copy running-config
startup-config

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
98
 Configuring Layer 3 Interfaces
Configuring a Subinterface on a Port-Channel Interface

Example
• This example shows how to create a subinterface:
switch# configure terminal
switch(config)# interface ethernet 2/1.1
switch(config-if)# ip address 192.0.2.1/8
switch(config-if)# encapsulation dot1Q 33
switch(config-if)# copy running-config startup-config

• The output of the show interface eth command is enhanced for the subinterfaces as shown
in the following :
switch# show interface ethernet 1/2.1
Ethernet1/2.1 is down (Parent Interface Admin down)
admin state is down, Dedicated Interface, [parent interface is Ethernet1/2]
Hardware: 40000 Ethernet, address: 0023.ac67.9bc1 (bia 4055.3926.61d4)
Internet Address is 10.10.10.1/24
MTU 1500 bytes, BW 40000000 Kbit, DLY 10 usec
reliability 255/255, txload 1/255, rxload 1/255
Auto-mdix is turned off
EtherType is 0x8100
L3 in Switched:
ucast: 0 pkts, 0 bytes - mcast: 0 pkts, 0 bytes
L3 out Switched:
ucast: 0 pkts, 0 bytes - mcast: 0 pkts, 0 bytes

Configuring a Subinterface on a Port-Channel Interface

You can configure one or more subinterfaces on a port-channel interface.

Note Subinterfaces on a port-channel interface do not support multicast routing, router ACLs, QoS, policy-based
routing (PBR), SPAN, or ERSPAN.

Before you begin

Configure the parent interface as a port-channel interface. (7.0(3)I1(2) and later)
See the “Configuring Port Channels” chapter.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface port-channel channel-id.number Creates a subinterface and enters subinterface

configuration mode.
Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
99
 Configuring Layer 3 Interfaces
Configuring a VLAN Interface

Command or Action Purpose

switch(config)# interface port-channel
100.1
switch(config-subif)#

Step 3 [ip address ip-address/length | ipv6 address • Configures an IP address for this
ipv6-address/length] subinterface. See the Cisco Nexus 9000
Series NX-OS Unicast Routing
Example:
Configuration Guide for more information
switch(config-subif)# ip address on IP addresses.
192.0.2.1/8

Example: • Configures an IPv6 address for this

subinterface. See the Cisco Nexus 9000
switch(config-subif)# ipv6 address
2001:0DB8::1/8
Series NX-OS Unicast Routing
Configuration Guide for more information
on IPv6 addresses.

Step 4 encapsulation dot1Q vlan-id Configures IEEE 802.1Q VLAN encapsulation

on the subinterface. The range is from 2 to
Example:
4093.
switch(config-subif)# encapsulation dot1Q
33

Step 5 show interfaces (Optional) Displays the Layer 3 interface

statistics.
Example:
switch(config-subif)# show interfaces
ethernet 2/1.1

Step 6 copy running-config startup-config (Optional) Saves the configuration change.

Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to create a subinterface:

switch# configure terminal

switch(config)# interface port-channel 115.3
switch(config-subif)# ip address 141.143.101.2/24
switch(config-subif)# encapsulation dot1q 3
switch(config-subif)# copy running-config startup-config

Configuring a VLAN Interface

You can create VLAN interfaces to provide inter-VLAN routing.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
100
Configuring Layer 3 Interfaces
Configuring a VLAN Interface

Procedure

Command or Action Purpose

Step 1 configure terminal Enters configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 feature interface-vlan Enables VLAN interface mode.

Example:
switch(config)# feature interface-vlan

Step 3 interface vlan number Creates a VLAN interface. The number range
is from 1 to 4094.
Example:
switch(config)# interface vlan 10
switch(config-if)#

Step 4 [ip address ip-address/length | ipv6 address • Configures an IP address for this VLAN
ipv6-address/length] interface. See the Cisco Nexus 9000 Series
NX-OS Unicast Routing Configuration
Example:
Guide for more information on IP
switch(config-if)# ip address 192.0.2.1/8 addresses.
Example:
• Configures an IPv6 address for this VLAN
switch(config-if)# ipv6 address interface. See the Cisco Nexus 9000 Series
2001:0DB8::1/8
NX-OS Unicast Routing Configuration
Guide for more information on IPv6
addresses.

Step 5 show interface vlan number (Optional) Displays the Layer 3 interface
statistics.
Example:
switch(config-if)# show interface vlan
10

Step 6 no shutdown (Optional) Clears the errors on the interfaces

where policies correspond with hardware
Example:
policies. This command allows policy
switch(config)# int e3/1 programming to continue and the port to come
switch(config)# no shutdown
up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Step 7 copy running-config startup-config (Optional) Saves the configuration change.

Example:
switch(config-if)# copy running-config
startup-config

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
101
 Configuring Layer 3 Interfaces
Enabling Layer 3 Retention During VRF Membership Change

Example
This example shows how to create a VLAN interface:
switch# configure terminal
switch(config)# feature interface-vlan
switch(config)# interface vlan 10
switch(config-if)# ip address 192.0.2.1/8
switch(config-if)# copy running-config startup-config

Enabling Layer 3 Retention During VRF Membership Change

The following steps enable the retention of the Layer 3 configuration when changing the VRF membership
on the interface.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters configuration mode.
Example:

switch# configure terminal

switch(config)#

Step 2 system vrf-member-change retain-l3-config Enables Layer 3 configuration retention during

VRF membership change.
Example:
Note To disable the retention of the Layer
switch(config)# system vrf-member-change 3 configuration, use the no system
retain-l3-config vrf-member-change
retain-l3-config command.
Warning: Will retain L3 configuration
when vrf member change on interface.

Configuring a Loopback Interface

You can configure a loopback interface to create a virtual interface that is always up.

Before you begin

Ensure that the IP address of the loopback interface is unique across all routers on the network.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters configuration mode.
Example:
switch# configure terminal
switch(config)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
102
 Configuring Layer 3 Interfaces
Configuring IP Unnumbered on an Ethernet Interface

Command or Action Purpose

Step 2 interface loopback instance Creates a loopback interface. The range is from
0 to 1023.
Example:
switch(config)# interface loopback 0
switch(config-if)#

Step 3 [ip address ip-address/length | ipv6 address • Configures an IP address for this interface.
ipv6-address/length] See the Cisco Nexus 9000 Series NX-OS
Unicast Routing Configuration Guide for
Example:
more information about IP addresses.
switch(config-if)# ip address 192.0.2.1/8
• Configures an IPv6 address for this
Example:
interface. See the Cisco Nexus 9000 Series
switch(config-if)# ipv6 address NX-OS Unicast Routing Configuration
2001:0DB8::1/8
Guide for more information about IPv6
addresses.

Step 4 show interface loopback instance (Optional) Displays the loopback interface
statistics.
Example:
switch(config-if)# show interface
loopback 0

Step 5 copy running-config startup-config (Optional) Saves the configuration change.

Example:
switch(config-if)# copy running-config
startup-config

Example
This example shows how to create a loopback interface:
switch# configure terminal
switch(config)# interface loopback 0
switch(config-if)# ip address 192.0.2.1/8
switch(config-if)# copy running-config startup-config

Configuring IP Unnumbered on an Ethernet Interface

You can configure the IP unnumbered feature on an ethernet interface.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
103
 Configuring Layer 3 Interfaces
Configuring OSPF for an IP Unnumbered Interface

Command or Action Purpose

Step 2 interface ethernet slot/port Enters interface configuration mode.
Example:
switch(config)# interface ethernet 1/1
switch(config-if)#

Step 3 medium p2p Configures the interface medium as point to

point.
Example:
switch(config-if)# medium p2p

Step 4 ip unnumbered type number Enables IP processing on an interface without

assigning an explicit IP address to the interface.
Example:
switch(config-if)# ip unnumbered loopback type and number specify another interface on
100 which the router has an assigned IP address.
The interface specified cannot be another
unnumbered interface.
Note type is limited to loopback.
(7.0(3)I3(1) and later)

Configuring OSPF for an IP Unnumbered Interface

You can configure OSPF for an IP unnumbered loopback interface.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface ethernet slot/port Enters interface configuration mode.

Example:
switch(config)# interface ethernet
1/20.1
switch(config-if)#

Step 3 encapsulation dot1Q vlan-id Configures IEEE 802.1Q VLAN encapsulation

on the subinterface. The range is from 2 to
Example:
4093.
switch(config-if)# encapsulation dot1Q
100

Step 4 medium p2p Configures the interface medium as point to

point.
Example:
switch(config-if)# medium p2p

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
104
Configuring Layer 3 Interfaces
Configuring OSPF for an IP Unnumbered Interface

Command or Action Purpose

Step 5 ip unnumbered type number Enables IP processing on an interface without
assigning an explicit IP address to the
Example:
interface.
switch(config-if)# ip unnumbered
loopback 101 type and number specify another interface on
which the router has an assigned IP address.
The interface specified cannot be another
unnumbered interface.
Note type is limited to loopback.
(7.0(3)I3(1) and later)

Step 6 (Optional) ip ospf authentication Specifies the authentication type for interface.
Example:
switch(config-if)# ip ospf
authentication

Step 7 (Optional) ip ospf authentication-key Specifies the authentication password for

password OSPF authentication.
Example:
switch(config-if)# ip ospf
authentication 3 b7bdf15f62bbd250

Step 8 ip router ospf instance area area-number Configures routing process for IP on an
interface and specifies an area.
Example:
switch(config-if)# ip router ospf 100 Note The ip router ospf command is
area 0.0.0.1 required for both the unnumbered
and the numbered interface.

Step 9 no shutdown Brings up the interface (administratively).

Example:
switch(config-if)# no shutdown

Step 10 interface loopback instance Creates a loopback interface. The range is from
0 to 1023.
Example:
switch(config)# interface loopback 101

Step 11 ip address ip-address/length Configures an IP address for the interface.

Example:
switch(config-if)# 192.168.101.1/32

Step 12 ip router ospf instance area area-number Configures routing process for IP on an
interface and specifies an area.
Example:
switch(config-if)# ip router ospf 100 Note The ip router ospf command is
area 0.0.0.1 required for both the unnumbered
and the numbered interface.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
105
 Configuring Layer 3 Interfaces
Configuring ISIS for an IP Unnumbered Interface

Configuring ISIS for an IP Unnumbered Interface

You can configure ISIS for an IP unnumbered loopback interface.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 feature isis Enables ISIS.

Example:
Switch(config)# feature isis

Step 3 router isis area-tag Assigns a tag to an IS-IS process and enters
router configuration mode.
Example:
Switch(config)# router isis 100

Step 4 net network-entity-title Configures the network entity title (NET) on

the device.
Example:
Switch(config-router)# net
49.0001.0100.0100.1001.00

Step 5 end Exit router configuration mode.

Example:
Switch(config-router)# end

Step 6 interface ethernet slot/port Enters interface configuration mode.

Example:
switch(config)# interface ethernet
1/20.1

Step 7 encapsulation dot1Q vlan-id Configures IEEE 802.1Q VLAN encapsulation

on the subinterface. The range is from 2 to
Example:
4093.
switch(config-subif)# encapsulation
dot1Q 100

Step 8 medium p2p Configures the interface medium as point to

point.
Example:
switch(config-subif)# medium p2p

Step 9 ip unnumbered type number Enables IP processing on an interface without

assigning an explicit IP address to the
Example:
interface.
switch(config-if)# ip unnumbered
loopback 101

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
106
 Configuring Layer 3 Interfaces
Configuring PBR on SVI on the Gateway

Command or Action Purpose

type and number specify another interface on
which the router has an assigned IP address.
The interface specified cannot be another
unnumbered interface.
Note type is limited to loopback.
(7.0(3)I3(1) and later)

Step 10 ip router isis area-tag Enables ISIS on the unnumbered interface.

Example:
switch(config-subif)# ip router isis
100

Step 11 no shutdown Brings up the interface (administratively).

Example:
switch(config-subif)# no shutdown

Configuring PBR on SVI on the Gateway

This procedure configures PBR on the primary SVI interface in the gateway.

Note Steps 2 through 6 are needed if you want to configure a PBR policy on the unnumbered Primary/Secondary
VLAN interfaces. This is not mandatory for IP unnumbered on the SVI feature.

Procedure

Command or Action Purpose

Step 1 configure terminal Enter global configuration mode.
Example:
switch# configure terminal

Step 2 ip access-list list-name Configure access list.

Example:
switch(config)# ip access-list
pbr-sample

Step 3 permit tcp host ipaddr host ipaddr eq Specify the packets to forward on a specific
port-number port.
Example:
switch(config-acl)# permit tcp host
10.1.1.1 host 192.168.2.1 eq 80

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
107
 Configuring Layer 3 Interfaces
Configuring PBR on SVI on the Gateway

Command or Action Purpose

Step 4 exit Exit configuration mode.
Example:
switch(config-acl)# exit

Step 5 route-map route-map-name Create a route-map or enter route-map

command mode.
Example:
switch(config)# route-map pbr-sample

Step 6 match ip address access-list-name Match values from the routing table.
Example:
switch(config-route-map)# match ip
address pbr-sample

Step 7 set ip next-hop addr1 Set IP address of the next hop.

Example:
switch(config-route-map)# set ip
next-hop 192.168.1.1

Step 8 exit Exit command mode.

Example:
switch(config-route-map)# exit

Step 9 interface vlan vlan-id Creates a VLAN interface and enters interface
configuration mode. The range is from 1 and
Example:
4094.This is the primary VLAN.
switch(config)# interface vlan 2003

Step 10 ip address ip-addr Configures an IP address for the interface.

Example:
switch(config-if)# ip address 10.0.0.1/8

Step 11 no ip redirects Needs to be configured on all unnumbered

primary and secondary VLAN interfaces.
Example:
switch(config-if)# no ip redirects

Step 12 (Optional) ip policy route-map pbr-sample Enter this command if you want to apply a
PBR policy on the unnumbered
Example:
Primary/Secondary VLAN interface.
switch(config-if)# ip policy route-map
pbr-sample

Step 13 exit Exit command mode.

Example:
switch(config-if)# exit

Step 14 hsrp version 2 Set the HSRP version.

Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
108
 Configuring Layer 3 Interfaces
Configuring IP Unnumbered on SVI Secondary VLAN on the Gateway

Command or Action Purpose

switch(config-if)# hsrp version 2

Step 15 hsrpgroup-num Set the HSRP group number.

Example:
switch(config-if)# hsrp 200

Step 16 name name-val Configure the redundancy name string.

Example:
switch(config-if-hsrp)# name primary

Step 17 ip ip-addr Configures an IP address.

Example:
switch(config-if-hsrp)# ip 10.0.0.100

Step 18 no shutdown Negates shutdown.

Example:
switch(config-if-hsrp)# no shutdown

Configuring IP Unnumbered on SVI Secondary VLAN on the Gateway

This procedure configures IP unnumbered on the secondary SVI in the gateway.
Beginning with Cisco NX-OS Release 7.0(3)I7(1), this feature is supported on Cisco Nexus 93108TC-EX,
93180LC-EX and 93180YC-EX switches; and Cisco Nexus 9500 series switches with X9736C-EX,
X97160YC-EX and X9732C-EX line cards.

Procedure

Command or Action Purpose

Step 1 configure terminal Enter configuration mode.
Example:
switch# configure terminal

Step 2 interface vlan vlan-list Creates a VLAN interface and enters interface
configuration mode. The range is from 1 to
Example:
4094. This is the secondary VLAN.
switch(config)# interface vlan 2001

Step 3 ip unnumbered vlan primary-vlan-id Enables IP processing on an interface without

assigning an explicit IP address to an interface.
Example:
switch(config-if)# ip unnumbered vlan
2003

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
109
 Configuring Layer 3 Interfaces
Assigning an Interface to a VRF

Command or Action Purpose

Step 4 (Optional) ip policy route-map pbr-sample Enter this command if you want to apply a
PBR policy on the unnumbered
Example:
Primary/Secondary VLAN interface.
switch(config-if)# ip policy route-map
pbr-sample

Step 5 no ip redirects Needs to be configured on all unnumbered

primary and secondary VLAN interfaces.
Example:
switch(config-if)# no ip redirects

Step 6 hsrp version 2 Set the HSRP version.

Example:
switch(config-if)# hsrp version 2

Step 7 hsrp group-num Set the HSRP group number.

Example:
switch(config-if)# hsrp 200

Step 8 follow name Configure the group to be followed.

Example:
switch(config-if-hsrp)# follow primary

Step 9 ip ip-addr Enters HRSP IPv4 and sets the virtual IP

address.
Example:
switch(config-if-hsrp)# ip 10.0.0.100

Step 10 no shutdown Negate shutdown.

Example:
switch(config-if-hsrp)# no shutdown

Assigning an Interface to a VRF

You can add a Layer 3 interface to a VRF.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface interface-type number Enters interface configuration mode.

Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
110
 Configuring Layer 3 Interfaces
Configuring a MAC-Embedded IPv6 Address

Command or Action Purpose

switch(config)# interface loopback 0
switch(config-if)#

Step 3 vrf member vrf-name Adds this interface to a VRF.

Example:
switch(config-if)# vrf member
RemoteOfficeVRF

Step 4 ip address ip-prefix/length Configures an IP address for this interface. You

must do this step after you assign this interface
Example:
to a VRF.
switch(config-if)# ip address
192.0.2.1/16

Step 5 show vrf [vrf-name] interface interface-type (Optional) Displays VRF information.
number
Example:
switch(config-vrf)# show vrf Enterprise
interface loopback 0

Step 6 copy running-config startup-config (Optional) Saves the configuration change.

Example:
switch(config-if)# copy running-config
startup-config

Example
This example shows how to add a Layer 3 interface to the VRF:
switch# configure terminal
switch(config)# interface loopback 0
switch(config-if)# vrf member RemoteOfficeVRF
switch(config-if)# ip address 209.0.2.1/16
switch(config-if)# copy running-config startup-config

Configuring a MAC-Embedded IPv6 Address

Beginning with 7.0(3)I2(1), you can configure a MAC-embedded IPv6 (MEv6) address.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
111
 Configuring Layer 3 Interfaces
Configuring a MAC-Embedded IPv6 Address

Command or Action Purpose

Step 2 interface type slot/port Enters the interface configuration mode for the
specified interface.
Example:
switch(config)# interface ethernet 1/3
switch(config-if)#

Step 3 no switchport Configures the interface as a Layer 3 interface

and deletes any configuration specific to Layer
Example:
2 on this interface.
switch(config-if)# no switchport
Note To convert a Layer 3 interface back
into a Layer 2 interface, use the
switchport command.

Step 4 mac-address ipv6-extract Extracts the MAC address embedded in the

IPv6 address configured on the interface.
Example:
switch(config-if)# mac-address Note The MEv6 configuration is currently
ipv6-extract not supported with the EUI-64
format of IPv6 address.

Step 5 ipv6 address ip-address/length Configures an IPv6 address for this interface.
Example:
switch(config-if)# ipv6 address
2002:1::10/64

Step 6 ipv6 nd mac-extract [exclude nud-phase] Extracts the next-hop MAC address embedded
in a next-hop IPv6 address.
Example:
switch(config-if)# ipv6 nd mac-extract The exclude nud-phase option blocks packets
during the ND phase only. When the exclude
nud-phase option is not specified, packets are
blocked during both ND and neighbor
unreachability detection (NUD) phases.

Step 7 (Optional) show ipv6 icmp interface type Displays IPv6 Internet Control Message
slot/port Protocol version 6 (ICMPv6) interface
information.
Example:
switch(config-if)# show ipv6 icmp
interface ethernet 1/3

Step 8 (Optional) copy running-config startup-config Copies the running configuration to the startup
configuration.
Example:
switch(config-if)# copy running-config
startup-config

Example
This example shows how to configure a MAC-embedded IPv6 address with ND mac-extract enabled:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
112
Configuring Layer 3 Interfaces
Configuring a MAC-Embedded IPv6 Address

switch# configure terminal

switch(config)# interface ethernet 1/3
switch(config-if)# no switchport
switch(config-if)# mac-address ipv6-extract
switch(config-if)# ipv6 address 2002:1::10/64
switch(config-if)# ipv6 nd mac-extract
switch(config-if)# show ipv6 icmp interface ethernet 1/3
ICMPv6 Interfaces for VRF "default"
Ethernet1/3, Interface status: protocol-up/link-up/admin-up
IPv6 address: 2002:1::10
IPv6 subnet: 2002:1::/64
IPv6 interface DAD state: VALID
ND mac-extract : Enabled
ICMPv6 active timers:
Last Neighbor-Solicitation sent: 00:01:39
Last Neighbor-Advertisement sent: 00:01:40
Last Router-Advertisement sent: 00:01:41
Next Router-Advertisement sent in: 00:03:34
Router-Advertisement parameters:
Periodic interval: 200 to 600 seconds
Send "Managed Address Configuration" flag: false
Send "Other Stateful Configuration" flag: false
Send "Current Hop Limit" field: 64
Send "MTU" option value: 1500
Send "Router Lifetime" field: 1800 secs
Send "Reachable Time" field: 0 ms
Send "Retrans Timer" field: 0 ms
Suppress RA: Disabled
Suppress MTU in RA: Disabled
Neighbor-Solicitation parameters:
NS retransmit interval: 1000 ms
ICMPv6 error message parameters:
Send redirects: true
Send unreachables: false
ICMPv6-nd Statisitcs (sent/received):
RAs: 3/0, RSs: 0/0, NAs: 2/0, NSs: 7/0, RDs: 0/0
Interface statistics last reset: never

This example shows how to configure a MAC-embedded IPv6 address with ND mac-extract (excluding
NUD phase) enabled:
switch# configure terminal
switch(config)# interface ethernet 1/5
switch(config-if)# no switchport
switch(config-if)# mac-address ipv6-extract
switch(config-if)# ipv6 address 2002:2::10/64
switch(config-if)# ipv6 nd mac-extract exclude nud-phase
switch(config-if)# show ipv6 icmp interface ethernet 1/5
ICMPv6 Interfaces for VRF "default"
Ethernet1/5, Interface status: protocol-up/link-up/admin-up
IPv6 address: 2002:2::10
IPv6 subnet: 2002:2::/64
IPv6 interface DAD state: VALID
ND mac-extract : Enabled (Excluding NUD Phase)
ICMPv6 active timers:
Last Neighbor-Solicitation sent: 00:06:45
Last Neighbor-Advertisement sent: 00:06:46
Last Router-Advertisement sent: 00:02:18
Next Router-Advertisement sent in: 00:02:24
Router-Advertisement parameters:
Periodic interval: 200 to 600 seconds
Send "Managed Address Configuration" flag: false
Send "Other Stateful Configuration" flag: false

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
113
 Configuring Layer 3 Interfaces
Configuring a DHCP Client on an Interface

Send "Current Hop Limit" field: 64

Send "MTU" option value: 1500
Send "Router Lifetime" field: 1800 secs
Send "Reachable Time" field: 0 ms
Send "Retrans Timer" field: 0 ms
Suppress RA: Disabled
Suppress MTU in RA: Disabled
Neighbor-Solicitation parameters:
NS retransmit interval: 1000 ms
ICMPv6 error message parameters:
Send redirects: true
Send unreachables: false
ICMPv6-nd Statisitcs (sent/received):
RAs: 6/0, RSs: 0/0, NAs: 2/0, NSs: 7/0, RDs: 0/0
Interface statistics last reset: never

Configuring a DHCP Client on an Interface

You can configure the DHCP client on an SVI, a management interface, or a physical Ethernet interface for
IPv4 or IPv6 address

Procedure

Command or Action Purpose

Step 1 switch# configure terminal Enters global configuration mode.

Step 2 switch(config)# interface ethernet type Creates a physical Ethernet interface, a

slot/port | mgmt mgmt-interface-number | vlan management interface, or a VLAN interface.
vlan id
The range of vlan id is from 1 to 4094.

Step 3 switch(config-if)# [no] ipv6 address Prepares for request to the DHCP server.
use-link-local-only
Note This command is only required for
an IPv6 address.

Step 4 switch(config-if)# [no] [ip | ipv6] address dhcp Requests the DHCP server for an IPv4 or IPv6
address.
The no form of this command removes any
address that was acquired.

Step 5 (Optional) switch(config)# copy Saves the change persistently through reboots
running-config startup-config and restarts by copying the running
configuration to the startup configuration.

Example
This example shows how to configure the IP address of a DHCP client on an SVI:
switch# configure terminal
switch(config)# interface vlan 15
switch(config-if)# ip address dhcp

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
114
 Configuring Layer 3 Interfaces
Verifying the Layer 3 Interfaces Configuration

This example shows how to configure an IPv6 address of a DHCP client on a management interface:
switch# configure terminal
switch(config)# interface mgmt 0
switch(config-if)# ipv6 address use-link-local-only
switch(config-if)# ipv6 address dhcp

Verifying the Layer 3 Interfaces Configuration

To display the Layer 3 configuration, perform one of the following tasks:

Command Purpose

show interface ethernet slot/port Displays the Layer 3 interface configuration, status,
and counters (including the 5-minute exponentially
decayed moving average of inbound and outbound
packet and byte rates).

show interface ethernet slot/port brief Displays the Layer 3 interface operational status.

show interface ethernet slot/port capabilities Displays the Layer 3 interface capabilities, including
port type, speed, and duplex.

show interface ethernet slot/port description Displays the Layer 3 interface description.

show interface ethernet slot/port status Displays the Layer 3 interface administrative status,
port mode, speed, and duplex.

show interface ethernet slot/port.number Displays the subinterface configuration, status, and
counters (including the f-minute exponentially
decayed moving average of inbound and outbound
packet and byte rates).

show interface port-channel channel-id.number Displays the port-channel subinterface configuration,

status, and counters (including the 5-minute
exponentially decayed moving average of inbound
and outbound packet and byte rates).

show interface loopback number Displays the loopback interface configuration, status,
and counters.

show interface loopback number brief Displays the loopback interface operational status.

show interface loopback number description Displays the loopback interface description.

show interface loopback number status Displays the loopback interface administrative status
and protocol status.

show interface vlan number Displays the VLAN interface configuration, status,
and counters.

show interface vlan number brief Displays the VLAN interface operational status.

show interface vlan number description Displays the VLAN interface description.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
115
 Configuring Layer 3 Interfaces
Monitoring the Layer 3 Interfaces

Command Purpose

show interface vlan number status Displays the VLAN interface administrative status
and protocol status.

show ip interface brief Displays interface address and interface status

(numbered/unnumbered).

show ip route Displays routes learned via OSPF or ISIS. (Includes

addresses for best unicast and multicast next-hop.)

Monitoring the Layer 3 Interfaces

Use the following commands to display Layer 3 statistics:

Command Purpose

load- interval {interval seconds {1 | 2 | 3}} Cisco Nexus 9000 Series devices set three different
sampling intervals to bit-rate and packet-rate statistics.
The range for VLAN network interface is 60 to 300
seconds, and the range for Layer interfaces is 30 to
300 seconds.

show interface ethernet slot/port counters Displays the Layer 3 interface statistics (unicast,
multicast, and broadcast).

show interface ethernet slot/port counters brief Displays the Layer 3 interface input and output
counters.

show interface ethernet errors slot/port detailed Displays the Layer 3 interface statistics. You can
[all] optionally include all 32-bit and 64-bit packet and
byte counters (including errors).

show interface ethernet errors slot/port counters Displays the Layer 3 interface input and output errors.
errors

show interface ethernet errors slot/port counters Displays the Layer 3 interface counters reported by
snmp SNMP MIBs.

show interface ethernet slot/port.number counters Displays the subinterface statistics (unicast, multicast,
and broadcast).

show interface port-channel channel-id.number Displays the port-channel subinterface statistics

counters (unicast, multicast, and broadcast).

show interface loopback number counters Displays the loopback interface input and output
counters (unicast, multicast, and broadcast).

show interface loopback number detailed [all] Displays the loopback interface statistics. You can
optionally include all 32-bit and 64-bit packet and
byte counters (including errors).

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
116
 Configuring Layer 3 Interfaces
Configuration Examples for Layer 3 Interfaces

Command Purpose

show interface loopback number counters errors Displays the loopback interface input and output
errors.

show interface vlan number counters Displays the VLAN interface input and output
counters (unicast, multicast, and broadcast).

show interface vlan number counters detailed [all] Displays the VLAN interface statistics. You can
optionally include all Layer 3 packet and byte counters
(unicast and multicast).

show interface vlan number counters snmp Displays the VLAN interface counters reported by
SNMP MIBs.

Configuration Examples for Layer 3 Interfaces

This example shows how to configure Ethernet subinterfaces:
interface ethernet 2/1.10
description Layer 3
ip address 192.0.2.1/8

This example shows how to configure a loopback interface:

interface loopback 3
ip address 192.0.2.2/32

Example of Changing VRF Membership for an Interface

• Enable Layer 3 configuration retention when changing VRF membership.

switch# configure terminal

switch(config)# system vrf-member-change retain-l3-config

Warning: Will retain L3 configuration when vrf member change on interface.

• Verify Layer 3 retention.

switch# show running-config | include vrf-member-change

system vrf-member-change retain-l3-config

• Configure the SVI interface with Layer 3 configuration as VRF "blue".

switch# configure terminal

switch(config)# show running-config interface vlan 2002

interface Vlan2002
description TESTSVI
no shutdown
mtu 9192
vrf member blue
no ip redirects
ip address 192.168.211.2/27

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
117
 Configuring Layer 3 Interfaces
Example of Changing VRF Membership for an Interface

ipv6 address 2620:10d:c041:12::2/64

ipv6 link-local fe80::1
ip router ospf 1 area 0.0.0.0
ipv6 router ospfv3 1 area 0.0.0.0
hsrp version 2
hsrp 2002
preempt delay minimum 300 reload 600
priority 110 forwarding-threshold lower 1 upper 110
ip 192.168.211.1
hsrp 2002 ipv6
preempt delay minimum 300 reload 600
priority 110 forwarding-threshold lower 1 upper 110
ip 2620:10d:c041:12::1

• Change the SVI interface VRF to "red".

switch# configure terminal

Enter configuration commands, one per line. End with CNTL/Z.

switch(config)# interface vlan 2002
switch(config-if)# vrf member red

Warning: Retain-L3-config is on, deleted and re-added L3 config on interface Vlan2002

• Verify SVI interface after VRF change.

switch# configure terminal

switch(config)# show running-config interface vlan 2002

interface Vlan2002
description TESTSVI
no shutdown
mtu 9192
vrf member red
no ip redirects
ip address 192.168.211.2/27
ipv6 address 2620:10d:c041:12::2/64
ipv6 link-local fe80::1
ip router ospf 1 area 0.0.0.0
ipv6 router ospfv3 1 area 0.0.0.0
hsrp version 2
hsrp 2002
preempt delay minimum 300 reload 600
priority 110 forwarding-threshold lower 1 upper 110
ip 192.168.211.1
hsrp 2002 ipv6
preempt delay minimum 300 reload 600
priority 110 forwarding-threshold lower 1 upper 110
ip 2620:10d:c041:12::1

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
118
 Configuring Layer 3 Interfaces
Related Documents

Note • When changing the VRF, the Layer 3 configuration retention affects:
• Physical Interface
• Loopback Interface
• SVI Interface
• Sub-interface
• Tunnel Interface
• Port-Channel

• When changing the VRF, the existing Layer 3 configuration is deleted and reapplied. All routing protocols,
such as OSPF/ISIS/EIGRP/HSRP, go down in the old VRF and come up in the new VRF.
• Direct/Local IPv4/IPv6 addresses are removed from the old VRF and installed in the new VRF.
• Some traffic loss might occur during the VRF change.

Related Documents
Related Documents Document Title

IP Cisco Nexus 9000 Series NX-OS Unicast Routing

Configuration Guide

VLANs Cisco Nexus 9000 Series NX-OS Layer 2 Switching

Configuration Guide

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
119
 Configuring Layer 3 Interfaces
Related Documents

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
120
 CHAPTER 6
Configuring Bidirectional Forwarding Detection
• About BFD, on page 121
• Prerequisites for BFD, on page 123
• Guidelines and Limitations, on page 124
• Default Settings, on page 126
• Configuring BFD, on page 127
• Configuring BFD Support for Routing Protocols, on page 140
• Configuring BFD Interoperability, on page 150
• Verifying the BFD Configuration, on page 153
• Monitoring BFD, on page 154
• Configuration Examples for BFD, on page 154
• Related Documents, on page 155
• RFCs, on page 155

About BFD
BFD is a detection protocol designed to provide fast forwarding-path failure detection times for media types,
encapsulations, topologies, and routing protocols. You can use BFD to detect forwarding path failures at a
uniform rate, rather than the variable rates for different protocol hello mechanisms. BFD makes network
profiling and planning easier and reconvergence time consistent and predictable.
BFD provides subsecond failure detection between two adjacent devices and can be less CPU-intensive than
protocol hello messages because some of the BFD load can be distributed onto the data plane on supported
modules.

Asynchronous Mode
Cisco NX-OS supports the BFD asynchronous mode, which sends BFD control packets between two adjacent
devices to activate and maintain BFD neighbor sessions between the devices. You configure BFD on both
devices (or BFD neighbors). Once BFD has been enabled on the interfaces and on the appropriate protocols,
Cisco NX-OS creates a BFD session, negotiates BFD session parameters, and begins to send BFD control
packets to each BFD neighbor at the negotiated interval. The BFD session parameters include the following:
• Desired minimum transmit interval—The interval at which this device wants to send BFD hello messages.
• Required minimum receive interval—The minimum interval at which this device can accept BFD hello
messages from another BFD device.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
121
 Configuring Bidirectional Forwarding Detection
BFD Detection of Failures

• Detect multiplier—The number of missing BFD hello messages from another BFD device before this
local device detects a fault in the forwarding path.

The following figure shows how a BFD session is established. The figure shows a simple network with two
routers running Open Shortest Path First (OSPF) and BFD. When OSPF discovers a neighbor (1), it sends a
request to the local BFD process to initiate a BFD neighbor session with the OSPF neighbor router (2). The
BFD neighbor session with the OSPF neighbor router is now established (3).
Figure 6: Establishing a BFD Neighbor Relationship

BFD Detection of Failures

Once a BFD session has been established and timer negotiations are complete, BFD neighbors send BFD
control packets that act in the same manner as an IGP hello protocol to detect liveliness, except at a more
accelerated rate. BFD detects a failure, but the protocol must take action to bypass a failed peer.
BFD sends a failure detection notice to the BFD-enabled protocols when it detects a failure in the forwarding
path. The local device can then initiate the protocol recalculation process and reduce the overall network
convergence time.
The following figure shows what happens when a failure occurs in the network (1). The BFD neighbor session
with the OSPF neighbor router is torn down (2). BFD notifies the local OSPF process that the BFD neighbor
is no longer reachable (3). The local OSPF process tears down the OSPF neighbor relationship (4). If an
alternative path is available, the routers immediately start converging on it.

Note Note The BFD failure detection occurs in less than a second, which is much faster than OSPF Hello messages
could detect the same failure.

Figure 7: Tearing Down an OSPF Neighbor Relationship

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
122
 Configuring Bidirectional Forwarding Detection
Distributed Operation

Distributed Operation
Cisco NX-OS can distribute the BFD operation to compatible modules that support BFD. This process offloads
the CPU load for BFD packet processing to the individual modules that connect to the BFD neighbors. All
BFD session traffic occurs on the module CPU. The module informs the supervisor when a BFD failure is
detected.

BFD Echo Function

The BFD echo function sends echo packets from the forwarding engine to the remote BFD neighbor. The
BFD neighbor forwards the echo packet back along the same path in order to perform detection; the BFD
neighbor does not participate in the actual forwarding of the echo packets. The echo function and the forwarding
engine are responsible for the detection process. BFD can use the slow timer to slow down the asynchronous
session when the echo function is enabled and reduce the number of BFD control packets that are sent between
two BFD neighbors. Also, the forwarding engine tests the forwarding path on the remote (neighbor) system
without involving the remote system, so there is less interpacket delay variability and faster failure detection
times.
The echo function is without asymmetry when both BFD neighbors are running echo function.

Security
Cisco NX-OS uses the packet Time to Live (TTL) value to verify that the BFD packets came from an adjacent
BFD peer. For all asynchronous and echo request packets, the BFD neighbor sets the TTL value to 255 and
the local BFD process verifies the TTL value as 255 before processing the incoming packet. For the echo
response packet, BFD sets the TTL value to 254.
You can configure SHA-1 authentication of BFD packets.

High Availability
BFD supports stateless restarts. After a reboot or supervisor switchover, Cisco NX-OS applies the running
configuration and BFD immediately sends control packets to the BFD peers.

Virtualization Support
BFD supports virtual routing and forwarding instances (VRFs). VRFs exist within virtual device contexts
(VDCs). By default, Cisco NX-OS places you in the default VDC and default VRF.

Prerequisites for BFD

BFD has the following prerequisites:
• You must enable the BFD feature.
• Disable Internet Control Message Protocol (ICMP) redirect messages on BFD-enabled interfaces.
• Disable the IP packet verification check for identical IP source and destination addresses.
• See other detailed prerequisites that are listed with the configuration tasks.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
123
 Configuring Bidirectional Forwarding Detection
Guidelines and Limitations

Guidelines and Limitations

BFD has the following configuration guidelines and limitations:
• Beginning with Cisco NX-OS Release 7.0(3)I7(3), the QSFP 40/100-G BiDi comes up in the highest
possible speed available on the port. For example, in the Cisco Nexus 93180LC-EX switch it comes up
as 40 G in the first 28 ports and 100 G in the last 4 ports. If you need to connect to 40-G SR4 BiDi, the
speed on the 40/100-G BiDi needs to be set to 40 G.
• BFD is not supported on unnumbered interfaces. Beginning with Cisco NX-OS Release 7.0(3)I7(6), the
BFD process is prevented from running on unnumbered interfaces.
• Beginning with Cisco NX-OS Release 7.0(3)I7(5), the QSFP-40/100-SRBD comes up in the speed of
100-G on Cisco Nexus N9K-C93180YC-FX and N9K-C93180TC-FX switches.
• The dual rate BiDi does not support auto speed sensing. With speed auto, the QSFP always defaults to
100 G rather than the speed of the link partner.
This feature is supported on the following Cisco Nexus switches:
• 9336C-FX
• 9364C
• 93180YC-EX
• 93180YC-FX
• 9732C-EX
• 9732C-FX
• 9736C-FX

• show commands with the internal keyword are not supported.

• Starting with Release 7.0(3)I5(1), BFD per-member link support is added on Cisco Nexus 9000 Series
switches.
• BFD supports BFD version 1.
• BFD supports IPv4 and IPv6.
• BFD supports OSPFv3.
• BFD supports IS-ISv6.
• BFD supports BGPv6.
• BFD supports EIGRPv6.
• BFD supports only one session per address family, per interface.
• BFD supports single-hop BFD.
• BFD for BGP supports single-hop EBGP and iBGP peers.
• BFD supports keyed SHA-1 authentication.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
124
Configuring Bidirectional Forwarding Detection
Guidelines and Limitations

• BFD supports the following Layer 3 interfaces—physical interfaces, port channels, subinterfaces, and
VLAN interfaces.
• BFD depends on a Layer 3 adjacency information to discover topology changes, including Layer 2
topology changes. A BFD session on a VLAN interface (SVI) may not be up after the convergence of
the Layer 2 topology if there is no Layer 3 adjacency information available.
• For BFD on a static route between two devices, both devices must support BFD. If one or both of the
devices do not support BFD, the static routes are not programmed in the Routing Information Base (RIB).
• Port channel configuration limitations:
• For Layer 3 port channels used by BFD, you must enable LACP on the port channel.
• For Layer 2 port channels used by SVI sessions, you must enable LACP on the port channel.

• SVI limitations:
• An ASIC reset causes traffic disruption for other ports and it can cause the SVI sessions on the other
ports to flap. For example, if the carrier interface is a virtual port channel (vPC), BFD is not supported
over the SVI interface and it could cause a trigger for an ASIC reset. When a BFD session is over
SVI using virtual port channel (vPC) peer-link, the BFD echo function is not supported. You must
disable the BFD echo function for all sessions over SVI between vPC peer nodes.
An SVI on the Cisco Nexus series switches should not be configured to establish a BFD neighbor
adjacency with a device connected to it via a vPC. This is because the BFD keepalives from the
neighbour, if sent over the vPC member link connected to the vPC peer-switch, do not reach this
SVI causing the BFD adjacency to fail.
• When you change the topology (for example, add or delete a link into a VLAN, delete a member
from a Layer 2 port channel, and so on), the SVI session could be affected. It may go down first
and then come up after the topology discovery is finished.
• BFD over FEX HIF interfaces is not supported.
• When a BFD session is over SVI using virtual port-channel (vPC) peer-link (either BCM or GEM
based ports), the BFD echo function is not supported. You must disable the BFD echo function for
all sessions over SVI between vPC peer nodes using the no bfd echo command at the SVI
configuration level.

Tip If you do not want the SVI sessions to flap and you need to change the topology,
you can disable the BFD feature before making the changes and reenable BFD
after the changes have been made. You can also configure the BFD timer to be
a large value (for example, 5 seconds), and change it back to a fast timer after
the above events complete.

• When you configure the BFD Echo function on the distributed Layer 3 port channels, reloading a member
module flaps the BFD session hosted on that module, which results in a packet loss.
If you connect the BFD peers directly without a Layer 2 switch in between, you can use the BFD per-link
mode as an alternative solution.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
125
 Configuring Bidirectional Forwarding Detection
Default Settings

Note Using BFD per-link mode and subinterface optimization simultaneously on a

Layer 3 port channel is not supported.

• When you specify a BFD neighbor prefix in the clear {ip | ipv6} route prefix command, the BFD echo
session will flap.
• The clear {ip | ipv6} route * command causes BFD echo sessions to flap.
• HSRP for IPv4 is supported with BFD.
• BFD packets generated by the Cisco NX-OS device linecards are sent with COS 6/DSCP CS6. The
DSCP/COS values for BFD packets are not user configurable.
• When configuring BFDv6 in no-bfd-echo mode, it is recommended to run with timers of 150 ms with a
multiplier of 3.
• BFDv6 is not supported for VRRPv3 and HSRP for v6.
• IPv6 eigrp bfd cannot be disabled on an interface.
• Port channel configuration notes:
• When the BFD per-link mode is configured, the BFD echo function is not supported. You must
disable the BFD echo function using the no bfd echo command before configuring the bfd per-link
command.
• Configuring BFD per-link with link-local is not supported.

Default Settings
The following table lists the default settings for BFD parameters.

Table 9: Default BFD Parameters

Parameters Default

BFD feature Disabled

Required minimum receive interval 50 milliseconds

Desired minimum transmit interval 50 milliseconds

Detect multiplier 3

Echo function Enabled

Mode Asynchronous

Port-channel Logical mode (one session per source-destination pair

address)

Slow timer 2000 milliseconds

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
126
 Configuring Bidirectional Forwarding Detection
Configuring BFD

Parameters Default

Startup timer (7.0(3)I2(1) and later) 5 seconds

Configuring BFD
Configuration Hierarchy
You can configure BFD at the global level and at the interface level. The interface configuration overrides
the global configuration.
For physical ports that are members of a port channel, the member port inherits the primary port channel BFD
configuration.

Task Flow for Configuring BFD

Follow these steps in the following sections to configure BFD:
• Enabling the BFD Feature.
• Configuring Global BFD Parameters or Configuring BFD on an Interface.

Enabling the BFD Feature

You must enable the BFD feature before you can configure BFD on an interface and protocol.

Note Use the no feature bfd command to disable the BFD feature and remove all associated configuration.

Command Purpose

no feature bfd Disables the BFD feature and

removes all associated
Example:
configuration.
switch(config)# no feature bfd

Procedure

Command or Action Purpose

Step 1 configure terminal Enters configuration mode.
Example:
switch# configure terminal
switch(config)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
127
 Configuring Bidirectional Forwarding Detection
Configuring Global BFD Parameters

Command or Action Purpose

Step 2 feature bfd Enables the BFD feature.
Example:
switch(config)# feature bfd

Step 3 show feature | include bfd (Optional) Displays enabled and disabled
features.
Example:
switch(config)# show feature | include
bfd

Step 4 copy running-config startup-config (Optional) Saves the configuration change.

Example:
switch(config)# copy running-config
startup-config

Configuring Global BFD Parameters

You can configure the BFD session parameters for all BFD sessions on the device. The BFD session parameters
are negotiated between the BFD peers in a three-way handshake.
See the Configuring BFD on an Interface section to override these global session parameters on an interface.

Before you begin

Enable the BFD feature.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 bfd interval mintx min_rx msec multiplier Configures the BFD session parameters for all
value BFD sessions on the device. This command
overrides these values by configuring the BFD
Example:
session parameters on an interface. The mintx
switch(config)# bfd interval 50 min_rx and msec range is from 50 to 999 milliseconds
50 multiplier 3
and the default is 50. The multiplier range is
from 1 to 50. The multiplier default is 3.

Step 3 bfd slow-timer [interval] Configures the slow timer used in the echo
function. This value determines how fast BFD
Example:
starts up a new session and at what speed the
switch(config)# bfd slow-timer 2000 asynchronous sessions use for BFD control
packets when the echo function is enabled. The
slow-timer value is used as the new control

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
128
 Configuring Bidirectional Forwarding Detection
Configuring BFD on an Interface

Command or Action Purpose

packet interval, while the echo packets use the
configured BFD intervals. The echo packets are
used for link failure detection, while the control
packets at the slower rate maintain the BFD
session. The range is from 1000 to 30000
milliseconds. The default is 2000.

Step 4 [no] bfd startup-timer [seconds] Configures the BFD startup timer, which delays
the startup time for BFD sessions in order to
Example:
give the routes that are being used by local and
switch(config)# bfd startup-timer 20 remote routers time to settle down in the
hardware. Using this feature can prevent BFD
flaps in higher scale scenarios. The range is
from 0 to 30 seconds. The default is 5 seconds.
The bfd startup-timer 0 command disables the
BFD startup timer.
The no bfd startup-timer command sets the
BFD startup timer to 5 seconds (the default
value).
Important bfd startup-timer command
applicable for 7.0(3)I2(1) and later.

Step 5 bfd echo-interface loopback interface number Configures the interface used for Bidirectional
Forwarding Detection (BFD) echo frames. This
Example:
command changes the source address for the
switch(config)# bfd echo-interface echo packets to the one configured on the
loopback 1 3
specified loopback interface. The interface
number range is from 0 to 1023.

Step 6 show running-config bfd (Optional) Displays the BFD running

configuration.
Example:
switch(config)# show running-config bfd

Step 7 copy running-config startup-config (Optional) Saves the configuration change.

Example:
switch(config)# copy running-config
startup-config

Configuring BFD on an Interface

You can configure the BFD session parameters for all BFD sessions on an interface. The BFD session
parameters are negotiated between the BFD peers in a three-way handshake.
This configuration overrides the global session parameters for the configured interface.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
129
 Configuring Bidirectional Forwarding Detection
Configuring BFD on an Interface

Before you begin

Ensure that Internet Control Message Protocol (ICMP) redirect messages are disabled on BFD-enabled
interfaces. Use the no ip redirects command or the no ipv6 redirects command on the interface.
Enable the BFD feature. See the Enabling the BFD Feature section.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface int-if Enters interface configuration mode. Use the ?

keyword to display the supported interfaces.
Example:
switch(config)# interface ethernet 2/1
switch(config-if)#

Step 3 bfd interval mintx min_rx msec multiplier Configures the BFD session parameters for all
value BFD sessions on the device. This command
overrides these values by configuring the BFD
Example:
session parameters on an interface. The mintx
switch(config-if)# bfd interval 50 and msec range is from 50 to 999 milliseconds
min_rx 50 multiplier 3
and the default is 50. The multiplier range is
from 1 to 50. The multiplier default is 3.

Step 4 bfd authentication keyed-sha1 keyid id key (Optional) Configures SHA-1 authentication
ascii_key for all BFD sessions on the interface. The
ascii_key string is a secret key shared among
Example:
BFD peers. The id value, a number between 0
switch(config-if)# bfd authentication and 255, is assigned to this particular ascii_key.
keyed-sha1 keyid 1 ascii_key cisco123
BFD packets specify the key by id, allowing
the use of multiple active keys.
To disable SHA-1 authentication on the
interface, use the no form of the command.

Step 5 show running-config bfd (Optional) Displays the BFD running

configuration.
Example:
switch(config-if)# show running-config
bfd

Step 6 copy running-config startup-config (Optional) Saves the configuration change.

Example:
switch(config-if)# copy running-config
startup-config

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
130
 Configuring Bidirectional Forwarding Detection
Configuring BFD on a Port Channel

Configuring BFD on a Port Channel

You can configure the BFD session parameters for all BFD sessions on a port channel. If per-link mode is
used for Layer 3 port channels, BFD creates a session for each link in the port channel and provides an
aggregate result to client protocols. For example, if the BFD session for one link on a port channel is up, BFD
informs client protocols, such as OSPF, that the port channel is up. The BFD session parameters are negotiated
between the BFD peers in a three-way handshake.
This configuration overrides the global session parameters for the configured port channel. The member ports
of the port channel inherit the port channel BFD session parameters.

Before you begin

Ensure that you enable LACP on the port channel before you enable BFD.
Ensure that Internet Control Message Protocol (ICMP) redirect messages are disabled on BFD-enabled
interfaces. Use the no ip redirects command on the interface.
Enable the BFD feature. See the Enabling the BFD Feature section.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface port-channel number Enters port-channel configuration mode. Use

the ? keyword to display the supported number
Example:
range.
switch(config)# interface port-channel
2
switch(config-if)#

Step 3 bfd per-link Configures the BFD sessions for each link in
the port channel.
Example:
switch(config-if)# bfd per-link

Step 4 bfd interval mintx min_rx msec multiplier (Optional) Configures the BFD session
value parameters for all BFD sessions on the port
channel. This command overrides these values
Example:
by configuring the BFD session parameters.
switch(config-if)# bfd interval 50 The mintx and msec range is from 50 to 999
min_rx 50 multiplier 3
milliseconds and the default is 50. The
multiplier range is from 1 to 50. The multiplier
default is 3.

Step 5 bfd authentication keyed-sha1 keyid id key (Optional) Configures SHA-1 authentication
ascii_key for all BFD sessions on the interface. The
ascii_key string is a secret key shared among
Example:
BFD peers. The id value, a number between 0

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
131
 Configuring Bidirectional Forwarding Detection
Configuring the BFD Echo Function

Command or Action Purpose

switch(config-if)# bfd authentication and 255, is assigned to this particular ascii_key.
keyed-sha1 keyid 1 ascii_key cisco123 BFD packets specify the key by id, allowing
the use of multiple active keys.
To disable SHA-1 authentication on the
interface, use the no form of the command.

Step 6 show running-config bfd (Optional) Displays the BFD running

configuration.
Example:
switch(config-if)# show running-config
bfd

Step 7 copy running-config startup-config (Optional) Saves the configuration change.

Example:
switch(config-if)# copy running-config
startup-config

Configuring the BFD Echo Function

You can configure the BFD echo function on one or both ends of a BFD-monitored link. The echo function
slows down the required minimum receive interval, based on the configured slow timer. The
RequiredMinEchoRx BFD session parameter is not set to zero if the echo function is disabled in compliance
with RFC 5880. The slow timer becomes the required minimum receive interval if the echo function is enabled.

Before you begin

Enable the BFD feature. See the Enabling the BFD Feature section.
Configure the BFD session parameters. See the Configuring Global BFD Parameters section on or the
Configuring BFD on an Interface section.
Ensure that Internet Control Message Protocol (ICMP) redirect messages are disabled on BFD-enabled
interfaces. Use the no ip redirects command on the interface.
Ensure that the IP packet verification check for identical IP source and destination addresses is disabled. Use
the no hardware ip verify address identical command. See the Cisco Nexus 9000 Series NX-OS Unicast
Routing Configuration Guide for more information about this command.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 bfd slow-timer echo-interval Configures the slow timer used in the echo
function. This value determines how fast BFD
Example:
starts up a new session and is used to slow down
switch(config)# bfd slow-timer 2000

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
132
 Configuring Bidirectional Forwarding Detection
Configuring Per-Member Link BFD Sessions

Command or Action Purpose

the asynchronous sessions when the BFD echo
function is enabled. This value overwrites the
required minimum receive interval when the
echo function is enabled. The range is from
1000 to 30000 milliseconds. The default is
2000.

Step 3 interface int-if Enters interface configuration mode. Use the ?

keyword to display the supported interfaces.
Example:
switch(config)# interface ethernet 2/1
switch(config-if)#

Step 4 bfd echo Enables the echo function. The default is

enabled.
Example:
switch(config-if)# bfd echo

Step 5 show running-config bfd (Optional) Displays the BFD running

configuration.
Example:
switch(config-if)# show running-config
bfd

Step 6 copy running-config startup-config (Optional) Saves the configuration change.

Example:
switch(config-if)# copy running-config
startup-config

Configuring Per-Member Link BFD Sessions

Starting with Release 7.0(3)I5(1), BFD per-member link support is added on Cisco Nexus 9000 Series switches.
See the following sections for more information.

BFD Enhancement to Address Per-link Efficiency

The Bidirectional Forwarding (BFD) enhancement to address per-link efficiency, called as IETF Micro BFD,
lets you configure the individual BFD sessions on every Link Aggregation Group (LAG) member interfaces
(as defined in RFC 7130).
With this enhancement, the BFD sessions run on each member link of the port-channel. If BFD detects a link
failure, the member link is removed from the forwarding table. This mechanism delivers faster failure detection
as the BFD sessions are created on an individual port-channel interface.
The BFD sessions running on member links of the port-channel are called as Micro BFD sessions. You can
configure RFC 7130 BFD over main port-channel interface, that performs bandwidth monitoring over LAG
by having one Micro BFD session over each member. If any of the member port goes down, the port is removed
from the forwarding table and this prevents traffic disruption on that member.
Micro BFD sessions are supported for both LACP and non-LACP based-port channels. For more information
on how to configure Micro BFD sessions, see Configuring Micro BFD Sessions.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
133
 Configuring Bidirectional Forwarding Detection
Limitations of the IETF Bidirectional Forwarding Detection

Limitations of the IETF Bidirectional Forwarding Detection

See the following limitations of the IETF Bidirectional Forwarding Detection:
• BFD Limitations
• It cannot co-exist with BFD over logical port-channels or proprietary BFD per-member links. BFD
IPv6 logical/proprietary per-link session is also not supported when BFD IETF IPv4 is configured
on PC.
• IETF BFD IPv6 is not supported.
• Echo functionality is not supported for Micro-BFD sessions.
• Port-channel interfaces should be directly connected between two switches that are running the BFD
sessions. No intermediate Layer 2 switches are expected.

• EthPCM/LACP Limitations
• If a LACP port-channel has members in hot-standby state, BFD failure in one of the active links
may not cause the hot-standby link to come up directly. Once the active link with BFD failure goes
down, the hot-standby member becomes active. However, it may not be able to prevent the
port-channel from going down before the hot-standby link comes up, in cases where port-channel
min-link condition is hit.

• General Limitations:
• It is supported only on Layer 3 port-channels.
• It is not supported on the following:
• vPC
• Layer 3 sub-interfaces
• Layer 2 port-channels/Layer 2 Fabric Path
• FPC/HIF PC
• Layer 3 sub-interfaces
• VSI over port-channels

Guidelines for Migration/Configuration of IETF Per-Member Sessions:

See the following guidelines for migration/configuration of IETF per-member sessions:
• The logical BFD sessions that are created using the routing protocols over port-channel sub-interfaces
(where RFC 7130 cannot run) are still supported. The main port-channel interface however does not
support both logical and RFC 7130 sessions that co-exist. It can support only either of them.
• You can configure RFC 7130 BFD over the main port-channel interface that perform bandwidth monitoring
over the LAG by having one Micro-BFD session over each member. If any of the member port goes
down, BFD notifies it to the port-channel manager that removes the port from the LTL, thereby preventing
blackholing of the traffic on that member.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
134
 Configuring Bidirectional Forwarding Detection
Configuring Port Channel Interface

• If the minimum number of links required to have the port-channel operationally up is not met in the
above case, the port-channel is brought down by the port-channel manager. This in turn brings down the
port-channel sub-interfaces if they are configured and thereby the logical BFD session also comes down
notifying the routing protocol.
• When you are using RFC 7130 on the main port-channel and logical BFD on the sub-interfaces, the
logical BFD session should be run with lesser aggressive timers than the RFC 7130 BFD session. You
can have RFC 7130 configured on the port-channel interface or you can have it configured in conjunction
with the logical BFD sessions on the port-channel sub-interfaces.
• When a proprietary per-link is configured, enabling IETF Micro-BFD sessions is not allowed on a port
channel and vice-versa. You have to remove the proprietary per-link configuration. Current implementation
of proprietary per-link does not allow changing the configuration (no per-link), if there is any BFD session
that is bootstrapped by the applications. You need to remove the BFD tracking on the respective
applications and remove per-link configuration. The migration path from the proprietary per-link to IETF
Micro-BFD is as follows:
• Remove the BFD configuration on the applications.
• Remove the per-link configuration.
• Enable the IETF Micro-BFD command.
• Enable BFD on the applications.

The same migration path can be followed for proprietary BFD to IETF Micro-BFD on the main
port-channel interface.

Configuring Port Channel Interface

Before you begin

Ensure that the BFD feature is enabled.

Procedure

Step 1 switch(config)# interface port-channel port-number

Configures interface port-channel.

Step 2 switch(config)# no switchport

Configures interface as Layer 3 port-channel.

What to do next
• Configuring BFD Start Timer
• Enabling IETF Per-link BFD

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
135
 Configuring Bidirectional Forwarding Detection
Configuring BFD Start Timer

Configuring BFD Start Timer

Complete the following steps to configure the BFD start timer:

Procedure

switch(config)# interface port-channelport-number

Configures the BFD start timer for a port-channel.
Note The default value is infinite (that is no timer is running). For start timer to work, configure start
timer value before completing the port-channel BFD configurations (for example, before the
port-channel bfd track-member-link and port-channel bfd destination are configured for Layer 3
port-channel interface with the active members).

What to do next
• Enabling IETF Per-link BFD
• Configuring BFD Destination IP Address

Enabling IETF Per-link BFD

Procedure

switch(config-if)# port-channel bfd track-member-link

Enables IETF BFD on port-channel interface.

What to do next
• Configuring BFD Destination IP Address
• Verifying Micro BFD Session Configurations

Configuring BFD Destination IP Address

Complete the following steps to configure the BFD destination IP address:

Procedure

switch(config-if)# port-channel bfd destinationip-address

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
136
 Configuring Bidirectional Forwarding Detection
Verifying Micro BFD Session Configurations

Configures an IPv4 address to be used for the BFD sessions on the member links.

What to do next
• Verifying Micro BFD Sessions Configuration

Verifying Micro BFD Session Configurations

Use the following commands to verify the Micro BFD session configurations.

Procedure

Step 1 Displays the port-channel and port-channel member operational state.

switch# show port-channel summary

Step 2 switch# show bfd neighbors

Displays Micro BFD sessions on port-channel members.

Step 3 switch# show bfd neighbors details

Displays BFD session for a port channel interface and the associated Micro BFD sessions on members.

Step 4 switch# show tech-support bfd

Displays the technical support information for BFD.

Step 5 switch# show tech-support lacp all

Displays the technical support information for Ethernet Port Manager, Ethernet Port-channel Manager, and
LACP.

Step 6 switch# show running-config interface port-channel port-channel-number

Displays the running configuration information of the port-channel interface.

Examples: Configuring Micro BFD Sessions

See the following examples for configuring Micro BFD sessions.

Configuring Micro BFD Sessions

In this example, the following topology is used.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
137
 Configuring Bidirectional Forwarding Detection
Examples: Configuring Micro BFD Sessions

Figure 8: Configuring Micro BFD Session

The sample configuration of switch 1 is as follows:

feature bfd
configure terminal
interface port-channel 10
port-channel bfd track-member-link
port-channel bfd destination 10.1.1.2
port-channel bfd start 60
ip address 10.1.1.1/24

The sample configuration of switch 2 is as follows:

feature bfd
configure terminal
interface port-channel 10
port-channel bfd track-member-link
port-channel bfd destination 10.1.1.1
port-channel bfd start 60
ip address 10.1.1.2/24

Verifying Micro BFD Sessions Configuration

The following example displays the show output of the show running-config interface
port-channel<port-channel>, show port-channel summary, show bfd neighbors vrf
internet_routes, and show bfd neighbors interface port-channel <port-channel> vrf
internet_routes details commands.
switch# show running-config interface port-channel 1001

!Command: show running-config interface port-channel1001

!Time: Fri Oct 21 09:08:00 2016

version 7.0(3)I5(1)

interface port-channel1001
no switchport
vrf member internet_routes
port-channel bfd track-member-link
port-channel bfd destination 40.4.1.2
ip address 40.4.1.1/24
ipv6 address 2001:40:4:1::1/64

switch# show por

port-channel port-profile
switch# show port-channel summary

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
138
Configuring Bidirectional Forwarding Detection
Examples: Configuring Micro BFD Sessions

Flags: D - Down P - Up in port-channel (members)

I - Individual H - Hot-standby (LACP only)
s - Suspended r - Module-removed
b - BFD Session Wait
S - Switched R - Routed
U - Up (port-channel)
p - Up in delay-lacp mode (member)
M - Not in use. Min-links not met
--------------------------------------------------------------------------------
Group Port- Type Protocol Member Ports
Channel
--------------------------------------------------------------------------------
1001 Po1001(RU) Eth LACP Eth1/11/1(P) Eth1/11/2(P) Eth1/12/1(P)
Eth1/12/2(P)
switch# show bfd neighbors vrf internet_routes

OurAddr NeighAddr LD/RD RH/RS Holdown(mult)

State Int Vrf
40.4.1.1 40.4.1.2 1090519041/0 Up N/A(3) Up
Po1001 internet_routes
40.4.1.1 40.4.1.2 1090519042/1090519051 Up 819(3) Up
Eth1/12/1 internet_routes
40.4.1.1 40.4.1.2 1090519043/1090519052 Up 819(3) Up
Eth1/12/2 internet_routes
40.4.1.1 40.4.1.2 1090519044/1090519053 Up 819(3) Up
Eth1/11/1 internet_routes
40.4.1.1 40.4.1.2 1090519045/1090519054 Up 819(3) Up
Eth1/11/2 internet_routes
switch#

switch# show bfd neighbors interface port-channel 1001 vrf internet_routes details

OurAddr NeighAddr LD/RD RH/RS Holdown(mult)

State Int Vrf
40.4.1.1 40.4.1.2 1090519041/0 Up N/A(3) Up
Po1001 internet_routes

Session state is Up
Local Diag: 0
Registered protocols: eth_port_channel
Uptime: 1 days 11 hrs 4 mins 8 secs
Hosting LC: 0, Down reason: None, Reason not-hosted: None
Parent session, please check port channel config for member info
switch#

switch# show bfd neighbors interface ethernet 1/12/1 vrf internet_routes details

OurAddr NeighAddr LD/RD RH/RS Holdown(mult)

State Int Vrf
40.4.1.1 40.4.1.2 1090519042/1090519051 Up 604(3) Up
Eth1/12/1 internet_routes

Session state is Up and not using echo function

Local Diag: 0, Demand mode: 0, Poll bit: 0, Authentication: None
MinTxInt: 100000 us, MinRxInt: 100000 us, Multiplier: 3
Received MinRxInt: 300000 us, Received Multiplier: 3
Holdown (hits): 900 ms (0), Hello (hits): 300 ms (458317)
Rx Count: 427188, Rx Interval (ms) min/max/avg: 19/1801/295 last: 295 ms ago
Tx Count: 458317, Tx Interval (ms) min/max/avg: 275/275/275 last: 64 ms ago
Registered protocols: eth_port_channel
Uptime: 1 days 11 hrs 4 mins 24 secs
Last packet: Version: 1 - Diagnostic: 0
State bit: Up - Demand bit: 0
Poll bit: 0 - Final bit: 0

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
139
 Configuring Bidirectional Forwarding Detection
Configuring BFD Support for Routing Protocols

Multiplier: 3 - Length: 24
My Discr.: 1090519051 - Your Discr.: 1090519042
Min tx interval: 300000 - Min rx interval: 300000
Min Echo interval: 300000 - Authentication bit: 0
Hosting LC: 1, Down reason: None, Reason not-hosted: None
Member session under parent interface Po1001

switch# show bfd neighbors interface ethernet 1/12/2 vrf internet_routes details

OurAddr NeighAddr LD/RD RH/RS Holdown(mult)

State Int Vrf
40.4.1.1 40.4.1.2 1090519043/1090519052 Up 799(3) Up
Eth1/12/2 internet_routes

Session state is Up and not using echo function

Local Diag: 0, Demand mode: 0, Poll bit: 0, Authentication: None
MinTxInt: 100000 us, MinRxInt: 100000 us, Multiplier: 3
Received MinRxInt: 300000 us, Received Multiplier: 3
Holdown (hits): 900 ms (0), Hello (hits): 300 ms (458336)
Rx Count: 427207, Rx Interval (ms) min/max/avg: 19/1668/295 last: 100 ms ago
Tx Count: 458336, Tx Interval (ms) min/max/avg: 275/275/275 last: 251 ms ago
Registered protocols: eth_port_channel
Uptime: 1 days 11 hrs 4 mins 30 secs
Last packet: Version: 1 - Diagnostic: 0
State bit: Up - Demand bit: 0
Poll bit: 0 - Final bit: 0
Multiplier: 3 - Length: 24
My Discr.: 1090519052 - Your Discr.: 1090519043
Min tx interval: 300000 - Min rx interval: 300000
Min Echo interval: 300000 - Authentication bit: 0
Hosting LC: 1, Down reason: None, Reason not-hosted: None
Member session under parent interface Po1001
switch#

Configuring BFD Support for Routing Protocols

Configuring BFD on BGP
You can configure BFD for the Border Gateway Protocol (BGP).

Before you begin

Enable the BFD feature. See the Enabling the BFD Feature section.
Configure the BFD session parameters. See the Configuring Global BFD Parameters section or the Configuring
BFD on an Interface section.
Enable the BGP feature. See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide for
more information.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
140
 Configuring Bidirectional Forwarding Detection
Configuring BFD on EIGRP

Procedure

Command or Action Purpose

Step 1 configure terminal Enters configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 router bgp as-number Enables BGP and assigns the AS number to the
local BGP speaker. The AS number can be a
Example:
16-bit integer or a 32-bit integer in the form of
switch(config)# router bgp 64496 a higher 16-bit decimal number and a lower
switch(config-router)#
16-bit decimal number in xx.xx format.

Step 3 neighbor (ip-address | ipv6-address) remote-as Configures the IPv4 or IPv6 address and AS
as-number number for a remote BGP peer. The ip-address
format is x.x.x.x. The ipv6-address format is
Example:
A:B::C:D.
switch(config-router)# neighbor
209.165.201.1 remote-as 64497
switch(config-router-neighbor)#

Step 4 update-source interface Allows BGP sessions to use the primary IP

address from a particular interface as the local
Example:
address when forming a BGP session with a
switch(config-router-neighbor)# neighbor and enables BGP to register as a client
update-source ethernet 2/1
with BFD.

Step 5 show running-config bgp (Optional) Displays the BGP running

configuration.
Example:
switch(config-router-neighbor)# show
running-config bgp

Step 6 copy running-config startup-config (Optional) Saves the configuration change.

Example:
switch(config-router-neighbor)# copy
running-config startup-config

Configuring BFD on EIGRP

You can configure BFD for the Enhanced Interior Gateway Routing Protocol (EIGRP).

Before you begin

Enable the BFD feature. See the Enabling the BFD Feature section.
Configure the BFD session parameters. See the Configuring Global BFD Parameters section or the Configuring
BFD on an Interface section.
Enable the EIGRP feature. See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide
for more information.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
141
 Configuring Bidirectional Forwarding Detection
Configuring BFD on OSPF

Procedure

Command or Action Purpose

Step 1 configure terminal Enters configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 router eigrp instance-tag Creates a new EIGRP process with the
configured instance tag. The instance tag can
Example:
be any case-sensitive, alphanumeric string up
switch(config)# router eigrp Test1 to 20 characters.
switch(config-router)#
If you configure an instance-tag that does not
qualify as an AS number, you must use the
autonomous-system to configure the AS
number explicitly or this EIGRP instance will
remain in the shutdown state.

Step 3 bfd [ipv4 | ipv6] (Optional) Enables BFD for all EIGRP
interfaces.
Example:
switch(config-router-neighbor)# bfd ipv4

Step 4 interface int-if Enters interface configuration mode. Use the ?

keyword to display the supported interfaces.
Example:
switch(config-router-neighbor)# interface
ethernet 2/1
switch(config-if)#

Step 5 ip eigrp instance-tag bfd (Optional) Enables or disables BFD on an

EIGRP interface. The instance tag can be any
Example:
case-sensitive, alphanumeric string up to 20
switch(config-if)# ip eigrp Test1 bfd characters.
The default is disabled.

Step 6 show ip eigrp [vrf vrf-name] [ interfaces if] (Optional) Displays information about EIGRP.
The vrf-name can be any case-sensitive,
Example:
alphanumeric string up to 32 characters.
switch(config-if)# show ip eigrp

Step 7 copy running-config startup-config (Optional) Saves the configuration change.

Example:
switch(config-if)# copy
running-config startup-config

Configuring BFD on OSPF

You can configure BFD for the Open Shortest Path First.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
142
Configuring Bidirectional Forwarding Detection
Configuring BFD on OSPF

Before you begin

Enable the BFD feature. See the Enabling the BFD Feature section.
Configure the BFD session parameters. See the Configuring Global BFD Parameters section or the Configuring
BFD on an Interface section.
Enable the OSPF feature. See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide for
more information.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 router ospf instance-tag Creates a new OSPF instance with the
configured instance tag. The instance tag can
Example:
be any case-sensitive, alphanumeric string up
switch(config)# router ospf 200 to 20 characters.
switch(config-router)#

Step 3 bfd [ipv4 | ipv6] (Optional) Enables BFD for all OSPF interfaces.
Example:
switch(config-router)# bfd

Step 4 interface int-if Enters interface configuration mode. Use the ?

keyword to display the supported interfaces.
Example:
switch(config-router)# interface
ethernet 2/1
switch(config-if)#

Step 5 ip ospf bfd (Optional) Enables or disables BFD on an OSPF

interface. The default is disabled.
Example:
switch(config-if)# ip ospf bfd

Step 6 show ip ospf [vrf vrf-name] [ interfaces if] (Optional) Displays information about OSPF.
The vrf-name can be any case-sensitive,
Example:
alphanumeric string up to 32 characters.
switch(config-if)# show ip ospf

Step 7 copy running-config startup-config (Optional) Saves the configuration change.

Example:
switch(config-if)# copy
running-config startup-config

Example Configurations for BFD on OSPF

Example configuration where BFD is enabled under a non-default VRF (OSPFv3 neighbors in vrf3).

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
143
 Configuring Bidirectional Forwarding Detection
Configuring BFD on IS-IS

configure terminal
router ospfv3 10
vrf vrf3
bfd

Configuring BFD on IS-IS

You can configure BFD for the Intermediate System-to-Intermediate System (IS-IS) protocol.

Before you begin

Enable the BFD feature. See the Enabling the BFD Feature section.
Configure the BFD session parameters. See the Configuring Global BFD Parameters section or the Configuring
BFD on an Interface section.
Enable the IS-IS feature. See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide for
more information.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 router isis instance-tag Creates a new IS-IS instance with the
configured instance tag.
Example:

switch(config)# router isis 100

switch(config-router)# net
49.0001.1720.1600.1001.00
switch(config-router)# address-family
ipv6 unicast

Step 3 bfd [ipv4 | ipv6] (Optional) Enables BFD for all OSPF interfaces.
Example:
switch(config-router)# bfd

Step 4 interface int-if Enters interface configuration mode. Use the ?

keyword to display the supported interfaces.
Example:
switch(config-router)# interface
ethernet 2/1
switch(config-if)#

Step 5 isis bfd (Optional) Enables or disables BFD on an IS-IS

interface. The default is disabled.
Example:
switch(config-if)# isis bfd

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
144
 Configuring Bidirectional Forwarding Detection
Configuring BFD on HSRP

Command or Action Purpose

Step 6 show isis [vrf vrf-name] [ interface if] (Optional) Displays information about IS-IS.
The vrf-name can be any case-sensitive,
Example:
alphanumeric string up to 32 characters.
switch(config-if)# show isis

Step 7 copy running-config startup-config (Optional) Saves the configuration change.

Example:
switch(config-if)# copy
running-config startup-config

Example Configurations for BFD on IS-IS

Example configuration for IS-IS where BFD is enabled under IPv4 and an IPv6 address family.
configure terminal
router isis isis-1
bfd
address-family ipv6 unicast
bfd

Configuring BFD on HSRP

You can configure BFD for the Hot Standby Router Protocol (HSRP). The active and standby HSRP routers
track each other through BFD. If BFD on the standby HSRP router detects that the active HSRP router is
down, the standby HSRP router treats this event as an active time rexpiry and takes over as the active HSRP
router.
The show hsrp detail command shows this event as BFD@Act-down or BFD@Sby-down.

Before you begin

Enable the BFD feature. See the Enabling the BFD Feature section.
Configure the BFD session parameters. See the Configuring Global BFD Parameters section or the Configuring
BFD on an Interface section.
Enable the HSRP feature. See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide for
more information.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
145
 Configuring Bidirectional Forwarding Detection
Configuring BFD on VRRP

Command or Action Purpose

Step 2 hsrp bfd all-interfaces (Optional) Enables or disables BFD on all
HSRP interfaces. The default is disabled.
Example:
switch# hsrp bfd all-interfaces

Step 3 interface int-if Enters interface configuration mode. Use the ?

keyword to display the supported interfaces.
Example:
switch(config-router)# interface
ethernet 2/1
switch(config-if)#

Step 4 hsrp bfd (Optional) Enables or disables BFD on an

HSRP interface. The default is disabled.
Example:
switch(config-if)# hsrp bfd

Step 5 show running-config hsrp (Optional) Displays the HSRP running

configuration.
Example:
switch(config-if)# show running-config
hsrp

Step 6 copy running-config startup-config (Optional) Saves the configuration change.

Example:
switch(config-if)# copy
running-config startup-config

Configuring BFD on VRRP

You can configure BFD for the Virtual Router Redundancy Protocol (VRRP). The active and standby VRRP
routers track each other through BFD. If BFD on the standby VRRP router detects that the active VRRP router
is down, the standby VRRP router treats this event as an active time rexpiry and takes over as the active VRRP
router.
The show vrrp detail command shows this event as BFD@Act-down or BFD@Sby-down.

Before you begin

Enable the BFD feature. See the Enabling the BFD Feature section.
Configure the BFD session parameters. See the Configuring Global BFD Parameters section or the Configuring
BFD on an Interface section.
Enable the VRRP feature. See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide
for more information.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
146
 Configuring Bidirectional Forwarding Detection
Configuring BFD on PIM

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface int-if Enters interface configuration mode. Use the ?

keyword to display the supported interfaces.
Example:
switch(config)# interface
ethernet 2/1
switch(config-if)#

Step 3 vrrp group-no Specifies the VRRP group number.

Example:
switch(config-if)# vrrp 2

Step 4 vrrp bfd address Enables or disables BFD on a VRRP interface.

The default is disabled.
Example:
switch(config-if)# vrrp bfd

Step 5 show running-config vrrp (Optional) Displays the VRRP running

configuration.
Example:
switch(config-if)# show running-config
vrrp

Step 6 copy running-config startup-config (Optional) Saves the configuration change.

Example:
switch(config-if)# copy
running-config startup-config

Configuring BFD on PIM

You can configure BFD for the Protocol Independent Multicast (PIM) protocol.

Before you begin

Enable the BFD feature. See the Enabling the BFD Feature section.
Enable the PIM feature. See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide for
more information.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
147
 Configuring Bidirectional Forwarding Detection
Configuring BFD on Static Routes

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 ip pim bfd Enables BFD for PIM.

Example:
switch(config)# ip pim bfd

Step 3 interface int-if Enters interface configuration mode. Use the ?

keyword to display the supported interfaces.
Example:
switch(config)# interface
ethernet 2/1
switch(config-if)#

Step 4 ip pim bfd-instance [disable] (Optional) Enables or disables BFD on a PIM

interface. The default is disabled.
Example:
switch(config-if)# ip pim bfd-instance

Step 5 show running-config pim (Optional) Displays the PIM running

configuration.
Example:
switch(config)# show running-config pim

Step 6 copy running-config startup-config (Optional) Saves the configuration change.

Example:
switch(config)# copy
running-config startup-config

Configuring BFD on Static Routes

You can configure BFD for static routes on an interface. You can optionally configure BFD on a static route
within a virtual routing and forwarding (VRF) instance.

Before you begin

Enable the BFD feature. See the Enabling the BFD Feature section.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
148
 Configuring Bidirectional Forwarding Detection
Disabling BFD on an Interface

Command or Action Purpose

switch# configure terminal
switch(config)#

Step 2 vrf context vrf-name (Optional) Enters VRF configuration mode.

Example:
switch(config)# vrf context Red
switch(config-vrf)#

Step 3 ip route route interface {nh-address | Creates a static route Use the ? keyword to
nh-prefix} display the supported interfaces.
Example:
switch(config-vrf)# ip route 192.0.2.1
ethernet 2/1 192.0.2.4

Step 4 ip route static bfd interface {nh-address | Enables BFD for all static routes on an interface.
nh-prefix} Use the? keyword to display the supported
interfaces.
Example:
switch(config-vrf)# ip route static bfd
ethernet 2/1 192.0.2.4

Step 5 show ip route static [vrf vrf-name] (Optional) Displays the static routes.
Example:
switch(config-vrf)# show ip route static
vrf Red

Step 6 copy running-config startup-config (Optional) Saves the configuration change.

Example:
switch(config-vrf)# copy
running-config startup-config

Disabling BFD on an Interface

You can selectively disable BFD on an interface for a routing protocol that has BFD enabled at the global or
VRF level.
To disable BFD on an interface, use one of the following commands in interface configuration mode:

Command Purpose

ip eigrp instance-tag bfd disable Disables BFD on an EIGRP interface. The instance
tag can be any case-sensitive, alphanumeric string up
Example:
to 20 characters.
switch(config-if)# ip eigrp Test1 bfd
disable

ip ospf bfd disable Disables BFD on an OSPFv2 interface.

Example:
switch(config-if)# ip ospf bfd disable

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
149
 Configuring Bidirectional Forwarding Detection
Configuring BFD Interoperability

Command Purpose

isis bfd disable Disables BFD on an IS-IS interface.

Example:
switch(config-if)# isis bfd disable

Disabling BFD on an Interface

Example configuration where BFD is disabled per interface.
configure terminal
interface port-channel 10
no ip redirects
ip address 22.1.10.1/30
ipv6 address 22:1:10::1/120
no ipv6 redirects
ip router ospf 10 area 0.0.0.0
ip ospf bfd disable /*** disables IPv4 BFD session for OSPF
ospfv3 bfd disable /*** disables IPv6 BFD session for OSPFv3

Configuring BFD Interoperability

Configuring BFD Interoperability in Cisco NX-OS Devices in a Point-to-Point
Link
Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface port-channel int-if Enters interface configuration mode. Use the ?
keyword to display the supported interfaces.
Example:
switch(config-if)# interface ethernet
2/1

Step 3 ip ospf bfd Enables BFD on an OSPFv2 interface. The

default is disabled.
Example:
switch(config-if)# ip ospf bfd OSPF is used as an example. You can enable
BFD of any of the supported protocols.

Step 4 no ip redirects Prevents the device from sending redirects.

Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
150
 Configuring Bidirectional Forwarding Detection
Configuring BFD Interoperability in Cisco NX-OS Devices in a Switch Virtual Interface

Command or Action Purpose

switch(config-if)# no ip redirects

Step 5 bfd interval mintx min_rx msec multiplier Configures the BFD session parameters for all
value BFD sessions on the port channel. This
command overrides these values by configuring
Example:
the BFD session parameters. The mintx and
switch(config-if)# bfd interval 50 msec range is from 50 to 999 milliseconds and
min_rx 50 multiplier 3
the default is 50. The multiplier range is from
1 to 50. The multiplier default is 3.

Step 6 exit Exits interface configuration mode and returns

to EXEC mode.
Example:
switch(config-if)# exit

Configuring BFD Interoperability in Cisco NX-OS Devices in a Switch Virtual

Interface
Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface port-channel vlan vlan-id Creates a dynamic Switch Virtual Interface
(SVI).
Example:
switch(config)# interface vlan 998
switch(config-if)#

Step 3 bfd interval mintx min_rx msec multiplier Configures the BFD session parameters for all
value BFD sessions on the device. The mintx and
msec range is from 50 to 999 milliseconds and
Example:
the default is 50. The multiplier range is from
switch(config-if)# bfd interval 50 1 to 50. The multiplier default is 3.
min_rx 50 multiplier 3

Step 4 no ip redirects Prevents the device from sending redirects.

Example:
switch(config-if)# no ip redirects

Step 5 ip address ip-address/length Configures an IP address for this interface.

Example:
switch(config-if)# ip address
10.1.0.253/24

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
151
 Configuring Bidirectional Forwarding Detection
Configuring BFD Interoperability in Cisco NX-OS Devices in Logical Mode

Command or Action Purpose

Step 6 ip ospf bfd Enables BFD on an OSPFv2 interface. The
default is disabled.
Example:
switch(config-if)# ip ospf bfd

Step 7 exit Exits interface configuration mode and returns

to EXEC mode.
Example:
switch(config-if)# exit

Configuring BFD Interoperability in Cisco NX-OS Devices in Logical Mode

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface port-channel type Enters port channel configuration mode. Use
number.subinterface-id the ? keyword to display the supported number
range.
Example:
switch(config-if)# interface port-channel
50.2

Step 3 bfd interval mintx min_rx msec multiplier Configures the BFD session parameters for all
value BFD sessions on the port channel. The mintx
and msec range is from 50 to 999 milliseconds
Example:
and the default is 50. The multiplier range is
switch(config-if)# bfd interval 50 from 1 to 50. The multiplier default is 3.
min_rx 50 multiplier 3

Step 4 no ip redirects Prevents the device from sending redirects.

Example:
switch(config-if)# no ip redirects

Step 5 ip ospf bfd Enables BFD on an OSPFv2 interface. The

default is disabled.
Example:
switch(config-if)# ip ospf bfd OSPF is used as an example. You can enable
BFD of any of the supported protocols.

Step 6 exit Exits interface configuration mode and returns

to EXEC mode.
Example:
switch(config-if)# exit

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
152
 Configuring Bidirectional Forwarding Detection
Verifying BFD Interoperability in a Cisco Nexus 9000 Series Device

Verifying BFD Interoperability in a Cisco Nexus 9000 Series Device

The following example shows how to verify BFD interoperability in a Cisco Nexus 9000 Series device.

switch# show bfd neighbors details

OurAddr NeighAddr LD/RD RH/RS Holdown(mult) State Int
Vrf
10.1.1.1 10.1.1.2 1140850707/2147418093 Up 6393(4) Up Vlan2121
default
Session state is Up and using echo function with 50 ms interval
Local Diag: 0, Demand mode: 0, Poll bit: 0, Authentication: None
MinTxInt: 50000 us, MinRxInt: 2000000 us, Multiplier: 3
Received MinRxInt: 2000000 us, Received Multiplier: 4
Holdown (hits): 8000 ms (0), Hello (hits): 2000 ms (108)
Rx Count: 92, Rx Interval (ms) min/max/avg: 347/1996/1776 last: 1606 ms ago
Tx Count: 108, Tx Interval (ms) min/max/avg: 1515/1515/1515 last: 1233 ms ago
Registered protocols: ospf
Uptime: 0 days 0 hrs 2 mins 44 secs
Last packet: Version: 1 - Diagnostic: 0
State bit: Up - Demand bit: 0
Poll bit: 0 - Final bit: 0
Multiplier: 4 - Length: 24
My Discr.: 2147418093 - Your Discr.: 1140850707
Min tx interval: 2000000 - Min rx interval: 2000000
Min Echo interval: 1000 - Authentication bit: 0
Hosting LC: 10, Down reason: None, Reason not-hosted: None

switch# show bfd neighbors details

OurAddr NeighAddr LD/RD RH/RS Holdown(mult) State Int
Vrf
10.0.2.1 10.0.2.2 1140850695/131083 Up 270(3) Up Po14.121
default
Session state is Up and not using echo function
Local Diag: 0, Demand mode: 0, Poll bit: 0, Authentication: None
MinTxInt: 50000 us, MinRxInt: 50000 us, Multiplier: 3
Received MinRxInt: 100000 us, Received Multiplier: 3
Holdown (hits): 300 ms (0), Hello (hits): 100 ms (3136283)
Rx Count: 2669290, Rx Interval (ms) min/max/avg: 12/1999/93 last: 29 ms ago
Tx Count: 3136283, Tx Interval (ms) min/max/avg: 77/77/77 last: 76 ms ago
Registered protocols: ospf
Uptime: 2 days 21 hrs 41 mins 45 secs
Last packet: Version: 1 - Diagnostic: 0
State bit: Up - Demand bit: 0
Poll bit: 0 - Final bit: 0
Multiplier: 3 - Length: 24
My Discr.: 131083 - Your Discr.: 1140850695
Min tx interval: 100000 - Min rx interval: 100000
Min Echo interval: 0 - Authentication bit: 0
Hosting LC: 8, Down reason: None, Reason not-hosted: None

Verifying the BFD Configuration

To display BFD configuration information, perform one of the following:

Command Purpose

show running-config bfd Displays the running BFD configuration.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
153
 Configuring Bidirectional Forwarding Detection
Monitoring BFD

Command Purpose

show startup-config bfd Displays the BFD configuration that will be applied
on the next system startup.

Monitoring BFD
Use the following commands to display BFD:

Command Purpose

show bfd neighbors [application name] [details] Displays information about BFD for a supported
application, such as BGP or OSPFv2.

show bfd neighbors [interface int-if] [details] Displays information about BGP sessions on an
interface.

show bfd neighbors [dest-ip ip-address] [src-ip Displays information about the specified BGP session
ip-address][details] on an interface.

show bfd neighbors [vrf vrf-name] [details] Displays information about BFD for a VRF.

show bfd [ipv4 | ipv6] [neighbors] Displays information about IPv4 neighbors or IPv6
neighbors.

Configuration Examples for BFD

This example shows how to configure BFD for OSPFv2 on Ethernet 2/1, using the default BFD session
parameters:

feature bfd
feature ospf
router ospf Test1
interface ethernet 2/1
ip ospf bfd
no shutdown

This example shows how to configure BFD for all EIGRP interfaces, using the default BFD session parameters:

feature bfd
feature eigrp
bfd interval 100 min_rx 100 multiplier 4
router eigrp Test2
bfd

This example shows how to configure BFDv6:

feature bfd
feature ospfv3
router ospfv3 Test1

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
154
 Configuring Bidirectional Forwarding Detection
Show Example for BFD

interface Ethernet2/7
ipv6 router ospfv3 Test1 area 0.0.0.0
ospfv3 bfd
no shutdown

Show Example for BFD

This example shows results of the show bfd ipv6 neighbors details command.

#show bfd ipv6 neighbors details

OurAddr NeighAddr
LD/RD RH/RS Holdown(mult) State Int
Vrf
cc:10::2 cc:10::1
1090519335/1090519260 Up 5692(3) Up Po1
default

Session state is Up and using echo function with 250 ms interval

Local Diag: 0, Demand mode: 0, Poll bit: 0, Authentication: None
MinTxInt: 250000 us, MinRxInt: 2000000 us, Multiplier: 3
Received MinRxInt: 2000000 us, Received Multiplier: 3
Holdown (hits): 6000 ms (4), Hello (hits): 2000 ms (205229)
Rx Count: 227965, Rx Interval (ms) min/max/avg: 124/1520/1510 last: 307 ms ago
Tx Count: 205229, Tx Interval (ms) min/max/avg: 1677/1677/1677 last: 587 ms ago
Registered protocols: bgp
Uptime: 3 days 23 hrs 31 mins 13 secs
Last packet: Version: 1 - Diagnostic: 0
State bit: Up - Demand bit: 0
Poll bit: 0 - Final bit: 0
Multiplier: 3 - Length: 24
My Discr.: 1090519260 - Your Discr.: 1090519335
Min tx interval: 250000 - Min rx interval: 2000000
Min Echo interval: 250000 - Authentication bit: 0
Hosting LC: 1, Down reason: None, Reason not-hosted: None

Related Documents
Related Topic Document Title

BFD commands Cisco Nexus 9000 Series NX-OS Unicast Routing

Configuration Guide

RFCs
RFC Title

RFC 5880 Bidirectional Forwarding Detection (BFD)

RFC 5881 BFD for IPv4 and IPv6 (Single Hop)

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
155
 Configuring Bidirectional Forwarding Detection
RFCs

RFC Title

RFC 7130 Bidirectional Forwarding Detection (BFD) on Link

Aggregation Group (LAG) Interfaces

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
156
 CHAPTER 7
Configuring Port Channels
This chapter describes how to configure port channels and to apply and configure the Link Aggregation
Control Protocol (LACP) for more efficient use of port channels in the Cisco NX-OS devices.
On a single switch, the port-channel compatibility parameters must be the same among all the port-channel
members on the physical switch.
• About Port Channels, on page 157
• Port Channels, on page 158
• Port-Channel Interfaces, on page 159
• Basic Settings, on page 159
• Compatibility Requirements, on page 160
• Load Balancing Using Port Channels, on page 162
• Symmetric Hashing, on page 163
• Guidelines and Limitations for ECMP, on page 163
• Resilient Hashing, on page 164
• GTP Tunnel Load Balancing, on page 164
• LACP, on page 165
• Prerequisites for Port Channeling, on page 171
• Guidelines and Limitations, on page 171
• Default Settings, on page 173
• Configuring Port Channels, on page 174

About Port Channels

A port channel is an aggregation of multiple physical interfaces that creates a logical interface. You can bundle
up to 32 individual active links into a port channel to provide increased bandwidth and redundancy. Port
channeling also load balances traffic across these physical interfaces. The port channel stays operational as
long as at least one physical interface within the port channel is operational.
You can create a Layer 2 port channel by bundling compatible Layer 2 interfaces, or you can create Layer 3
port channels by bundling compatible Layer 3 interfaces. You cannot combine Layer 2 and Layer 3 interfaces
in the same port channel.
You can also change the port channel from Layer 3 to Layer 2. See the Configuring Layer 2 Interfaces chapter
for information about creating Layer 2 interfaces.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
157
 Configuring Port Channels
Port Channels

A Layer 2 port channel interface and it's member ports can have different STP parameters. Changing the STP
parameters of the port channel does not impact the STP parameters of the member ports because a port channel
interface takes precedence if the member ports are bundled.

Note After a Layer 2 port becomes part of a port channel, all switchport configurations must be done on the port
channel; you can no longer apply switchport configurations to individual port-channel members. You cannot
apply Layer 3 configurations to an individual port-channel member either; you must apply the configuration
to the entire port channel.

You can use static port channels, with no associated aggregation protocol, for a simplified configuration.
For more flexibility, you can use the Link Aggregation Control Protocol (LACP), which is defined in IEEE
802.3ad. When you use LACP, the link passes protocol packets. You cannot configure LACP on shared
interfaces.
See the LACP Overview section for information about LACP.

Port Channels
A port channel bundles physical links into a channel group to create a single logical link that provides the
aggregate bandwidth of up to 32 physical links. If a member port within a port channel fails, the traffic
previously carried over the failed link switches to the remaining member ports within the port channel.
However, you can enable the LACP to use port channels more flexibly. Configuring port channels with LACP
and static port channels require a slightly different procedure (see the “Configuring Port Channels” section).

Note The device does not support Port Aggregation Protocol (PAgP) for port channels.

Each port can be in only one port channel. All the ports in a port channel must be compatible; they must use
the same speed and duplex mode (see the “Compatibility Requirements” section). When you run static port
channels with no aggregation protocol, the physical links are all in the on channel mode; you cannot change
this mode without enabling LACP (see the “Port-Channel Modes” section).
You can create port channels directly by creating the port-channel interface, or you can create a channel group
that acts to aggregate individual ports into a bundle. When you associate an interface with a channel group,
the software creates a matching port channel automatically if the port channel does not already exist. In this
instance, the port channel assumes the Layer 2 or Layer 3 configuration of the first interface. You can also
create the port channel first. In this instance, the Cisco NX-OS software creates an empty channel group with
the same channel number as the port channel and takes the default Layer 2 or Layer 3 configuration, as well
as the compatibility configuration (see the “Compatibility Requirements” section).

Note The port channel is operationally up when at least one of the member ports is up and that port’s status is
channeling. The port channel is operationally down when all member ports are operationally down.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
158
 Configuring Port Channels
Port-Channel Interfaces

Port-Channel Interfaces
The following shows port-channel interfaces.
Figure 9: Port-Channel Interfaces

You can classify port-channel interfaces as Layer 2 or Layer 3 interfaces. In addition, you can configure Layer
2 port channels in either access or trunk mode. Layer 3 port-channel interfaces have routed ports as channel
members.
You can configure a Layer 3 port channel with a static MAC address. If you do not configure this value, the
Layer 3 port channel uses the router MAC of the first channel member to come up. See the Cisco Nexus 9000
Series NX-OS Layer 2 Switching Configuration Guide for information about configuring static MAC addresses
on Layer 3 port channels.
See the "Configuring Layer 2 Interfaces" chapter for information about configuring Layer 2 ports in access
or trunk mode and the "Configuring Layer 3 Interfaces" chapter for information about configuring Layer 3
interfaces and subinterfaces.

Basic Settings
You can configure the following basic settings for the port-channel interface:
• Bandwidth—Use this setting for informational purposes only; this setting is to be used by higher-level
protocols.
• Delay—Use this setting for informational purposes only; this setting is to be used by higher-level protocols.
• Description
• Duplex

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
159
 Configuring Port Channels
Compatibility Requirements

• IP addresses
• Maximum Transmission Unit (MTU)
• Shutdown
• Speed

Compatibility Requirements
When you add an interface to a channel group, the software checks certain interface attributes to ensure that
the interface is compatible with the channel group. For example, you cannot add a Layer 3 interface to a Layer
2 channel group. The Cisco NX-OS software also checks a number of operational attributes for an interface
before allowing that interface to participate in the port-channel aggregation.
The compatibility check includes the following operational attributes:
• Network layer
• (Link) speed capability
• Speed configuration
• Duplex capability
• Duplex configuration
• Port mode
• Access VLAN
• Trunk native VLAN
• Tagged or untagged
• Allowed VLAN list
• MTU size
• SPAN—Cannot be a SPAN source or a destination port
• Storm control
• Flow-control capability
• Flow-control configuration
• Media type, either copper or fiber

Use the show port-channel compatibility-parameters command to see the full list of compatibility checks
that the Cisco NX-OS uses.
You can only add interfaces configured with the channel mode set to on to static port channels, and you can
only add interfaces configured with the channel mode as active or passive to port channels that are running
LACP. You can configure these attributes on an individual member port. If you configure a member port with
an incompatible attribute, the software suspends that port in the port channel.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
160
Configuring Port Channels
Compatibility Requirements

Alternatively, you can force ports with incompatible parameters to join the port channel if the following
parameters are the same:
• (Link) speed capability
• Speed configuration
• Duplex capability
• Duplex configuration
• Flow-control capability
• Flow-control configuration

When the interface joins a port channel, some of its individual parameters are removed and replaced with the
values on the port channel as follows:
• Bandwidth
• Delay
• Extended Authentication Protocol over UDP
• VRF
• IP address
• MAC address
• Spanning Tree Protocol
• NAC
• Service policy
• Access control lists (ACLs)

Many interface parameters remain unaffected when the interface joins or leaves a port channel as follows:
• Beacon
• Description
• CDP
• LACP port priority
• Debounce
• UDLD
• MDIX
• Rate mode
• Shutdown
• SNMP trap

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
161
 Configuring Port Channels
Load Balancing Using Port Channels

Note When you delete the port channel, the software sets all member interfaces as if they were removed from the
port channel.

See the “LACP Marker Responders” section for information about port-channel modes.

Load Balancing Using Port Channels

The Cisco NX-OS software load balances traffic across all operational interfaces in a port channel by hashing
the addresses in the frame to a numerical value that selects one of the links in the channel. Port channels
provide load balancing by default. Port-channel load balancing uses MAC addresses, IP addresses, or Layer
4 port numbers to select the link. Port-channel load balancing uses either source or destination addresses or
ports, or both source and destination addresses or ports.
You can configure the load- balancing mode to apply to all port channels that are configured on the entire
device. You can configure one load-balancing mode for the entire device. You cannot configure the
load-balancing method per port channel.
You can configure the type of load-balancing algorithm used. You can choose the load-balancing algorithm
that determines which member port to select for egress traffic by looking at the fields in the frame.
The default load-balancing mode for Layer 3 interfaces is the source and destination IP L4 ports, and the
default load-balancing mode for non-IP traffic is the source and destination MAC address. Use the port-channel
load-balance command to set the load-balancing method among the interfaces in the channel-group bundle.
The default method for Layer 2 packets is src-dst-mac. The default method for Layer 3 packets is src-dst
ip-l4port.
You can configure the device to use one of the following methods to load balance across the port channel:
• Destination MAC address
• Source MAC address
• Source and destination MAC address
• Destination IP address
• Source IP address
• Source and destination IP address
• Source TCP/UDP port number
• Destination TCP/UDP port number
• Source and destination TCP/UDP port number
• GRE inner IP headers with source, destination and source-destination

Non-IP and Layer 3 port channels both follow the configured load-balancing method, using the source,
destination, or source and destination parameters. For example, when you configure load balancing to use the
source IP address, all non-IP traffic uses the source MAC address to load balance the traffic while the Layer
3 traffic load balances the traffic using the source IP address. Similarly, when you configure the destination

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
162
 Configuring Port Channels
Symmetric Hashing

MAC address as the load-balancing method, all Layer 3 traffic uses the destination IP address while the non-IP
traffic load balances using the destination MAC address.
The unicast and multicast traffic is load-balanced across port-channel links based on configured load-balancing
algorithm displayed in show port-channel load-balancing command output.
The multicast traffic uses the following methods for load balancing with port channels:
• Multicast traffic with Layer 4 information—Source IP address, source port, destination IP address,
destination port
• Multicast traffic without Layer 4 information—Source IP address, destination IP address
• Non-IP multicast traffic—Source MAC address, destination MAC address

Note Devices that run Cisco IOS can optimize the behavior of the member ports ASICs if a failure of a single
member occurred by running the port-channel hash-distribution command. The Cisco Nexus 9000 Series
device performs this optimization by default and does not require or support this command. Cisco NX-OS
does support the customization of the load-balancing criteria on port channels through the port-channel
load-balance command for the entire device.

Symmetric Hashing
To be able to effectively monitor traffic on a port channel, it is essential that each interface connected to a
port channel receives both forward and reverse traffic flows. Normally, there is no guarantee that the forward
and reverse traffic flows will use the same physical interface. However, when you enable symmetric hashing
on the port channel, bidirectional traffic is forced to use the same physical interface and each physical interface
in the port channel is effectively mapped to a set of flows.
When symmetric hashing is enabled, the parameters used for hashing, such as the source and destination IP
address, are normalized before they are entered into the hashing algorithm. This process ensures that when
the parameters are reversed (the source on the forward traffic becomes the destination on the reverse traffic),
the hash output is the same. Therefore, the same interface is chosen.
Only the following load-balancing algorithms support symmetric hashing:
• src-dst ip
• src-dst ip-l4port

Guidelines and Limitations for ECMP

You might observe that load balancing with Layer 2/Layer 3 GW flows are not load balanced equally among
all links when the switch comes up initially after reload. There are two CLIs to change the ECMP hash
configuration in the hardware. The two CLI commands are mutually exclusive.
• Enter the port-channel load-balance [src | src-dst | dst] mac command for MAC-based only hash.
• For hash based on IP/Layer 4 ports, enter either the ip load-share or port-channel load-balance
command.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
163
 Configuring Port Channels
Resilient Hashing

• The port-channel load-balance command can overwrite the ip load-share command. It is better to
enter the port-channel load-balance command which helps to set both the IP and MAC parameters.
• There are no options to force the hashing algorithm based on the IP/Layer 4 port. The default MAC
configuration is always programmed as a part of the port channel configuration.
• ECMP resilient hashing is not supported for traffic flows over tunnel.

Resilient Hashing
With the exponential increase in the number of physical links used in data centers, there is also the potential
for an increase in the number of failed physical links. In static hashing systems that are used for load balancing
flows across members of port channels or Equal Cost Multipath (ECMP) groups, each flow is hashed to a
link. If a link fails, all flows are rehashed across the remaining working links. This rehashing of flows to links
results in some packets being delivered out of order even for those flows that were not hashed to the failed
link.
This rehashing also occurs when a link is added to the port channel or Equal Cost Multipath (ECMP) group.
All flows are rehashed across the new number of links, which results in some packets being delivered out of
order. Resilient hashing supports only unicast traffic.
Resilient hashing maps flows to physical ports. In case a link fails, the flows assigned to the failed link are
redistributed uniformly among the working links. The existing flows through the working links are not rehashed
and their packets are not delivered out of order.
Resilient hashing is supported only for ECMP groups and not on part channel interfaces. Resiliency is
guaranteed only upon deletion of a member in an ECMP group, not on adding a member to the ECMP group.
When a link is added to the port channel or ECMP group, some of the flows hashed to the existing links are
rehashed to the new link, but not across all existing links.
Resilient hashing also occurs when a link is added to the port channel or Equal Cost Multipath (ECMP) group.
Resilient hashing is supported on both Layer 2 port-channel member links and Layer 3 ECMP paths on the
routing table.
Resilient hashing supports IPv4 and IPv6 known unicast traffic, but it does not support IPv4 multicast traffic.

Note Resilient hashing is supported on Network Forwarding Engine (NFE) based Cisco Nexus 9300 platform
switches and Cisco Nexus 9500 platform switches. (NX-OS 7.0(3)I3(1) release and later).

Note Resilient hashing is not supported on port-channel members.

GTP Tunnel Load Balancing

GPRS Tunneling Protocol (GTP) is used mainly to deliver mobile data on wireless networks via Cisco Nexus
9000 Series switches as the core router. When two routers carrying GTP traffic are connected with link
bundling, the traffic is required to be distributed evenly between all bundle members.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
164
 Configuring Port Channels
LACP

To achieve load balancing, Cisco Nexus 9000 Series switches use 5-tuple load balancing mechanism. The
load balancing mechanism takes into account the source IP, destination IP, protocol, Layer 4 resource and
destination port (if traffic is TCP or UDP) fields from the packet. In the case of GTP traffic, a limited number
of unique values for these fields restrict the equal distribution of traffic load on the tunnel.
In order to avoid polarization for GTP traffic in load balancing, a tunnel endpoint identifier (TEID) in the
GTP header is used instead of a UDP port number. Since the TEID is unique per tunnel, traffic can be evenly
load balanced across multiple links in the bundle.
Beginning Cisco Nexus Release 7.0(3)I7(3), GTP Tunnel Load Balancing is supported on Cisco Nexus
9300-EX platform switches.
Beginning Cisco Nexus Release 7.0(3)I7(4), GTP Tunnel Load Balancing is supported on Cisco Nexus
9300-FX and 9364C platform switches.
This feature overrides the source and destination port information with the 32-bit TEID value that is present
in GTPU packets.
GTP tunnel load balancing feature adds support for:
• GTP with IPv4/IPv6 transport header on physical interface
• GTP traffic over TE tunnel
• GTPU with UDP port 2152

The ip load-sharing address source-destination gtpu command enables the GTP tunnel load balancing.
To know the egress interface for GTP traffic after load balancing, use show cef {ipv4 | ipv6} exact-route
command with TEID in place of L4 protocol source and destination port number. Use 16MSBist of TEID in
source port and 16LSBits of TEID in destination port.

LACP
LACP allows you to configure up to 16 interfaces into a port channel.

LACP Overview
The Link Aggregation Control Protocol (LACP) for Ethernet is defined in IEEE 802.1AX and IEEE 802.3ad.
This protocol controls how physical ports are bundled together to form one logical channel.

Note You must enable LACP before you can use LACP. By default, LACP is disabled. See the “Enabling LACP”
section for information about enabling LACP.

The system automatically takes a checkpoint before disabling the feature, and you can roll back to this
checkpoint. See the Cisco Nexus 9000 Series NX-OS System Management Configuration Guide for information
about rollbacks and checkpoints.
The following figure shows how individual links can be combined into LACP port channels and channel
groups as well as function as individual links.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
165
 Configuring Port Channels
Port-Channel Modes

Figure 10: Individual Links Combined into a Port Channel

With LACP, you can bundle up to 32 interfaces in a channel group.

Note When you delete the port channel, the software automatically deletes the associated channel group. All member
interfaces revert to their original configuration.

Note If you downgrade a Cisco Nexus 9500 series switch that is configured to use LACP vPC convergence feature,
that runs Cisco NX-OS Release 7.0(3)I7(5) to a lower release, the configuration is removed. You must configure
the LACP vPC convergence feature again when you upgrade the switch.

You cannot disable LACP while any LACP configurations are present.

Port-Channel Modes
Individual interfaces in port channels are configured with channel modes. When you run static port channels
with no aggregation protocol, the channel mode is always set to on. After you enable LACP globally on the
device, you enable LACP for each channel by setting the channel mode for each interface to either active or
passive. You can configure channel mode for individual links in the LACP channel group when you are
adding the links to the channel group

Note You must enable LACP globally before you can configure an interface in either the active or passive channel
mode.

The following table describes the channel modes.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
166
Configuring Port Channels
Port-Channel Modes

Table 10: Channel Modes for Individual Links in a Port Channel

Channel Mode Description

passive The LACP is enabled on this port channel and the

ports are in a passive negotiating state. Ports responds
to LACP packets that it receives but does not initiate
LACP negotiation.

active The LACP is enabled on this port channel and the

ports are in an active negotiating state. Ports initiate
negotiations with other ports by sending LACP
packets.

on The LACP is disabled on this port channel and the

ports are in a non-negotiating state. The on state of
the port channel represents the static mode.
The port will not verify or negotiate port channel
memberships. If you attempt to change the channel
mode to active or passive before enabling LACP, the
device displays an error message. When an LACP
attempts to negotiate with an interface in the on state,
it does not receive any LACP packets and becomes
an individual link with that interface, it does not join
the LACP channel group. The on state is the default
port-channel mode

Both the passive and active modes allow LACP to negotiate between ports to determine if they can form a
port channel based on criteria such as the port speed and the trunking state.The passive mode is useful when
you do not know whether the remote system, or partner, supports LACP.
Two devices can form an LACP port channel when their ports are in different LACP modes if the modes are
compatible as in the following example:

Table 11: Channel Modes Compatibility

Device 1 > Port-1 Device 2 > Port-2 Result

Active Active Can form a port channel.

Active Passive Can form a port channel.

Passive Passive Cannot form a port channel because no ports can initiate
negotiation.

On Active Cannot form a port channel because LACP is enabled

only on one side.

On Passive Cannot form a port channel because LACP is not

enabled.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
167
 Configuring Port Channels
LACP ID Parameters

LACP ID Parameters
This section describes the LACP parameters.

LACP System Priority

Each system that runs LACP has an LACP system priority value. You can accept the default value of 32768
for this parameter, or you can configure a value between 1 and 65535. LACP uses the system priority with
the MAC address to form the system ID and also uses the system priority during negotiation with other devices.
A higher system priority value means a lower priority.

Note The LACP system ID is the combination of the LACP system priority value and the MAC address.

LACP Port Priority

Each port that is configured to use LACP has an LACP port priority. You can accept the default value of
32768 for the LACP port priority, or you can configure a value between 1 and 65535. LACP uses the port
priority with the port number to form the port identifier.
LACP uses the port priority to decide which ports should be put in standby mode when there is a limitation
that prevents all compatible ports from aggregating and which ports should be put into active mode. A higher
port priority value means a lower priority for LACP. You can configure the port priority so that specified
ports have a lower priority for LACP and are most likely to be chosen as active links, rather than hot-standby
links.

LACP Administrative Key

LACP automatically configures an administrative key value equal to the channel-group number on each port
configured to use LACP. The administrative key defines the ability of a port to aggregate with other ports. A
port’s ability to aggregate with other ports is determined by these factors:
• Port physical characteristics, such as the data rate and the duplex capability
• Configuration restrictions that you establish

LACP Marker Responders

You can dynamically redistribute the data traffic by using port channels. This redistribution might result from
a removed or added link or a change in the load-balancing scheme. Traffic redistribution that occurs in the
middle of a traffic flow can cause misordered frames.
LACP uses the Marker Protocol to ensure that frames are not duplicated or reordered due to this redistribution.
The Marker Protocol detects when all the frames of a given traffic flow are successfully received at the remote
end. LACP sends Marker PDUs on each of the port-channel links. The remote system responds to the Marker
PDU once it receives all the frames received on this link prior to the Marker PDU. The remote system then
sends a Marker Responder. Once the Marker Responders are received by the local system on all member links
of the port channel, the local system can redistribute the frames in the traffic flow with no chance of misordering.
The software supports only Marker Responders.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
168
 Configuring Port Channels
LACP-Enabled and Static Port Channels Differences

LACP-Enabled and Static Port Channels Differences

The following table summarizes the major differences between port channels with LACP enabled and static
port channels.

Table 12: Port Channels with LACP Enabled and Static Port Channels

Configurations Port Channels with LACP Enabled Static Port Channels

Protocol applied Enable globally Not applicable

Channel mode of links Can be either: Can only be On

• Active
• Passive

Maximum number of links in 32 32

channel

LACP Compatibility Enhancements

When a Cisco Nexus 9000 Series device is connected to a non-Nexus peer, its graceful failover defaults may
delay the time that is taken to bring down a disabled port or cause traffic from the peer to be lost. To address
these conditions, the lacp graceful-convergence command was added.
By default, LACP sets a port to suspended state if it does not receive an LACP PDU from the peer. lacp
suspend-individual is a default configuration on Cisco Nexus 9000 series switches. This command puts the
port in suspended state if it does not receive any LACP PDUs. In some cases, although this feature helps in
preventing loops created due to misconfigurations, it can cause servers fail to boot up because they require
LACP to logically bring up the port. You can put a port into an individual state by using the no lacp
suspend-individual. Port in individual sate takes attributes of the individual port based on the port
configuration.
LACP port-channels exchange LACP PDUs for quick bundling of links when connecting a server and a switch.
However, the links go into suspended state when the PDUs are not received.
The delayed LACP feature enables one port-channel member, the delayed-LACP port, to come up first as a
member of a regular port-channel before LACP PDUs are received. After it is connected in LACP mode,
other members, the auxiliary LACP ports, are brought up. This avoids having the links becoming suspended
when PDUs are not received.
Which port in the port-channel comes up first depends on the port-priority value of the ports. A member link
in a port channel with lowest priority value, will come come up first as a LACP delayed port. Regardless of
the operational status of the links, the configured priority of a LACP port is used to select the delayed-lacp
port
This feature supports Layer 2 port channels, trunk mode spanning tree, and vPC and has the following
limitations:
• Using no lacp suspend-individual and lacp mode delay on a same port channel is not recommended
because it can put non-lacp delayed ports in individual state. As a best practice, you must avoid combining
these two configurations.
• Not supported on Layer 3 port channels.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
169
 Configuring Port Channels
LACP Port-Channel Minimum Links and MaxBundle

• Not supported on Cisco Nexus 9500 Switches and FEX HIF and FEX fabric ports.

LACP Port-Channel Minimum Links and MaxBundle

A port channel aggregates similar ports to provide increased bandwidth in a single manageable interface.
The introduction of the minimum links and maxbundle feature further refines LACP port-channel operation
and provides increased bandwidth in one manageable interface.
The LACP port-channel minimum links feature does the following:
• Configures the minimum number of ports that must be linked up and bundled in the LACP port channel.
• Prevents the low-bandwidth LACP port channel from becoming active.
• Causes the LACP port channel to become inactive if there are few active members ports to supply the
required minimum bandwidth.

The LACP MaxBundle defines the maximum number of bundled ports allowed in a LACP port channel.
The LACP MaxBundle feature does the following:
• Defines an upper limit on the number of bundled ports in an LACP port channel.
• Allows hot-standby ports with fewer bundled ports. (For example, in an LACP port channel with five
ports, you can designate two of those ports as hot-standby ports.)

Note The minimum links and maxbundle feature works only with LACP port channels. However, the device allows
you to configure this feature in non-LACP port channels, but the feature is not operational.

LACP Fast Timers

You can change the LACP timer rate to modify the duration of the LACP timeout. Use the lacp rate command
to set the rate at which LACP control packets are sent to an LACP-supported interface. You can change the
timeout rate from the default rate (30 seconds) to the fast rate (1 second). This command is supported only
on LACP-enabled interfaces. To configure the LACP fast time rate, see the “Configuring the LACP Fast
Timer Rate” section.
ISSU and ungraceful switchovers are not supported with LACP fast timers.

Virtualization Support
You must configure the member ports and other port channel-related configuration from the virtual device
context (VDC) that contains the port channel and member ports. You can use the numbers from 1 to 4096 in
each VDC to number the port channels.
All ports in one port channel must be in the same VDC. When you are using LACP, all possible 8 active ports
and all possible 8 standby ports must be in the same VDC.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
170
 Configuring Port Channels
High Availability

Note You must configure load balancing using port channels in the default VDC. See the “Load Balancing Using
Port Channels” section for more information about load balancing.

High Availability
Port channels provide high availability by load balancing traffic across multiple ports. If a physical port fails,
the port channel is still operational if there is an active member in the port channel. You can bundle ports
from different modules and create a port channel that remains operational even if a module fails because the
settings are common across the module.
Port channels support stateful and stateless restarts. A stateful restart occurs on a supervisor switchover. After
the switchover, the Cisco NX-OS software applies the runtime configuration after the switchover.
The port channel goes down if the operational ports fall below the configured minimum links number.

Note See the Cisco Nexus 9000 Series NX-OS High Availability and Redundancy Guide for complete information
about high-availability features.

Prerequisites for Port Channeling

Port channeling has the following prerequisites:
• You must be logged onto the device.
• All ports for a single port channel must be either Layer 2 or Layer 3 ports.
• All ports for a single port channel must meet the compatibility requirements. See the “Compatibility
Requirements” section for more information about the compatibility requirements.
• You must configure load balancing from the default VDC.

Guidelines and Limitations

Port channeling has the following configuration guidelines and limitations:
• show commands with the internal keyword are not supported.
• The LACP port-channel minimum links and maxbundle feature is not supported for host interface port
channels.
• You must enable LACP before you can use that feature.
• You can configure multiple port channels on a device.
• Do not put shared and dedicated ports into the same port channel. (See the “Configuring Basic Interface
Parameters” chapter for information about shared and dedicated ports.)

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
171
 Configuring Port Channels
Guidelines and Limitations

• For Layer 2 port channels, ports with different STP port path costs can form a port channel if they are
compatibly configured with each other. See the “Compatibility Requirements” section for more information
about the compatibility requirements.
• In STP, the port-channel cost is based on the aggregated bandwidth of the port members.
• After you configure a port channel, the configuration that you apply to the port channel interface affects
the port channel member ports. The configuration that you apply to the member ports affects only the
member port where you apply the configuration.
• LACP does not support half-duplex mode. Half-duplex ports in LACP port channels are put in the
suspended state.
• Do not configure ports that belong to a port channel group as private VLAN ports. While a port is part
of the private VLAN configuration, the port channel configuration becomes inactive.
• Channel member ports cannot be a source or destination SPAN port.
• Port-channels are not supported on generation 1 100G line cards (N9K-X9408PC-CFP2) or generic
expansion modules (N9K-M4PC-CFP2).
• Port-channels are supported on devices with generation 2 (and later) 100G interfaces. (7.0(3)I3(1) and
later)
• The port channel might be affected by the limitations of the Application Leaf Engine (ALE) uplink ports
on Cisco Nexus 9300 and 9500 Series devices:Limitations for ALE Uplink Ports
• Resilient hashing for port channels is not supported on Cisco Nexus 9200, Cisco Nexus 9300-EX, and
Cisco Nexus 9500 switches with 9700-EX line cards.
• In Cisco NX-OS Release 7.0(3)I4(1), resilient hashing (port-channel load-balancing resiliency) and
VXLAN configurations are not compatible with VTEPs using ALE uplink ports.

Note Resilient hashing is disabled by default.

• The maximum number of subinterfaces for a port is 511. The maximum number of subinterfaces for a
satellite/FEX port is 63.
• On a Cisco Nexus 92300YC switch, the first 24 ports that are part of the same quadrant. All the ports in
the same quadrant must have same speed. Having different speed on ports in a quadrant is not supported.
Following are the first 24 ports on the Cisco Nexus 92300YC switch that share same quadrant:
• 1,4,7,10
• 2,5,8,11
• 3,6,9,12
• 13,16,19,22
• 14,17,20,23
• 15,18,21,24

• On a Cisco Nexus 9500 switch with a X96136YC-R line card, the ports 17–48 are part of the same
quadrant. Ports in the same quadrant must have same speed (1/10G or 25G) on all ports. Having different

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
172
 Configuring Port Channels
Default Settings

speed on ports in a quadrant is not supported. If you set different speed in any of the ports in a quadrant,
the ports go into error disable state. Interfaces in same quadrant are:
• 17–20
• 21–24
• 25–28
• 29–32
• 33–36
• 37–40
• 41–44
• 45–48

• Beginning Cisco NX-OS Release 7.0(3)I7(8), you can use interface port-channel all command to select
all the available parent port channel interfaces in the system and then apply any subcommand to all the
member ports.

Note The subcommands impacts the FEX and VPC ports as well. For example, a shut
subcommand may shutdown any connected FEX or vPC ports.

• If you have layer 2 and layer 3 port channels in a switch some commands may not be applicable to both
the layer 2 and layer 3 port channels. In those instances, the configuration may not be applied properly.
• Use the interface port-channel all command only on similar type of port channels in a switch. This
command works on parent interfaces. However the command may not work if port channel sub interfaces
are present.

Default Settings
The following table lists the default settings for port-channel parameters.

Table 13: Default Port-Channel Parameters

Parameters Default

Port channel Admin up

Load balancing method for Layer 3 interfaces Source and destination IP address

Load balancing method for Layer 2 interfaces Source and destination MAC address

Load balancing per module Disabled

LACP Disabled

Channel mode on

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
173
 Configuring Port Channels
Configuring Port Channels

Parameters Default

LACP system priority 32768

LACP port priority 32768

Minimum links for LACP 1

Maxbundle 32

Minimum links for FEX fabric port channel 1

Configuring Port Channels

Note See the "Configuring Basic Interface Parameters” chapter for information about configuring the maximum
transmission unit (MTU) for the port-channel interface. See the “Configuring Layer 3 Interfaces” chapter for
information about configuring IPv4 and IPv6 addresses on the port-channel interface.

Note If you are familiar with the Cisco IOS CLI, be aware that the Cisco NX-OS commands for this feature might
differ from the Cisco IOS commands that you would use.

Creating a Port Channel

You can create a port channel before you create a channel group. The software automatically creates the
associated channel group.

Note When the port channel is created before the channel group, the port channel should be configured with all of
the interface attributes that the member interfaces are configured with. Use the switchport mode trunk
{allowed vlan vlan-id | native vlan-id} command to configure the members.

This is required only when the channel group members are Layer 2 ports (switchport) and trunks (switchport
mode trunk).

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
174
Configuring Port Channels
Creating a Port Channel

Note Use the no interface port-channel command to remove the port channel and delete the associated channel
group.

Command Purpose

no interface port-channel channel-number Removes the port channel and deletes the
associated channel group.
Example:
switch(config)# no interface port-channel 1

Before you begin

Enable LACP if you want LACP-based port channels.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface port-channel channel-number Specifies the port-channel interface to

configure, and enters the interface configuration
Example:
mode. The range is from 1 to 4096. The Cisco
switch(config)# interface port-channel NX-OS software automatically creates the
1
switch(config-if)
channel group if it does not already exist.

Step 3 show port-channel summary (Optional) Displays information about the port
channel.
Example:
switch(config-router)# show port-channel
summary

Step 4 no shutdown (Optional) Clears the errors on the interfaces

and VLANs where policies correspond with
Example:
hardware policies. This command allows policy
switch# configure terminal programming to continue and the port to come
switch(config)# int e3/1
switch(config-if)# no shutdown
up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Step 5 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
175
 Configuring Port Channels
Adding a Layer 2 Port to a Port Channel

Example
This example shows how to create a port channel:
switch# configure terminal
switch (config)# interface port-channel 1

See the “Compatibility Requirements” section for details on how the interface configuration changes
when you delete the port channel.

Adding a Layer 2 Port to a Port Channel

You can add a Layer 2 port to a new channel group or to a channel group that already contains Layer 2 ports.
The software creates the port channel associated with this channel group if the port channel does not already
exist.

Note Use the no channel-group command to remove the port from the channel group.

Command Purpose

no channel-group Removes the port from the channel group.

Example:
switch(config)# no channel-group

Before you begin

Enable LACP if you want LACP-based port channels.
All Layer 2 member ports must run in full-duplex mode and at the same speed

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface type slot/port Specifies the interface that you want to add to
a channel group, and enters the interface
Example:
configuration mode.
switch(config)# interface ethernet 1/4
switch(config-if)#

Step 3 switchport Configures the interface as a Layer 2 access

port.
Example:
switch(config)# switchport

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
176
Configuring Port Channels
Adding a Layer 2 Port to a Port Channel

Command or Action Purpose

Step 4 switchport mode trunk (Optional) Configures the interface as a Layer
2 trunk port.
Example:
switch(config)# switchport mode trunk

Step 5 switchport trunk {allowed vlan vlan-id | (Optional) Configures necessary parameters for
native vlan-id} a Layer 2 trunk port.
Example:
switch(config)# switchport trunk native
3
switch(config-if)#

Step 6 channel-group channel-number [force] [mode Configures the port in a channel group and sets
{on | active | passive}] the mode. The channel-number range is from 1
to 4096. This command creates the port channel
Example:
associated with this channel group if the port
• switch(config-if)# channel-group 5 channel does not already exist. All static
port-channel interfaces are set to mode on. You
• switch(config-if)# channel-group 5
force must set all LACP-enabled port-channel
interfaces to active or passive. The default
mode is on.
(Optional) Forces an interface with some
incompatible configurations to join the channel.
The forced interface must have the same speed,
duplex, and flow control settings as the channel
group.
Note The force option fails if the port has
a QoS policy mismatch with the
other members of the port channel.

Step 7 show interface type slot/port (Optional) Displays interface information.

Example:
switch# show interface port channel 5

Step 8 no shutdown (Optional) Clears the errors on the interfaces

and VLANs where policies correspond with
Example:
hardware policies. This command allows policy
switch# configure terminal programming to continue and the port to come
switch(config)# int e3/1
switch(config-if)# no shutdown
up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Step 9 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
177
 Configuring Port Channels
Adding a Layer 3 Port to a Port Channel

Example
This example shows how to add a Layer 2 Ethernet interface 1/4 to channel group 5:
switch# configure terminal
switch (config)# interface ethernet 1/4
switch(config-if)# switchport
switch(config-if)# channel-group 5

Adding a Layer 3 Port to a Port Channel

You can add a Layer 3 port to a new channel group or to a channel group that is already configured with Layer
3 ports. The software creates the port channel associated with this channel group if the port channel does not
already exist.
If the Layer 3 port that you are adding has a configured IP address, the system removes that IP address before
adding the port to the port channel. After you create a Layer 3 port channel, you can assign an IP address to
the port-channel interface.

Note Use the no channel-group command to remove the port from the channel group. The port reverts to its original
configuration. You must reconfigure the IP addresses for this port.

Command Purpose

no channel-group Removes the port from the channel group.

Example:
switch(config)# no channel-group

Before you begin

Enable LACP if you want LACP-based port channels.
Remove any IP addresses configured on the Layer 3 interface.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface type slot/port Specifies the interface that you want to add to
a channel group, and enters the interface
Example:
configuration mode.
switch(config)# interface ethernet 1/4
switch(config-if)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
178
Configuring Port Channels
Adding a Layer 3 Port to a Port Channel

Command or Action Purpose

Step 3 no switchport Configures the interface as a Layer 3 port.
Example:
switch(config-if)# no switchport

Step 4 channel-group channel-number [force] [mode Configures the port in a channel group and sets
{on | active | passive}] the mode. The channel-number range is from 1
to 4096. The Cisco NX-OS software creates the
Example:
port channel associated with this channel group
• switch(config-if)# channel-group 5 if the port channel does not already exist.
• switch(config-if)# channel-group 5 (Optional) Forces an interface with some
force incompatible configurations to join the channel.
The forced interface must have the same speed,
duplex, and flow control settings as the channel
group.

Step 5 show interface type slot/port (Optional) Displays interface information.

Example:
switch# show interface ethernet 1/4

Step 6 no shutdown (Optional) Clears the errors on the interfaces

and VLANs where policies correspond with
Example:
hardware policies. This command allows policy
switch# configure terminal programming to continue and the port to come
switch(config)# int e3/1
switch(config-if)# no shutdown
up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Step 7 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to add a Layer 3 Ethernet interface 1/5 to channel group 6 in on mode:
switch# configure terminal
switch (config)# interface ethernet 1/5
switch(config-if)# switchport
switch(config-if)# channel-group 6

This example shows how to create a Layer 3 port-channel interface and assign the IP address:
switch# configure terminal
switch (config)# interface port-channel 4
switch(config-if)# ip address 192.0.2.1/8

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
179
 Configuring Port Channels
Applying Port Channel Configuration to all Configured Port Channels

Applying Port Channel Configuration to all Configured Port Channels

You can select all the available port channel's parent interfaces that are already configured in the system and
then apply that configuration to the member ports of all the configured port channels.

Before you begin

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface port-channel all Changes the config mode to IF range.

Example:
switch(config)# interface port-channel
all
switch(config-if-range)#

Step 3 where Apply the configuration specified to all port

channels in the running-configuration.
Example:
switch(config-if-range)# where
switch(config-if-range)#

Step 4 shutdown Shuts down the local traffic on all those

interfaces selected by the command interface
Example:
port-channel all To return the interface to its
switch(config-if-range)# shutdown default operational state, use the no form of this
switch(config-if-range)#
command.

Step 5 show running-config interface port-channel Displays the running configuration of the port
all channels for which the new configuration is
applied.
Example:
switch(config-if-range)# show
running-config interface port-channel
all
switch(config-if-range)#

Example
This example shows how to apply a port channel configuration to all member port channels:

switch(config)# iinterface port-channel all

switch(config-if-range)# where
conf; interface port-channel1-2 admin@switch%default
switch(config-if-range)#
switch(config-if-range)# shutdown

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
180
 Configuring Port Channels
Configuring the Bandwidth and Delay for Informational Purposes

switch(config-if-range)#
switch(config-if-range)# sh run int po all

See the “Compatibility Requirements” section for details on how the interface configuration changes
when you delete the port channel.

Configuring the Bandwidth and Delay for Informational Purposes

The bandwidth of the port channel is determined by the number of total active links in the channel.
You configure the bandwidth and delay on port-channel interfaces for informational purposes.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface port-channel channel-number Specifies the port-channel interface that you
want to configure, and enters the interface
Example:
mode.
switch(config)# interface port-channel
2
switch(config-if)#

Step 3 bandwidth value Specifies the bandwidth, which is used for

informational purposes. The range is from 1 to
Example:
3,200,000,000 kbs. The default value depends
switch(config-if)# bandwidth 60000000 on the total active interfaces in the channel
switch(config-if)#
group.

Step 4 delay value Specifies the throughput delay, which is used

for informational purposes. The range is from
Example:
1 to 16,777,215 tens of microseconds. The
switch(config-if)# delay 10000 default value is 10 microseconds.
switch(config-if)#

Step 5 exit Exits the interface mode and returns to the

configuration mode.
Example:
switch(config-if)# exit
switch(config)#

Step 6 show interface port-channel channel-number (Optional) Displays interface information for
the specified port channel.
Example:
switch# show interface port-channel 2

Step 7 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
181
 Configuring Port Channels
Shutting Down and Restarting the Port-Channel Interface

Command or Action Purpose

switch(config)# copy running-config
startup-config

Example
This example shows how to configure the informational parameters of the bandwidth and delay for
port channel 5:
switch# configure terminal
switch (config)# interface port-channel 5
switch(config-if)# bandwidth 60000000
switch(config-if)# delay 10000
switch(config-if)#

Shutting Down and Restarting the Port-Channel Interface

You can shut down and restart the port-channel interface. When you shut down a port-channel interface, no
traffic passes and the interface is administratively down.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface port-channel channel-number Specifies the port-channel interface that you
want to configure, and enters the interface
Example:
mode.
switch(config)# interface port-channel
2
switch(config-if)#

Step 3 shutdown Shuts down the interface. No traffic passes and

the interface displays as administratively down.
Example:
The default is no shutdown.
switch(config-if)# shutdown
switch(config-if)# Note Use the no shutdown command to
open the interface.
The interface displays as
administratively up. If there are no
operational problems, traffic passes.
The default is no shutdown.

Step 4 exit Exits the interface mode and returns to the

configuration mode.
Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
182
 Configuring Port Channels
Configuring a Port-Channel Description

Command or Action Purpose

switch(config-if)# exit
switch(config)#

Step 5 show interface port-channel channel-number (Optional) Displays interface information for
the specified port channel.
Example:
switch(config-router)# show interface
port-channel 2

Step 6 no shutdown (Optional) Clears the errors on the interfaces

and VLANs where policies correspond with
Example:
hardware policies. This command allows policy
switch# configure terminal programming to continue and the port to come
switch(config)# int e3/1
switch(config-if)# no shutdown
up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Step 7 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to bring up the interface for port channel 2:
switch# configure terminal
switch (config)# interface port-channel 2
switch(config-if)# no shutdown

Configuring a Port-Channel Description

You can configure a description for a port channel.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface port-channel channel-number Specifies the port-channel interface that you
want to configure, and enters the interface
Example:
mode.
switch(config)# interface port-channel
2
switch(config-if)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
183
 Configuring Port Channels
Configuring the Speed and Duplex Settings for a Port-Channel Interface

Command or Action Purpose

Step 3 description Allows you to add a description to the
port-channel interface. You can use up to 80
Example:
characters in the description. By default, the
switch(config-if)# description description does not display; you must configure
engineering
switch(config-if)# this parameter before the description displays
in the output.

Step 4 exit Exits the interface mode and returns to the

configuration mode.
Example:
switch(config-if)# exit
switch(config)#

Step 5 show interface port-channel channel-number (Optional) Displays interface information for
the specified port channel.
Example:
switch# show interface port-channel 2

Step 6 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to add a description to port channel 2:
switch# configure terminal
switch (config)# interface port-channel 2
switch(config-if)# description engineering

Configuring the Speed and Duplex Settings for a Port-Channel Interface

You can configure the speed and duplex settings for a port-channel interface.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface port-channel channel-number Specifies the port-channel interface that you
want to configure, and enters the interface
Example:
mode.
switch(config)# interface port-channel
2
switch(config-if)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
184
 Configuring Port Channels
Configuring Load Balancing Using Port Channels

Command or Action Purpose

Step 3 speed {10 | 100 | 1000 | auto} Sets the speed for the port-channel interface.
The default is auto for autonegotiation.
Example:
switch(config-if)# speed auto
switch(config-if)#

Step 4 duplex {auto | full | half} Sets the duplex for the port-channel interface.
The default is auto for autonegotiation.
Example:
switch(config-if)# speed auto
switch(config-if)#

Step 5 exit Exits the interface mode and returns to the

configuration mode.
Example:
switch(config-if)# exit
switch(config)#

Step 6 show interface port-channel channel-number (Optional) Displays interface information for
the specified port channel.
Example:
switch# show interface port-channel 2

Step 7 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to set port channel 2 to 100 Mb/s:
switch# configure terminal
switch (config)# interface port-channel 2
switch(config-if)# speed 100

Configuring Load Balancing Using Port Channels

You can configure the load-balancing algorithm for port channels that applies to the entire device.

Note Use the no port-channel load-balance command to restore the default load-balancing algorithm of
source-dest-mac for non-IP traffic and source-dest-ip for IP traffic.

Command Purpose

no port-channel load-balance Restores the default load-balancing algorithm.

Example:
switch(config)# no port-channel load-balance

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
185
 Configuring Port Channels
Enabling LACP

Before you begin

Enable LACP if you want LACP-based port channels.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 port-channel load-balance method {dst ip | Specifies the load-balancing algorithm for the
dst ip-gre | dst ip-l4port | dst ip-l4port-vlan device. The range depends on the device. The
| dst ip-vlan | dst l4port | dst mac | src ip | src default for Layer 3 is src-dst ip-l4port for both
ip-gre | src ip-l4port | src ip-l4port-vlan | src IPv4 and IPv6, and the default for non-IP is
ip-vlan | src l4port | src mac | src-dst ip | src-dst mac.
src-dst ip-gre | src-dst ip-l4port [symmetric]
Note Only the following load-balancing
| src-dst ip-l4port-vlan | src-dst ip-vlan |
algorithms support symmetric
src-dst l4port | src-dst mac} [fex {fex-range
hashing:
| all}] [rotate rotate]
• src-dst ip
Example:
• switch(config)# port-channel • src-dst ip-l4port
load-balance src-dst mac
switch(config)#

• switch(config)# no port-channel
load-balance src-dst mac
switch(config)#

Step 3 show port-channel load-balance (Optional) Displays the port-channel

load-balancing algorithm.
Example:
switch(config-router)# show port-channel
load-balance

Step 4 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Enabling LACP
LACP is disabled by default; you must enable LACP before you begin LACP configuration. You cannot
disable LACP while any LACP configuration is present.
LACP learns the capabilities of LAN port groups dynamically and informs the other LAN ports. Once LACP
identifies correctly matched Ethernet links, it group the links into a port channel. The port channel is then
added to the spanning tree as a single bridge port.
To configure LACP, you must do the following:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
186
 Configuring Port Channels
Configuring LACP Port-Channel Port Modes

• Enable LACP globally by using the feature lacp command.

• You can use different modes for different interfaces within the same LACP-enabled port channel. You
can change the mode between active and passive for an interface only if it is the only interface that is
designated to the specified channel group.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 feature lacp Enables LACP on the device.

Example:
switch(config)# feature lacp

Step 3 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to enable LACP:
switch# configure terminal
switch (config)# feature lacp

Configuring LACP Port-Channel Port Modes

After you enable LACP, you can configure the channel mode for each individual link in the LACP port channel
as active or passive. This channel configuration mode allows the link to operate with LACP.
When you configure port channels with no associated aggregation protocol, all interfaces on both sides of the
link remain in the on channel mode.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
187
 Configuring Port Channels
Configuring LACP Port-Channel Minimum Links

Command or Action Purpose

Step 2 interface type slot/port Specifies the interface that you want to add to
a channel group, and enters the interface
Example:
configuration mode.
switch(config)# interface ethernet 1/4
switch(config-if)#

Step 3 channel-group number mode {active | on | Specifies the port mode for the link in a port
passive} channel. After LACP is enabled, you configure
each link or the entire channel as active or
Example:
passive.
switch(config-if)# channel-group 5 mode
active When you run port channels with no associated
aggregation protocol, the port-channel mode is
always on.
The default port-channel mode is on.

Step 4 show port-channel summary (Optional) Displays summary information about

the port channels.
Example:
switch(config-if)# show port-channel
summary

Step 5 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to set the LACP-enabled interface to the active port-channel mode for
Ethernet interface 1/4 in channel group 5:
switch# configure terminal
switch (config)# interface ethernet 1/4
switch(config-if)# channel-group 5 mode active

Configuring LACP Port-Channel Minimum Links

You can configure the LACP minimum links feature. Although minimum links and maxbundles work only
in LACP, you can enter the CLI commands for these features for non-LACP port channels, but these commands
are nonoperational.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
188
Configuring Port Channels
Configuring LACP Port-Channel Minimum Links

Note Use the no lacp min-links command to restore the default port-channel minimum links configuration.

Command Purpose

no lacp min-links Restores the default port-channel minimum

links configuration.
Example:
switch(config)# no lacp min-links

Before you begin

Ensure that you are in the correct port-channel interface.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface port-channel number Specifies the interface to configure, and enters
the interface configuration mode.
Example:
switch(config)# interface port-channel
3
switch(config-if)#

Step 3 lacp min-links number Specifies the port-channel interface to configure

the number of minimum links. The range is
Example:
from 1 to 16.
switch(config-if)# lacp min-links 3

Step 4 show running-config interface port-channel (Optional) Displays the port-channel minimum
number links configuration.
Example:
switch(config-if)# show running-config
interface port-channel 3

Example
This example shows how to configure the minimum number of port-channel member interfaces to
be up/active for the port-channel to be up/active:
switch# configure terminal
switch(config)# interface port-channel 3
switch(config-if)# lacp min-links 3

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
189
 Configuring Port Channels
Configuring the LACP Port-Channel MaxBundle

Configuring the LACP Port-Channel MaxBundle

You can configure the LACP maxbundle feature. Although minimum links and maxbundles work only in
LACP, you can enter the CLI commands for these features for non-LACP port channels, but these commands
are nonoperational.

Note Use the no lacp max-bundle command to restore the default port-channel max-bundle configuration.

Command Purpose

no lacp max-bundle Restores the default port-channel max-bundle

configuration.
Example:
switch(config)# no lacp max-bundle

Before you begin

Ensure that you are in the correct port-channel interface.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface port-channel number Specifies the interface to configure, and enters
the interface configuration mode.
Example:
switch(config)# interface port-channel
3
switch(config-if)#

Step 3 lacp max-bundle number Specifies the port-channel interface to configure

max-bundle.
Example:
switch(config-if)# lacp max-bundle The default value for the port-channel
max-bundle is 16. The allowed range is from 1
to 32.
Note Even if the default value is 16, the
number of active members in a port
channel is the minimum of the
pc_max_links_config and
pc_max_active_members that is
allowed in the port channel.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
190
 Configuring Port Channels
Configuring the LACP Fast Timer Rate

Command or Action Purpose

Step 4 show running-config interface port-channel (Optional) Displays the port-channel
number max-bundle configuration.
Example:
switch(config-if)# show running-config
interface port-channel 3

Example
This example shows how to configure the port channel interface max-bundle:
switch# configure terminal
switch(config)# interface port-channel 3
switch(config-if)# lacp max-bundle 3

Configuring the LACP Fast Timer Rate

You can change the LACP timer rate to modify the duration of the LACP timeout. Use the lacp rate command
to set the rate at which LACP control packets are sent to an LACP-supported interface. You can change the
timeout rate from the default rate (30 seconds) to the fast rate (1 second). This command is supported only
on LACP-enabled interfaces.

Note We do not recommend changing the LACP timer rate. HA and SSO are not supported when the LACP fast
rate timer is configured.

Note Configuring lacp rate fast is not recommended on the vPC peer-links. When lacp rate fast is configured on
the vPC peer-link member interfaces, an alert is displayed in the syslog messages only when the LACP logging
level is set to 5.

Before you begin

Ensure that you have enabled the LACP feature.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface type slot/port Specifies the interface to configure and enters
the interface configuration mode.
Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
191
 Configuring Port Channels
Configuring the LACP System Priority

Command or Action Purpose

switch(config)# interface ethernet 1/4
switch(config-if)#

Step 3 lacp rate fast Configures the fast rate (one second) at which
LACP control packets are sent to an
Example:
LACP-supported interface.
switch(config-if)# lacp rate fast
To reset the timeout rate to its default, use the
no form of the command.

Example
This example shows how to configure the LACP fast rate on Ethernet interface 1/4:
switch# configure terminal
switch (config)# interface ethernet 1/4
switch(config-if)# lacp rate fast

This example shows how to restore the LACP default rate (30 seconds) on Ethernet interface 1/4.
switch# configure terminal
switch (config)# interface ethernet 1/4
switch(config-if)# no lacp rate fast

Configuring the LACP System Priority

The LACP system ID is the combination of the LACP system priority value and the MAC address.

Before you begin

Enable LACP.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 lacp system-priority priority Configures the system priority for use with
LACP. Valid values are from 1 through 65535,
Example:
and higher numbers have a lower priority. The
switch(config)# lacp system-priority default value is 32768.
40000
Note Each VDC has a different LACP
system ID because the software adds
the MAC address to this configured
value.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
192
 Configuring Port Channels
Configuring the LACP Port Priority

Command or Action Purpose

Step 3 show lacp system-identifier (Optional) Displays the LACP system identifier.
Example:
switch(config-if)# show lacp
system-identifier

Step 4 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to set the LACP system priority to 2500:
switch# configure terminal
switch(config)# lacp system-priority 2500

Configuring the LACP Port Priority

When you enable LACP, you can configure each link in the LACP port channel for the port priority.

Before you begin

Enable LACP.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface type slot/port Specifies the interface that you want to add to
a channel group, and enters the interface
Example:
configuration mode.
switch(config)# interface ethernet 1/4
switch(config-if)#

Step 3 lacp port-priority priority Configures the port priority for use with LACP.
Valid values are from 1 through 65535, and
Example:
higher numbers have a lower priority. The
switch(config-if)# lacp port-priority default value is 32768.
40000

Step 4 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
193
 Configuring Port Channels
Configuring LACP System MAC and Role

Command or Action Purpose

switch(config-if)# copy running-config
startup-config

Example
This example shows how to set the LACP port priority for Ethernet interface 1/4 to 40000:
switch# configure terminal
switch (config)# interface ethernet 1/4
switch(config-if)# lacp port-priority 40000

Configuring LACP System MAC and Role

You can configure the MAC address used by the LACP for protocol exchanges and the optional role. By
default, the LACP uses the VDC MAC address. By default, the role is primary.
Use the no lacp system-mac command to make LACP use the default (VDC) MAC address and default role.
This procedure is supported on the Cisco Nexus 9336C-FX2, 93300YC-FX2, and 93240YC-FX2-Z switches.

Before you begin

LACP must be enabled.

Procedure

Command or Action Purpose

Step 1 configure terminal Enter global configuration mode.
Example:
switch# configure terminal

Step 2 lacp system-mac mac-address role role-value Specifies the MAC address to use in the LACP
protocol exchanges. The role is optional.
Example:
Primary is the default.
switch(config)# lacp system-mac
000a.000b.000c role primary
switch(config)# lacp system-mac
000a.000b.000c role secondary

Step 3 (Optional) show lacp system-identifier Displays the configured MAC address.
Example:
switch(config)# show lacp
system-identifier

Step 4 copy running-config startup-config Copies the running configuration to the startup
configuration.
Example:
switch(config)# copy running-config
startup-config

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
194
 Configuring Port Channels
Disabling LACP Graceful Convergence

Example
The following example shows how to configure the role of a switch as primary.
Switch1# sh lacp system-identifier
32768,0-b-0-b-0-b
Switch1# sh run | grep lacp
feature lacp
lacp system-mac 000b.000b.000b role primary

The following example shows how to configure the role of a switch as secondary.
Switch2# sh lacp system-identifier
32768,0-b-0-b-0-b
Switch2# sh run | grep lacp
feature lacp
lacp system-mac 000b.000b.000b role secondary

Disabling LACP Graceful Convergence

By default, LACP graceful convergence is enabled. In situations where you need to support LACP
interoperability with devices where the graceful failover defaults may delay the time taken for a disabled port
to be brought down or cause traffic from the peer to be lost, you can disable convergence. If the downstream
access switch is not a Cisco Nexus device, disable the LACP graceful convergence option.

Note The port channel has to be in the administratively down state before the command can be run.

Before you begin

Enable LACP.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface port-channel number Specifies the port channel interface to configure
and enters the interface configuration mode.
Example:
switch(config)# interface port-channel
1
switch(config-if)#

Step 3 shutdown Administratively shuts down the port channel.

Example:
switch(config-if) shutdown

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
195
 Configuring Port Channels
Reenabling LACP Graceful Convergence

Command or Action Purpose

Step 4 no lacp graceful-convergence Disables LACP graceful convergence on the
port channel.
Example:
switch(config-if)# no lacp
graceful-convergence

Step 5 no shutdown Brings the port channel administratively up.

Example:
switch(config-if) no shutdown

Step 6 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to disable LACP graceful convergence on a port channel:
switch# configure terminal
switch (config)# interface port-channel 1
switch(config-if)# shutdown
switch(config-if)# no lacp graceful-convergence
switch(config-if)# no shutdown

Reenabling LACP Graceful Convergence

If the default LACP graceful convergence is once again required, you can reenable convergence.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface port-channel number Specifies the port channel interface to configure
and enters the interface configuration mode.
Example:
switch(config)# interface port-channel
1
switch(config-if)#

Step 3 shutdown Administratively shuts down the port channel.

Example:
switch(config-if) shutdown

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
196
 Configuring Port Channels
Disabling LACP Suspend Individual

Command or Action Purpose

Step 4 lacp graceful-convergence Enables LACP graceful convergence on the port
channel.
Example:
switch(config-if)# lacp
graceful-convergence

Step 5 no shutdown Brings the port channel administratively up.

Example:
switch(config-if) no shutdown

Step 6 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to enable LACP graceful convergence on a port channel:
switch# configure terminal
switch (config)# interface port-channel 1
switch(config-if)# shutdown
switch(config-if)# lacp graceful-convergence
switch(config-if)# no shutdown

Disabling LACP Suspend Individual

LACP sets a port to the suspended state if it does not receive an LACP PDU from the peer. This process can
cause some servers to fail to boot up as they require LACP to logically bring up the port.

Note You should only enter the lacp suspend-individual command on edge ports. The port channel has to be in
the administratively down state before you can use this command.

Before you begin

Enable LACP.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
197
 Configuring Port Channels
Reenabling LACP Suspend Individual

Command or Action Purpose

Step 2 interface port-channel number Specifies the port channel interface to configure
and enters the interface configuration mode.
Example:
switch(config)# interface port-channel
1
switch(config-if)#

Step 3 shutdown Administratively shuts down the port channel.

Example:
switch(config-if) shutdown

Step 4 no lacp suspend-individual Disables LACP individual port suspension

behavior on the port channel.
Example:
switch(config-if)# no lacp
suspend-individual

Step 5 no shutdown Brings the port channel administratively up.

Example:
switch(config-if) no shutdown

Step 6 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to disable LACP individual port suspension on a port channel:
switch# configure terminal
switch (config)# interface port-channel 1
switch(config-if)# shutdown
switch(config-if)# no lacp suspend-individual
switch(config-if)# no shutdown

Reenabling LACP Suspend Individual

You can reenable the default LACP individual port suspension.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
198
 Configuring Port Channels
Configuring Delayed LACP

Command or Action Purpose

Step 2 interface port-channel number Specifies the port channel interface to configure
and enters the interface configuration mode.
Example:
switch(config)# interface port-channel
1
switch(config-if)#

Step 3 shutdown Administratively shuts down the port channel.

Example:
switch(config-if) shutdown

Step 4 lacp suspend-individual Enables LACP individual port suspension

behavior on the port channel.
Example:
switch(config-if)# lacp
suspend-individual

Step 5 no shutdown Brings the port channel administratively up.

Example:
switch(config-if) no shutdown

Step 6 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to reenable the LACP individual port suspension on a port channel:
switch# configure terminal
switch (config)# interface port-channel 1
switch(config-if)# shutdown
switch(config-if)# lacp suspend-individual
switch(config-if)# no shutdown

Configuring Delayed LACP

The delayed LACP feature enables one port channel member, the delayed LACP port, to come up first as a
member of a regular port channel before LACP PDUs are received. You configure the delayed LACP feature
using the lacp mode delaycommand on a port channel followed by configuring the LACP port priority on a
one member port of the port channel.

Note For vPC, you must enable the delayed LACP on both vPC switches.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
199
 Configuring Port Channels
Configuring Delayed LACP

Note For vPC, when the delayed LACP port is on the primary switch and the primary switch fails to boot, you need
to remove the vPC configuration on the delayed LACP port-channel of the acting primary switch and flap the
port-channel for a new port to be chosen as the delayed LACP port on the existing port-channel.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.

Step 2 interface port-channel number Specifies the port channel interface to configure
and enters the interface configuration mode.

Step 3 lacp mode delay Enables delayed LACP.

Note To disable delayed LACP, use the
no lacp mode delay command.

Complete the configuration of the delayed

LACP by configuring the LACP port priority.
See the "Configuring the LACP Port Priority"
section for details.
The priority of a LACP port determines the
election of the delayed LACP port. The port
with the lowest numerical priority is elected.
When two or more ports have the same best
priority, the VDC system MAC is used to
determine which vPC is used. Then within a
non-vPC switch or the elected vPC switch, the
smallest of the ethernet port names is used.
When the delayed LACP feature is configured
and made effective with a port channel flap, the
delayed LACP port operates as a member of a
regular port channel, allowing data to be
exchanged between the server and switch. After
receiving the first LACP PDU, the delayed
LACP port transitions from a regular port
member to a LACP port member.
Note The election of the delayed LACP
port is not complete or effective until
the port channel flaps on the switch
or at a remote server.

Example
The following example configures delayed LACP.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
200
 Configuring Port Channels
Configuring Port Channel Hash Distribution

switch# config terminal

switch(config)# interface po 1
switch(config-if)# lacp mode delay

switch# config terminal

switch(config)# interface ethernet 1/1
switch(config-if)# lacp port-priority 1
switch(config-if)# channel-group 1 mode active

The following example disables delayed LACP.

switch# config terminal

switch(config)# interface po 1
switch(config-if)# no lacp mode delay

Configuring Port Channel Hash Distribution

Cisco NX-OS supports the adaptive and fixed hash distribution configuration for both global and port-channel
levels. This option minimizes traffic disruption by minimizing Result Bundle Hash (RBH) distribution changes
when members come up or go down so that flows that are mapped to unchange RBH values continue to flow
through the same links. The port-channel level configuration overrules the global configuration. The default
configuration is adaptive globally, and there is no configuration for each port channel, so there is no change
during an ISSU. No ports are flapped when the command is applied, and the configuration takes effect at the
next member link change event. Both modes work with RBH module or non-module schemes.
During an ISSD to a lower version that does not support this feature, you must disable this feature if the fixed
mode command is being used globally or if there is a port-channel level configuration.

Configuring Port Channel Hash Distribution at the Global Level

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 no port-channel hash-distribution {adaptive Specifies the port-channel hash distribution at

| fixed} the global level.
Example: The default is adaptive mode.
switch(config)# port-channel The command does not take effect until the next
hash-distribution adaptive
switch(config)#
member link event (link down/up/no
shutdown/shutdown). (Do you still want to
continue(y/n)? [yes])

Step 3 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
201
 Configuring Port Channels
Configuring Port Channel Hash Distribution at the Port Channel Level

Command or Action Purpose

switch(config)# copy running-config
startup-config

Example
This example shows how to configure hash distribution at the global level:
switch# configure terminal
switch(config)# no port-channel hash-distribution fixed

Configuring Port Channel Hash Distribution at the Port Channel Level

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface port-channel {channel-number | Specifies the interface to configure, and enters
range} the interface configuration mode.
Example:
switch# interface port-channel 4
switch(config-if)#

Step 3 no port-channel port hash-distribution Specifies the port-channel hash distribution at

{adaptive | fixed} the port channel level.
Example: There is no default.
switch(config-if)# port-channel port The command does not take effect until the next
hash-distribution adaptive
switch(config-if)
member link event (link down/up/no
shutdown/shutdown). (Do you still want to
continue(y/n)? [yes])

Step 4 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch(config)# copy running-config
startup-config

Example
This example shows how to configure hash distribution as a global-level command:
switch# configure terminal
switch(config)# no port-channel hash-distribution fixed

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
202
 Configuring Port Channels
Enabling ECMP Resilient Hashing

Enabling ECMP Resilient Hashing

Resilient ECMP ensures minimal impact to the existing flows when members are deleted from an ECMP
group. This is achieved by replicating the existing members in a round-robin fashion at the indices that were
previously occupied by the deleted members.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal

Step 2 hardware profile ecmp resilient Enables ECMP resilient hashing and displays
the following: Warning: The command will
Example:
take effect after next reload.
switch(config)# hardware profile ecmp
resilient

Step 3 copy running-config startup-config Copies the running configuration to the startup
configuration.
Example:
switch(config)# copy running-config
startup-config

Step 4 reload Reboots the switch.

Example:
switch(config)# reload

Disabling ECMP Resilient Hashing

Before you begin
ECMP resilient hashing is enabled.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal

Step 2 no hardware profile ecmp resilient Disables ECMP resilient hashing and displays
the following: Warning: The command will
Example:
take effect after next reload.
switch(config)# no hardware profile ecmp
resilient

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
203
 Configuring Port Channels
Configuring ECMP Load Balancing based on Inner NVGRE Header

Command or Action Purpose

Step 3 copy running-config startup-config Copies the running configuration to the startup
configuration.
Example:
switch(config)# copy running-config
startup-config

Step 4 reload Reboots the switch.

Example:
switch(config)# reload

Configuring ECMP Load Balancing based on Inner NVGRE Header

Configuring the ECMP load-sharing algorithm based on inner NVGRE header enables hashing for Layer 3
ECMP paths and port-channels. To configure the ECMP load-sharing algorithm based on inner NVGRE
header, use the following command in global configuration mode:

Note ECMP load balancing based on inner NVGRE header is supported only on N9K-C9504-FM-E,
N9K-C9508-FM-E, N9K-C9516-FM-E, N9K-C9516-FM-E2 devices.

Procedure

Command or Action Purpose

Step 1 ip load-sharing address { source-destination Configures the ECMP load-sharing algorithm
[port source-destination | inner all for data traffic.
infra-mode]} [universal-id seed ] [rotate
The following options are available for IP load
rotate ] [concatenation]
sharing configuration based on inner NVGRE
Example: header:
ip load-sharing address • The universal-id option sets the random
source-destination port
source-destination inner all infra-mode seed for the hash algorithm and shifts the
flow from one link to another.
You do not need to configure the universal
ID. Cisco NX-OS chooses the universal
ID if you do not configure it. The
universal-id range is from 1 to
4294967295.
• The rotate option causes the hash
algorithm to rotate the link picking
selection so that it does not continually
choose the same link across all nodes in
the network. It does so by influencing the
bit pattern for the hash algorithm. This
option shifts the flow from one link to
another and load balances the already

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
204
 Configuring Port Channels
Verifying the ECMP Resilient Hashing Configuration

Command or Action Purpose

load-balanced (polarized) traffic from the
first ECMP level across multiple links.
If you specify a rotate value, the 64-bit
stream is interpreted starting from that bit
position in a cyclic rotation. The rotate
range is from 1 to 63, and the default is 32.
• The concatenation option ties together
the hash tag values for ECMP and the hash
tag values for port channels in order to use
a stronger 64-bit hash. If you do not use
this option, you can control ECMP
load-balancing and port-channel
load-balancing independently. The default
is disabled.

The infra-mode option is not needed for Cisco

Nexus N9K-C9516-FM-E2 based switch.
Caution The VXLAN and infra-mode option
cannot coexist. You must disable
feature nv overlay before enabling
this option.

Caution This configuration enables inner

header based hashing for GRE traffic
on port channel and ECMP.

Step 2 (Optional) show ip load-sharing Displays the ECMP load-sharing algorithm for
data traffic.
Example:
switch(config)# show ip load-sharing
IPv4/IPv6 ECMP load sharing:
Universal-id (Random Seed): 398176901
Load-share mode : address
source-destination port
source-destination inner all infra-mode
GRE-Outer hash is disabled
Concatenation is disabled
Rotate: 32

Verifying the ECMP Resilient Hashing Configuration

To display ECMP Resilient Hashing configuration information, perform one of the following tasks:

Command Purpose
switch(config)# show running-config | grep Displays the enabled status.
"hardware profile ecmp resilient
hardware profile ecmp resilient
switch(config)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
205
 Configuring Port Channels
Verifying the Port-Channel Configuration

Command Purpose
switch(config)# show running-config | grep Displays the disabled status.
"hardware profile ecmp resilient
switch(config)#

Verifying the Port-Channel Configuration

To display port-channel configuration information, perform one of the following tasks:

Command Purpose

show interface port-channel channel-number Displays the status of a port-channel interface.

show feature Displays enabled features.

load- interval {interval seconds {1 | 2 | 3}} Sets three different sampling intervals to bit-rate and
packet-rate statistics.

show port-channel compatibility-parameters Displays the parameters that must be the same among
the member ports in order to join a port channel.

show port-channel database [interface Displays the aggregation state for one or more
port-channel channel-number] port-channel interfaces.

show port-channel load-balance Displays the type of load balancing in use for port
channels.

show port-channel summary Displays a summary for the port-channel interfaces.

show port-channel traffic Displays the traffic statistics for port channels.

show port-channel usage Displays the range of used and unused channel
numbers.

show lacp {counters [interface port-channel Displays information about LACP.

channel-number] | [interface type/slot] | neighbor
[interface port-channel channel-number] |
port-channel [interface port-channel
channel-number] | system-identifier]]}

show running-config interface port-channel Displays information about the running configuration
channel-number of the port-channel.

Monitoring the Port-Channel Interface Configuration

Use the following commands to display port-channel interface configuration information.

Command Purpose

clear counters interface port-channel Clears the counters.

channel-number

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
206
 Configuring Port Channels
Example Configurations for Port Channels

Command Purpose

clear lacp counters [interface port-channel Clears the LACP counters.

channel-number]

load- interval {interval seconds {1 | 2 | 3}} Sets three different sampling intervals to bit-rate and
packet-rate statistics.

show interface counters [module module] Displays input and output octets unicast packets,
multicast packets, and broadcast packets.

show interface counters detailed [all] Displays input packets, bytes, and multicast and output
packets and bytes.

show interface counters errors [module module] Displays information about the number of error
packets.

show lacp counters Displays statistics for LACP.

Example Configurations for Port Channels

This example shows how to create an LACP port channel and add two Layer 2 interfaces to that port channel:

switch# configure terminal

switch (config)# feature lacp
switch (config)# interface port-channel 5
switch (config-if)# interface ethernet 1/4
switch(config-if)# switchport
switch(config-if)# channel-group 5 mode active
switch(config-if)# lacp port priority 40000
switch(config-if)# interface ethernet 1/7
switch(config-if)# switchport
switch(config-if)# channel-group 5 mode

This example shows how to add two Layer 3 interfaces to a channel group. The Cisco NX-OS software
automatically creates the port channel:

switch# configure terminal

switch (config)# interface ethernet 1/5
switch(config-if)# no switchport
switch(config-if)# no ip address
switch(config-if)# channel-group 6 mode active
switch (config)# interface ethernet 2/5
switch(config-if)# no switchport
switch(config-if)# no ip address
switch(config-if)# channel-group 6 mode active
switch (config)# interface port-channel 6
switch(config-if)# ip address 192.0.2.1/8

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
207
 Configuring Port Channels
Related Documents

Related Documents
Related Topic Document Title

System management Cisco Nexus 9000 Series NX-OS System Management

Configuration Guide

High availability Cisco Nexus 9000 Series NX-OS High Availability

and Redundancy Guide

Licensing Cisco NX-OS Licensing Guide

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
208
 CHAPTER 8
Configuring vPCs
This chapter describes how to configure virtual port channels (vPCs) on Cisco NX-OS devices.
• Information About vPCs, on page 209
• Guidelines and Limitations, on page 235
• Best Practices for Layer 3 and vPC Configuration, on page 238
• Default Settings, on page 247
• Configuring vPCs, on page 247
• Verifying the vPC Configuration, on page 275
• Monitoring vPCs, on page 276
• Configuration Examples for vPCs, on page 276
• Related Documents, on page 278

Information About vPCs

vPC Overview
A virtual port channel (vPC) allows links that are physically connected to two different Cisco Nexus 9000
Series devices to appear as a single port channel by a third device (see figure). The third device can be a
switch, server, or any other networking device that supports port channels. A vPC can provide Layer 2
multipathing, which allows you to create redundancy and increase the bisectional bandwidth by enabling
multiple parallel paths between nodes and allowing load balancing traffic.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
209
 Configuring vPCs
vPC Overview

Figure 11: vPC Architecture

You can use only Layer 2 port channels in the vPC. You configure the port channels by using one of the
following:
• No protocol
• Link Aggregation Control Protocol (LACP)

When you configure the port channels in a vPC—including the vPC peer link channel—without using LACP,
each device can have up to 32 active links in a single port channel. When you configure the port channels in
a vPC—including the vPC peer link channels—using LACP, each device can have 32 active links and eight
standby links in a single port channel. (See the “vPC Interactions with Other Features” section for more
information on using LACP and vPCs.)

Note You must enable the vPC feature before you can configure or run the vPC functionality.

After you enable the vPC functionality, you create the peer-keepalive link, which sends heartbeat messages
between the two vPC peer devices.
You can create a vPC peer link by configuring a port channel on one Cisco Nexus 9000 Series chassis by
using two or more Ethernet ports higher speed than 1-Gigabit Ethernet. To ensure that you have the correct
hardware to enable and run a vPC, enter the show hardware feature-capability command. If you see an X
across from the vPC in your command output, your hardware cannot enable the vPC feature.
We recommend that you configure the vPC peer link Layer 2 port channels as trunks. On another Cisco Nexus
9000 Series chassis, you configure another port channel again using two or more Ethernet ports with spped
higher than 1-Gigabit in the dedicated port mode. Connecting these two port channels creates a vPC peer link
in which the two linked Cisco Nexus devices appear as one device to a third device. The third device, or
downstream device, can be a switch, server, or any other networking device that uses a regular port channel
to connect to the vPC.
For modular Cisco Nexus 9500 switches, we recommend that you configure the vPC peer links on dedicated
ports of different modules to reduce the possibility of a failure. For the best resiliency scenario, use at least
two modules.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
210
 Configuring vPCs
Hitless vPC Role Change

You can use any of the interfaces of the Nexus 9000 device for the vPC peer link. If you must configure all
the vPC peer links and core-facing interfaces on a single module, you should configure a track object that is
associated with the Layer 3 link to the core and on all the links on the vPC peer link on both vPC peer devices.
The vPC domain includes both vPC peer devices, the vPC peer-keepalive link, the vPC peer link, and all of
the port channels in the vPC domain connected to the downstream device. You can have only one vPC domain
ID on each device.
In this version, you can connect each downstream device to a single vPC domain ID using a single port channel.

Note Devices attached to a vPC domain using port channels should be connected to both of vPC peers.

A vPC (see figure) provides the following benefits:

• Allows a single device to use a port channel across two upstream devices
• Eliminates Spanning Tree Protocol (STP) blocked ports
• Provides a loop-free topology
• Uses all available uplink bandwidth
• Provides fast convergence if either the link or a device fails
• Provides link-level resiliency
• Assures high availability

Figure 12: vPC Interfaces

Hitless vPC Role Change

A virtual port channel (vPC) allows links that are physically connected to two different Cisco Nexus 9000
Series devices to appear as a single port channel. The vPC role change feature enables you switch vPC roles
between vPC peers without impacting traffic flow. The vPC role switching is done based on the role priority

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
211
 Configuring vPCs
vPC Terminology

value of the device under the vPC domain. A vPC peer device with lower role priority is selected as the primary
vPC device during the vPC Role switch. You can use the vpc role preempt command to switch vPC role
between peers.
For information about how to configure Hitless vPC Role Change, see Configuring Hitless vPC Role Change,
on page 273.

vPC Terminology
The terminology used in vPCs is as follows:
• vPC—The combined port channel between the vPC peer devices and the downstream device.
• vPC peer device—One of a pair of devices that are connected with the special port channel known as
the vPC peer link.
• vPC peer link—The link used to synchronize states between the vPC peer devices. Both ends must be
on 10-Gigabit Ethernet or 40-Gigabit Ethernet interfaces.
• vPC member port—An interface that belongs to a vPC.
• Host vPC port—A Fabric Extender host interfaces that belongs to a vPC.
• vPC domain—This domain includes both vPC peer devices, the vPC peer-keepalive link, and all of the
port channels in the vPC connected to the downstream devices. It is also associated to the configuration
mode that you must use to assign vPC global parameters.
• vPC peer-keepalive link—The peer-keepalive link monitors the vitality of a vPC peer Cisco Nexus 9000
Series device. The peer-keepalive link sends configurable, periodic keepalive messages between vPC
peer devices.
We recommend that you associate a peer-keepalive link to a separate virtual routing and forwarding
(VRF) instance that is mapped to a Layer 3 interface in each vPC peer device. If you do not configure a
separate VRF, the system uses the management VRF by default. However, if you use the management
interfaces for the peer-keepalive link, you must put a management switch connected to both the active
and standby management ports on each vPC peer device (see figure).

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
212
 Configuring vPCs
vPC Peer Link Overview

Figure 13: Separate Switch Required to Connect Management Ports for vPC Peer-Keepalive Link

No data or synchronization traffic moves over the vPC peer-keepalive link; the only traffic on this link
is a message that indicates that the originating switch is operating and running a vPC.
• vPC member port—Interfaces that belong to the vPCs.
• Dual-active— Both vPC peers act as primary. This situation occurs when the peer-keepalive and peer-link
go down when both the peers are still active. In this case, the secondary vPC assumes that the primary
vPC is inactive and acts as the primary vPC.
• Recovery—When the peer-keepalive and the peer-link come up, one switch becomes the secondary vPC.
On the switch that becomes the secondary vPC, the vPC links go down and come back up.

vPC Peer Link Overview

You can have only two devices as vPC peers; each device can serve as a vPC peer to only one other vPC peer.
The vPC peer devices can also have non-vPC links to other devices.
See the following figure for invalid vPC peer configurations.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
213
 Configuring vPCs
vPC Peer Link Overview

Figure 14: vPC Peer Configurations That Are Not Allowed

To make a valid configuration, you first configure a port channel on each device and then configure the vPC
domain. You assign the port channel on each device as a peer link, using the same vPC domain ID. For
redundancy, we recommend that you should configure at least two of the dedicated ports into the port channel
because if one of the interfaces in the vPC peer link fails, the device automatically falls back to use another
interface in the peer link.

Note We recommend that you configure the Layer 2 port channels in trunk mode.

Many operational parameters and configuration parameters must be the same in each device connected by a
vPC peer link (see the Compatibility Parameters for vPC Interfaces section). Because each device is completely
independent on the management plane, you must ensure that the devices are compatible on the critical
parameters. vPC peer devices have separate control planes. After configuring the vPC peer link, you should
display the configuration on each vPC peer device to ensure that the configurations are compatible.

Note You must ensure that the two devices connected by the vPC peer link have certain identical operational and
configuration parameters. For more information on required configuration consistency, see the Compatibility
Parameters for vPC Interfaces section.

When you configure the vPC peer link, the vPC peer devices negotiate that one of the connected devices is
the primary device and the other connected device is the secondary device (see the “Configuring vPCs”
section). The Cisco NX-OS software uses the lowest MAC address to elect the primary device. The software
takes different actions on each device—that is, the primary and secondary—only in certain failover conditions.
If the primary device fails, the secondary device becomes the new primary device when the system recovers,
and the previously primary device is now the secondary device.
You can also configure which of the vPC devices is the primary device. Changing the priority of the vPC peer
devices can cause the interfaces in your network to go up and down. If you want to configure the role priority
again to make one vPC device the primary device, configure the role priority on both the primary vPC device
with a lower priority value and the secondary vPC device with the higher value. Then, shut down the port
channel that is the vPC peer link on both devices by entering the shutdown command, and finally reenable
the port channel on both devices by entering the no shutdown command.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
214
 Configuring vPCs
Features That You Must Manually Configure on the Primary and Secondary Devices

Note We recommend that you use two different modules for redundancy on each vPC peer device on each vPC
peer link.

The software keeps all traffic that forwards across the vPC peer devices as local traffic. A packet that ingresses
the port channel uses one of the local links rather than moving across the vPC peer link. Unknown unicast,
multicast, and broadcast traffic (including STP BPDUs) are flooded across the vPC peer link. The software
keeps the multicast forwarding state synchronized on both of the vPC peer devices.
You can configure any of the standard load-balancing schemes on both the vPC peer link devices and the
downstream device (see the Configuring Port Channels chapter for information about load balancing).
Configuration information flows across the vPC peer links using the Cisco Fabric Services over Ethernet
(CFSoE) protocol. (See the CFSoE, on page 231 section for more information about CFSoE.)
All MAC addresses for those VLANs configured on both devices are synchronized between vPC peer devices.
The software uses CFSoE for this synchronization. (See the CFSoE, on page 231 section for information about
CFSoE.)
If the vPC peer link fails, the software checks the status of the remote vPC peer device using the peer-keepalive
link, which is a link between vPC peer devices that ensures that both devices are up. If the vPC peer device
is up, the secondary vPC device disables all vPC ports on its device, to prevent loops and disappearing or
flooding traffic. The data then forwards down the remaining active links of the port channel.
The software learns of a vPC peer device failure when the keepalive messages are not returned over the
peer-keepalive link.
Use a separate link (vPC peer-keepalive link) to send configurable keepalive messages between the vPC peer
devices. The keepalive messages on the vPC peer-keepalive link determines whether a failure is on the vPC
peer link only or on the vPC peer device. The keepalive messages are used only when all the links in the peer
link fail. See the “Peer-Keepalive Link and Messages” section for information about the keepalive message.

Features That You Must Manually Configure on the Primary and Secondary Devices
You must manually configure the following features to conform to the primary/secondary mapping of each
of the vPC peer devices:
• STP root—Configure the primary vPC peer device as the STP primary root device and configure the
vPC secondary device to be the STP secondary root device. See the “vPC Peer Links and STP” section
for more information about vPCs and STP.
• We recommend that you configure the vPC peer link interfaces as STP network ports so that Bridge
Assurance is enabled on all vPC peer links.
• We recommend that you configure Rapid per VLAN Spanning Tree plus (PVST+) so that the
primary device is the root for all VLANs and configure Multiple Spanning Tree (MST) so that the
primary device is the root for all instances.

• Layer 3 VLAN network interface—Configure Layer 3 connectivity from each vPC peer device by
configuring a VLAN network interface for the same VLAN from both devices.
• HSRP active—If you want to use Hot Standby Router Protocol (HSRP) and VLAN interfaces on the
vPC peer devices, configure the primary vPC peer device with the HSRP active highest priority. Configure
the secondary device to be the HSRP standby and ensure that you have VLAN interfaces on each vPC

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
215
 Configuring vPCs
Configuring Layer 3 Backup Routes on a vPC Peer Link

device that are in the same administrative and operational mode. (See the “vPC Peer Links and Routing”
section for more information on vPC and HSRP.)

While you configure Unidirectional Link Detection (UDLD), note the following recommendations:
• If LACP is used as port-channel aggregation protocol, UDLD is not required in a vPC domain.
• If LACP is not used as the port-channel aggregation protocol (static port-channel), use UDLD in normal
mode on vPC member ports.
• If STP is used without Bridge Assurance and if LACP is not used, use UDLD in normal mode on vPC
orphan ports.

Configuring Layer 3 Backup Routes on a vPC Peer Link

You can use VLAN network interfaces on the vPC peer devices to link to Layer 3 of the network for such
applications as HSRP and PIM. Ensure that you have a VLAN network interface configured on each peer
device and that the interface is connected to the same VLAN on each device. Also, each VLAN interface must
be in the same administrative and operational mode. For more information about configuring VLAN network
interfaces, see the “Configuring Layer 3 Interfaces” chapter.
If a failover occurs on the vPC peer link, the VLAN interfaces on the vPC peer devices are also affected. If
a vPC peer link fails, the system brings down associated VLAN interfaces on the secondary vPC peer device.
You can ensure that specified VLAN interfaces do not go down on the vPC secondary device when the vPC
peer link fails.

Peer-Keepalive Link and Messages

The Cisco NX-OS software uses the peer-keepalive link between the vPC peers to transmit periodic,
configurable keepalive messages. You must have Layer 3 connectivity between the peer devices to transmit
these messages; the system cannot bring up the vPC peer link unless the peer-keepalive link is already up and
running.

Note We recommend that you associate the vPC peer-keepalive link to a separate VRF mapped to a Layer 3 interface
in each vPC peer device. If you do not configure a separate VRF, the system uses the management VRF and
management ports by default. Do not use the peer link itself to send and receive vPC peer-keepalive messages.

If one of the vPC peer devices fails, the vPC peer device on the other side of the vPC peer link senses the
failure by not receiving any peer-keepalive messages. The default interval time for the vPC peer-keepalive
message is 1 second, and you can configure the interval between 400 milliseconds and 10 seconds.
You can configure a hold-timeout value with a range of 3 to 10 seconds; the default hold-timeout value is 3
seconds. This timer starts when the vPC peer link goes down. During this hold-timeout period, the secondary
vPC peer device ignores vPC peer-keepalive messages, which ensures that network convergence occurs before
a vPC action takes place. The purpose of the hold-timeout period is to prevent false-positive cases.
You can also configure a timeout value with a range of 3 to 20 seconds; the default timeout value is 5 seconds.
This timer starts at the end of the hold-timeout interval. During the timeout period, the secondary vPC peer
device checks for vPC peer-keepalive hello messages from the primary vPC peer device. If the secondary
vPC peer device receives a single hello message, that device disables all vPC interfaces on the secondary vPC
peer device.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
216
 Configuring vPCs
vPC Peer-Gateway

The difference between the hold-timeout and the timeout parameters is as follows:
• During the hold-timeout, the vPC secondary device does not take any action based on any keepalive
messages received, which prevents the system taking action when the keepalive might be received just
temporarily, such as if a supervisor fails a few seconds after the peer link goes down.
• During the timeout, the vPC secondary device takes action to become the vPC primary device if no
keepalive message is received by the end of the configured interval.

See the “Configuring vPC Keepalive Link and Messages” section for information about configuring the timer
for the keepalive messages.

Note Ensure that both the source and destination IP addresses used for the peer-keepalive messages are unique in
your network and these IP addresses are reachable from the VRF associated with the vPC peer-keepalive link.
Peer-keepalive IP addresses must be global unicast addresses. Link-local addresses are not supported.

Use the command-line interface (CLI) to configure the interfaces you are using the vPC peer-keepalive
messages as trusted ports. Leave the precedence at the default (6) or configure it higher.

vPC Peer-Gateway
You can configure vPC peer devices to act as the gateway even for packets that are destined to the vPC peer
device’s MAC address.
Use the peer-gateway command to configure this feature.

Note The peer-gateway exclude-vlan command that is used when configuring a VLAN interface for Layer 3
backup routing on vPC peer devices is not supported.

Some network-attached storage (NAS) devices or load balancers might have features that help to optimize
the performances of particular applications. These features enable the device to avoid a routing-table lookup
when responding to a request that originated from a host that is not locally attached to the same subnet. Such
devices might reply to traffic using the MAC address of the sender Cisco Nexus 9000 Series device rather
than the common HSRP gateway. This behavior is noncomplaint with some basic Ethernet RFC standards.
Packets that reach a vPC device for the nonlocal router MAC address are sent across the peer link and could
be dropped by the built in vPC loop avoidance mechanism if the final destination is behind another vPC.
The vPC peer-gateway capability allows a vPC switch to act as the active gateway for packets that are addressed
to the router MAC address of the vPC peer. This feature enables local forwarding of packets without the need
to cross the vPC peer link. In this scenario, the feature optimizes use of the peer link and avoids potential
traffic loss.
Configuring the peer-gateway feature must be done on both primary and secondary vPC peers and is
nondisruptive to the operations of the device or to the vPC traffic. The vPC peer-gateway feature can be
configured globally under the vPC domain submode.
When you enable this feature, Cisco NX-OS automatically disables IP redirects on all interface VLANs
mapped over a vPC VLAN to avoid generation of IP redirect messages for packets switched through the peer
gateway router.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
217
 Configuring vPCs
vPC Domain

Packets that arrive at the peer-gateway vPC device have their Time to Live (TTL) decremented, so that packets
carrying a TTL of 1 might get dropped in transit due to TTL expiration. You should take this situation into
account when the peer-gateway feature is enabled and particular network protocols that source packets with
a TTL of 1 operate on a vPC VLAN.

vPC Domain
You can use the vPC domain ID to identify the vPC peer links and the ports that are connected to the vPC
downstream devices.
The vPC domain is also a configuration mode that you use to configure the keepalive messages and other
vPC peer link parameters rather than accept the default values. See the “Configuring vPCs” section for more
information about configuring these parameters.
To create a vPC domain, you must first create a vPC domain ID on each vPC peer device using a number
from 1 to 1000. You can have only one vPC domain per vPC peer.
You must explicitly configure the port channel that you want to act as the peer link on each device. You
associate the port channel that you made a peer link on each device with the same vPC domain ID to form a
single vPC domain. Within this domain, the system provides a loop-free topology and Layer 2 multipathing.
You can only configure these port channels and vPC peer links statically. You can configure the port channels
and vPC peer links either using LACP or no protocol. We recommend that you use LACP with the interfaces
in active mode to configure port channels in each vPC, which ensures an optimized, graceful recovery in a
port-channel failover scenario and provides configuration checks against configuration mismatches among
the port channels themselves.
The vPC peer devices use the vPC domain ID that you configure to automatically assign a unique vPC system
MAC address. Each vPC domain has a unique MAC address that is used as a unique identifier for the specific
vPC-related operations, although the devices use the vPC system MAC addresses only for link-scope operations,
such as LACP. We recommend that you create each vPC domain within the contiguous Layer 2 network with
a unique domain ID. You can also configure a specific MAC address for the vPC domain, rather than having
the Cisco NX-OS software assign the address.
See the “vPC and Orphan Ports” section for more information about displaying the vPC MAC table.
After you create a vPC domain, the Cisco NX-OS software creates a system priority for the vPC domain. You
can also configure a specific system priority for the vPC domain.

Note When manually configuring the system priority, you must ensure that you assign the same priority value on
both vPC peer devices. If the vPC peer devices have different system priority values, vPC does not come up.

vPC Topology
The following figure shows a basic configuration in which the Cisco Nexus 9000 Series device ports are
directly connected to another switch or host and are configured as part of a port channel that becomes part of
a vPC.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
218
Configuring vPCs
vPC Topology

Figure 15: Switch vPC Topology

In the figure, vPC 20 is configured on port channel 20, which has Eth1/10 on the first device and Eth2/1 on
the second as member ports.
You can configure a vPC from the peer devices through Fabric Extenders (FEXs) as shown in the figure.
Figure 16: FEX Straight-Through Topology (Host vPC)

In the figure, each FEX is single-homed (straight-through FEX topology) with a Cisco Nexus 9000 Series
device. The host interfaces on this FEX are configured as port channels and those port channels are configured
as vPCs. Eth101/1/1 and Eth102/1/5 are configured as members of PO200, and PO200 is configured for vPC
200.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
219
 Configuring vPCs
Compatibility Parameters for vPC Interfaces

In both topologies, port channels P020 and P0200 must be configured identically on the peer switches and
configuration synchronization is used to synchronize the configurations of the vPC switches.
See the Cisco Nexus 2000 Series NX-OS Fabric Extender Configuration Guide for Cisco Nexus 9000 Series
Switches for more information about configuring FEX ports.

Compatibility Parameters for vPC Interfaces

Many configuration and operational parameters must be identical on all interfaces in the vPC. We recommend
that you configure the Layer 2 port channels that you use for the vPC peer link in trunk mode.
After you enable the vPC feature and configure the peer link on both vPC peer devices, Cisco Fabric Services
(CFS) messages provide a copy of the configuration on the local vPC peer device configuration to the remote
vPC peer device. The system then determines whether any of the crucial configuration parameters differ on
the two devices. (See the “vPC and Orphan Ports” section for more information about CFS.)

Note Enter the show vpc consistency-parameterscommand to display the configured values on all interfaces in
the vPC. The displayed configurations are only those configurations that would limit the vPC peer link and
vPC from coming up.

Note The port channel compatibility parameters must be the same for all the port channel members on the physical
switch. You cannot configure shared interfaces to be part of a vPC.

The compatibility check process for vPCs differs from the compatibility check for regular port channels.
See the “Configuring Port Channels” chapter for information about regular port channels.

Configuration Parameters That Must Be Identical

The configuration parameters in this section must be configured identically on both devices of the vPC peer
link; otherwise, the vPC moves fully or partially into a suspended mode.

Note You must ensure that all interfaces in the vPC have the identical operational and configuration parameters
listed in this section.

Note Enter the show vpc consistency-parameters command to display the configured values on all interfaces in
the vPC. The displayed configurations are only those configurations that would limit the vPC peer link and
vPC from coming up.

The devices automatically check for compatibility for some of these parameters on the vPC interfaces. The
per-interface parameters must be consistent per interface, and the global parameters must be consistent globally:
• Port-channel mode: on, off, or active (port-channel mode can, however, be active/passive on each side
of the vPC peer)

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
220
 Configuring vPCs
Configuration Parameters That Should Be Identical

• Link speed per channel

• Duplex mode per channel
• Trunk mode per channel:
• Native VLAN
• VLANs allowed on trunk
• Tagging of native VLAN traffic

• Spanning Tree Protocol (STP) mode

• STP region configuration for Multiple Spanning Tree
• Enable/disable state per VLAN
• STP global settings:
• Bridge Assurance setting
• Port type setting
• Loop Guard settings

• STP interface settings:

• Port type setting
• Loop Guard
• Root Guard

• Maximum Transmission Unit (MTU)

If any of these parameters are not enabled or defined on either device, the vPC consistency check ignores
those parameters.

Note To ensure that none of the vPC interfaces are in the suspend mode, enter the show vpc brief and show vpc
consistency-parameters commands and check the syslog messages.

Configuration Parameters That Should Be Identical

When any of the following parameters are not configured identically on both vPC peer devices, a
misconfiguration might cause undesirable behavior in the traffic flow:
• MAC aging timers
• Static MAC entries
• VLAN interface—Each device on the end of the vPC peer link must have a VLAN interface configured
for the same VLAN on both ends and they must be in the same administrative and operational mode.
Those VLANs configured on only one device of the peer link do not pass traffic using the vPC or peer
link. You must create all VLANs on both the primary and secondary vPC devices, or the VLAN will be
suspended.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
221
 Configuring vPCs
Consequences of Parameter Mismatches

• All ACL configurations and parameters

• Quality of Service (QoS) configuration and parameters
• STP interface settings:
• BPDU Filter
• BPDU Guard
• Cost
• Link type
• Priority
• VLANs (Rapid PVST+)

• Port security
• Cisco Trusted Security (CTS)
• Dynamic Host Configuration Protocol (DHCP) snooping
• Network Access Control (NAC)
• Dynamic ARP Inspection (DAI)
• IP source guard (IPSG)
• Internet Group Management Protocol (IGMP) snooping
• Hot Standby Routing Protocol (HSRP)
• Protocol Independent Multicast (PIM)
• All routing protocol configurations

To ensure that all the configuration parameters are compatible, we recommend that you display the
configurations for each vPC peer device once you configure the vPC.

Consequences of Parameter Mismatches

You can configure the graceful consistency check feature, which suspends only the links on the secondary
peer device when a mismatch is introduced in a working vPC. This feature is configurable only in the CLI
and is enabled by default.
The graceful consistency-check command is configured by default.
As part of the consistency check of all parameters from the list of parameters that must be identical, the system
checks the consistency of all VLANs.
The vPC remains operational, and only the inconsistent VLANs are brought down. This per-VLAN consistency
check feature cannot be disabled and does not apply to Multiple Spanning Tree (MST) VLANs.

vPC Number
Once you have created the vPC domain ID and the vPC peer link, you create port channels to attach the
downstream device to each vPC peer device. That is, you create one port channel to the downstream device

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
222
 Configuring vPCs
Moving Other Port Channels into a vPC

from the primary vPC peer device and you create another port channel to the downstream device from the
secondary peer device.

Note We recommend that you configure the ports on the downstream devices that connect to a host or a network
device that is not functioning as a switch or a bridge as STP edge ports.

On each vPC peer device, you assign a vPC number to the port channel that connects to the downstream
device. You will experience minimal traffic disruption when you are creating vPCs. To simplify the
configuration, you can assign the vPC ID number to every port channel to be the same as the port channel
itself (that is, vPC ID 10 for port channel 10).

Note The vPC number that you assign to the port channel that connects to the downstream device from the vPC
peer device must be identical on both vPC peer devices.

Moving Other Port Channels into a vPC

Note You must attach a downstream device using a port channel to both vPC peer devices.

To connect to the downstream device, you create a port channel to the downstream device from the primary
vPC peer device and you create another port channel to the downstream device from the secondary peer device.
On each vPC peer device, you assign a vPC number to the port channel that connects to the downstream
device. You will experience minimal traffic disruption when you are creating vPCs.

vPC Object Tracking

Note We recommend that you configure the vPC peer links on dedicated ports of different modules on Cisco Nexus
9500 devices. This is recommended to reduce the possibility of a failure. For the best resiliency scenario, use
at least two modules.

vPC object tracking is used to prevent traffic black-holing in case of failure of a module where both peer-link
and uplinks to the core resides. By tracking interface feature can suspend vPC on affected switch and prevent
traffic black-holing.
If you must configure all the vPC peer links and core-facing interfaces on a single module, you should configure,
using the command-line interface, a track object and a track list that is associated with the Layer 3 link to the
core and on all vPC peer links on both vPC peer devices. You use this configuration to avoid dropping traffic
if that particular module goes down because when all the tracked objects on the track list go down, the system
does the following:
• Stops the vPC primary peer device sending peer-keepalive messages, which forces the vPC secondary
peer device to take over.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
223
 Configuring vPCs
vPC Object Tracking

• Brings down all the downstream vPCs on that vPC peer device, which forces all the traffic to be rerouted
in the access switch toward the other vPC peer device.

Once you configure this feature and if the module fails, the system automatically suspends all the vPC links
on the primary vPC peer device and stops the peer-keepalive messages. This action forces the vPC secondary
device to take over the primary role and all the vPC traffic to go to this new vPC primary device until the
system stabilizes.
You should create a track list that contains all the links to the core and all the vPC peer links as its object.
Enable tracking for the specified vPC domain for this track list. Apply this same configuration to the other
vPC peer device. See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide for information
about configuring object tracking and track lists.

Note This example uses Boolean OR in the track list and forces all traffic to the vPC peer device only for a complete
module failure. If you want to trigger a switchover when any core interface or peer link goes down, use a
Boolean AND in the torack list below.

To configure a track list to switch over a vPC to the remote peer when all related interfaces on a single module
fail, follow these steps:
1. Configure track objects on an interface (Layer 3 to core) and on a port channel (vPC peer link).

switch(config-if)# track 35 interface ethernet 8/35 line-protocol

switch(config-track)# track 23 interface ethernet 8/33 line-protocol
switch(config)# track 55 interface port-channel 100 line-protocol

2. Create a track list that contains all the interfaces in the track list using the Boolean OR to trigger when
all objects fail.

switch(config)# track 44 list boolean OR

switch(config-track)# object 23
switch(config-track)# object 35
switch(config-track)# object 55
switch(config-track)# end

3. Add this track object to the vPC domain:

switch(config)# vpc domain 1

switch(config-vpc-domain)# track 44

4. Display the track object:

switch# show vpc brief

Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 1
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status: success
vPC role : secondary
Number of vPCs configured : 52
Track object : 44

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
224
 Configuring vPCs
vPC Interactions with Other Features

vPC Peer-link status

---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ --------------------------------------------------
1 Po100 up 1-5,140
vPC status
----------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
-- ---- ------ ----------- -------------------------- ------------
1 Po1 up success success 1-5,140

This example shows how to display information about the track objects:

switch# show track brief

Track Type Instance Parameter State Last
Change
23 Interface Ethernet8/33 Line Protocol UP 00:03:05
35 Interface Ethernet8/35 Line Protocol UP 00:03:15
44 List ----- Boolean
or UP 00:01:19
55 Interface port-channel100 Line Protocol UP 00:00:34

vPC Interactions with Other Features

vPC and LACP
LACP uses the system MAC address of the vPC domain to form the LACP Aggregation Group (LAG) ID
for the vPC. (See the “Configuring Port Channels” chapter for information about LAG-ID and LACP.)
You can use LACP on all the vPC port channels, including those channels from the downstream device. We
recommend that you configure LACP with active mode on the interfaces on each port channel on the vPC
peer devices. This configuration allows you to more easily detect compatibility between devices, unidirectional
links, and multihop connection, and provides dynamic reaction to run-time changes and link failures.
We recommend that you manually configure the system priority on the vPC peer link devices to ensure that
the vPC peer link devices have a higher LACP priority than the downstream connected devices. A lower
numerical value system priority means a higher LACP priority.

Note When manually configuring the system priority, you must ensure that you assign the same priority value on
both vPC peer devices. If the vPC peer devices have different system priority values, vPC does not come up.

vPC Peer Links and STP

Although vPCs provide a loop-free Layer 2 topology, STP is still required to provide a fail-safe mechanism
to protect against any incorrect or defective cabling or possible misconfiguration. When you first bring up a
vPC, STP reconverges. STP treats the vPC peer link as a special link and always includes the vPC peer link
in the STP active topology.
We recommend that you set all the vPC peer link interfaces to the STP network port type so that Bridge
Assurance is automatically enabled on all vPC peer links. We also recommend that you do not enable any of

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
225
 Configuring vPCs
vPC Peer Links and STP

the STP enhancement features on vPC peer links. If the STP enhancements are already configured, they do
not cause any problems for the vPC peer links..
When you are running both MST and Rapid PVST+, ensure that the PVST simulation feature is correctly
configured.
See the Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration Guide for information about STP
enhancement features and PVST simulation.

Note You must configure a list of parameters to be identical on the vPC peer devices on both sides of the vPC peer
link. See the “Compatibility Parameters for vPC Interfaces” section for information about these required
matched settings.

STP is distributed; that is, the protocol continues running on both vPC peer devices. However, the configuration
on the vPC peer device elected as the primary device controls the STP process for the vPC interfaces on the
secondary vPC peer device.
The primary vPC device synchronizes the STP state on the vPC secondary peer device using Cisco Fabric
Services over Ethernet (CFSoE). See the “vPC and Orphan Ports” section for information about CFSoE.
The STP process for vPC also relies on the periodic keepalive messages to determine when one of the connected
devices on the peer link fails. See the “Peer-Keepalive Link and Messages” section for information about
these messages.
The vPC manager performs a proposal/handshake agreement between the vPC peer devices that set the primary
and secondary devices and coordinates the two devices for STP. The primary vPC peer device then controls
the STP protocol on both the primary and secondary devices. We recommend that you configure the primary
vPC peer device as the STP primary root device and configure the secondary VPC device to be the STP
secondary root device.
If the primary vPC peer device fails over to the secondary vPC peer device, there is no change in the STP
topology.
The BPDUs uses the MAC address set for the vPC for the STP bridge ID in the designated bridge ID field.
The vPC primary device sends these BPDUs on the vPC interfaces.
You must configure both ends of vPC peer link with the identical STP configuration for the following
parameters:
• STP global settings:
• STP mode
• STP region configuration for MST
• Enable/disable state per VLAN
• Bridge Assurance setting
• Port type setting
• Loop Guard settings

• STP interface settings:

• Port type setting

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
226
 Configuring vPCs
vPC Peer Switch

• Loop Guard
• Root Guard

Note If any of these parameters are misconfigured, the Cisco NX-OS software suspends all interfaces in the vPC.
Check the syslog and enter the show vpc brief command to see if the vPC interfaces are suspended.

Ensure that the following STP interface configurations are identical on both sides of the vPC peer links or
you may see unpredictable behavior in the traffic flow:
• BPDU Filter
• BPDU Guard
• Cost
• Link type
• Priority
• VLANs (PVRST+)

Note Display the configuration on both sides of the vPC peer link to ensure that the settings are identical.

You can use the show spanning-tree command to display information about the vPC when that feature is
enabled. See the Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration Guide for an example.

Note We recommend that you configure the ports on the downstream devices as STP edge ports. You should
configure all host ports connected to a switch as STP edge ports. See the Cisco Nexus 9000 Series NX-OS
Layer 2 Switching Configuration Guide for more information about STP port types.

vPC Peer Switch

The vPC peer switch feature was added to Cisco NX-OS to address performance concerns around STP
convergence. This feature allows a pair of Cisco Nexus 9000 Series devices to appear as a single STP root in
the Layer 2 topology. This feature eliminates the need to pin the STP root to the vPC primary switch and
improves vPC convergence if the vPC primary switch fails.
To avoid loops, the vPC peer link is excluded from the STP computation. In vPC peer switch mode, STP
BPDUs are sent from both vPC peer devices to avoid issues related to STP BPDU timeout on the downstream
switches, which can cause traffic disruption.
This feature can be used with the pure peer switch topology in which the devices all belong to the vPC.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
227
 Configuring vPCs
vPC and ARP or ND

Note Peer-switch feature is supported on networks that use vPC and STP-based redundancy is not supported. If the
vPC peer-link fail in a hybrid peer-switch configuration, you can lose traffic. In this scenario, the vPC peers
use the same STP root ID as well as the same bridge ID. The access switch traffic is split in two with half
going to the first vPC peer and the other half to the second vPC peer. With peer link failure, there is no impact
to the north/south traffic but the east/west traffic is lost.

See the Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration Guide for information about STP
enhancement features and Rapid PVST+.

vPC and ARP or ND

A feature was added to Cisco NX-OS to address table synchronization across vPC peers using the reliable
transport mechanism of the Cisco Fabric Service over Ethernet (CFSoE) protocol. You must enable the ip
arp synchronize and ipv6 nd synchronize commands to support faster convergence of address tables between
the vPC peers. This convergence overcomes the delay that occurs in ARP table restoration for IPv4 or ND
table restoration for IPv6 when the peer link port channel flaps or when a vPC peer comes back online.

vPC Multicast—PIM, IGMP, and IGMP Snooping

The Cisco NX-OS software for the Nexus 9000 Series devices supports the following on a vPC:
• PIM Any Source Multicast (ASM).
• PIM Source-Specific Multicast (SSM) (7.0(3)I4(1) and later).

Note The Cisco NX-OS software does not support Bidirectional (BIDR) on a vPC.

The software keeps the multicast forwarding state synchronized on both of the vPC peer devices. The IGMP
snooping process on a vPC peer device shares the learned group information with the other vPC peer device
through the vPC peer link; the multicast states are always synchronized on both vPC peer devices. The PIM
process in vPC mode ensures that only one of the vPC peer devices forwards the multicast traffic to the
receivers.
Each vPC peer is a Layer 2 or Layer 3 device. Multicast traffic flows from only one of the vPC peer devices.
You might see duplicate packets in the following scenarios:
• Orphan hosts
• When the source and receivers are in the Layer 2 vPC cloud in different VLANs with multicast routing
enabled and a vPC member link goes down.

You might see negligible traffic loss in the following scenarios:

• When you reload the vPC peer device that is forwarding the traffic.
• When you restart PIM on the vPC peer device that is forwarding the traffic.

Ensure that you dual-attach all Layer 3 devices to both vPC peer devices. If one vPC peer device goes down,
the other vPC peer device continues to forward all multicast traffic normally.
The following outlines vPC PIM and vPC IGMP/IGMP snooping:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
228
 Configuring vPCs
Multicast PIM Dual DR (Proxy DR )

• vPC PIM—The PIM process in vPC mode ensures that only one vPC peer device forwards multicast
traffic. The PIM process in vPC mode synchronizes the source state with both vPC peer devices and
elects which vPC peer device forwards the traffic.
• vPC IGMP/IGMP snooping—The IGMP process in vPC mode synchronizes the designated router (DR)
information on both vPC peer devices. Dual DRs are available for IGMP when you are in vPC mode.
Dual DRs are not available when you are not in vPC mode, because both vPC peer devices maintain the
multicast group information between the peers.

Note A PIM neighbor relationship between a vPC VLAN (a VLAN that is carried on a vPC peer link) and a
downstream vPC-attached Layer 3 device is not supported, which can result in dropped multicast packets. If
a PIM neighbor relationship is required with a downstream Layer 3 device, a physical Layer 3 interface must
be used instead of a vPC interface.

You should enable or disable IGMP snooping identically on both vPC peer devices, and all the feature
configurations should be identical. IGMP snooping is on by default.

Note The following commands are not supported in vPC mode:

• ip pim spt-threshold infinity
• ip pim use-shared-tree-only

See the Cisco Nexus 9000 Series NX-OS Multicast Routing Configuration Guide for more information about
multicasting.

Multicast PIM Dual DR (Proxy DR )

By default, a multicast router sends PIM joins upstream only if it has interested receivers. These interested
receivers can either be IGMP hosts (they communicate through IGMP reports) or other multicast routers (they
communicate through PIM joins).
In the Cisco NX-OS vPC implementation, PIM works in dual designated router (DR) mode. That is, if a vPC
device is a DR on a vPC SVI outgoing interface (OIF), its peer automatically assumes the proxy DR role.
IGMP adds an OIF (the report is learned on that OIF) to the forwarding if the OIF is a DR. With dual DRs,
both vPC devices have an identical (*,G) entry with respect to the vPC SVI OIFs as shown in this example:

VPC Device1:
------------
(*,G)
oif1 (igmp)
VPC Device2:
------------
(*,G)
oif1 (igmp)

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
229
 Configuring vPCs
IP PIM PRE-BUILD SPT

IP PIM PRE-BUILD SPT

When the multicast source is in a Layer 3 cloud (outside the vPC domain), one vPC peer is elected as the
forwarder for the source. This forwarder election is based on the metrics to reach the source. If there is a tie,
the vPC primary is chosen as the forwarder. Only the forwarder has the vPC OIFs in its associated (S,G) and
the nonforwarder (S,G) has 0 OIFs. Therefore, only the forwarder sends PIM (S,G) joins toward the source
as shown in this example:

VPC Device1 (say this is Forwarder for Source 'S'):

------------
(*,G)
oif1 (igmp)
(S,G)
oif1 (mrib)
VPC Device2:
------------
(*,G)
oif1 (igmp)
(S,G)
NULL

In the case of a failure (for example, a Layer 3 Reverse Path Forwarding (RPF) link on the forwarder becomes
inoperable or the forwarder gets reloaded), if the current nonforwarder ends up becoming the forwarder, it
has to start sending PIM joins for (S,G) toward the source to pull the traffic. Depending upon the number of
hops to reach the source, this operation might take some time (PIM is a hop-by-hop protocol).
To eliminate this issue and get better convergence, use the ip pim pre-build-spt command. This command
enables PIM send joins even if the multicast route has 0 OIFs. In a vPC device, the nonforwarder sends PIM
(S,G) joins upstream toward the source. The downside is that the link bandwidth upstream from the
nonforwarder gets used for the traffic that is ultimately dropped by it. The benefits that result with better
convergence far outweigh the link bandwidth usage. Therefore, we recommend that you use this command
if you use vPCs.

vPC Peer Links and Routing

The First Hop Redundancy Protocols (FHRPs) interoperate with vPCs. The Hot Standby Routing Protocol
(HSRP), and Virtual Router Redundancy Protocol (VRRP) all interoperate with vPCs. We recommend that
you dual-attach all Layer 3 devices to both vPC peer devices.
The primary FHRP device responds to ARP requests, even though the secondary vPC device forwards the
data traffic.
To simplify initial configuration verification and vPC/HSRP troubleshooting, you can configure the primary
vPC peer device with the FHRP active router highest priority.
In addition, you can use the priority command in the if-hsrp configuration mode to configure failover thresholds
for when a group state enabled on a vPC peer link is in standby or in listen state. You can configure lower
and upper thresholds to prevent the interface from going up and down.
VRRP acts similarly to HSRP when running on vPC peer devices. You should configure VRRP the same way
that you configure HSRP.
When the primary vPC peer device fails over to the secondary vPC peer device, the FHRP traffic continues
to flow seamlessly.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
230
 Configuring vPCs
CFSoE

We recommend that you configure routing adjacency between the two vPC peer devices to act as a backup
routing path. If one vPC peer device loses Layer 3 uplinks, the vPC can redirect the routed traffic to the other
vPC peer device and leverage its active Layer 3 uplinks.
You can configure the inter-switch link for a backup routing path in the following ways:
• Create a Layer 3 link between the two vPC peer devices.
• Use the non-VPC VLAN trunk with a dedicated VLAN interface.
• Use a vPC peer link with a dedicated VLAN interface.

We do not recommend that you configure the burnt-in MAC address option (use-bia) for HSRP or manually
configure virtual MAC addresses for any FHRP protocol in a vPC environment because these configurations
can adversely affect vPC load balancing. The HSRP use-bia option is not supported on vPCs. When you are
configuring custom MAC addresses, you must configure the same MAC address on both vPC peer devices.
You can use the delay restore command to configure a restore timer that delays the vPC coming back up
until after the peer adjacency forms and the VLAN interfaces are back up. This feature enables you to avoid
packet drops when the routing tables might not be converged before the vPC is once again passing traffic.
Use the delay restore command to configure this feature.
To delay the VLAN interfaces on the restored vPC peer device from coming up, use the interfaces-vlan
option of the delay restore command.
See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide for more information about
FHRPs and routing.

CFSoE
The Cisco Fabric Services over Ethernet (CFSoE) is a reliable state transport mechanism that is used to
synchronize the actions of the vPC peer devices. CFSoE carries messages and packets for many features linked
with vPC, such as STP and IGMP. Information is carried in CFS/CFSoE protocol data units (PDUs).
When you enable the vPC feature, the device automatically enables CFSoE, and you do not have to configure
anything. CFSoE distributions for vPCs do not need the capabilities to distribute over IP or the CFS regions.
You do not need to configure anything for the CFSoE feature to work correctly on vPCs.
The CFSoE transport is local to each VDC.
You can use the show mac address-table command to display the MAC addresses that CFSoE synchronizes
for the vPC peer link.

Note Do not enter the no cfs eth distribute or the no cfs distribute command. You must enable CFSoE for vPC
functionality. If you do enter either of these commands with vPC enabled, the system displays an error message.

When you enter the show cfs application command, the output displays “Physical-eth,” which shows the
applications that are using CFSoE.
CFS also transports data over TCP/IP. See the Cisco Nexus 9000 Series NX-OS System Management
Configuration Guide for more information about CFS over IP.

Note The software does not support CFS regions.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
231
 Configuring vPCs
vPC and Orphan Ports

vPC and Orphan Ports

When a device that is not vPC-capable connects to each peer, the connected ports are known as orphan ports
because they are not members of a vPC. The device’s link to one peer will be active (forwarding) and the
other link will be standby (blocking) due to STP.
If a peer link failure or restoration occurs, an orphan port’s connectivity might be bound to the vPC failure
or restoration process. For example, if a device’s active orphan port connects to the secondary vPC peer, the
device loses any connections through the primary peer if a peer link failure occurs and the vPC ports are
suspended by the secondary peer. If the secondary peer were to also suspend the active orphan port, the
device’s standby port becomes active, provides a connection to the primary peer, and restores connectivity.
You can configure in the CLI that specific orphan ports are suspended by the secondary peer when it suspends
its vPC ports and are restored when the vPC is restored.

Virtualization Support
All ports in a given vPC must be in the same VDC. This version of the software supports only one vPC domain
per VDC. You can use the numbers from 1 to 4096 in each VDC to number the vPC.

vPC Recovery After an Outage

In a data center outage, both the vPC peer in vPC domain get reloaded. Occasionally only one peer can be
restored. With no functioning peer-keepalive or peer link, the vPC cannot function normally, a method might
be available to allow vPC services to use only the local ports of the functional peer.

Autorecovery
You can configure the Cisco Nexus 9000 Series device to restore vPC services when its peer fails to come
online by using the auto-recovery command. You must save this setting in the startup configuration. On
reload, if the peer link is down and three consecutive peer-keepalive messages are lost, the secondary device
assumes the primary STP role and the primary LACP role. The software reinitializes the vPCs, bringing up
its local ports. Because there are no peers, the consistency check is bypassed for the local vPC ports. The
device elects itself to be the STP primary regardless of its role priority and also acts as the primary device for
LACP port roles.

vPC Peer Roles After a Recovery

When the other peer device completes its reload and adjacency forms, the following process occurs:
1. The first vPC peer maintains its current role to avoid any transition reset to other protocols. The peer
accepts the other available role.
2. When an adjacency forms, consistency checks are performed and appropriate actions are taken.

High Availability
During an In-Service Software Upgrade (ISSU), the software reload process on the first vPC device locks its
vPC peer device by using CFS messaging over the vPC communications channel. Only one device at a time
is upgraded. When the first device completes its upgrade, it unlocks its peer device. The second device then
performs the upgrade process, locking the first device as it does so. During the upgrade, the two vPC devices

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
232
 Configuring vPCs
vPC Forklift Upgrade Scenario

temporarily run different releases of Cisco NX-OS, however the system functions correctly because of its
backward compatibility support.

Note See the Cisco Nexus 9000 Series NX-OS High Availability and Redundancy Guide for complete information
about high-availability features.

vPC Forklift Upgrade Scenario

The following procedure describes a scenario of migrating pair of Cisco Nexus 9500 switches in a vPC domain
to a different pair of Cisco Nexus 9500 switches with a same type of line cards. Migrating from Cisco Nexus
9504 switches to Cisco Nexus 9508 switches for the need of more interfaces is a typical example of such
migration. The following migration scenarios are not supported:
• Migration of Cisco Nexus 9500 switches with a different set of line cards. For example, from a Cisco
Nexus 9500 switches with N9K-X94xx line card to Cisco Nexus 9500 switches with N9K-X97xx line
card.
• Migration between different generations of Cisco Nexus 9300 switches. For example, migration from
Cisco Nexus N9K-C9372PX to Cisco Nexus N9K-93180YC-EX switches
• Having different generations of Cisco Nexus 9000 switches in a vPC domain is not supported

Considerations for a vPC forklift upgrade:

• vPC Role Election and Sticky-bit
When the two vPC systems are joined to form a vPC domain, priority decides which device is the vPC
primary and which is the vPC secondary. When the primary device is reloaded, the system comes back
online and connectivity to the vPC secondary device (now the operational primary) is restored. The
operational role of the secondary device (operational primary) does not change (to avoid unnecessary
disruptions). This behavior is achieved with a sticky-bit, where the sticky information is not saved in the
startup configuration. This method makes the device that is up and running win over the reloaded device.
Hence, the vPC primary becomes the vPC operational secondary. Sticky-bit is also set when a vPC node
comes up with peer-link and peer-keepalive down and it becomes primary after the auto recovery period.
• vPC Delay Restore
The delay restore timer is used to delay the vPC from coming up on the restored vPC peer device after
a reload when the peer adjacency is already established.
To delay the VLAN interfaces on the restored vPC peer device from coming up, use the interfaces-vlan
option of the delay restore command.
• vPC Auto-Recovery
During a data center power outage when both vPC peer switches go down, if only one switch is restored,
the auto-recovery feature allows that switch to assume the role of the primary switch and the vPC links
come up after the auto-recovery time period. The default auto-recovery period is 240 seconds.

The following example is a migration scenario that replaces vPC peer nodes Node1 and Node2 with New_Node1
and New_Node2.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
233
 Configuring vPCs
vPC Forklift Upgrade Scenario

Migration Step Expected Behavior Node1 Node1 Node2 Node2

Configured Operational Configured Operational
role ( Ex: role role (Ex: role
role priority role priority
100) 200)

1 Initial state Traffic is primary Primary secondary Secondary

forwarded by both
Sticky bit: Sticky bit:
vPC peers – Node1
False False
and Node2.
Node1 is primary
and Node2 is
secondary.

2 Node2 replacement – Traffic converged primary Primary secondary Secondary

Shut all vPCs and on Primary vPC
Sticky bit: Sticky bit:
uplinks on Node2. peer Node1.
False False
Peer-link and vPC
peer-keepalive are in
administrative up
state.

3 Remove Node2. Node1 will primary Primary n/a n/a

continue to
Sticky bit:
forward traffic.
False

4 Configure New_Node2 will primary Primary secondary Secondary

New_Node2. Copy come up as
Sticky bit: Sticky bit:
the configuration to secondary. Node1
False False
startup config. vPC continue to be
peer-link and primary.
peer-keepalive in
Traffic will
administrative up
continue to be
state.
forwarded on
Power off Node01.
New_Node2.
Make all connections.
Power on
New_Node2.

5 Bring up all vPCs and Traffic will be primary Primary secondary Secondary
uplink ports on forwarded by both
Sticky bit: Sticky bit:
New_Node2. Node 1 and
False False
New_Node2.

6 Node1 replacement - Traffic will primary Primary secondary Secondary

converge on
Shut vPCs and Sticky bit: Sticky bit:
New_Node2.
uplinks on Node1. False False

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
234
 Configuring vPCs
Guidelines and Limitations

Migration Step Expected Behavior Node1 Node1 Node2 Node2

Configured Operational Configured Operational
role ( Ex: role role (Ex: role
role priority role priority
100) 200)

7 Remove Node1. New_Node2 will n/a n/a secondary Primary

become secondary,
Sticky bit:
operational
True
primary and sticky
bit will be set to
True.

8 Configure New_Node1 will primary Secondary secondary Primary

New_Node1. Copy come up as
Sticky bit: Sticky bit:
running to startup. primary,
False True
operational
Power off the new
secondary.
Node1. Make all
connections. Power
on New_Node1.

9 Bring up all vPCs and Traffic will be primary Secondary secondary Primary
uplink ports on forwarded by both
Sticky bit: Sticky bit:
New_Node1. New Node1 and
False True
new Node2.

Note If you prefer to have the configured secondary node as the operational secondary and the configured primary
as the operational primary, then Node2 can be reloaded at the end of the migration. This is optional and does
not have any functional impact.

Guidelines and Limitations

vPCs have the following configuration guidelines and limitations:
• Make sure that both vPC peers are in the same mode (regular mode or enhanced mode) before performing
a non-disruptive upgrade.

Note vPC peering between an enhanced ISSU mode (boot mode lxc) configured switch
and a non-enhanced ISSU mode switch is not supported.

• show commands with the internal keyword are not supported.

• Cisco Nexus 9000 Series switches do not support NAT on vPC topology.
• The spanning-tree pseudo-information command is not available on Cisco Nexus 92160 and Cisco
Nexus 93180 switches starting from Release 7.0(3)I4(1).

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
235
 Configuring vPCs
Guidelines and Limitations

• vPC peers must run the same Cisco NX-OS release. During a software upgrade, make sure to upgrade
the primary vPC peer first.
• All ports for a given vPC must be in the same VDC.
• You must enable vPCs before you can configure them.
• You must configure the peer-keepalive link and messages before the system can form the vPC peer link.
• Only Layer 2 port channels can be in vPCs.
• You must configure both vPC peer devices; the configuration is not sent from one device to the other.
• To configure multilayer (back-to-back) vPCs, you must assign unique vPC domain ID for each respective
vPC.
• Check that the necessary configuration parameters are compatible on both sides of the vPC peer link.
See the “Compatibility Parameters for vPC Interfaces” section for information about compatibility
recommendations.
• You might experience minimal traffic disruption while configuring vPCs.
• The software does not support BIDR PIM on vPCs.
• The software does not support DHCP snooping, DAI, or IPSG in a vPC environment; DHCP Relay is
supported.
• The software does not support CFS regions.
• Port security is not supported on port channels.
• When peer-switch features are configured under vpc domain configuration mode on two Cisco Nexus
9000 Series switches, the spanning-tree root changes even for VLANs that are not enabled on the vPC
peer-link. Both the switches act as one system with one MAC address as the bridge address. This is true
even for non-vPC mst-instance or VLANs. Therefore, a non vPC peer-link between the two switches
gets blocked as a backup link. This is an expected behavior.
• We recommend that you configure all the port channels in the vPC using LACP with the interfaces in
active mode.
• Back-to-back, multilayer vPC topologies require unique domain IDs on each respective vPC.
• Having the same Hot Standby Router Protocol (HSRP)/Virtual Router Redundancy Protocol (VRRP)
group on all nodes on a double sided vPC is supported on Cisco NX-OS 7.0(3)I2(1) and later releases.
• When migrating from a pair of spine nodes to a pair of Cisco Nexus 9000 devices, the HSRP priority
should be configured so that the Cisco Nexus 9000 vPC peers are in Active/Standby state. There is no
support for Cisco Nexus 9000 vPC peers in HSRP state to be in Active/Listen state, or Standby/Listen
state. (7.(0)I2(2) or later)
• When using vPCs, we recommend that you use default timers for FHRP (HSRP, VRRP), and PIM
configurations. There is no advantage in convergence times when using aggressive timers in vPC
configurations.
• If you configure open shortest path first (OSPF) in a vPC environment, use the following timer commands
in router configuration mode on the core switch to ensure fast OSPF convergence when a vPC peer link
is shut down:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
236
Configuring vPCs
Guidelines and Limitations

switch (config-router)# timers throttle spf 1 50 50

switch (config-router)# timers lsa-arrival 10

See the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide for further details about
OSPF.
• BFD for VRRP/HSRP is not supported in a vPC environment.
• The STP port cost is fixed to 200 in a vPC environment.
• Jumbo frames are enabled by default on the vPC peer link.
• To accommodate increased traffic when the vPC goes down and traffic needs to cross the peer-link, it
is a best practice to use multiple high bandwidth interfaces (such as the 40G interfaces for the Cisco
Nexus 9000) across linecards for the peer-link.
• The vpc orphan-ports suspend command also applies to ports in non-vPC VLANs and Layer 3 ports.
However, it is recommended to be used with ports in VPC VLANs.
• Starting in NX-OS 7.0(3)I5(2), FEX-AA (dual-homed FEX) and FEX-ST (FEX straight-thru) topologies
(FEX-AA and FEX-ST) are supported. The following mixing is not supported as the parent switches:
• Cisco Nexus 9300-EX and 9300 switches
• Cisco Nexus 9300 and 9500 switches
• Cisco Nexus 9300-EX and 9500 switches

• Starting in NX-OS 7.0(3)I2(2), when configuring vPCs, the behavior previously provided by using the
ip pim pre-build-spt command has now been enabled automatically by default and cannot be disabled.
• Starting in NX-OS 7.0(3)I2(2), a vPC port channel member link that is operating in Individual state will
be flapped while checking for VLAN inconsistencies. To avoid having the link flapped during server
provisioning, disable the VPC graceful consistency check with the no graceful consistency-check
command.
The following example disables the VPC graceful consistency check:

switch# conf t
Enter configuration commands, one per line. End with CNTL/Z.

switch(config)# vpc domain 1

switch(config-vpc-domain)# no graceful consistency-check

• vPC STP hitless role change feature is supported only from Cisco Nexus 9000 Release 7.0(3)I7(1)
onwards.
• vPC role change can be performed from either of the peer devices.
• You can connect a pair of switches such as Cisco Nexus 9300 Series switches or Cisco Nexus 9500
Series switches in a vPC domain, that will for a vPC to a network node or to a host.
• Switches in a vPC domain, peer switches must be of the same type, for example, you can connect
a pair of Cisco Nexus 9300 series switches, but you cannot connect a Cisco Nexus 9300 series
switch to a Cisco Nexus 9500 series switch in a vPC domain.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
237
 Configuring vPCs
Best Practices for Layer 3 and vPC Configuration

• vPC peer switch families must also be of the same type, for example, you can connect a pair of
Cisco Nexus 9300-EX switches, but you cannot connect a Cisco Nexus 9300-EX switch to a Cisco
Nexus 9300-FX switch in a vPC domain.

• If the original secondary device has higher role priority value than the original primary device, role
swapping cannot be performed. Change the role priority on either vPC device so that the value of the
original secondary device is lower than the original primary one. To view the existing role of a device,
use the show vpc role command on local and peer switch.
• Always check the existing configured role priority before configuring vPC hitless role change feature
• In a vPC domain, enable the peer-switch command, where both vPC peers have same STP priorities, and
ensure it is operational before issuing a role change. If you do not enable the peer-switch command, it
can lead to convergence issues. Use show spanning-tree summary | grep peer command to verify
whether the peer vPC switch is operational or not.
• All the devices that are attached to a vPC domain must be dual homed.
• The first generation Broadcom based Nexus 9300 series switches and Nexus 9500 series line-cards does
not support policy based routing route map with a set ip next-hop statement where the egress interface
is the vPC peer link while the vPC convergence TCAM region is allocated. This limitation does not apply
to cloud scale based Nexus 9000 series devices such as Cisco Nexus 9200 switches, 9300 switches with
EX/FX/FX2 line-cards and Nexus 9500 platform switches with 9700-EX/FX line-cards.
• Beginning with Cisco NX-OS Release 7.0(3)I5(1), Layer 3 over vPC is supported on Cisco Nexus 9000
Series switches for Layer 3 unicast communication only. Layer 3 over vPC is not supported for Layer 3
multicast traffic. For more information see the Best Practices for Layer 3 and vPC Configuration section

Best Practices for Layer 3 and vPC Configuration

This section describes best practices for using and configuring Layer 3 with vPC.

Layer 3 and vPC Configuration Overview

When a Layer 3 device is connected to a vPC domain through a vPC, it has the following views:
• At Layer 2, the Layer 3 device sees a unique Layer 2 switch presented by the vPC peer devices.
• At Layer 3, the Layer 3 device sees two distinct Layer 3 devices (one for each vPC peer device).

vPC is a Layer 2 virtualization technology, so at Layer 2, both vPC peer devices present themselves as a
unique logical device to the rest of the network.
There is no virtualization technology at Layer 3, so each vPC peer device is seen as a distinct Layer 3 device
by the rest of the network.
The following figure illustrates the two different Layer 2 and Layer 3 views with vPC.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
238
 Configuring vPCs
Supported Topologies for Layer 3 and vPC

Figure 17: Different Views for vPC Peer Devices

Supported Topologies for Layer 3 and vPC

This section contains examples of Layer 3 and vPC network topologies.
There are two approaches for Layer 3 and vPC interactions. The first one is by using dedicated Layer 3 links
to connect the Layer 3 devices to each vPC peer device. The second one is by allowing the Layer 3 devices
to peer with the SVIs defined on each of the vPC peer device, on a dedicated VLAN that is carried on the
vPC connection. The following sections describe all the supported topologies leveraging the elements that
are described in the legends in the following figure.
Figure 18: Legend

Peering with an External Router Using Layer 3 Links

This example shows a topology that uses Layer 3 links to connect a Layer 3 device to the Cisco Nexus 9000
switches that are part of the a vPC domain

Note Interconnecting the two entities together in this way allows to support Layer 3 unicast and multicast
communication.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
239
 Configuring vPCs
Peering Between vPC Devices for a Backup Routing Path

Figure 19: Peering with an External Router Using Layer 3 Links

Layer 3 devices can initiate Layer 3 routing protocol adjacencies with both vPC peer devices.
One or multiple Layer 3 links can be used to connect a Layer 3 device to each vPC peer device. Cisco Nexus
9000 series devices support Layer 3 Equal Cost Multipathing (ECMP) with up to 16 hardware load-sharing
paths per prefix. Traffic from a vPC peer device to a Layer 3 device can be load-balanced across all the Layer
3 links interconnecting the two devices together.
Using Layer 3 ECMP on the Layer 3 device can effectively use all Layer 3 links from the device to the vPC
domain. Traffic from a Layer 3 device to the vPC domain can be load-balanced across all the Layer 3 links
interconnecting the two entities together.
Follow these guidelines when connecting a Layer 3 device to the vPC domain using Layer 3 links:
• Use separate Layer 3 links to connect Layer 3 devices to the vPCdomain. Each link represents a
point-to-point Layer 3 connection and should get assigned an IP address taken from a small IP subnet
(/30 or /31).
• If the Layer 3 peering is required for multiple VRFs, it is recommended to define multiple sub-interfaces,
each mapped to an individual VRF.

Peering Between vPC Devices for a Backup Routing Path

This example shows peering between the two vPC peer devices with a Layer 3 backup routed path. If the
Layer 3 uplinks on vPC peer device 1 or vPC peer device 2 fail, the path between the two peer devices is used
to redirect traffic to the switch that has the Layer 3 uplinks in the up state.
The Layer 3 backup routing path can be implemented using a dedicated interface VLAN (such as SVI) over
the vPC peer-link or by using dedicated Layer 2 or Layer 3 links across the two vPC peer devices.
Figure 20: Peering Between vPC Devices for a Backup Routing Path

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
240
 Configuring vPCs
Direct Layer 3 Peering Between Routers

Direct Layer 3 Peering Between Routers

In this scenario, the Nexus 9000 devices part of the vPC domain are simply used as a Layer 2 transit path to
allow the routers connected to them to establish Layer 3 peering and communication.
Figure 21: Peering Between Routers

The Layer 3 devices can peer with each other in following two methods. Peering also depends on the specific
device deployed for this role.
• Defining a VLAN network interface (SVI) for a VLAN that is extended between the Layer 3 devices
through the intermediate Cisco Nexus 9000 vPC peer switches.
• Defining a Layer 3 port-channel interface on each Layer 3 device and establishing a point-to-point Layer
3 peering.

Note In deployments where the Layer 3 peering must be established for multiple VRFs, the first method require
the definition on the Layer 3 devices of a VLAN (and SVI) per VRF. For the second method, it is possible to
create a Layer 3 port-channel subinterface per VRF

Peering Between Two Routers with vPC Devices as Transit Switches

This example is similar to the peering between routers topology. In this case also, the Cisco Nexus 9000
devices that are part of the same vPC domain are only used as Layer 2 transit paths. The difference here is
that there are two pairs of Cisco Nexus 9000 switches. Each switch that is connected with a Layer 3 device
using a vPC connection, also establishes a back-to-back vPC connection between them. The difference is that
the vPC domains are only used as Layer 2 transit paths.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
241
 Configuring vPCs
Peering with an External Router on Parallel Interconnected Routed Ports

Figure 22: Peering Between Two Routers with vPC Devices as Transit Switches

This topology is commonly used when you want to establish connectivity between separate data centers that
are interconnected with direct links (dark fibers or DWDM circuits). The two pairs of Cisco Nexus 9000
switches, in this case, provide only Layer 2 extension services, allowing the Layer 3 devices to peer with each
other at Layer 3.

Peering with an External Router on Parallel Interconnected Routed Ports

When you require both routed and bridged traffic, use individual Layer 3 links for routed traffic and a separate
Layer 2 port-channel for bridged traffic, as shown in following figure.
The Layer 2 links are used for bridged traffic (traffic staying in the same VLAN) or inter-VLAN traffic
(assuming vPC domain hosts the interface VLAN and associated HSRP configuration).
The Layer 3 links are used for routing protocol peering adjacency with each vPC peer device.
The purpose of this topology is to attract specific traffic to go through the Layer 3 device. Layer 3 links are
also used to carry routed traffic from a Layer 3 device to the vPC domain.
Figure 23: Peering with an External Router on Parallel Interconnected Routed Ports

Peering between vPC Switch Pairs on Parallel Interconnected Routed Ports

An alternative design to what is shown in the previous section (Peering Between Two Routers with vPC
Devices as Transit Switches), uses two pairs of Cisco Nexus 9000 switches that are deployed in each data
center for providing both Layer 2 and Layer 3 extension services. When routing protocol peering adjacency
is required to be established between the two pairs of Cisco Nexus 9000 devices, the best practice is to add
dedicated Layer 3 links between the two sites as shown in the following example.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
242
 Configuring vPCs
Peering Over a PC Interconnection and Dedicated Interswitch Link Using non-vPC VLAN

Figure 24: Peering Over a vPC Interconnection on Parallel Interconnected Routed Ports

The back-to-back vPC connection between the two data centers carry bridged traffic or inter-VLAN traffic
while the dedicated Layer 3 links carry the routed traffic across the two sites.

Peering Over a PC Interconnection and Dedicated Interswitch Link Using non-vPC VLAN
This example shows when the Layer 3 device is single-attached to the vPC domain, you can use a non-vPC
VLAN with a dedicated inter-switch link to establish the routing protocol peering adjacency between the
Layer 3 device and each vPC peer device. However, the non-vPC VLAN must be configured to use a static
MAC that is different than the vPC VLAN.

Note Configuring the vPC VLAN (and vPC peer-link) for this purpose is not supported.

Figure 25: Peering Over a PC Interconnection and Dedicated Interswitch Link Using non-vPC VLAN

Peering Directly Over a vPC Connection

Beginning with Cisco NX-OS Release 7.0(3)I5(1), an alternative method has been introduced to establish
Layer 3 peering between a Layer 3 router and a pair of Cisco Nexus 9000 vPC switches.

Note Peering directly over a vPC connection is supported only for Layer 3 unicast communication but not for Layer
3 multicast traffic. If you require Layer 3 multicast, you must establish peering over dedicated Layer 3 links

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
243
 Configuring vPCs
Peering Directly Over a vPC Connection

Figure 26: Supported: Peering Over a vPC Interconnection Where the Router Peers with Both the vPC Peers.

In this scenario, the Layer 3 peering between the external router and the Cisco Nexus 9000 switches that are
part of a same vPC domain is established directly on a VLAN carried on the vPC connection. The external
router in this case peers with SVI interfaces defined on each vPC device. As for the scenario shown in previous
figure 12, the external router could use an SVI or a Layer 3 Port-Channel to peer with the vPC devices (multiple
SVIs or Port-Channel subinterfaces could be used for a multi-VRF deployment).
This deployment model requires configuring layer3 peer-router command as part of the vPC domain. You
can adopt the same approach for establishing Layer 2 and Layer 3 connectivity on a vPC back-to-back
connection established between two separate pairs of vPC switches.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
244
 Configuring vPCs
Configuring Layer 3 over vPC

Figure 27: Supported: Peering Over a vPC Interconnection Where Each Nexus Device Peers with Two vPC Peers.

In this deployment model, SVI interfaces in the same VLAN is configured on all the four Cisco Nexus 9000
switches to establish routing peering and connectivity between them.

Configuring Layer 3 over vPC

Beginning with Cisco NX-OS Release 7.0(3)I5(1), Layer 3 over vPC is supported on Cisco Nexus 9000 Series
switches.

Before you begin

Ensure that the peer-gateway feature is enabled and it is configured on both the peers and both the peers run
an image that supports Layer 3 over vPC. If you enter the layer3 peer-router command without enabling the
peer-gateway feature, a syslog message is displayed recommending you to enable the peer-gateway feature.
Ensure that the peer link is up.

Procedure

Command or Action Purpose

Step 1 switch# configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 switch(config)# vpc domain domain-id Creates a vPC domain if it does not already
exist, and enters the vpc-domain configuration
Example:
mode. There is no default; the range is from <1
switch(config)# vpc domain 5 to 1000>.
switch(config-vpc-domain)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
245
 Configuring vPCs
Configuring Layer 3 over vPC

Command or Action Purpose

Step 3 switch(config-vpc-domain)#layer3 peer-router Enables the Layer 3 device to form peering
adjacency with both the peers.
Note Configure this command in both the
peers. If you configure this command
only on one of the peers or you
disable it on one peer, the
operational state of layer 3
peer-router gets disabled. You get a
notification when there is a change
in the operational state.

Step 4 switch(config-vpc-domain)# exit Exits the vpc-domain configuration mode.

Step 5 (Optional) switch#show vpc brief Displays brief information about each vPC
domain.

Step 6 (Optional) switch#copy running-config Copies the running configuration to the startup
startup-config configuration.

Example
The following example shows how to configure Layer 3 over vPC feature:
switch# configure terminal
switch(config)# vpc domain 5
switch(config-vpc-domain)# layer3 peer-router

switch(config-vpc-domain)# exit

switch(config)#

This example shows how to verify if the Layer 3 over vPC feature is configured. The Operational
Layer3 Peer is enabled or disabled depending up on how the operational state of Layer 3 over vPC
is configured.
switch# show vpc brief

vPC domain id : 5
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : failed
Type-2 consistency status : success
vPC role : secondary
Number of vPCs configured : 2
Peer Gateway : Enabled
Peer gateway excluded VLANs : -
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled (timeout = 240 seconds)
Operational Layer3 Peer : Enabled

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
246
 Configuring vPCs
Default Settings

Default Settings
The following table lists the default settings for vPC parameters.

Table 14: Default vPC Parameters

Parameters Default

vPC system priority 32667

vPC peer-keepalive message Disabled

vPC peer-keepalive interval 1 second

vPC peer-keepalive timeout 5 seconds

vPC peer-keepalive UDP port 3200

Configuring vPCs

Note You must use these procedures on both devices on both sides of the vPC peer link. You configure both of the
vPC peer devices using these procedures.

This section describes how to configure vPCs using the command-line interface (CLI).

Note If you are familiar with the Cisco IOS CLI, be aware that the Cisco NX-OS commands for this feature might
differ from the Cisco IOS commands that you would use.

Enabling vPCs
You must enable the feature vPC before you can configure and use vPCs.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 feature vpc Enables vPCs on the device.

Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
247
 Configuring vPCs
Disabling vPCs

Command or Action Purpose

switch(config)# feature vpc

Step 3 exit Exits global configuration mode.

Example:
switch(config)# exit
switch#

Step 4 show feature (Optional) Displays which features are enabled

on the device.
Example:
switch# show feature

Step 5 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch# copy running-config
startup-config

Example
This example shows how to enable the vPC feature:
switch# configure terminal
switch(config)# feature vpc
switch(config)# exit
switch(config)#

Disabling vPCs

Note When you disable the vPC functionality, the device clears all the vPC configurations.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 no feature vpc Disables vPCs on the device.

Example:
switch(config)# no feature vpc

Step 3 exit Exits global configuration mode.

Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
248
 Configuring vPCs
Creating a vPC Domain and Entering vpc-domain Mode

Command or Action Purpose

switch(config)# exit
switch#

Step 4 show feature (Optional) Displays which features are enabled

on the device.
Example:
switch# show feature

Step 5 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch# copy running-config
startup-config

Example
This example shows how to disable the vPC feature:
switch# configure terminal
switch(config)# no feature vpc
switch(config)# exit
switch#

Creating a vPC Domain and Entering vpc-domain Mode

You can create a vPC domain and put the vPC peer link port channels into the identical vPC domain on both
vPC peer devices. Use a unique vPC domain number throughout a single vPC domain . This domain ID is
used to automatically to form the vPC system MAC address.
You can also use this command to enter vpc-domain command mode.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 vpc domain domain-id [shut | no shut] Creates a vPC domain on the device, and enters
vpc-domain configuration mode for
Example:
configuration purposes. There is no default; the
switch(config)# vpc domain 5 range is from 1 to 1000.
switch(config-vpc-domain)#

Step 3 exit Exits vpc-domain configuration mode.

Example:
switch(config)# exit
switch#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
249
 Configuring vPCs
Configuring a vPC Keepalive Link and Messages

Command or Action Purpose

Step 4 show vpc brief (Optional) Displays brief information about
each vPC domain.
Example:
switch# show vpc brief

Step 5 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch# copy running-config
startup-config

Example
This example shows how to enter the vpc-domain command mode to configure an existing vPC
domain:

switch# configure terminal

switch(config)# vpc domain 5
switch(config-vpc-domain)# exit
switch(config)#

Configuring a vPC Keepalive Link and Messages

You can configure the destination IP for the peer-keepalive link that carries the keepalive messages. Optionally,
you can configure other parameters for the keepalive messages.

Note You must configure the vPC peer-keepalive link before the system can form the vPC peer link.

Note We recommend that you configure a separate VRF instance and put a Layer 3 port from each vPC peer device
into that VRF for the vPC peer-keepalive link. Do not use the peer link itself to send vPC peer-keepalive
messages. For information about creating and configuring VRFs, see the Cisco Nexus 9000 Series NX-OS
Unicast Routing Configuration Guide. Ensure that both the source and destination IP addresses use for the
peer-keepalive message are unique in your network. The management port and management VRF are the
defaults for these keepalive messages.

Before you begin

Ensure that you have enabled the vPC feature.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
250
Configuring vPCs
Configuring a vPC Keepalive Link and Messages

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 vpc domain domain-id [shut | no shut] Creates a vPC domain on the device, and enters
vpc-domain configuration mode.
Example:
switch(config)# vpc domain 5
switch(config-vpc-domain)#

Step 3 peer-keepalive destination ipaddress Configures the IPv4 and IPv6 addresses for the
[hold-timeout secs | interval msecs {timeout remote end of the vPC peer-keepalive link.
secs} | {precedence {prec-value | network |
Note The system does not form the vPC
internet | critical | flash-override | flash |
peer link until you configure a vPC
immediate priority | routine}} | tos {tos-value
peer-keepalive link.
| max-reliability | max-throughput |
min-delay | min-monetary-cost | normal}}
Note You may get the following error
|tos-byte tos-byte-value} | source ipaddress |
message if you do not specify the
vrf {name | management vpc-keepalive}]
source IP address when you
Example: configure an IPv6 address for the
switch(config-vpc-domain)# peer-keepalive remote end of the vPC
destination 172.28.230.85 peer-keepalive link.
switch(config-vpc-domain)#
Cannot configure IPV6
peer-keepalive without source
IPV6 address

The management ports and VRF are the

defaults.
Note We recommend that you configure
a separate VRF and use a Layer 3
port from each vPC peer device in
that VRF for the vPC peer-keepalive
link. For more information about
creating and configuring VRFs, see
the Cisco Nexus 9000 Series NX-OS
Unicast Routing Configuration
Guide.

Step 4 exit Exits global configuration mode.

Example:
switch(config)# exit
switch#

Step 5 show vpc statistics (Optional) Displays information about the

configuration for the keepalive messages.
Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
251
 Configuring vPCs
Creating a vPC Peer Link

Command or Action Purpose

switch# show vpc statistics

Step 6 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch# copy running-config
startup-config

Example
For more information about configuring VRFs, see the Cisco Nexus 9000 Series NX-OS Unicast
Routing Configuration Guide.
This example shows how to configure the destination and source IP address and VRF for the
vPC-peer-keepalive link:
switch# configure terminal
switch(config)# vpc domain 100
switch(config-vpc-domain)# peer-keepalive destination 172.168.1.2 source 172.168.1.1 vrf
vpc-keepalive
switch(config-vpc-domain)# exit
switch#

Creating a vPC Peer Link

You create the vPC peer link by designating the port channel that you want on each device as the peer link
for the specified vPC domain. We recommend that you configure the Layer 2 port channels that you are
designating as the vPC peer link in trunk mode and that you use two ports on separate modules on each vPC
peer device for redundancy.

Before you begin

Ensure that you have enabled the vPC feature.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface port-channel channel-number Selects the port channel that you want to use as
the vPC peer link for this device, and enters
Example:
interface configuration mode.
switch(config)# interface port-channel
20
switch(config-if)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
252
 Configuring vPCs
Configuring a vPC Peer-Gateway

Command or Action Purpose

Step 3 switchport mode trunk (Optional) Configures this interface in trunk
mode.
Example:
switch(config-if)# switchport mode trunk

Step 4 switchport trunk allowed vlan vlan-list (Optional) Configures the permitted VLAN list.
Example:
switch(config-if)# switchport trunk
allowed vlan 1-120,201-3967

Step 5 vpc peer-link Configures the selected port channel as the vPC
peer link, and enters vpc-domain configuration
Example:
mode.
switch(config-if)# vpc peer-link
switch(config-vpc-domain)#

Step 6 exit Exits vpc-domain configuration mode.

Example:
switch(config)# exit
switch#

Step 7 show vpc brief (Optional) Displays information about each

vPC, including information about the vPC peer
Example:
link.
switch# show vpc brief

Step 8 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch# copy running-config
startup-config

Example
This example shows how to configure a vPC peer link:
switch# configure terminal
switch(config)# interface port-channel 20
switch(config-if)# switchport mode
switch(config-if)# switchport mode trunk
switch(config-if)# switchport trunk allowed vlan 1-120,201-3967
switch(config-if)# vpc peer-link
switch(config-vpc-domain)# exit
switch(config)#

Configuring a vPC Peer-Gateway

You can configure vPC peer devices to act as the gateway for packets that are destined to the vPC peer device's
MAC address.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
253
 Configuring vPCs
Configuring Fast Convergence

Before you begin

Ensure that you have enabled the vPC feature.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 vpc domain domain-id [shut | no shut] Creates a vPC domain if it does not already
exist, and enters vpc-domain configuration
Example:
mode.
switch(config-if)# vpc domain 5
switch(config-vpc-domain)#

Step 3 peer-gateway Enables Layer 3 forwarding for packets destined

to the peer's gateway MAC address.
Example:
switch(config-vpc-domain)# peer-gateway

Note Disable IP redirects on all

interface-vlans of this vPC domain
for correct operation of this feature.

Step 4 exit Exits vpc-domain configuration mode.

Example:
switch(config)# exit
switch#

Step 5 show vpc brief (Optional) Displays information about each

vPC, including information about the vPC peer
Example:
link.
switch# show vpc brief

Step 6 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch# copy running-config
startup-config

Configuring Fast Convergence

Beginning with Cisco NX-OS Release 7.0(3)I7(1), fast convergence feature is supported on Cisco Nexus
9000 Series platforms. You can enable or disable the vPC optimizations using this command. To achieve
faster convergence, you must enable [no] fast-convergence on both vPC peers to achieve fast-convergence.
The optimization is archived on secondary switch, vPC member ports and orphan ports with vpc orphan-ports
suspend command is configured. In case the peer link fails, these ports will be suspended immediately and
traffic will be forwarded to primary vPC peer only to improve convergence.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
254
 Configuring vPCs
Configuring LACP vPC Convergence

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.

Step 2 switch(config) # vpc domain <domain> Configure the VPC domain number.

Step 3 switch(config) # peer-switch Define the peer switch.

Step 4 switch(config) # show vpc peer-keepalive Displays information about the peer keepalive
messages

Step 5 switch(config) # delay restore { time } Number of seconds to delay bringing up the
restored vPC peer device. The range is from 1
to 3600.

Step 6 switch(config) # peer-gateway To enable Layer 3 forwarding for packets

destined to the gateway MAC address of the
virtual Port Channel (vPC), use the
peer-gateway command. To disable Layer 3
forwarding packets, use the no form of this
command.
.

Step 7 switch(config) # delay restore orphan-port Number of seconds to delay bringing up the
restored device's orphan port

Step 8 switch(config-vpc-domain)# fast-convergence Configure vPC fast convergence.

Configuring LACP vPC Convergence

Beginning with Cisco NX-OS Release 7.0(3)I7(1), Link Aggregation Control Protocol (LACP) vPC convergence
feature is supported on Cisco Nexus 9200 and 9300 Series Switches. You can configure LACP vPC convergence
feature for more efficient use of port channels by reducing convergence time of vPC port channel for member
link going down and first member bring up.
Beginning with Cisco NX-OS Release 7.0(3)I7(5), Link Aggregation Control Protocol (LACP) vPC convergence
feature is supported on Cisco Nexus 9500 Series Switches with 9700-EX and 9700-FX line cards. This feature
is not supported on Nexus 9500 with 9400, 9500, and 9600 and 9600-R line cards.
When you configure LACP vPC convergence on a Cisco Nexus 9000 switch, it waits until all the VLANs are
initialized and programmed and then send LACP sync PDU, which will start sending traffic to the VPC domain
without drops. You may configure the lacp vpc-convergence command in a VXLAN and non-VXLAN
environments that have vPC port-channels to hosts that support LACP.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
255
 Configuring vPCs
Configuring a Graceful Consistency Check

Command or Action Purpose

Step 2 switch(config) # interface {type/slot | Specifies an interface to configure, and enters
portchannel number} interface configuration mode.

Step 3 switch(config-if) # lacp vpc-convergence Configure LACP convergence. Reduce the

convergence time of the vPC port channel for
member link going down and first member
bring up.
Note You must enable this command on
both the vPC peer switches. This
command must be configured only
on PortFast ports (vPC port channels
on which the spanning-tree port type
edge [trunk] is enabled).

Note In a vPC environment, when this

command is not configured on vPC
port-channel interfaces to devices
that support LACP, and if one of the
vPC peers is reloaded or if one of the
links is brought up, the link(s)
connected to the vPC peer switch in
the 'up' state will remain active and
forward traffic. The other link(s)
may go down and will transition to
the 'up' state. The links transitioning
to the 'up' state starts initializing
VLANs. When the VLANs are
initialized, LACP sync PDUs are
sent per initialized VLAN, which
will bring port-channel into 'up' state
and that leads to traffic black-hole
for non-idealized VLANs.

Configuring a Graceful Consistency Check

You can configure the graceful consistency check feature, which is enabled by default. Unless this feature is
enabled, the vPC is completely suspended when a mismatch in a mandatory compatibility parameter is
introduced in a working vPC. When this feature is enabled, only the links on the secondary peer device are
suspended. See the “Compatibility Parameters for vPC Interfaces” section for information about consistent
configurations on the vPCs.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
256
 Configuring vPCs
Checking the Configuration Compatibility on a vPC Peer Link

Command or Action Purpose

switch# configure terminal
switch(config)#

Step 2 vpc domain domain-id [shut | no shut] Creates a vPC domain if it does not already
exist, and enters vpc-domain configuration
Example:
mode.
switch(config-if)# vpc domain 5
switch(config-vpc-domain)#

Step 3 graceful consistency-check Specifies that only the links on the secondary
peer device are suspended when a mismatch is
Example:
detected in a mandatory compatibility
switch(config-vpc-domain)# graceful parameter.
consistency-check
Use the no form of this command to disable the
feature.

Step 4 exit Exits vpc-domain configuration mode.

Example:
switch(config)# exit
switch#

Step 5 show vpc brief (Optional) Displays information on the vPCs.

Example:
switch# show vpc brief

Example
This example shows how to enable the graceful consistency check feature:
switch# configure terminal
switch(config)# vpc domain 5
switch(config-vpc-domain)# graceful consistency-check
switch(config-vpc-domain)# exit
switch(config)#

Checking the Configuration Compatibility on a vPC Peer Link

After you have configured the vPC peer link on both vPC peer devices, check that the configurations are
consistent on all vPC interfaces. See the “Compatibility Parameters for vPC Interfaces” section for information
about consistent configurations on the vPCs.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
257
 Configuring vPCs
Moving Other Port Channels into a vPC

Command or Action Purpose

Step 2 show vpc consistency-parameters {global | (Optional) Displays the status of those
interface port-channel channel-number} parameters that must be consistent across all
vPC interfaces.
Example:
switch(config)# show vpc
consistency-parameters global
switch(config)#

Example
This example shows how to check that the required configurations are compatible across all the vPC
interfaces:
switch# configure terminal
switch(config)# show vpc consistency-parameters global
switch(config)#

Note Messages regarding the vPC interface configuration compatibility are also logged to the syslog.

Moving Other Port Channels into a vPC

We recommend that you attach the vPC domain downstream port channel to two devices for redundancy.
To connect to the downstream device, you create a port channel from the downstream device to the primary
vPC peer device and you create another port channel from the downstream device to the secondary peer device.
On each vPC peer device, you assign a vPC number to the port channel that connects to the downstream
device. You will experience minimal traffic disruption when you are creating vPCs.

Before you begin

Ensure that you have enabled the vPC feature.
Ensure that you are using a Layer 2 port channel.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface port-channel channel-number Selects the port channel that you want to put
into the vPC to connect to the downstream
Example:
device, and enters interface configuration mode.
switch(config)# interface port-channel
20
switch(config-if)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
258
 Configuring vPCs
Manually Configuring a vPC Domain MAC Address

Command or Action Purpose

Step 3 vpc number Configures the selected port channel into the
vPC to connect to the downstream device. You
Example:
can use any module in the device for these port
switch(config-if)# vpc 5 channels. The range is from 1 and 4096.
switch(config-vpc-domain)#
Note The vPC number that you assign to
the port channel connecting to the
downstream device from the vPC
peer device must be identical on both
vPC peer devices.

Step 4 exit Exits vpc-domain configuration mode.

Example:
switch(config)# exit
switch#

Step 5 show vpc brief (Optional) Displays information on the vPCs.

Example:
switch# show vpc brief

Step 6 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch# copy running-config
startup-config

Example
This example shows how to configure a port channel to connect to the downstream device:
switch# configure terminal
switch(config)# interface port-channel 20
switch(config-if)# vpc 5
switch(config-if)# exit
switch(config)#

Manually Configuring a vPC Domain MAC Address

When you create a vPC domain, the Cisco NX-OS software automatically creates a vPC system MAC address,
which is used for operations that are confined to the link-scope, such as LACP. However, you might choose
to configure the vPC domain MAC address manually.

Before you begin

Ensure that you have enabled the vPC feature.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
259
 Configuring vPCs
Manually Configuring the System Priority

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 vpc domain domain-id [shut | no shut] Enters the vPC domain number that you want
to configure. The system enters vpc-domain
Example:
configuration mode.
switch(config)# vpc domain 5
switch(config-vpc-domain)#

Step 3 system-mac mac-address Enters the MAC address that you want for the
specified vPC domain in the following format:
Example:
aaaa.bbbb.cccc.
switch(config-vpc-domain)# system-mac
23fb.4ab5.4c4e
switch(config-vpc-domain)#

Step 4 exit Exits vpc-domain configuration mode.

Example:
switch(config-vpc-domain)# exit
switch#

Step 5 show vpc role (Optional) Displays the vPC system MAC
address.
Example:
switch# show vpc brief

Step 6 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch# copy running-config
startup-config

Example
This example shows how to manually configure a vPC domain MAC address:
switch# configure terminal
switch(config)# vpc domain 5
switch(config-vpc-domain)# system-mac 13gb.4ab5.4c4e
switch(config-vpc-domain)# exit
switch(config)#

Manually Configuring the System Priority

When you create a vPC domain, the system automatically creates a vPC system priority. However, you can
also manually configure a system priority for the vPC domain.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
260
Configuring vPCs
Manually Configuring the System Priority

Note We recommend that you manually configure the vPC system priority when you are running LACP to ensure
that the vPC peer devices are the primary devices on LACP. When you manually configure the system priority,
ensure that you configure the same priority value on both vPC peer devices. If these values do not match, vPC
does not come up.

Before you begin

Ensure that you have enabled the vPC feature.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 vpc domain domain-id [shut | no shut] Enters the vPC domain number that you want
to configure. The system enters vpc-domain
Example:
configuration mode.
switch(config)# vpc domain 5
switch(config-vpc-domain)#

Step 3 system-priority priority Enters the system priority that you want for the
specified vPC domain. The range of values is
Example:
from 1 to 65535. The default value is 32667.
switch(config-vpc-domain)#
system-priority 4000
switch(config-vpc-domain)#

Step 4 exit Exits vpc-domain configuration mode.

Example:
switch(config-vpc-domain)# exit
switch#

Step 5 show vpc role (Optional) Displays the vPC system priority.
Example:
switch# show vpc role

Step 6 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch# copy running-config
startup-config

Example
This example shows how to manually configure the vPC domain system priority:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
261
 Configuring vPCs
Manually Configuring the vPC Peer Device Role

switch# configure terminal

switch(config)# vpc domain 5
switch(config-vpc-domain)# system-priority 4000
switch(config-vpc-domain)# exit
switch(config)#

Manually Configuring the vPC Peer Device Role

By default, the Cisco NX-OS software elects a primary and secondary vPC peer device after you configure
the vPC domain and both sides of the vPC peer link. However, you might want to elect a specific vPC peer
device as the primary device for the vPC. Then, you would manually configure the role value for the vPC
peer device that you want as the primary device to be lower than the other vPC peer device.
vPCs do not support role preemption. If the primary vPC peer device fails, the secondary vPC peer device
takes over to become operationally the vPC primary device. However, the original operational roles are not
restored if the formerly primary vPC comes up again.

Before you begin

Ensure that you have enabled the vPC feature.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 vpc domain domain-id [shut | no shut] Enters the vPC domain number that you want
to configure. The system enters vpc-domain
Example:
configuration mode.
switch(config)# vpc domain 5
switch(config-vpc-domain)#

Step 3 role priority priority Enters the role priority that you want for the
vPC system priority. The range of values is
Example:
from 1 to 65636, and the default value is 32667.
switch(config-vpc-domain)# role priority A lower value means that this switch has a
4
switch(config-vpc-domain)#
better chance of being the primary vPC.

Step 4 exit Exits vpc-domain configuration mode.

Example:
switch(config)# exit
switch#

Step 5 show vpc role (Optional) Displays the vPC system priority.
Example:
switch# show vpc role

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
262
 Configuring vPCs
Configuring vPC Object Tracking Tracking Feature on a Single-Module vPC

Command or Action Purpose

Step 6 copy running-config startup-config (Optional) Copies the running configuration to
the startup configuration.
Example:
switch# copy running-config
startup-config

Example
This example shows how to manually configure the role priority of the vPC peer device:
switch# configure terminal
switch(config)# vpc domain 5
switch(config-vpc-domain)# role priority 4
switch(config-vpc-domain)# exit
switch(config)#

Configuring vPC Object Tracking Tracking Feature on a Single-Module vPC

If you must configure all the vPC peer links and core-facing interfaces on a single module, you should configure
a track object and a track list that is associated with the Layer 3 link to the core and on all the links on the
vPC peer link on both primary vPC peer devices. Once you configure this feature and if the primary vPC peer
device fails, the system automatically suspends all the vPC links on the primary vPC peer device. This action
forces all the vPC traffic to the secondary vPC peer device until the system stabilizes.
You must put this configuration on both vPC peer devices. Additionally, you should put the identical
configuration on both vPC peer devices because either device can become the operationally primary vPC peer
device.

Before you begin

Ensure that you have enabled the vPC feature.
Ensure that you have configured the track object and the track list. Ensue that you assign all interfaces that
connect to the core and to the vPC peer link to the track-list object on both vPC peer devices.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 vpc domain domain-id [shut | no shut] Enters the vPC domain number that you want
to configure, and enters vpc-domain
Example:
configuration mode.
switch(config)# vpc domain 5
switch(config-vpc-domain)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
263
 Configuring vPCs
Configuring for Recovery After an Outage

Command or Action Purpose

Step 3 track track-object-id Adds the previously configured track-list object
with its associated interfaces to the vPC domain.
Example:
See the Cisco Nexus 9000 Series NX-OS
switch(config-vpc-domain)# track object Unicast Routing Configuration Guide for
23
switch(config-vpc-domain)#
information about configuring object tracking
and track lists.

Step 4 exit Exits vpc-domain configuration mode.

Example:
switch(config-vpc-domain)# exit
switch#

Step 5 show vpc brief (Optional) Displays information about the

tracked objects.
Example:
switch# show vpc brief

Step 6 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch# copy running-config
startup-config

Example
This example shows how to put the previously configured track-list object into the vPC domain on
the vPC peer device:
switch# configure terminal
switch(config)# vpc domain 5
switch(config-vpc-domain)# track object 5
switch(config-vpc-domain)# exit
switch(config)#

Configuring for Recovery After an Outage

If an outage occurs, the vPC waits for a peer adjacency to form on a switch reload. This situation can result
in an unacceptably long service disruption. You can configure the Cisco Nexus 9000 Series device to restore
vPC services when its peer fails to come on line.

Configuring Reload Restore

The reload restore command and procedure described in this section is deprecated. We recommend that you
use the auto-recovery command and procedure described in the “Configuring an Autorecovery” section.
You can configure the Cisco Nexus 9000 Series device to restore vPC services when its peer fails to come
online by using the reload restore command.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
264
Configuring vPCs
Configuring Reload Restore

Before you begin

Ensure that you have enabled the vPC feature.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 vpc domain domain-id [shut | no shut] Enters the vPC domain number that you want
to configure, and enters vpc-domain
Example:
configuration mode.
switch(config)# vpc domain 5
switch(config-vpc-domain)#

Step 3 reload restore [delay time-out] Configures the vPC to assume its peer is not
functional and to bring up the vPC. The default
Example:
delay is 240 seconds. You can configure a
switch(config-vpc-domain)# reload restore time-out delay from 240 to 3600 seconds.
Use the no form of the command to reset the
vPC to its default settings.

Step 4 exit Exits vpc-domain configuration mode.

Example:
switch(config-vpc-domain)# exit
switch#

Step 5 show running-config vpc (Optional) Displays information about the vPC,
specifically the reload status.
Example:
switch# show running-config vpc

Step 6 show vpc consistency-parameters interface (Optional) Displays information about the vPC
port-channel number consistency parameters for the specified
interface.
Example:
switch# show vpc consistency-parameters
interface port-channel 1

Step 7 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch# copy running-config Note To ensure the reload feature is
startup-config enabled, you should perform this
step.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
265
 Configuring vPCs
Configuring an Autorecovery

Example
This example shows how to set the vPC reload restore feature and save it in the switch startup
configuration:
switch# configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
switch(config)# vpc domain 5
switch(config-vpc-domain)# reload restore

Warning:
Enables restoring of vPCs in a peer-detached state after reload, will wait for 240
seconds (by default) to determine if peer is un-reachable

switch(config-vpc-domain)# exit
switch(config)# exit
switch# copy running-config startup-config
switch# show running-config vpc

!Command: show running-config vpc

!Time: Wed Mar 24 18:43:54 2010
version 5.0(2)
feature vpc
logging level vpc 6
vpc domain 5
reload restore

This example shows how to examine the consistency parameters:

switch# show vpc consistency-parameters interface port-channel 1

Legend:
Type 1 : vPC will be suspended in case of mismatch
Name Type Local Value Peer Value
------------- ---- ----------- ---------------
STP Port Type 1 Default -
STP Port Guard 1 None -
STP MST Simulate PVST 1 Default -
mode 1 on -
Speed 1 1000 Mb/s -
Duplex 1 full -
Port Mode 1 trunk -
Native Vlan 1 1 -
MTU 1 1500 -
Allowed VLANs - 1-3967,4048-4093
Local suspended VLANs

Configuring an Autorecovery
You can configure the Cisco Nexus 9000 Series device to restore vPC services when its peer fails to come
online by using the auto-recovery command.
You can configure the Cisco Nexus 9000 Series device to restore vPC services on the secondary vPC peer
when its vPC primary peer fails and bringing down peer-keepalive and peer-link, by using the auto-recovery
command. In case of failure of primary switch where both peer-keepalive and peer-links are down secondary
switch will suspend vPC member. However, after 3 missed keepalive heartbeats secondary switch resumes
the role of a primary switch and bring up vPC member ports. The auto-recovery reload restore command
can be used in scenarios when vPC primary switch reloads, where secondary switch resumes the role of the
vPC primary and bring ip VPC member ports.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
266
Configuring vPCs
Configuring an Autorecovery

Before you begin

Ensure that you have enabled the vPC feature.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 vpc domain domain-id [shut | no shut] Enters the vPC domain number that you want
to configure, and enters vpc-domain
Example:
configuration mode.
switch(config)# vpc domain 5
switch(config-vpc-domain)#

Step 3 auto-recovery [reload-delay time] Configures the vPC to assume its peer is not
functional and to bring up the vPC, and
Example:
specifies the time to wait after a reload to restore
switch(config-vpc-domain)# auto-recovery the vPC. The default delay is 240 seconds. You
can configure a delay from 240 to 3600 seconds.
Use the no form of the command to reset the
vPC to its default settings.

Step 4 exit Exits vpc-domain configuration mode.

Example:
switch(config-vpc-domain)# exit
switch#

Step 5 show running-config vpc (Optional) Displays information about the vPC,
specifically the reload status.
Example:
switch# show running-config vpc

Step 6 show vpc consistency-parameters interface (Optional) Displays information about the vPC
port-channel number consistency parameters for the specified
interface.
Example:
switch# show vpc consistency-parameters
interface port-channel 1

Step 7 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch# copy running-config Note To ensure the autorecovery feature
startup-config is enabled, you should perform this
step.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
267
 Configuring vPCs
Configuring the Suspension of Orphan Ports

Example
This example shows how to set the vPC autorecovery feature and save it in the switch startup
configuration:
switch# configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
switch(config)# vpc domain 5
switch(config-vpc-domain)# auto-recovery
switch(config-vpc-domain)# auto-recovery auto-recovery reload-delay 100

Warning:
Enables restoring of vPCs in a peer-detached state after reload, will wait for 240
seconds to determine if peer is un-reachable

switch(config-vpc-domain)# exit
switch(config)# exit
switch# copy running-config startup-config

Configuring the Suspension of Orphan Ports

When a device that is not vPC-capable connects to each peer, the connected ports are known as orphan ports
because they are not members of a vPC. You can explicitly declare physical interfaces as orphan ports to be
suspended (shut down) by the secondary peer when it suspends its vPC ports in response to a peer link or
peer-keepalive failure. The orphan ports are restored when the vPC is restored.

Note You can configure vPC orphan port suspension only on physical ports, portchannels. However, you cannot
configure the same on individual port channel member ports.

Before you begin

Ensure that you have enabled the vPC feature.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 show vpc orphan-ports (Optional) Displays a list of the orphan ports.
Example:
switch# show vpc orphan-ports

Step 3 interface type slot/port Specifies an interface to configure, and enters

interface configuration mode.
Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
268
 Configuring vPCs
Configuring Delay Restore on an Orphan Port

Command or Action Purpose

switch(config)# interface ethernet 3/1
switch(config-if)#

Step 4 vpc orphan-port suspend Configures the selected interface as a vPC

orphan port to be suspended by the secondary
Example:
peer in the case of a vPC failure.
switch(config-if)# vpc orphan-ports
suspend

Step 5 exit Exits interface configuration mode.

Example:
switch(config-if)# exit
switch#

Step 6 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch# copy running-config
startup-config

Example
This example shows how to configure an interface as a vPC orphan port to be suspended by the
secondary peer in the case of a vPC failure:
switch# configure terminal
switch(config)# interface ethernet 3/1
switch(config-if)# vpc orphan-ports suspend
switch(config-if)# exit
switch(config)#

Configuring Delay Restore on an Orphan Port

You can configure delay restore orphan-port command on Cisco Nexus 9000 Series switches to configure
a restore timer that delays the bringing up of restored device's orphan port starting from Cisco NX-OS Release
7.0(3)I7(1).

Note The delay restore orphan-port command applies only to interfaces that has vpc orphan-port suspend command
configured. Other orphan ports may not delay bringing up devices.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.

Step 2 switch(config) # vpc domain <domain> Configure the VPC domain number.

Step 3 switch(config) # peer-switch Define the peer switch.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
269
 Configuring vPCs
Configuring the vPC Peer Switch

Command or Action Purpose

Step 4 switch(config) # show vpc peer-keepalive Displays information about the peer keepalive
messages

Step 5 switch(config) # delay restore { time } Number of seconds to delay bringing up the
restored vPC peer device. The range is from 1
to 3600.

Step 6 switch(config) # peer-gateway To enable Layer 3 forwarding for packets

destined to the gateway MAC address of the
virtual Port Channel (vPC), use the
peer-gateway command. To disable Layer 3
forwarding packets, use the no form of this
command.
.

Step 7 switch(config) # delay restore orphan-port Number of seconds to delay bringing up the
restored device's orphan port

Configuring the vPC Peer Switch

You can configure the Cisco Nexus 9000 Series device to make a pair of vPC devices appear as a single STP
root in the Layer 2 topology.

Configuring a Pure vPC Peer Switch Topology

You can configure a pure vPC peer switch topology by using the peer-switch command and then setting the
best possible (lowest) spanning tree bridge priority value.

Before you begin

Ensure that you have enabled the vPC feature.

Note When using a non-VPC dedicated trunk link between the VPC peers, the non-VPC VLANs should have a
different global priority on the peers to prevent STP from blocking the VLANs.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
270
Configuring vPCs
Configuring a Pure vPC Peer Switch Topology

Command or Action Purpose

Step 2 vpc domain domain-id [shut | no shut] Enters the vPC domain number that you want
to configure, and enters vpc-domain
Example:
configuration mode.
switch(config)# vpc domain 5
switch(config-vpc-domain)#

Step 3 peer-switch Enables the vPC switch pair to appear as a

single STP root in the Layer 2 topology.
Example:
switch(config-vpc-domain)# peer-switch Use the no form of the command to disable the
peer switch vPC topology.

Step 4 spanning-tree vlan vlan-range priority value Configures the bridge priority of the VLAN.
Valid values are multiples of 4096. The default
Example:
value is 32768.
switch(config)# spanning-tree vlan 1
priority 8192

Step 5 exit Exits vpc-domain configuration mode.

Example:
switch(config-vpc-domain)# exit
switch#

Step 6 show spanning-tree summary (Optional) Displays a summary of the spanning

tree port states including the vPC peer switch.
Example:
switch# show spanning-tree summary

Step 7 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch# copy running-config
startup-config

Example
This example shows how to configure a pure vPC peer switch topology:
switch# configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
switch(config)# vpc domain 5
switch(config-vpc-domain)# peer-switch

2010 Apr 28 14:44:44 switch %STP-2-VPC_PEERSWITCH_CONFIG_ENABLED: vPC peer-switch

configuration is enabled. Please make sure to configure spanning tree "bridge" priority as
per recommended guidelines to make vPC peer-switch operational.

switch(config-vpc-domain)# spanning-tree vlan 1 priority 8192

switch(config-vpc-domain)# exit
switch(config)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
271
 Configuring vPCs
Configuring a Hybrid vPC Peer Switch Topology

Configuring a Hybrid vPC Peer Switch Topology

You can configure a hybrid vPC and non-vPC peer switch topology by using the spanning-tree
pseudo-information command to change the designated bridge ID so that it meets the STP VLAN-based
load-balancing criteria and then change the root bridge ID priority to a value that is better than the best bridge
priority. You then enable the peer switch.

Before you begin

Ensure that you have enabled the vPC feature.
When using a non-VPC dedicated trunk link between the VPC peers, the non-VPC VLANs should have a
different pseudo root priority on the peers to prevent STP from blocking the VLANs.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 spanning-tree pseudo-information Configures the spanning tree pseudo

information.
Example:
switch(config)# spanning-tree
pseudo-information
switch(config-pseudo)#

Step 3 vlan vlan-id designated priority priority Configures the designated bridge priority of the
VLAN. Valid values are multiples of 4096 from
Example:
0 to 61440.
switch(config-pseudo)# vlan 1 designated
priority 8192

Step 4 vlan vlan-id root priority priority Configures the root bridge priority of the
VLAN. Valid values are multiples of 4096 from
Example:
0 to 61440.
switch(config-pseudo)# vlan 1 root
priority 4096

Step 5 vpc domain domain-id [shut | no shut] Enters the vPC domain number that you want
to configure, and enters vpc-domain
Example:
configuration mode.
switch(config)# vpc domain 5
switch(config-vpc-domain)#

Step 6 peer-switch Enables the vPC switch pair to appear as a

single STP root in the Layer 2 topology.
Example:
switch(config-vpc-domain)# peer-switch Use the no form of the command to disable the
peer switch vPC topology.

Step 7 exit Exits vpc-domain configuration mode.

Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
272
 Configuring vPCs
Configuring Hitless vPC Role Change

Command or Action Purpose

switch(config-vpc-domain)# exit
switch#

Step 8 show spanning-tree summary (Optional) Displays a summary of the spanning

tree port states including the vPC peer switch.
Example:
switch# show spanning-tree summary

Step 9 copy running-config startup-config (Optional) Copies the running configuration to

the startup configuration.
Example:
switch# copy running-config
startup-config

Example
This example shows how to configure a hybrid vPC peer switch topology:
switch# configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
switch(config)# spanning-tree pseudo-information
switch(config-pseudo)# vlan 1 designated priority 8192
switch(config-pseudo)# vlan 1 root priority 4096
switch(config-pseudo)# vpc domain 5
switch(config-vpc-domain)# peer-switch
switch(config-vpc-domain)# exit
switch(config)#

Configuring Hitless vPC Role Change

Complete these steps to enable hitless vPC role change.

Before you begin

• • Ensure that the vPC feature is enabled.
• • Ensure that the vPC peer link is up
• • Verify the role priority of devices

Procedure

Command or Action Purpose

Step 1 vpc role preempt Enable hitless vPC role change.
Example:
switch# vpc role preempt
switch(config)#

Step 2 show vpc role (Optional) Verify hitless vPC role change
feature.
Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
273
 Configuring vPCs
Use Case Scenario for vPC Role Change

Command or Action Purpose

switch(config)# show vpc role

Example
This example on how to configure hitless vPC role change:
switch# show vpc rolevPC Role status
----------------------------------------------------
vPC role : secondary
vPC system-mac : 00:23:04:ee:be:01
vPC system-priority : 32667
vPC local system-mac : 8c:60:4f:03:84:41
vPC local role-priority : 32668
vPC peer system-mac : 8c:60:4f:03:84:43
vPC peer role-priority : 32667

! Configure vPC hitless role change on the device!

switch(config)# vpc role preempt

! The following is an output from the show vpc role command after the
vPC hitless feature is configured
switch(config)# show vpc role
vPC Role status
----------------------------------------------------
vPC role : primary
vPC system-mac : 00:00:00:00:00:00
vPC system-priority : 32667
vPC local system-mac : 8c:60:4f:03:84:41
vPC local role-priority : 32666
vPC peer system-mac : 8c:60:4f:03:84:43
vPC peer role-priority : 32667

switch(config)#

Use Case Scenario for vPC Role Change

The hitless vPC role change feature can be used in the following scenarios:
• Role change request—When you want to change the roles of the peer devices in a vPC domain.
• Primary switch reload—When the devices comes up after a reload and roles are defined, you can use the
hitless vPC role change feature to restore the roles. For example, after a reload if the primary device
takes the role of operational secondary and the secondary device takes the role of primary operational,
you can change the vPC peer roles to their original defined roles using the vpc role preempt command.

Note Always check the existing device role priority before switching vPC role.

• Dual-active recovery—In a dual-active recovery scenario, the vPC primary switch continues to be
(operational) primary, but the vPC secondary switch becomes the targeted primary switch and keeps its
vPC member ports up. You can use the vPC hitless feature and restore the device roles. After the
Dual-active recovery, if one side is operational primary and the other side operational secondary, then
you can use the vpc role preempt command to restore the device roles to be primary and secondary

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
274
 Configuring vPCs
Enabling STP to Use the Cisco MAC Address

Enabling STP to Use the Cisco MAC Address

This procedure enables STP to use the Cisco MAC address (00:26:0b:xx:xx:xx).

Before you begin

Ensure that you have enabled the vPC feature.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal

Step 2 vpc domain domain-id Creates a vPC domain if it does not already
exist, and enters vpc-domain configuration
Example:
mode.
switch(config)# vpc domain 5

Step 3 [no] mac-address bpdu source version 2 Enables STP to use the Cisco MAC address
(00:26:0b:xx:xx:xx) as the source address of
Example:
BPDUs generated on vPC ports.
switch(config-vpc-domain)# mac-address
bpdu source version 2

Step 4 exit Exits vpc-domain configuration mode.

Example:
switch(config-vpc-domain)# exit

Step 5 (Optional) copy running-config startup-config Copies the running configuration to the startup
configuration.
Example:
switch(confg)# copy running-config
startup-config

Verifying the vPC Configuration

To display vPC configuration information, perform one of the following tasks:

Command Purpose

show feature Displays whether the vPC is enabled or not.

show vpc brief Displays brief information about the vPCs.

show vpc consistency-parameters Displays the status of those parameters that must be
consistent across all vPC interfaces.

show running-config vpc Displays running configuration information for vPCs.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
275
 Configuring vPCs
Monitoring vPCs

Command Purpose

show port-channel capacity Displays how many port channels are configured and
how many are still available on the device.

show vpc statistics Displays statistics about the vPCs.

show vpc peer-keepalive Displays information about the peer-keepalive

messages.

show vpc role Displays the peer status, the role of the local device,
the vPC system MAC address and system priority,
and the MAC address and priority for the local vPC
device.

Monitoring vPCs
Use the show vpc statisticscommand to display vPC statistics.

Note This command displays the vPC statistics only for the vPC peer device that you are working on.

Configuration Examples for vPCs

The following example shows how to configure vPC on device A as shown in the figure:
Figure 28: vPC Configuration Example

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
276
Configuring vPCs
Configuration Examples for vPCs

1. Enable vPC and LACP.

switch# configure terminal
switch(config)# feature vPC
switch(config)# feature lacp

2. (Optional) Configure one of the interfaces that you want to be a peer link in the dedicated port mode.
switch(config)# interface ethernet 7/1,
ethernet 7/3, ethernet 7/5. ethernet 7/7
switch(config-if)# shutdown
switch(config-if)# exit
switch(config)# interface ethernet 7/1

switch(config-if)# no shutdown
switch(config-if)# exit
switch(config)#

3. (Optional) Configure the second, redundant interface that you want to be a peer link in the dedicated
port mode.
switch(config)# interface ethernet 7/2, ethernet 7/4,
ethernet 7/6. ethernet 7/8
switch(config-if)# shutdown
switch(config-if)# exit
switch(config)# interface ethernet 7/2

switch(config-if)# no shutdown
switch(config-if)# exit
switch(config)#

4. Configure the two interfaces (for redundancy) that you want to be in the peer link to be an active Layer
2 LACP port channel.
switch(config)# interface ethernet 7/1-2
switch(config-if)# switchport
switch(config-if)# switchport mode trunk
switch(config-if)# switchport trunk allowed vlan 1-50
switch(config-if)# switchport trunk native vlan 20
switch(config-if)# channel-group 20 mode active
switch(config-if)# exit

5. Create and enable the VLANs.

switch(config)# vlan 1-50
switch(config-vlan)# no shutdown
switch(config-vlan)# exit

6. Create a separate VRF for the vPC peer-keepalive link and add a Layer 3 interface to that VRF.
switch(config)# vrf context pkal
switch(config-vrf)# exit
switch(config)# interface ethernet 8/1
switch(config-if)# vrf member pkal
switch(config-if)# ip address 172.23.145.218/24
switch(config-if)# no shutdown
switch(config-if)# exit

7. Create the vPC domain and add the vPC peer-keepalive link.
switch(config)# vpc domain 1
switch(config-vpc-domain)# peer-keepalive
destination 172.23.145.217 source 172.23.145.218 vrf pkal
switch(config-vpc-domain)# exit

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
277
 Configuring vPCs
Related Documents

8. Configure the vPC peer link.

switch(config)# interface port-channel 20
switch(config-if)# switchport mode trunk
switch(config-if)# switchport trunk allowed vlan 1-50
switch(config-if)# vpc peer-link
switch(config-if)# exit
switch(config)#

9. Configure the interface for the port channel to the downstream device of the vPC.
switch(config)# interface ethernet 7/9
switch(config-if)# switchport mode trunk
switch(config-if)# allowed vlan 1-50
switch(config-if)# native vlan 20
switch(config-if)# channel-group 50 mode active
switch(config-if)# exit
switch(config)# interface port-channel 50
switch(config-if)# vpc 50
switch(config-if)# exit
switch(config)#

10. Save the configuration.

switch(config)# copy running-config startup-config

Note If you configure the port channel first, ensure that it is a Layer 2 port channel.

Related Documents
Related Topic Related Topic

System management System management

High availability High availability

Release Notes Release Notes

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
278
 CHAPTER 9
Configuring IP Tunnels
This chapter describes how to configure IP tunnels using Generic Route Encapsulation (GRE) on Cisco NX-OS
devices.
• Information About IP Tunnels, on page 279
• Prerequisites for IP Tunnels, on page 281
• Guidelines and Limitations, on page 281
• Default Settings, on page 285
• Configuring IP Tunnels, on page 286
• Verifying the IP Tunnel Configuration, on page 298
• Configuration Examples for IP Tunneling, on page 298
• Related Documents, on page 299

Information About IP Tunnels

IP tunnels can encapsulate a same-layer or higher layer protocol and transport the result over IP through a
tunnel created between two devices.

IP Tunnel Overview
IP tunnels consists of the following three main components:
• Passenger protocol—The protocol that needs to be encapsulated. IPv4 is an example of a passenger
protocol.
• Carrier protocol—The protocol that is used to encapsulate the passenger protocol. Cisco NX-OS supports
GRE as a carrier protocol.
• Transport protocol—The protocol that is used to carry the encapsulated protocol. IPv4 is an example of
a transport protocol. An IP tunnel takes a passenger protocol, such as IPv4, and encapsulates that protocol
within a carrier protocol, such as GRE. The device then transmits this carrier protocol over a transport
protocol, such as IPv4.

You configure a tunnel interface with matching characteristics on each end of the tunnel.
You must enable the tunnel feature before you can configure it. The system automatically takes a checkpoint
prior to disabling the feature, and you can roll back to this checkpoint. See the Cisco Nexus 9000 Series
NX-OS System Management Configuration Guide for information about rollbacks and checkpoints.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
279
 Configuring IP Tunnels
GRE Tunnels

GRE Tunnels
You can use generic routing encapsulation (GRE) as the carrier protocol for a variety of passenger protocols.
The following figure shows the IP tunnel components for a GRE tunnel. The original passenger protocol
packet becomes the GRE payload and the device adds a GRE header to the packet. The device then adds the
transport protocol header to the packet and transmits it.
Figure 29: GRE PDU

Point-to-Point IP-in-IP Tunnel Encapsulation and Decapsulation

For 6.1(2)I3(4), 7.0(3)I1(2), and later, point-to-point IP-in-IP encapsulation and decapsulation is a type of
tunnel that you can create to send encapsulated packets from a source tunnel interface to a destination tunnel
interface. This type of tunnel will carry both inbound and outbound traffic.

Note The selection of IP-in-IP tunnel based on the PBR policy is not supported.

Multi-Point IP-in-IP Tunnel Decapsulation

For 6.1(2)I3(4), 7.0(3)I1(2), and later, multi-point IP-in-IP decapsulate-any is a type of tunnel that you can
create to decapsulate packets from any number of IP-in-IP tunnels to one tunnel interface. This tunnel will
not carry any outbound traffic. However, any number of remote tunnel endpoints can use a tunnel configured
this way as their destination.

Path MTU Discovery

Path maximum transmission unit (MTU) discovery (PMTUD) prevents fragmentation in the path between
two endpoints by dynamically determining the lowest MTU along the path from the packet's source to its
destination. PMTUD reduces the send MTU value for the connection if the interface receives information that
the packet would require fragmentation.
When you enable PMTUD, the interface sets the Don't Fragment (DF) bit on all packets that traverse the
tunnel. If a packet that enters the tunnel encounters a link with a smaller MTU than the MTU value for the
packet, the remote link drops the packet and sends an ICMP message back to the sender of the packet. This
message indicates that fragmentation was required (but not permitted) and provides the MTU of the link that
dropped the packet.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
280
 Configuring IP Tunnels
High Availability

Note PMTUD on a tunnel interface requires that the tunnel endpoint can receive ICMP messages generated by
devices in the path of the tunnel. Check that ICMP messages can be received before using PMTUD over
firewall connections.

High Availability
IP tunnels support stateful restarts. A stateful restart occurs on a supervisor switchover. After the switchover,
Cisco NX-OS applies the runtime configuration after the switchover.

Prerequisites for IP Tunnels

IP tunnels have the following prerequisites:
• You must be familiar with TCP/IP fundamentals to configure IP tunnels.
• You are logged on to the switch.
• You must enable the tunneling feature in a device before you can configure and enable any IP tunnels.

Guidelines and Limitations

IP tunnels have the following configuration guidelines and limitations:
• Guidelines for source-direct and ipv6ipv6-decapsulate-any options for tunnels:
• Beginning with Cisco NX-OS Release 7.0(3)I7(3), the source-direct command is supported on the
Cisco Nexus 9500 platform switches with the Application Spine Engine (ASE) and the Leaf Spine
Engine (LSE).
Cisco Nexus 9500 platform switches with the Network Forwarding Engine (NFE) do not support
the tunnel source direct command.
The tunnel source direct command with the tunnel mode ipv6ipv6 decapsulate-any command
on the Cisco Nexus 9500 platform switches is only supported in the MPLS heavy routing template.
• The IP tunnel supports the tunnel source CLI command with interface, IPv4 address, IPv6 address,
or IPv4 prefix. Beginning with Cisco NX-OS Release 7.0(3)I6(1), you can configure IP-in-IP tunnel
decapsulation on directly connected IP addresses (for example, physical interface, port-channel,
loopback, and SVI) using the new tunnel source direct CLI command. You can select the IP ECMP
links when there are multiple IP links between the two switches. A single tunnel interface can
decapsulate the tunneled packets whose outer destination IP is any of the IPv4 or IPv6 address that
is locally configured and it is operationally Up in the switch.
• Currently, tunnel mode ipip decapsulate-any is supported for decapsulating IPv4 payload over
IPv4 transport (IPv4inIPv4 packets). Beginning with Cisco NX-OS Release 7.0(3)I6(1), the new
CLI tunnel mode ipv6ipv6 decapsulate-any command is introduced to support IPv6 payload over
IPv6 transport (IPv6inIPv6 packets).

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
281
 Configuring IP Tunnels
Guidelines and Limitations

• The tunnel source direct and tunnel mode ipv6ipv6 decapsulate-any CLI commands are not
supported on Cisco Nexus 9500 platform switches with the Network Formation Engine (NFE).
• The tunnel source direct CLI command is supported only when an administrator uses the IP-in-IP
decapsulation to source route the packets through the network. The source-direct tunnel is always
operationally Up unless it is administratively shut down. The directly connected interfaces are
identified using the show ip route direct CLI command.
• The tunnel source direct CLI command is supported only on decapsulate-any tunnel modes, for
example, tunnel mode ipip decapsulate-any and tunnel mode ipv6ipv6 decapsulate-any.
• Auto-recovery for source-direct is not supported.
• For ipv6ipv6 decapsulate-any, inter-VRF is not supported. The tunnel interface VRF (iVRF) and
tunnel transport or forwarding VRF (fVRF) must be the same. Only one decapsulate-any tunnel
(irrespective of VRF) can be present in Cisco Nexus 9200 and 9300-EX Series switches.
• To enable IPv6 on ipv6ipv6 decap-any tunnel interface, configure a valid IPv6 address or configure
ipv6 address use-link-local-only under the tunnel interface.

• See the following hardware limitations on the maximum sources that can be accommodated on a source
direct tunnel and the related behavior:
• Source direct tunnel is now supported for Cisco Nexus 9000 Series switches with Network Forwarding
Engine (NFE), Application Spine Engine (ASE), and Leaf Spine Engine (LSE). Most of the
limitations are only in case of scaled SIP (number of total IP/IPv6 addresses on the interfaces (L3,
sub-interface, PC, PC-sub interfaces, loopback, SVI, and any secondary IP/IPv6 addresses.)
See the following sample use cases.
• Use Case 1: Non-deterministic behavior of which SIP gets installed if the number of IP/IPv6
interface scale is more.
Both the switches have 512 entries for tunnel SIP. With tunnel source, direct any IP or IPv6
address w.r.t ipip or ipv6ipv6 decap any with tunnel source gets installed in the above table.
The insertion of these entries is on a first come first serve basis without any CLI command to
control which interface IP addresses get installed. If the system has more number of IP/IPv6
interfaces to be installed, the behavior is non-deterministic (The behavior can change across
reload with interface flaps.)
• Use Case 2: The scale numbers are different in both switches and each has its own advantages
and disadvantages.
IPv4 individual scale can be more (up to 512) in case of switches with NFE but it is shared
with IPv6. In the switches with ASE and LSE, the IPv4 individual scale can be 256 but it is
not shared with IPv6.
Whenever the tunnel decap table gets filled, the TABLE_FULL error is displayed. If some
entry gets deleted after the table gets full, the table full error is cleared.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
282
Configuring IP Tunnels
Guidelines and Limitations

Table 15: Scale Numbers

Commands Switches with NFE: Table size Switches with ASE and LSE:
512, v4 takes 1 entry, v6 takes Table size 512, v4 takes 1
4 entries entry, v6 takes 2 entries
(paired index)

IPIP decap any with tunnel Shared between v4 and v6, v6 Dedicated 256
source direct takes 4 entries
v4 + 4 *v6 =512
Maximum entries can be 512
with no v6 entries

IPv6IPv6 decap any with Shared between v4 and v6, v6 Dedicated 128
tunnel source direct takes 4 entries
v4 + 4 *v6 =512
Maximum entries can be 128
with no v4 entries

• Use Case 3: Auto-recovery is not supported.

If any entry does not get installed in the hardware due to exhaustion of above table, removal
of an already installed IP/IPv6 from interfaces does not automatically trigger the addition of
the failed SIP in the table though the table has space now. You need to flap the tunnel interface
or IP interface to get them installed.
However, if an entry does not get installed in the hardware due to a duplicate entry (if there
was already a decap-any with one source present and now the source direct tunnel CLI
command is configured, there is a duplicate entry for the prior source configured) that was
taken care of by removing the entry only when both the tunnels get deleted.

• For Cisco Nexus 9000 Series switches with Network Forwarding Engine (NFE) and Application
Spine Engine (ASE), the syslog is different as the dedicated IPv4 and IPv6 decap antries are carved
in the syslog. If the tunnel-decap-table is full, the user gets a syslog as follows:

2017 Apr 6 12:18:04 switch %$ VDC-1 %$ %IPFIB-2-FIB_HW_IPV4_TUNNEL_DECAP_TABLE_FULL:

IPv4 tunnel decap hardware table full.
IP tunnel decapsulation may not work for some GRE/IPinIP traffic

2017 Apr 6 12:18:11 switch %$ VDC-1 %$ %IPFIB-2-FIB_HW_IPV6_TUNNEL_DECAP_TABLE_FULL:

IPv6 tunnel decap hardware table full.
IP tunnel decapsulation may not work for some GRE/IPinIP traffic

If the table is full and once some entry gets deleted from the table (due to an interface being
operationally down or removal of IP address), the clear syslog for the table is displayed. Note that
the deletion of the tunnel removes all the entries that are added as part of that tunnel.

2017 Apr 5 13:29:25 switch %$ VDC-1 %$

%IPFIB-2-FIB_HW_IPV4_TUNNEL_DECAP_TABLE_FULL_CLRD: IPv4 tunnel decap hardware table
full exception cleared

2017 Apr 4 19:41:22 switch %$ VDC-1 %$

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
283
 Configuring IP Tunnels
Guidelines and Limitations

%IPFIB-2-FIB_HW_IPV6_TUNNEL_DECAP_TABLE_FULL_CLRD: IPv6 tunnel decap hardware table

full exception cleared

• Beginning with Cisco NX-OS Release 7.0(3)I6(1), IP-in-IP tunnel decapsulation is supported on IPv6
enabled networks.

!
interface tunnel 1
ipv6 address use-link-local-only <<< enable IPv6
tunnel mode ipv6ipv6 decapsulate-any
tunnel source direct
description IPinIP Decapsulation Interface
mtu 1476
no shutdown

• The show commands with the internal keyword are not supported.
• Cisco NX-OS supports only the following protocols:
• IPv4 passenger protocol.
• GRE carrier protocol.

• Cisco NX-OS supports the following maximum number of tunnels:

• IP tunnels - 8 tunnels.
• GRE and IP-in-IP regular tunnels - 8 tunnels. (7.0(3)I1(2) and later)

• IP tunnels do not support access control lists (ACLs) or QoS policies.

• Cisco NX-OS supports the GRE header defined in IETF RFC 2784. Cisco NX-OS does not support
tunnel keys and other options from IETF RFC 1701.
• Cisco NX-OS does not support GRE tunnel keepalives.
• All unicast routing protocols are supported by IP tunnels.
• The IP tunnel interface cannot be configured to be a span source or destination.
• IP tunnels do not support PIM or other Multicast features and protocols. (7.0(3)I1(2) and later)
• The selection of IP-in-IP tunnel based on the PBR policy is not supported. (7.0(3)I1(2) and later)
• The feature tunnel feature on Cisco Nexus 9000 switches cannot co-exist with the VXLAN feature
feature nv overlay.
• IP tunnels are supported only in the default system routing mode and not in other modes. (7.0(3)I1(2)
and later)
• When configuring a tunnel interface to ipip mode, the maximum mtu value is 9192 (7.0(3)I2(2) and
earlier) or 9196 (7.0(3)I3(1) and later).
When downgrading from NX-OS 7.0(3)I3(1) (or later release) to an earlier release, with a tunnel interface
in ipip mode that has an mtu value of 9196, the mtu configuration is lost as a result of the downgrade
operation. As a best practice, adjust the mtu value to 9192 before commencing the downgrade to avoid
losing the mtu configuration.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
284
 Configuring IP Tunnels
Default Settings

• When configuring a tunnel interface to ipip mode, the default mtu value is 1476 (7.0(3)I3(1) and earlier)
or 1480 (7.0(3)I4(1) and later).
When downgrading from NX-OS 7.0(3)I4(1) (or later release) to an earlier release, with a tunnel interface
in ipip mode with no explicit mtu configuration, the mtu value changes as a result of the downgrade
operation from 1480 to 1476. As a best practice, adjust the mtu value to 1476 before commencing the
downgrade to avoid any changes to the mtu value.
When upgrading from NX-OS 7.0(3)I3(1) (or earlier release) to NX-OS 7.0(3)I4(1) (or later release),
with a tunnel interface in ipip mode with no explicit mtu configuration, the mtu value changes as a result
of the upgrade operation from 1476 to 1480. As a best practice, adjust the mtu value to 1480 before
commencing the upgrade to avoid any changes to the mtu value.
• On Cisco Nexus 9200 Series switches, GRE packets that are received on an IP-in-IP tunnel are not
dropped as expected and are instead forwarded to the packet destination.
• Tx packects originating from the switch, such as control pkts, are not included in Tx statistics.
• Tunnel destinations that are reachable over another tunnel are not supported.
• The consistency checker is not supported for routes over a tunnel.
• Non-IP routing protocols, such as isis, are not supported over IP-in-IP tunnels.
• RFC5549 is not be supported over tunnels.
• BGP adjacency over tunnel is not supported in a scenario where the tunnel interface and tunnel source
are in same VRF (example: VRF-A) and tunnel destination is reachable with route-leak from opposite
end (example: via VRF-B)
• Configuring two tunnel interfaces with the same source and destination address is not supported. Loopback
interfaces may be configured as the source addresses instead.
• Cisco Nexus 9200, 9300-EX, 9300-FX, 9300-FX2 series switches and Cisco Nexus 9500 platform
switches with 9700-EX/FX line cards may not have multiple tunnel interfaces in a single VRF that are
sourced from or destined to the same IP address. For example, a device may not have tunnel 0 and tunnel
1 interfaces in the default VRF that are sourced from the same IP address or interface.

Default Settings
The following table lists the default settings for IP tunnel parameters.

Table 16: Default IP Tunnel Parameters

Parameters Default

Path MTU discovery age timer 10 minutes

Path MTU discovery minimum MTU 64

Tunnel feature Disabled

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
285
 Configuring IP Tunnels
Configuring IP Tunnels

Configuring IP Tunnels

Note If you are familiar with the Cisco IOS CLI, be aware that the Cisco NX-OS commands for this feature might
differ from the Cisco IOS commands that you would use.

Enabling Tunneling
You must enable the tunneling feature before you can configure any IP tunnels.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 feature tunnel Allows the creation of a new tunnel interface.

Example: To disable the tunnel interface feature, use the
switch(config)# feature tunnel no form of this command.
switch(config-if)#
Note The feature tunnel command may
break the multicast functionality if
multicast heavy template is enforced.

Step 3 exit Exits the interface mode and returns to the

configuration mode.
Example:
switch(config-if)# exit
switch#

Step 4 show feature (Optional) Displays information about the

features enabled on the device.
Example:
switch(config-if)# show feature

Step 5 copy running-config startup-config (Optional) Saves this configuration change.

Example:
switch(config-if)# copy running-config
startup-config

Creating a Tunnel Interface

You can create a tunnel interface and then configure this logical interface for your IP tunnel.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
286
Configuring IP Tunnels
Creating a Tunnel Interface

Note Cisco NX-OS supports a maximum of 8 IP tunnels.

Note Use the no interface tunnel command to remove the tunnel interface and all associated configuration.

Command Purpose

no interface tunnel number Deletes the tunnel interface and the

associated configuration.
Example:
switch(config)# no interface tunnel 1

description string Configures a description for the tunnel.

Example:
switch(config-if)# description GRE tunnel

mtu value Sets the MTU of IP packets sent on an

interface.
Example:
switch(config-if)# mtu 1400

tunnel ttl value Sets the tunnel time-to-live value. The

range is from 1 to 255.
Example:
switch(config-if)# tunnel ttl 100

Before you begin

You can configure the tunnel source and the tunnel destination in different VRFs. Ensure that you have enabled
the tunneling feature.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface tunnel number Creates a new tunnel interface.

Example:
switch(config)# interface tunnel 1
switch(config-if)#

Step 3 tunnel mode {gre ip | ipip {ip | Sets this tunnel mode to GRE, ipip, or ipip
decapsulate-any}} decapsulate-only.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
287
 Configuring IP Tunnels
Creating an IP-in-IP Tunnel with a Netmask

Command or Action Purpose

The gre and ip keywords specify that GRE
encapsulation over IP will be used.
The ipip keyword specifies that IP-in-IP
encapsulation will be used. The optional
decapsulate-any keyword terminates IP-in-IP
tunnels at one tunnel interface. This keyword
creates a tunnel that will not carry any outbound
traffic. However, remote tunnel endpoints can
use a tunnel configured as their destination.

Step 4 tunnel source {ip-address |interface-name} Configures the source address for this IP tunnel.
The source can be specified by IP address or
Example:
logical interface name.
switch(config-if)# tunnel source
ethernet 1/2

Step 5 tunnel destination {ip-address |host-name} Configures the destination address for this IP
tunnel. The destination can be specified by IP
Example:
address or logical host name.
switch(config-if)# tunnel destination
192.0.2.1

Step 6 tunnel use-vrf vrf-name (Optional) Uses the configured VRF to look up
the tunnel IP destination address.
Example:
switch(config-if)# tunnel use-vrf blue

Step 7 show interfaces tunnel number (Optional) Displays the tunnel interface
statistics.
Example:
switch# show interfaces tunnel 1

Step 8 copy running-config startup-config (Optional) Saves this configuration change.

Example:
switch(config-if)# copy running-config
startup-config

Example
This example shows how to create a tunnel interface
switch# configure terminal
switch(config)# interface tunnel 1
switch(config-if)# tunnel source ethenet 1/2
switch(config-if)# tunnel destination 192.0.2.1
switch(config-if)# copy running-config startup-config

Creating an IP-in-IP Tunnel with a Netmask

Creating an IP-in-IP tunnel with a netmask allows you to specify a tunnel source subnet and a tunnel destination
subnet, and decap the packet if it matches. (7.0(3)I2(1) and later)

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
288
Configuring IP Tunnels
Creating an IP-in-IP Tunnel with a Netmask

• The IP-in-IP decap-any tunnel receives encapsulated packets from any number of IP-in-IP tunnels.
• With the netmask feature, the switch receives packets from IP addresses which comply with the netmasks.

Notes for the netmask feature:

• Routing protocols are not supported on an IP-in-IP tunnel created with a netmask.
• Encap is not supported with the netmask feature; only decap from a set of sources in the same subnet is
supported.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface tunnel number Creates a new tunnel interface.

Example:
switch(config)# interface tunnel 5
switch(config-if)#

Step 3 tunnel mode ipip [ip] Sets this tunnel mode to ipip.
The ipip keyword specifies that IP-in-IP
encapsulation will be used.

Step 4 tunnel source ip-address / mask_length Configures the source address for this IP tunnel.
The source is specified by IP address and length
Example:
of mask.
switch(config-if)# tunnel source
33.1.1.1 255.255.255.0

Step 5 tunnel destination ip-address / mask_length Configures the destination address for this IP
tunnel. The destination is specified by IP
Example:
address and length of mask.
switch(config-if)# tunnel destination
33.1.1.2 255.255.255.0

Step 6 (Optional) no shut Clears the interface.

Step 7 ip address ip-prefix/length Configures an IP address for this interface.

Example:
switch(config-if)# ip address 50.1.1.1/24

Example
The following example shows how to create an IP-in-IP tunnel with a netmask.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
289
 Configuring IP Tunnels
Configuring a Tunnel Interface

switch(config)# interface tunnel 10

switch(config-if)# tunnel mode ipip
switch(config-if)# tunnel source 33.1.1.2/24
switch(config-if)# tunnel destination 33.1.1.1/24
switch(config-if)# no shut
switch(config-if)# ip address 10.10.10.10/24
switch(config-if)# end
switch# show interface tunnel 10
Tunnel10 is up
Admin State: up
Internet address is 10.10.10.10/24
MTU 1476 bytes, BW 9 Kbit
Tunnel protocol/transport IPIP/IP
Tunnel source 33.1.1.2, destination 33.1.1.1
Transport protocol is in VRF "default"
Last clearing of "show interface" counters never
Tx
0 packets output, 0 bytes
Rx
0 packets input, 0 bytes

switch# show run interface tunnel 10

!Command: show running-config interface Tunnel10

!Time: Wed Aug 26 13:50:01 2015

version 7.0(3)I2(1)

interface Tunnel10
ip address 10.10.10.10/24
tunnel mode ipip ip
tunnel source 33.1.1.2 255.255.255.0
tunnel destination 33.1.1.1 255.255.255.0
no shutdown

Configuring a Tunnel Interface

You can set a tunnel interface to GRE tunnel mode, ipip mode, or ipip decapsulate-only mode. GRE mode is
the default tunnel mode. (7.0(3)I1(2) and later).
Beginning with Cisco NX-OS Release 7.0(3)I6(1), the tunnel source direct and tunnel mode ipv6ipv6
decapsulate-any CLI commands are supported on Cisco Nexus 9000 Series switches.

Note The tunnel source direct and tunnel mode ipv6ipv6 decapsulate-any CLI commands are not supported on
Cisco Nexus 9500 platform switches with the Network Forwarding Engine (NFE).

The new CLI tunnel mode ipv6ipv6 decapsulate-any command is introduced to support IPv6 payload over
IPv6 transport (IPv6inIPv6 packets). You can configure IP-in-IP tunnel decapsulation on directly connected
IP addresses (for example, physical interface, port-channel, loopback, and SVI) using the new tunnel source
direct CLI command.

Before you begin

Ensure that you have enabled the tunneling feature.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
290
Configuring IP Tunnels
Configuring a Tunnel Interface

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface tunnel number Creates a new tunnel interface.

Example:
switch(config)# interface tunnel 1
switch(config-if)#

Step 3 tunnel mode {gre ip | ipip | {ip | Sets this tunnel mode to GRE, ipip, or ipip
decapsulate-any}} decapsulate-only.
The gre and ip keywords specify that GRE
encapsulation over IP will be used.
The ipip keyword specifies that IP-in-IP
encapsulation will be used. The optional
decapsulate-any keyword terminates IP-in-IP
tunnels at one tunnel interface. This keyword
creates a tunnel that will not carry any outbound
traffic. However, remote tunnel endpoints can
use a tunnel configured as their destination.

Step 4 (Optional) tunnel mode ipv6ipv6 Supports IPv6 payload over IPv6 transport
decapsulate-any (IPv6inIPv6 packets) ( 7.0(3)I6(1) and later).
This step is applicable for IPv6 networks only.
Note This command is not supported on
Cisco Nexus 9500 platform switches.

Step 5 tunnel source direct Configures IP-in-IP tunnel decapsulation on

any directly connected IP addresses. Beginning
with Cisco NX-OS Release 7.0(3)I6(1), this
option is now supported only when the IP-in-IP
decapsulation is used to source route the packets
through the network.
Note This command is not supported on
Cisco Nexus 9500 platform switches
with the Network Forwarding Engine
(NFE).

Step 6 show interfaces tunnel number (Optional) Displays the tunnel interface
statistics.
Example:
switch(config-if)# show interfaces tunnel
1

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
291
 Configuring IP Tunnels
Configuring a Tunnel Interface

Command or Action Purpose

Step 7 mtu value Sets the maximum transmission unit (MTU) of
IP packets sent on an interface.
The range is from 64 to 9192 units.
Note When configuring tunnel mode
ipip, the range is dependent on
NX-OS release:
• 64 to 9192 units (7.0(3)I2(2)
and earlier)
• 64 to 9196 units (7.0(3)I3(1)
and later)

Step 8 copy running-config startup-config (Optional) Saves this configuration change.

Example:
switch(config-if)# copy running-config
startup-config

Example
This example shows how to create the tunnel interface to GRE:
switch# configure terminal
switch(config)# interface tunnel 1
switch(config-if)# tunnel mode gre ip
switch(config-if)# copy running-config startup-config

This example shows how to create an ipip tunnel:

switch# configure terminal
switch(config)# interface tunnel 1
switch(config-if)# tunnel mode ipip
switch(config-if)# mtu 1400
switch(config-if)# copy running-config startup-config
switch(config-if)# no shut

This example shows how to configure IP-in-IP tunnel decapsulation on directly connected IP addresses:
switch# configure terminal
switch(config)# interface tunnel 0
switch(config-if)# tunnel mode ipip ip
switch(config-if)# tunnel source direct
switch(config-if)# description IPinIP Decapsulation Interface
switch(config-if)# no shut

This example shows how to configure IP-in-IP tunnel decapsulation on IPv6 enabled networks:

!
interface tunnel 1
ipv6 address use-link-local-only <<< enable IPv6
tunnel mode ipv6ipv6 decapsulate-any
tunnel source direct
description IPinIP Decapsulation Interface
mtu 1476

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
292
 Configuring IP Tunnels
Configuring a GRE Tunnel

no shutdown

show running-config interface tunnel 1

interface Tunnel1
tunnel mode ipv6ipv6 decapsulate-any
tunnel source direct
no shutdown

show interface tunnel 1

Tunnel1 is up Admin State: up
MTU 1460 bytes, BW 9 Kbit
Tunnel protocol/transport IPv6/DECAPANY/IPv6
Tunnel source - direct
Transport protocol is in VRF "default"
Tunnel interface is in VRF "default"
Last clearing of "show interface" counters never
Tx 0 packets output, 0 bytes Rx 0 packets input, 0 bytes

Configuring a GRE Tunnel

You can set a tunnel interface to GRE tunnel mode. (6.1(2)I3(3) and 7.0(3)I1(1))

Note Cisco NX-OS supports only the GRE protocol for IPV4 over IPV4.

Before you begin

Ensure that you have enabled the tunneling feature.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface tunnel number Creates a new tunnel interface.

Example:
switch(config)# interface tunnel 1
switch(config-if)#

Step 3 tunnel mode gre ip Sets this tunnel mode to GRE.

Example:
switch(config-if)# tunnel mode gre ip

Step 4 show interfaces tunnel number (Optional) Displays the tunnel interface
statistics.
Example:
switch(config-if)# show interfaces tunnel
1

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
293
 Configuring IP Tunnels
Configuring a GRE Tunnel

Command or Action Purpose

Step 5 copy running-config startup-config (Optional) Saves this configuration change.
Example:
switch(config-if)# copy running-config
startup-config

Configuring a GRE Tunnel

GRE v6 tunnel is used to carry different types of packets over IPv6 transport. GREv6 tunnel carries only IPv4
payload. The tunnel CLIs are enhanced to select IPv6 tunnel and configure v6 tunnel source and destination.
(7.0(3)I2(1) and later)
You can set a tunnel interface to GRE tunnel mode, ipip mode, or ipip decapsulate-only mode. GRE mode is
the default tunnel mode.

Before you begin

Ensure that you have enabled the tunneling feature.

Procedure

Command or Action Purpose

Step 1 switch# configure terminal Enters global configuration mode.

Step 2 switch(config)# interface tunnel number Enters a tunnel interface configuration mode.

Step 3 switch(config-if)# tunnel mode {gre ip | ipip Sets this tunnel mode to GRE, ipip, or ipip
{ip | decapsulate-any}} decapsulate-only.
The gre and ip keywords specify that GRE
encapsulation over IP will be used.
The ipip keyword specifies that IP-in-IP
encapsulation will be used. The optional
decapsulate-any keyword terminates IP-in-IP
tunnels at one tunnel interface. This keyword
creates a tunnel that will not carry any outbound
traffic. However, remote tunnel endpoints can
use a tunnel configured as their destination.

Step 4 Required: switch(config-if)# tunnel use-vrf Configures tunnel VRF name.

vrf-name
Step 5 Required: switch(config-if)# ipv6 address IPv6 Configures the IPv6 address.
address
Note The tunnel source and the destination
addresses are still the same (IPv4
address.)

Step 6 (Optional) switch(config-if)# show interface Displays the tunnel interface statistics.
tunnel number

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
294
Configuring IP Tunnels
Configuring a GRE Tunnel

Command or Action Purpose

Step 7 switch(config-if)# mtu value Sets the maximum transmission unit (MTU) of
IP packets sent on an interface.

Step 8 (Optional) switch(config-if)# copy Saves the change persistently through reboots
running-config startup-config and restarts by copying the running
configuration to the startup configuration.

Example
This example displays how to configure IPv6 Payload over GRE v4 tunnel. Configure the tunnel
source, destination, IPv4 address, IPv6 address, and perform the no shut command. Once the GREv4
tunnel is created, you can configure v4 or v6 route via the tunnel:
switch# configure terminal
switch(config)# interface tunnel 10
switch(config)# tunnel source 11.1.1.1
switch(config)# tunnel destination 11.1.1.2
switch(config-if)# tunnel mode gre ip
switch(config-if)# tunnel use-vrf red
switch(config-if)# ip address 10.1.1.1/24
switch(config-if)# ipv6 address 2:2::2/64
switch(config-if)# no shut

switch(config)# ip route 50.1.1.0/24 tunnel 10

switch(config)# ipv6 route 2000:100::/64 tunnel 10

This example shows how to view the GRE v4 tunnel interface 10 and display IPv4 and IPv6 routes:
switch(config)# show int tunnel 10
Tunnel 10 is up
Admin State: up
Internet address(es):
10.1.1.1/24
1010::1/64
MTU 1476 bytes, BW 9 Kbit
Tunnel protocol/transport GRE/IP
Tunnel source 11.1.1.1, destination 11.1.1.2
Transport protocol is in VRF "default"

switch#show ipv6 route

…
2000:100::/64, ubest/mbest: 1/0, attached
*via Tunnel10, [1/0], 00:00:16, static

#show ip route
…
50.1.1.0/24, ubest/mbest: 1/0
*via Tunnel10, [1/0], 00:03:33, static

This example displays how to configure IPv4 payload over GRE v6 tunnel. Configure the tunnel
mode as GRE IPv6, tunnel v6 source and destination, IPv4 address, and perform the no shut command.
Once the GREv6 tunnel is created, you can configure v4 route via the tunnel:
switch# configure terminal
switch(config)# interface tunnel 20
switch(config-if)# tunnel mode gre ipv6

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
295
 Configuring IP Tunnels
Enabling Path MTU Discovery

switch(config)# tunnel source 1313::1

switch(config)# tunnel destination 1313::2
switch(config-if)# tunnel use-vrf red
switch(config-if)# ip address 20.1.1.1/24
switch(config-if)# no shut

switch(config)# ip route 100.1.1.0/24 tunnel 20

This example displays how to view the GREv6 tunnel interface 20:
show interface tunnel 20
Tunnel 20 is up
Admin State: up
Internet address is 20.1.1.1/24
MTU 1456 bytes, BW 9 Kbit
Tunnel protocol/transport GRE/IPv6
Tunnel source 1313::1, destination 1313::2
Transport protocol is in VRF "default"

#show ip route
…
100.1.1.0/24, ubest/mbest: 1/0
*via Tunnel20, [1/0], 00:01:00, static

red10# show interface brief | grep Tunnel

Tunnel10 up 10.1.1.1/24 GRE/IP 1476
Tunnel20 up 20.1.1.1/24 GRE/IPv6 1456

This example shows how to create an ipip tunnel:

switch# configure terminal
switch(config)# interface tunnel 1
switch(config-if)# tunnel mode ipip
switch(config-if)# mtu 1400
switch(config-if)# copy running-config startup-config
switch(config-if)# no shut

Enabling Path MTU Discovery

Use the tunnel path-mtu discovery command to enable path MTU discovery on a tunnel.

Procedure

Command or Action Purpose

Step 1 tunnel path-mtu-discovery age-timer min Enables Path MTU Discovery (PMTUD) on a
tunnel interface.
Example:
switch(config-if)# tunnel • min—Number of minutes. The range is
path-mtu-discovery age-timer 25 from 10 to 30. The default is 10.

Step 2 tunnel path-mtu-discovery min-mtu bytes Enables Path MTU Discovery (PMTUD) on a
tunnel interface.
Example:
switch(config-if)# tunnel • bytes—Minimum MTU recognized.
path-mtu-discovery min-mtu 1500
The range is from 64 to 9192. The default
is 64.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
296
 Configuring IP Tunnels
Assigning VRF Membership to a Tunnel Interface

Assigning VRF Membership to a Tunnel Interface

You can add a tunnel interface to a VRF.

Before you begin

Ensure that you have enabled the tunneling feature.
Assign the IP address for a tunnel interface after you have configured the interface for a VRF.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 interface tunnel number Enters interface configuration mode.

Example:
switch(config)# interface tunnel 0
switch(config-if)#

Step 3 vrf member vrf-name Adds this interface to a VRF.

Example:
switch(config-if)# vrf member
RemoteOfficeVRF

Step 4 ip address ip-prefix/length Configures an IP address for this interface. You

must do this step after you assign this interface
Example:
to a VRF.
switch(config-if)# ip address
192.0.2.1/16

Step 5 show vrf [vrf-name] interface interface-type (Optional) Displays VRF information.
number
Example:
switch(config-vrf)# show vrf Enterprise
interface tunnel 0

Step 6 copy running-config startup-config (Optional) Saves this configuration change.

Example:
switch# copy running-config
startup-config

Example
This example shows how to add a tunnel interface to the VRF:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
297
 Configuring IP Tunnels
Verifying the IP Tunnel Configuration

switch# configure terminal

switch(config)# interface tunnel 0
switch(config-if)# vrf member RemoteOfficeVRF
switch(config-if)# ip address 209.0.2.1/16
switch(config-if)# copy running-config startup-config

Verifying the IP Tunnel Configuration

To verify the IP tunnel configuration information, perform one of the following tasks:

Command Purpose

show interface tunnel number Displays the configuration for the tunnel interface
(MTU, protocol, transport, and VRF). Displays input
and output packets, bytes, and packet rates.

show interface tunnel number brief Displays the operational status, IP address,
encapsulation type, and MTU of the tunnel interface.

show interface tunnel number counters Displays interface counters of input/output packets.
Note The byte count displayed with the interface
counters include the internal header size.

show interface tunnel number description Displays the configured description of the tunnel
interface.

show interface tunnel number status Displays the operational status of the tunnel interface.

show interface tunnel number status err-disabled Displays the error disabled status of the tunnel
interface.

Configuration Examples for IP Tunneling

The following example shows a simple GRE tunnel. Ethernet 1/2 is the tunnel source for router A and the
tunnel destination for router B. Ethernet interface 2/1 is the tunnel source for router B and the tunnel destination
for router A.
Router A:

feature tunnel
interface tunnel 0
ip address 209.165.20.2/8
tunnel source ethernet 1/2
tunnel destination 192.0.2.2
tunnel mode gre ip
tunnel path-mtu-discovery 25 1500

interface ethernet 1/2

ip address 192.0.2.55/8

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
298
 Configuring IP Tunnels
Related Documents

Router B:

feature tunnel
interface tunnel 0
ip address 209.165.20.1/8
tunnel source ethernet 2/1
tunnel destination 192.0.2.55
tunnel mode gre ip

interface ethernet 2/1

ip address 192.0.2.2/8

Related Documents
Related Topic Document Title

IP Tunnel commands Cisco Nexus 9000 Series NX-OS Interfaces Command

Reference

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
299
 Configuring IP Tunnels
Related Documents

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
300
 CHAPTER 10
Configuring Q-in-Q VLAN Tunnels
• Information About Q-in-Q Tunnels, on page 301
• Guidelines and Limitations, on page 305
• Configuring Q-in-Q Tunnels and Layer 2 Protocol Tunneling, on page 307
• Configuring Q-in-Q Double Tagging, on page 314
• Verifying the Q-in-Q Configuration, on page 315
• Configuration Examples for Q-in-Q and Layer 2 Protocol Tunneling, on page 316

Information About Q-in-Q Tunnels

This chapter describes how to configure IEEE 802.1Q-in-Q VLAN tunnels and Layer 2 protocol tunneling
on Cisco NX-OS devices (7.0(3)F3(4), 7.0(3)I2(1) and later).
A Q-in-Q VLAN tunnel enables a service provider to segregate the traffic of different customers in their
infrastructure, while still giving the customer a full range of VLANs for their internal use by adding a second
802.1Q tag to an already tagged frame.

Q-in-Q Tunneling
Business customers of service providers often have specific requirements for VLAN IDs and the number of
VLANs to be supported. The VLAN ranges required by different customers in the same service-provider
network might overlap, and the traffic of customers through the infrastructure might be mixed. Assigning a
unique range of VLAN IDs to each customer would restrict customer configurations and could easily exceed
the VLAN limit of 4096 of the 802.1Q specification.

Note Q-in-Q is supported on port channels. To configure a port channel as an asymmetrical link, all ports in the
port channel must have the same tunneling configuration.

Using the 802.1Q tunneling feature, service providers can use a single VLAN to support customers who have
multiple VLANs. Customer VLAN IDs are preserved and the traffic from different customers is segregated
within the service-provider infrastructure even when they appear to be on the same VLAN. The 802.1Q
tunneling expands the VLAN space by using a VLAN-in-VLAN hierarchy and tagging the tagged packets.
A port configured to support 802.1Q tunneling is called a tunnel port. When you configure tunneling, you
assign a tunnel port to a VLAN that is dedicated to tunneling. Each customer requires a separate VLAN, but
that VLAN supports all of the customer’s VLANs.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
301
 Configuring Q-in-Q VLAN Tunnels
Q-in-Q Tunneling

Customer traffic that is tagged in the normal way with appropriate VLAN IDs come from an 802.1Q trunk
port on the customer device and into a tunnel port on the service-provider edge switch. The link between the
customer device and the edge switch is an asymmetric link because one end is configured as an 802.1Q trunk
port and the other end is configured as a tunnel port. You assign the tunnel port interface to an access VLAN
ID that is unique to each customer. See the figure below.
Figure 30: 802.1Q-in-Q Tunnel Ports

Packets that enter the tunnel port on the service-provider edge switch, which are already 802.1Q-tagged with
the appropriate VLAN IDs, are encapsulated with another layer of an 802.1Q tag that contains a VLAN ID
that is unique to the customer. The original 802.1Q tag from the customer is preserved in the encapsulated
packet. Therefore, packets that enter the service-provider infrastructure are double-tagged.
The outer tag contains the customer’s access VLAN ID (as assigned by the service provider), and the inner
VLAN ID is the VLAN of the incoming traffic (as assigned by the customer). This double tagging is called
tag stacking, Double-Q, or Q-in-Q as shown in the figure below.
Figure 31: Untagged, 802.1Q-Tagged, and Double-Tagged Ethernet Frames

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
302
 Configuring Q-in-Q VLAN Tunnels
Native VLAN Hazard

By using this method, the VLAN ID space of the outer tag is independent of the VLAN ID space of the inner
tag. A single outer VLAN ID can represent the entire VLAN ID space for an individual customer. This
technique allows the customer’s Layer 2 network to extend across the service provider network, potentially
creating a virtual LAN infrastructure over multiple sites.

Note Hierarchical tagging, or multi-level dot1q tagging Q-in-Q, is not supported.

Native VLAN Hazard

When configuring 802.1Q tunneling on an edge switch, you must use 802.1Q trunk ports for sending out
packets into the service-provider network. However, packets that go through the core of the service-provider
network might be carried through 802.1Q trunks, ISL trunks, or nontrunking links. When 802.1Q trunks are
used in these core switches, the native VLANs of the 802.1Q trunks must not match any native VLAN of the
dot1q-tunnel port on the same switch because traffic on the native VLAN is not tagged on the 802.1Q
transmitting trunk port.
In the figure below, VLAN 40 is configured as the native VLAN for the 802.1Q trunk port from Customer X
at the ingress edge switch in the service-provider network (Switch B). Switch A of Customer X sends a tagged
packet on VLAN 30 to the ingress tunnel port of Switch B in the service-provider network that belongs to
access VLAN 40. Because the access VLAN of the tunnel port (VLAN 40) is the same as the native VLAN
of the edge-switch trunk port (VLAN 40), the 802.1Q tag is not added to tagged packets that are received
from the tunnel port. The packet carries only the VLAN 30 tag through the service-provider network to the
trunk port of the egress-edge switch (Switch C) and is misdirected through the egress switch tunnel port to
Customer Y.
Figure 32: Native VLAN Hazard

These are a couple ways to solve the native VLAN problem:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
303
 Configuring Q-in-Q VLAN Tunnels
Information About Layer 2 Protocol Tunneling

• Configure the edge switch so that all packets going out an 802.1Q trunk, including the native VLAN,
are tagged by using the vlan dot1q tag native command. If the switch is configured to tag native VLAN
packets on all 802.1Q trunks, the switch accepts untagged packets but sends only tagged packets.

Note The vlan dot1q tag native command is a global command that affects the tagging
behavior on all trunk ports.

• Ensure that the native VLAN ID on the edge switch trunk port is not within the customer VLAN range.
For example, if the trunk port carries traffic of VLANs 100 to 200, assign the native VLAN a number
outside that range.

Information About Layer 2 Protocol Tunneling

Customers at different sites connected across a service-provider network need to run various Layer 2 protocols
to scale their topology to include all remote sites, as well as the local sites. The Spanning Tree Protocol (STP)
must run properly, and every VLAN should build a proper spanning tree that includes the local site and all
remote sites across the service-provider infrastructure. The Cisco Discovery Protocol (CDP) must be able to
discover neighboring Cisco devices from local and remote sites, and the VLAN Trunking Protocol (VTP)
must provide consistent VLAN configuration throughout all sites in the customer network.
Starting with Cisco NX-OS Release 7.0(3)I7(3), you can configure the switch to allow multi-tagged BPDUs
on a tunnel port. If you enable the l2protocol tunnel allow-double-tag command, when a multi-tagged
customer BPDU enters the tunnel port, the original 802.1Q tags from the customer traffic is preserved and an
outer VLAN tag (customer’s access VLAN ID, as assigned by the service-provider) is added in the encapsulated
packet. Therefore, BPDU packets that enter the service-provider infrastructure are multi tagged. When the
BPDUs leave the service-provider network, the outer tag is removed and the original multi-tagged BPDU is
sent to the customer network.
When protocol tunneling is enabled, edge switches on the inbound side of the service-provider infrastructure
encapsulate Layer 2 protocol packets with a special MAC address and send them across the service-provider
network. Core switches in the network do not process these packets, but forward them as normal packets.
Bridge protocol data units (BPDUs) for CDP, STP, or VTP cross the service-provider infrastructure and are
delivered to customer switches on the outbound side of the service-provider network. Identical packets are
received by all customer ports on the same VLANs.
If protocol tunneling is not enabled on 802.1Q tunneling ports, remote switches at the receiving end of the
service-provider network do not receive the BPDUs and cannot properly run STP, CDP, 802.1X, and VTP.
When protocol tunneling is enabled, Layer 2 protocols within each customer’s network are totally separate
from those running within the service-provider network. Customer switches on different sites that send traffic
through the service- provider network with 802.1Q tunneling achieve complete knowledge of the customer’s
VLAN.

Note Layer 2 protocol tunneling works by tunneling BPDUs in the software. A large number of BPDUs that come
into the supervisor will cause the CPU load to go up. You might need to make use of software rate limiters
to reduce the load on the supervisor CPU. See Configuring Thresholds for Layer 2 Protocol Tunnel Ports, on
page 313.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
304
 Configuring Q-in-Q VLAN Tunnels
Guidelines and Limitations

For example, in the figure below, Customer X has four switches in the same VLAN that are connected through
the service-provider network. If the network does not tunnel BPDUs, switches on the far ends of the network
cannot properly run the STP, CDP, 802.1X, and VTP protocols.
Figure 33: Layer 2 Protocol Tunneling

In the preceding example, STP for a VLAN on a switch in Customer X, Site 1 will build a spanning tree on
the switches at that site without considering convergence parameters based on Customer X’s switch in Site
2.
The figure below shows the resulting topology on the customer’s network when BPDU tunneling is not
enabled.
Figure 34: Virtual Network Topology Without BPDU Tunneling

Guidelines and Limitations

Q-in-Q tunnels and Layer 2 tunneling have the following configuration guidelines and limitations:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
305
 Configuring Q-in-Q VLAN Tunnels
Guidelines and Limitations

• Mapping multiple VLANs is supported from Cisco NX-OS Release 7.0(3)I5(1) and later versions.
• Multi-tagged BPDUs are supported on the Cisco Nexus 93108TC-EX and 93180YC-EX switches. We
support up to three tags.
• Selective Q-in-Q tunneling is not supported with multi-tagged BPDU.
• Only multi-tagged CDP and STP BPDUs are supported.
• The inner-most tag must always be 0x8100.
• Multiple selective Q-in-Q tags are not supported. That is, Q-in-Q does not support multiple SP tags on
a single interface.
• show commands with the internal keyword are not supported.
• Switches in the service-provider network must be configured to handle the increase in MTU size due to
Q-in-Q tagging.
• MAC address learning for Q-in-Q tagged packets is based on the outer VLAN (Service Provider VLAN)
tag. Packet forwarding issues might occur in deployments where a single MAC address is used across
multiple inner (customer) VLANs.
• Layer 3 and higher parameters cannot be identified in tunnel traffic (for example, Layer 3 destination
and source addresses). Tunneled traffic cannot be routed.
• Cisco Nexus 9000 Series devices can provide only MAC-layer ACL/QoS for tunnel traffic (VLAN IDs
and src/dest MAC addresses).
• You should use MAC address-based frame distribution.
• Asymmetrical links do not support the Dynamic Trunking Protocol (DTP) because only one port on the
link is a trunk. You must configure the 802.1Q trunk port on an asymmetrical link to trunk unconditionally.
• You cannot configure the 802.1Q tunneling feature on ports that are configured to support private VLANs.
Private VLAN are not required in these deployments.
• You must disable IGMP snooping on the tunnel VLANs.
• Control Plane Policing (CoPP) is not supported.
• You should enter the vlan dot1Q tag native command to maintain the tagging on the native VLAN and
drop untagged traffic. This command prevents native VLAN misconfigurations.
• You must manually configure the 802.1Q interfaces to be edge ports.
• IGMP snooping is not supported on the inner VLAN.
• Q-in-Q is not supported on the uplink ports of Cisco Nexus 9332PQ, 9372PX, 9372TX, and 93120TX
switches and Cisco Nexus 9396PX, 9396TX, and 93128TX switches with the N9K-M6PQ or N9K-M12PQ
generic expansion module (GEM).
• Q-in-Q tunnels might be affected by the limitations of the Application Leaf Engine (ALE) uplink ports
on Cisco Nexus 9300 and 9500 Series devices:Limitations for ALE Uplink Ports
• Q-in-Q tunneling is not supported on Application Spine Engine 2 (ASE2) and Application Spine Engine
3 (ASE3) based Cisco Nexus switches.
• Layer 2 protocol tunnelling with Link Aggregation Control Protocol (LACP) is not supported in Cisco
NX-OS Release 7.0(3)F3(4)

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
306
 Configuring Q-in-Q VLAN Tunnels
Configuring Q-in-Q Tunnels and Layer 2 Protocol Tunneling

• Q-in-Q tagging is not supported in Cisco NX-OS Release 7.0(3)F3(4).

• Layer 2 protocol tunneling is not supported on Cisco Nexus 9500 Series switches with N9K-X9636C-R,
N9K-X9636Q-R, N9K-X9636C-RX line cards.
• Beginning with 7.0(3)F3(4), on Cisco Nexus 9500 Series switches with N9K-X9636C-R, N9K-X9636Q-R,
N9K-X9636C-RX line cards, Q-in-Q is supported only on port or port-channel Layer 2 Access VLAN
Edge devices.

Configuring Q-in-Q Tunnels and Layer 2 Protocol Tunneling

Creating a 802.1Q Tunnel Port
You create the dot1q-tunnel port using the switchport mode command.

Note You must set the 802.1Q tunnel port to an edge port with the spanning-tree port type edge command. The
VLAN membership of the port is changed using the switchport access vlan vlan-id command.
You should disable IGMP snooping on the access VLAN allocated for the dot1q-tunnel port to allow multicast
packets to traverse the Q-in-Q tunnel.

The following CLI is mandatory only on LSE, EX, FX, FX2 based Cisco Nexus 9000 Series switches. For
seamless packet forwarding and preservation of all VLAN tags on pure transit boxes in the SP cloud that have
no Q-in-Q encapsulation or decapsulation requirement, configure the CLI command system dot1q-tunnel
transit . To remove the CLI, use no system dot1q-tunnel transit CLI command.
The caveats with the CLI that is executed on the switches are:
• L2 frames that egress out of the trunk ports are tagged even on the native VLAN on the port.
• Any other tunneling mechanisms, for instance, VXLAN and MPLS may not work with the CLI configured.

Before you begin

You must first configure the interface as a switchport.

Procedure

Command or Action Purpose

Step 1 switch# configure terminal Enters global configuration mode.

Step 2 switch(config)# interface ethernet slot/port Specifies an interface to configure, and enters
interface configuration mode.

Step 3 switch(config-if)# switchport Sets the interface as a Layer 2 switching port.

Step 4 switch(config-if)# switchport mode Creates a 802.1Q tunnel on the port. The port
dot1q-tunnel will go down and reinitialize (port flap) when
the interface mode is changed. BPDU filtering

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
307
 Configuring Q-in-Q VLAN Tunnels
Configuring VLAN Mapping for Selective Q-in-Q on a 802.1Q Tunnel Port

Command or Action Purpose

is enabled and CDP is disabled on tunnel
interfaces.

Step 5 (Optional) switch(config-if)# no switchport Disables the 802.1Q tunnel on the port.
mode dot1q-tunnel
Step 6 switch(config-if)# exit Exits configuration mode.

Step 7 (Optional) switch(config)# show dot1q-tunnel Displays all ports that are in dot1q-tunnel mode.
[interface if-range] Optionally, you can specify an interface or
range of interfaces to display.

Step 8 (Optional) switch(config)# no shutdown Clears the errors on the interfaces and VLANs
where policies correspond with hardware
policies. This command allows policy
programming to continue and the port to come
up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Step 9 (Optional) switch(config)# copy Copies the running configuration to the startup
running-config startup-config configuration.

Example
This example shows how to create an 802.1Q tunnel port:

switch# configure terminal

switch(config)# interface ethernet 7/1
switch(config-if)# switchport
switch(config-if)# switchport mode dot1q-tunnel
switch(config-if)# exit
switch(config)# exit
switch# show dot1q-tunnel

Configuring VLAN Mapping for Selective Q-in-Q on a 802.1Q Tunnel Port

To configure VLAN mapping for selective Q-in-Q on a 802.1Q tunnel port, complete the following steps.

Note You cannot configure one-to-one mapping and selective Q-in-Q on the same interface.

Procedure

Command or Action Purpose

Step 1 switch# configure terminal Enters global configuration mode.

Step 2 switch(config)# interface interface-id Enters interface configuration mode for the
interface connected to the service provider

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
308
 Configuring Q-in-Q VLAN Tunnels
Changing the EtherType for Q-in-Q

Command or Action Purpose

network. You can enter a physical interface or
an EtherChannel port channel.

Step 3 switch(config-if)# switchport mode Configure the interface as a tunnel port.

dot1q-tunnel
Step 4 switch(config-if)# switchport vlan mapping Enters the VLAN IDs to be mapped:
vlan-id-range dot1q-tunnel outer vlan-id
• vlan-id-range—The customer VLAN ID
range (C-VLAN) entering the switch from
the customer network. The range is from
1 to 4094. You can enter a string of
VLAN-IDs.
• outer vlan-id—Enter the outer VLAN ID
(S-VLAN) of the service provider network.
The range is from 1 to 4094.

Step 5 switch(config-if)# exit Exits the configuration mode.

Step 6 switch# show interfaces interface-id vlan Verifies the configuration.

mapping
Step 7 switch# copy running-config startup-config (Optional) Saves your entries in the
configuration file.

Use the no switchport vlan mapping vlan-id-range dot1q-tunnel outer vlan-id command to remove the
VLAN mapping configuration.
The following example shows how to configure selective Q-in-Q mapping on the port so that traffic with a
C-VLAN ID of 1 to 5 enters the switch with an S-VLAN ID of 100. The traffic of any other VLAN IDs is
dropped.

Example

switch(config)# interface gigabitethernet0/1

switch(config-if)# switchport vlan mapping 1-5 dot1q-tunnel 100

Switch(config-if)# exit

Changing the EtherType for Q-in-Q

You can change the 802.1Q EtherType value to be used for Q-in-Q encapsulation. The switch default EtherType
is 0x8100 for 802.1Q and Q-in-Q encapsulations. EtherType can be configured to 0x9100, 0x9200 and 0x88a8
on a per port basis by enabling the dot1q ethertype command on the switchport interface. You can configure
a custom EtherType field value on a port to support network devices that do not use the standard 0x8100
EtherType field value on 802.1Q-tagged

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
309
 Configuring Q-in-Q VLAN Tunnels
Changing the EtherType for Q-in-Q

Note You must set the EtherType only on the egress trunk interface that carries double tagged frames (the trunk
interface that connects the service providers). If you change the EtherType on one side of the trunk, you must
set the same value on the other end of the trunk (symmetrical configuration).

Note The EtherType 0x9100 is not supported on egress interfaces. The packets egressing the interface with an
EtherType 0X9100 configuration will egress with 0x8100 outer tag.

Important On Cisco Nexus 9300-EX, -FX, -FX2, and -GX switches, when 802.1q EtherType has changed on an interface,
the EtherType of all interfaces on the same slice will be changed to the configured value. This change is not
persistent after a reload of the switch and will revert to the EtherType value of the last port on the slice.

Caution The EtherType value you set affects all the tagged packets that go out on the interface (not just Q-in-Q packets).

Procedure

Command or Action Purpose

Step 1 switch# configure terminal Enters global configuration mode.

Step 2 switch(config)# interface ethernet slot/port Specifies an interface to configure, and enters
interface configuration mode.

Step 3 switch(config-if)# switchport Sets the interface as a Layer 2 switching port.

Step 4 switch(config-if)# switchport dot1q ethertype Sets the EtherType for the Q-in-Q tunnel on the
value port.

Step 5 (Optional) switch(config-if)# no switchport Resets the EtherType on the port to the default
dot1q ethertype value of 0x8100.

Step 6 switch(config-if)# exit Exits configuration mode.

Step 7 (Optional) switch(config)# no shutdown Clears the errors on the interfaces and VLANs
where policies correspond with hardware
policies. This command allows policy
programming to continue and the port to come
up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Step 8 (Optional) switch(config)# copy Copies the running configuration to the startup
running-config startup-config configuration.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
310
 Configuring Q-in-Q VLAN Tunnels
Enabling the Layer 2 Protocol Tunnel

Example
This example shows how to configure customer EtherType on a Layer 2 port:

switch# configure terminal

switch(config)# interface ethernet 7/1
switch(config-if)# switchport
switch(config-if)# switchport dot1q ethertype 0x9100
switch(config-if)# exit
switch(config)# exit
switch# show dot1q-tunnel

This example shows how to configure custom EtherType on an 802.1Q tunnel port:

switch# configure terminal

switch(config)# interface ethernet 1/1
switch(config-if)# switchport
switch(config-if)# switchport mode dot1q-tunnel
switch(config-if)# switchport dot1q ethertype 0x9100
switch(config-if)# switchport access vlan 30
switch(config-if)# exit
switch(config)# exit

Enabling the Layer 2 Protocol Tunnel

You can enable protocol tunneling on the 802.1Q tunnel port.

Procedure

Command or Action Purpose

Step 1 switch# configure terminal Enters global configuration mode.

Step 2 switch(config)# interface ethernet slot/port Specifies an interface to configure, and enters
interface configuration mode.

Step 3 switch(config-if)# switchport Sets the interface as a Layer 2 switching port.

Step 4 switch(config-if)# switchport mode Creates a 802.1Q tunnel on the port. The port
dot1q-tunnel will go down and reinitialize (port flap) when
the interface mode is changed. BPDU filtering
is enabled and CDP is disabled on tunnel
interfaces.

Step 5 switch(config-if)# l2protocol tunnel [cdp | stp Enables Layer 2 protocol tunneling. Optionally,
| lacp | lldp |vtp] you can enable CDP, STP, LACP, LLDP, or
VTP tunneling.

Step 6 (Optional) switch(config-if)# no l2protocol Disables protocol tunneling.

tunnel [cdp | stp | lacp | lldp |vtp]
Note Layer 2 protocol tunnel for LACP
and LLDP is only supported on
Cisco Nexus N9K-X9732C-EXM
modules.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
311
 Configuring Q-in-Q VLAN Tunnels
Configuring Global CoS for L2 Protocol Tunnel Ports

Command or Action Purpose

Step 7 switch(config-if)# exit Exits configuration mode.

Step 8 (Optional) switch(config)# no shutdown Clears the errors on the interfaces and VLANs
where policies correspond with hardware
policies. This command allows policy
programming to continue and the port to come
up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Step 9 (Optional) switch(config)# copy Copies the running configuration to the startup
running-config startup-config configuration.

Example
This example shows how to enable protocol tunneling on an 802.1Q tunnel port:

switch# configure terminal

switch(config)# interface ethernet 7/1
switch(config-if)# switchport
switch(config-if)# switchport mode dot1q-tunnel
switch(config-if)# l2protocol tunnel stp
switch(config-if)# exit
switch(config)# exit

Configuring Global CoS for L2 Protocol Tunnel Ports

You can specify a Class of Service (CoS) value globally so that ingress BPDUs on the tunnel ports are
encapsulated with the specified class.

Procedure

Command or Action Purpose

Step 1 switch# configure terminal Enters global configuration mode.

Step 2 switch(config)# l2protocol tunnel cos value Specifies a global CoS value on all Layer 2
protocol tunneling ports. The default cos-value
is 5.

Step 3 (Optional) switch(config)# no l2protocol Sets the global CoS value to default.
tunnel cos
Step 4 switch(config)# exit Exits configuration mode.

Step 5 (Optional) switch# no shutdown Clears the errors on the interfaces and VLANs
where policies correspond with hardware
policies. This command allows policy
programming to continue and the port to come
up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
312
 Configuring Q-in-Q VLAN Tunnels
Configuring Thresholds for Layer 2 Protocol Tunnel Ports

Command or Action Purpose

Step 6 (Optional) switch# copy running-config Copies the running configuration to the startup
startup-config configuration.

Example
This example shows how to specify a global CoS value for the purpose of Layer 2 protocol tunneling:

switch# configure terminal

switch(config)# l2protocol tunnel cos 6
switch(config)# exit

Configuring Thresholds for Layer 2 Protocol Tunnel Ports

You can specify the port drop and shutdown value for a Layer 2 protocol tunneling port.

Procedure

Command or Action Purpose

Step 1 switch# configure terminal Enters global configuration mode.

Step 2 switch(config)# interface ethernet slot/port Specifies an interface to configure, and enters
interface configuration mode.

Step 3 switch(config-if)# switchport Sets the interface as a Layer 2 switching port.

Step 4 switch(config-if)# switchport mode Creates a 802.1Q tunnel on the port.

dot1q-tunnel
Step 5 switch(config-if)# l2protocol tunnel Specifies the maximum number of packets that
drop-threshold [cdp | stp | vtp] can be processed on an interface before being
packets-per-sec dropped. Optionally, you can specify CDP,
STP, or VTP. Valid values for the packets are
from 1 to 4096.

Step 6 (Optional) switch(config-if)# no l2protocol Resets the threshold values to 0 and disables
tunnel drop-threshold [cdp | stp | vtp] the drop threshold.

Step 7 switch(config-if)# l2protocol tunnel Specifies the maximum number of packets that
shutdown-threshold [cdp | stp | vtp] can be processed on an interface. When the
packets-per-sec number of packets is exceeded, the port is put
in error-disabled state. Optionally, you can
specify CDP, STP, or VTP. Valid values for
the packets is from 1 to 4096.

Step 8 (Optional) switch(config-if)# no l2protocol Resets the threshold values to 0 and disables
tunnel shutdown-threshold [cdp | stp | vtp] the shutdown threshold.

Step 9 switch(config-if)# exit Exits configuration mode.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
313
 Configuring Q-in-Q VLAN Tunnels
Configuring Q-in-Q Double Tagging

Command or Action Purpose

Step 10 (Optional) switch(config)# no shutdown Clears the errors on the interfaces and VLANs
where policies correspond with hardware
policies. This command allows policy
programming to continue and the port to come
up. If policies do not correspond, the errors are
placed in an error-disabled policy state.

Step 11 (Optional) switch(config)# copy Copies the running configuration to the startup
running-config startup-config configuration.

Configuring Q-in-Q Double Tagging

Enable multi-tagging for STP and CDP BPDUs.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal

Step 2 interface interface Specifies the interface that you are configuring.
Example:
switch(config)# interface ethernet 7/1

Step 3 switchport Sets the interface as a Layer 2 switching port.

Example:
switch(config-if)# switchport

Step 4 switchport mode dot1q-tunnel Creates an 802.1Q tunnel on the port. The port
goes down and reinitializes (port flap) when the
Example:
interface mode is changed. BPDU filtering is
switch(config-if)# switchport mode enabled and CDP is disabled on tunnel
dot1q-tunnel
interfaces.

Step 5 l2protocol tunnel [cdp | stp] Enables Layer 2 protocol tunneling. Optionally,
you can enable CDP or STP.
Example:
switch(config-if)# l2protocol tunnel cdp

Step 6 (Optional) no l2protocol tunnel [cdp | stp] Disables protocol tunneling.

Example:
switch(config-if)# no l2protocol tunnel
stp

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
314
 Configuring Q-in-Q VLAN Tunnels
Verifying the Q-in-Q Configuration

Command or Action Purpose

Step 7 l2protocol tunnel allow-double-tag Enables multi-tagging for STP and CDP BPDUs
on the interface.
Example:
switch(config-if)# l2protocol tunnel
allow-double-tag

Step 8 (Optional) no l2protocol tunnel Disables multi-tagging for STP and CDP
allow-double-tag BPDUs on the interface.
Example:
switch(config-if)# no l2protocol tunnel
allow-double-tag

Step 9 exit Exits configuration mode.

Example:
switch(config-if)# exit

Example
This example shows how to enable multi-tagging for STP and CDP BPDUs:
switch# configure terminal
switch(config)# interface ethernet 7/1
switch(config-if)# switchport
switch(config-if)# switchport mode dot1q-tunnel
switch(config-if)# l2protocol tunnel cdp
switch(config-if)# l2protocol tunnel stp
switch(config-if)# l2protocol tunnel allow-double-tag
switch(config-if)# exit
switch(config)# exit
switch#

Verifying the Q-in-Q Configuration

Command Purpose

clear l2protocol tunnel counters [interface if-range] Clears all the statistics counters. If no interfaces are
specified, the Layer 2 protocol tunnel statistics are
cleared for all interfaces.

show dot1q-tunnel [interface if-range] Displays a range of interfaces or all interfaces that are
in dot1q-tunnel mode.

show l2protocol tunnel [interface if-range | vlan Displays Layer 2 protocol tunnel information for a
vlan-id] range of interfaces, for all dot1q-tunnel interfaces that
are part of a specified VLAN or all interfaces.

show l2protocol tunnel summary Displays a summary of all ports that have Layer 2
protocol tunnel configurations.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
315
 Configuring Q-in-Q VLAN Tunnels
Configuration Examples for Q-in-Q and Layer 2 Protocol Tunneling

Command Purpose

show running-config l2pt Displays the current Layer 2 protocol tunnel running
configuration.

Configuration Examples for Q-in-Q and Layer 2 Protocol

Tunneling
This example shows a service provider switch that is configured to process Q-in-Q for traffic coming in on
Ethernet 7/1. A Layer 2 protocol tunnel is enabled for STP BPDUs. The customer is allocated VLAN 10
(outer VLAN tag).

switch# configure terminal

Enter configuration commands, one per line. End with CNTL/Z.
switch(config)# vlan 10
switch(config-vlan)# no shutdown
switch(config-vlan)# no ip igmp snooping
switch(config-vlan)# exit
switch(config)# interface ethernet 7/1
switch(config-if)# switchport
switch(config-if)# switchport mode dot1q-tunnel
switch(config-if)# switchport access vlan 10
switch(config-if)# spanning-tree port type edge
switch(config-if)# l2protocol tunnel stp
switch(config-if)# no shutdown
switch(config-if)# exit
switch(config)# exit
switch#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
316
 CHAPTER 11
Configuring Smart-Channel
This chapter describes how to configure smart-channel on Cisco NX-OS devices.
This chapter includes the following sections:
• About Smart Channel, on page 317
• Topology Examples for Smart Channel , on page 318
• Prerequisites for Smart Channel, on page 320
• Guidelines and Limitations for Smart Channel, on page 321
• Default Settings for Smart Channel, on page 321
• Configuring Smart Channel, on page 321
• Verifying the Smart Channel Configuration, on page 324
• Configuration Examples for Smart Channel, on page 325

About Smart Channel

Smart channel is a hardware-based, multi-terabit solution for the Layer 2 traffic distribution, load balancing,
and redirection on the Cisco Nexus switches. This feature is supported on the Cisco Nexus 9372PX,
93108TC-EX, and the Cisco Nexus 9516 switches.

Note Smart channel feature is not supported on Cisco 9500 EX / FX line cards.

Smart channel is an aggregation of multiple physical links that creates a single logical link. You can bundle
up multiple physical links into a port group to provide an increased bandwidth (an aggregate of the multiple
physical links) and redundancy.
If one port within a smart channel fails, the traffic switches to the remaining ports in the smart channel.
Smart channel allows you to create a cluster of transparent mode appliances.

Smart Channel Features

The smart channel features are as follows:
• Multi-terabit solution at line rate
• Simplified provisioning and ease of deployment

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
317
 Configuring Smart-Channel
Benefits of Smart Channel

• Transparency to end device and stateless protocol benefits

• Removes the requirement for an expensive external load balancer

Benefits of Smart Channel

The benefits of smart channel are as follows:
• Simultaneous redirection and load balancing
• IP-stickiness and resiliency
• Health monitoring
• Removes the requirement for an expensive external load balancer
• Hashing does not depend on the wiring or the port numbering
• Every port on the switch is used for load balancing and traffic redirection
• Automatic failure handling of servers or appliances

Examples of the Deployment Use Cases

Examples of the deployment use cases for the smart channel feature are as follows:
• Load balances to a pool of firewalls.
• Scales the VDS-TC (video-caching) solution.
• Scales the transparent mode devices.

Topology Examples for Smart Channel

This section displays the following examples:
• Basic topology for smart channel
• Use case of a smart channel configuration
• Fail-action for resilient hashing

You can use the smart channel feature to load balance traffic to appliances used in a monitoring network. The
following figure shows the basic topology, where the traffic is sent to the appliances where you need to load
balance the traffic towards, such as the IPS or the IDS devices.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
318
Configuring Smart-Channel
Topology Examples for Smart Channel

Figure 35: Standard Topology for Smart Channel

The following example shows a typical use case of smart channel in a network where the traffic is spanned
from the production environment to the monitoring environment. In this example, we are using the Cisco
Nexus Data Broker to send copy of the monitoring traffic and scale monitoring networks.
Figure 36: Use Case for a Smart Channel Configuration

The following example shows the fail-action of a smart channel configuration:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
319
 Configuring Smart-Channel
Prerequisites for Smart Channel

Figure 37: Fail-Action of a Smart Channel Configuration

The following example shows the fail-action of a smart channel configuration:

Figure 38: Fail-Action of a Smart Channel Configuration

Prerequisites for Smart Channel

Smart channel has the following prerequisite:
• You must ensure that an enough TCAM size has been allocated to the VACL. To verify the TCAM size,
use the sh hardware access-list tcam region command. If the appropriate TCAM size is not allocated,
use the hardware access-list tcam region VACL size addtitional of 256 command to allocate the
appropriate TCAM size.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
320
 Configuring Smart-Channel
Guidelines and Limitations for Smart Channel

Guidelines and Limitations for Smart Channel

Smart channel has the following configuration guidelines and limitations:
• Cisco Nexus 9372PX, 93108TC-EX, and the Cisco Nexus 9516 switches support smart channel servicing.
• This feature is supported for the Cisco Nexus 9372PX, 93108TC-EX, and the Cisco Nexus 9516 switches.

Note Smart channel feature is not supported on Cisco 9500 EX / FX / R line cards.

• This feature is supported for the Cisco Nexus C93108TC-EX beginning Cisco NX-OS Release 9.2 (x).
• Smart channel does not support the vPC, port channel, and the L3 interfaces.
• Only the port group interfaces in a trunk or access mode are supported.
• You must not share the smart port-group to more than one service when the services have the access
configuration.
• Ensure that the TCAM size is equal to the sum of the number of the configured VLANs on the service
by the number of buckets.
• Ensure that the smart channel service does not have the same load balancing method (load-balance method
src ip) and the configuration of the (source filter ip any any).
• Ensure that the port-group to be added to the smart channel service has been configured.

Default Settings for Smart Channel

The following table lists the default settings for the smart channel parameter.

Table 17: Default Smart Channel Parameter

Parameters Default
Smart channel Disabled

Configuring Smart Channel

Enabling Smart Channel
You must enable the smart channel feature before you can configure the smart channel services on the device.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
321
 Configuring Smart-Channel
Configuring Port Groups

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 [no] feature smart-channel Enables or disables smart channel. By default,

smart channel is disabled.
Example:
switch(config)# feature smart-channel

Step 3 (Optional) show feature | grep smart-channel Displays the status of the smart channel
configuration.
Example:
switch(config-if)# show feature | grep
smart-channel

Step 4 (Optional) copy running-config startup-config Copies the running configuration to the startup
configuration.
Example:
switch(config)# copy running-config
startup-config

Configuring Port Groups

After you enable smart channel, you must create a port group and configure active interfaces on that group.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 [no] smart-channel port-group Creates or deletes a port group.

port-group-name
Example:
switch(config)# smart-channel port-group
Webservers
switch(config-port-group)#

Step 3 Required: interface ethernet type slot/port Configures the active interfaces on the port
group.
Example:

switch(config)# interface ethernet 3/1

switch(config-if)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
322
 Configuring Smart-Channel
Configuring the Smart Channel Service

Command or Action Purpose

Step 4 (Optional) copy running-config startup-config Copies the running configuration to the startup
configuration.
Example:
switch(config-port-group)# copy
running-config startup-config

Configuring the Smart Channel Service

To configure the smart channel service, you must do the following:
1. Configure the service name
2. Configure the VLAN/source/destination filters
3. Associate the port group to the smart channel service
4. Specify the load distribution scheme
5. Activate the smart channel service

Before you begin

You must enable the smart channel feature before you configure the smart channel service.

Note Beginning from Cisco Nexus NX-OS Release 9.3(3), feature Smart-channel is not supported. It is recommended
to take necessary actions while upgrading from any previous release to 9.3(2) or any newer versions.

Procedure

Command or Action Purpose

Step 1 configure terminal Enters global configuration mode.
Example:
switch# configure terminal
switch(config)#

Step 2 Required: [no] smart-channel service-name Configures or disables the smart channel
service.
Example:
switch(config)# smart-channel WebTraffic
switch(config-smart-channel)#

Step 3 [no] access vlan access-vlan | vlan vlan-range Configures a list of VLANs for the smart
channel service. While the access VLANs create
Example:
the smart channel in an access mode, the
switch(config-smart-channel)# access vlan VLANs in the VLAN range creates the smart
10-20
switch(config-port-group)# channel in the trunk mode.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
323
 Configuring Smart-Channel
Verifying the Smart Channel Configuration

Command or Action Purpose

Step 4 [no] port group port-group-name Associates a port-group with the smart channel
service.
Example:
switch(config-smart-channel)# port group
WEBSERVERS
switch(config-smart-channel)#

Step 5 [no] load-balance method [src | dst] Configures the load balancing method.
Example:
switch(config-smart-channel)#
load-balance method src-ip
switch(config-smart-channel)#

Step 6 [no] destination filter ip any [any] Configures the selected destination subnets.
Example:
switch(config-smart-channel)# destination
filter ip any any
switch(config-smart-channel)#

Step 7 [no] source filter [ipanyany] Configures the selected source subnets.
Example:
switch(config-smart-channel)# source
filter ip any any
switch(config-smart-channel)#

Step 8 [no] load-balance method [src | dst] Bucket Configures the load balancing method.
B
Example:
switch(config-smart-channel)#
load-balance method src Bucket 16
switch(config-port-group)#

Step 9 no shut Activates the smart channel service.

Example:
switch(config-smart-channel)# no shut
switch(config-smart-channel)#

Verifying the Smart Channel Configuration

To display the smart channel configuration, perform one of the following tasks:

Table 18: Smart Channel Show Commands

Command Purpose
show smart-channel service-name Displays the smart channel configuration status.

show vlan access-list vlan access-map name Displays the statistics for a smart channel service.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
324
 Configuring Smart-Channel
Configuration Examples for Smart Channel

Command Purpose
show running-config smart-channel [all] Displays the running configuration for smart channel.

Configuration Examples for Smart Channel

This example shows how to configure smart channel:

Enabling the smart channel feature.

switch# configure terminal
switch(config)# feature smart-channel

Configuring port-group.
switch(config)# smart-channel port-group pg1
switch(config-port-group)# interface ethernet 10/23
switch(config-pg-node)# interface ethernet 10/24

Configuring services.

Note Ensure that the port-group to be added to the smart channel service has been configured.

Service with Trunk configuration

--------------------------------

switch(config-smart-channel)# port-group pg1

switch(config-smart-channel)# vlan 67
switch(config-smart-channel)# load-balance method src ip
switch(config-smart-channel)# destination filter ip any any
switch(config-smart-channel)# no shut

Service with Access configuration

---------------------------------
switch(config-smart-channel)# port-group pg1
switch(config-smart-channel)# access vlan 68
switch(config-smart-channel)# load-balance method src ip
switch(config-smart-channel)# destination filter ip any any
switch(config-smart-channel)# no shut

Verifying the smart channel configuration.

switch(config-smart-channel)# show running-config smart-channel

version 7.0(3)I6(1)

smart-channel port-group pg1

interface Eth10/23
interface Eth10/24

smart-channel Service1
port-group pg1

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
325
 Configuring Smart-Channel
Configuration Examples for Smart Channel

vlan 67
destination filter ip any any
load-balance method src ip
no shutdown

Checking the status of the service and the port-group interfaces.

switch(config-smart-channel)# show smart-channel Service1

Legend:
ST(Status): ST-Standby,LF-Link Failed,PF-Probe Failed,PD-Peer Down,IA-Inactive

Name LB Scheme Status Buckets

-------------- ---------- -------- -------
Service1 src-ip ACTIVE 2

Port Group
--------------------------------------------------
pg1

Pool
------------------------------
Service1_smartc_pool
Source/Destination Filter Protocol Port
----------------------------------------------------------------
0.0.0.0 / 255.255.255.255 IP 0

Member-Interface Cfg-STS Weight Op-STS

------------------- ------- --------- --------
1 Ethernet10/23 OK 1

Bucket List
--------------------------------------------------------------------------
Service1_smartc_ip_1_bucket_1

Member-Interface Cfg-STS Weight Op-STS

------------------- ------- --------- --------
2 Ethernet10/24 OK 1

Bucket List
--------------------------------------------------------------------------
Service1_smartc_ip_1_bucket_2
================================================================
VACL config

switch# show running-config aclmgr

Checking the statistics.

switch# show vlan access-list Service1_smartc_pool

Checking the fail-action.

Note This is a default configuration and you are not required to perform any additional configurations.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
326
Configuring Smart-Channel
Configuration Examples for Smart Channel

switch# show smart-channel Service1

Legend:
ST(Status): ST-Standby,LF-Link Failed,PF-Probe Failed,PD-Peer Down,IA-Inactive

Name LB Scheme Status Buckets

-------------- ---------- -------- -------
Service1 src-ip ACTIVE 2

Port Group
--------------------------------------------------
pg1

Pool
------------------------------
Service1_smartc_pool
Source/Destination Filter Protocol Port
----------------------------------------------------------------
0.0.0.0 / 255.255.255.255 IP 0

Member-Interface Cfg-STS Weight Op-STS

------------------- ------- --------- --------
1 Ethernet10/23 LF 1

Bucket List
--------------------------------------------------------------------------
Service1_smartc_ip_1_bucket_1

Member-Interface Cfg-STS Weight Op-STS

------------------- ------- --------- --------
2 Ethernet10/24 OK 1

Bucket List
--------------------------------------------------------------------------
Service1_smartc_ip_1_bucket_2

======================================================================================
switch# show running-config aclmgr

=============
switch# show vlan access-list Service1_smartc_pool

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
327
 Configuring Smart-Channel
Configuration Examples for Smart Channel

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
328
 CHAPTER 12
Configuring Static and Dynamic NAT Translation
This chapter includes the following sections:
• Network Address Translation Overview, on page 329
• Information About Static NAT, on page 330
• Dynamic NAT Overview, on page 331
• Timeout Mechanisms, on page 332
• NAT Inside and Outside Addresses, on page 333
• Pool Support for Dynamic NAT, on page 333
• Static and Dynamic Twice NAT Overview, on page 334
• VRF Aware NAT, on page 334
• Guidelines and Limitations for Static NAT, on page 336
• Restrictions for Dynamic NAT, on page 337
• Guidelines and Limitations for Dynamic Twice NAT, on page 338
• Configuring Static NAT, on page 339
• Configuring Dynamic NAT, on page 346

Network Address Translation Overview

Network Address Translation (NAT) enables private IP internetworks that use nonregistered IP addresses to
connect to the Internet. NAT operates on a device, usually connecting two networks, and translates private
(not globally unique) IP addresses in the internal network into legal IP addresses before packets are forwarded
to another network. You can configure NAT to advertise only one IP address for the entire network to the
outside world. This ability provides additional security, effectively hiding the entire internal network behind
one IP address.
A device configured with NAT has at least one interface to the inside network and one to the outside network.
In a typical environment, NAT is configured at the exit router between a stub domain and a backbone. When
a packet leaves the domain, NAT translates the locally significant source IP address into a globally unique IP
address. When a packet enters the domain, NAT translates the globally unique destination IP address into a
local IP address. If more than one exit point exists, NAT configured at each point must have the same translation
table.
NAT is described in RFC 1631.
Beginning with 7.0(3)I2(1), static and dynamic NAT translation is supported.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
329
 Configuring Static and Dynamic NAT Translation
Information About Static NAT

Information About Static NAT

Static Network Address Translation (NAT) allows the user to configure one-to-one translations of the inside
local addresses to the outside global addresses. It allows both IP addresses and port number translations from
the inside to the outside traffic and the outside to the inside traffic. The Cisco Nexus device supports Hitless
NAT, which means that you can add or remove a NAT translation in the NAT configuration without affecting
the existing NAT traffic flows.
Static NAT creates a fixed translation of private addresses to public addresses. Because static NAT assigns
addresses on a one-to-one basis, you need an equal number of public addresses as private addresses. Because
the public address is the same for each consecutive connection with static NAT, and a persistent translation
rule exists, static NAT enables hosts on the destination network to initiate traffic to a translated host if an
access list exists that allows it .
With dynamic NAT and Port Address Translation (PAT), each host uses a different address or port for each
subsequent translation. The main difference between dynamic NAT and static NAT is that static NAT allows
a remote host to initiate a connection to a translated host if an access list exists that allows it, while dynamic
NAT does not.
The figure shows a typical static NAT scenario. The translation is always active so both translated and remote
hosts can originate connections, and the mapped address is statically assigned by the static command.
Figure 39: Static NAT

These are key terms to help you understand static NAT:

• NAT inside interface—The Layer 3 interface that faces the private network.
• NAT outside interface—The Layer 3 interface that faces the public network.
• Local address—Any address that appears on the inside (private) portion of the network.
• Global address—Any address that appears on the outside (public) portion of the network.
• Legitimate IP address—An address that is assigned by the Network Information Center (NIC) or service
provider.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
330
Configuring Static and Dynamic NAT Translation
Dynamic NAT Overview

• Inside local address—The IP address assigned to a host on the inside network. This address does not
need to be a legitimate IP address.
• Outside local address—The IP address of an outside host as it appears to the inside network. It does not
have to be a legitimate address, because it is allocated from an address space that can be routed on the
inside network.
• Inside global address—A legitimate IP address that represents one or more inside local IP addresses to
the outside world.
• Outside global address—The IP address that the host owner assigns to a host on the outside network.
The address is a legitimate address that is allocated from an address or network space that can be routed.

Dynamic NAT Overview

Dynamic Network Address Translation (NAT) translates a group of real IP addresses into mapped IP addresses
that are routable on a destination network. Dynamic NAT establishes a one-to-one mapping between
unregistered and registered IP addresses; however, the mapping can vary depending on the registered IP
address that is available at the time of communication.
A dynamic NAT configuration automatically creates a firewall between your internal network and outside
networks or the Internet. Dynamic NAT allows only connections that originate inside the stub domain—a
device on an external network cannot connect to devices in your network, unless your device has initiated the
contact.
Dynamic NAT translations do not exist in the NAT translation table until a device receives traffic that requires
translation. Dynamic translations are cleared or timed out when not in use to make space for new entries.
Usually, NAT translation entries are cleared when the ternary content addressable memory (TCAM) entries
are limited. The default minimum timeout for dynamic NAT translations is 30 minutes.

Note For Cisco NX-OS 7.0(3)I2(3) and earlier releases, the minimum value of the sampling-timeout in the ip nat
translation sampling-timeout command was reduced from 30 minutes to 15 minutes.

Note For Cisco NX-OS 7.0(3)I3(1) and later releases , the ip nat translation sampling-timeout command is not
supported. Statistics are collected every 60 seconds for the installed NAT policies. These statistics are used
to determine if the flow is active or not.

For Cisco NX-OS 7.0(3)I2(3) and earlier releases, the timeout of a dynamic NAT translation involves both
the sampling-timeout value and the TCP or UDP timeout value. The sampling-timeout specifies the time after
which the device checks for dynamic translation activity. It has a default value of 12 hours. All the other
timeouts start only after the sample-timeout times out. After the sampling-timeout, the device inspects the
packets that are hitting this translation. The checking happens for the TCP or UDP timeout period. If there
are no packets for the TCP or UDP timeout period, the translation is cleared. If activity is detected on the
translation, then the checking is stopped immediately and a sampling-timeout period begins.
For Cisco NX-OS 7.0(3)I2(3) and earlier releases, after waiting for this new sampling-timeout period, the
device checks for dynamic translation activity again. During an activity check the TCAM sends a copy of the
packet that matches the dynamic NAT translation to the CPU. If the Control Plane Policing (CoPP) is configured

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
331
 Configuring Static and Dynamic NAT Translation
Timeout Mechanisms

at a low threshold, the TCP or UDP packets might not reach the CPU, and the CPU considers this as inactivity
of the NAT translation.
Dynamic NAT supports Port Address Translation (PAT) and access control lists (ACLs). PAT, also known
as overloading, is a form of dynamic NAT that maps multiple unregistered IP addresses to a single registered
IP address by using different ports. Your NAT configuration can have multiple dynamic NAT translations
with same or different ACLs. However, for a given ACL, only one interface can be specified.

Timeout Mechanisms
For Cisco NX-OS 7.0(3)I2(3) and earlier releases, after dynamic NAT translations are created, they must be
cleared when not in use so that newer translations can be created, especially because the number of TCAM
entries is limited. The following NAT translation timeout timers are supported on the switch:
The following NAT translation timeout timers are supported on the switch:
• timeout—Timeout value for dynamic NAT translations.
The timeout value ranges from 60 seconds to 172800 seconds, including the sampling-timeout.
• sampling-timeout—Time after which the device checks for dynamic translation activity.
The timeout value ranges from 900 seconds to 172800 seconds.

The udp-timeout and the timeout value timers are triggered after the timeout configured for the ip nat
translation sampling-timeout command expires.

Note For Cisco NX-OS 7.0(3)I3(1) and later releases, there are three different options that can be configured for
aging:
• Time-out: This is applicable for all type of flows(both TCP and UDP).
• TCP TIME-OUT: This is applicable for only TCP flows.
• UDP TIME-OUT: This is applicable for only UDP flows.

Note For Cisco NX-OS 7.0(3)I3(1) and later releases, when you create dynamic entries without timeouts configured,
they take the default timeout of one hour (60 minutes). If you enter the clear ip nat translations all command
after configuring timeouts, the configured timeout take effect. A timeout can be configured from 60 to 172800
seconds.

Note When you create dynamic entries without timeouts configured, they take the default timeout of one hour (60
minutes). If you enter the clear ip nat translations all command after configuring timeouts, the configured
timeout take effect. A timeout can be configured from 60 to 172800 seconds.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
332
 Configuring Static and Dynamic NAT Translation
NAT Inside and Outside Addresses

NAT Inside and Outside Addresses

NAT inside refers to networks owned by an organization that must be translated. When NAT is configured,
hosts within this network will have addresses in one space (known as the local address space) that will appear
to those outside the network as being in another space (known as the global address space).
Similarly, NAT outside refers to those networks to which the stub network connects. They are not generally
under the control of the organization. Hosts in outside networks can be subject to translation and can have
local and global addresses.
NAT uses the following definitions:
• Local address—A local IP address that appears on the inside of a network.
• Global address—A global IP address that appears on the outside of a network.
• Inside local address—The IP address that is assigned to a host on the inside network. The address is
probably not a legitimate IP address assigned by the Internet Network Information Center (InterNIC) or
a service provider.
• Inside global address—A legitimate IP address (assigned by InterNIC or a service provider) that represents
one or more inside local IP addresses to the outside world.
• Outside local address—The IP address of an outside host as it appears to the inside network. The address
is not necessarily legitimate; it was allocated from the address space that is routable on the inside.
• Outside global address—The IP address that is assigned to a host on the outside network by the owner
of the host. The address was allocated from a globally routable address or a network space.

Pool Support for Dynamic NAT

Cisco NX-OS provides pool support for dynamic NAT. Dynamic NAT allows the configuration of a pool of
global addresses that can be used to dynamically allocate a global address from the pool for every new
translation. The addresses are returned to the pool after the session ages out or is closed. This allows for a
more efficient use of addresses based on requirements.
Support for PAT includes the use of the global address pool. This further optimizes IP address utilization.
PAT exhausts one IP address at a time with the use of port numbers. If no port is available from the appropriate
group and more than one IP address is configured, PAT moves to the next IP address and tries to allocate the
original source port again. This process continues until PAT runs out of available ports and IP addresses.
With dynamic NAT and PAT, each host uses a different address or port for each subsequent translation. The
main difference between dynamic NAT and static NAT is that static NAT allows a remote host to initiate a
connection to a translated host if an access list exists that allows it, while dynamic NAT does not.
When dynamic NAT is configured to use a pool of IP addresses, that are not locally available or configured
locally, the out-to-in traffic is considered as DEST MISS. Due to this behavior, the show system internal
access-list dest-miss stats command output displays increment in DEST MISS counters

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
333
 Configuring Static and Dynamic NAT Translation
Static and Dynamic Twice NAT Overview

Static and Dynamic Twice NAT Overview

When both the source IP address and the destination IP address are translated as a single packet that goes
through a Network Address Translation (NAT) device, it is referred to as twice NAT. Twice NAT is supported
for static and dynamic translations.
Twice NAT allows you to configure two NAT translations (one inside and one outside) as part of a group of
translations. These translations can be applied to a single packet as it flows through a NAT device. When you
add two translations as part of a group, both the individual translations and the combined translation take
effect.
A NAT inside translation modifies the source IP address and port number when a packet flows from inside
to outside. It modifies the destination IP address and port number when the packet returns from outside to
inside. NAT outside translation modifies the source IP address and port number when the packet flows from
outside to inside, and it modifies the destination IP address and port number when the packet returns from
inside to outside.
Without twice NAT, only one of the translation rules is applied on a packet, either the source IP address and
port number or the destination IP address and port number.
Static NAT translations that belong to the same group are considered for twice NAT configuration. If a static
configuration does not have a configured group ID, the twice NAT configuration will not work. All inside
and outside NAT translations that belong to a single group that is identified by the group ID are paired to
form twice NAT translations.
Dynamic twice NAT translations dynamically select the source IP address and port number information from
pre-defined ip nat pool or interface overload configurations. Packet filtration is done by configuring ACLs,
and traffic must originate from the dynamic NAT translation rule direction such that source translation is done
by using dynamic NAT rules.
Dynamic twice NAT allows you to configure two NAT translations (one inside and one outside) as part of a
group of translations. One translation must be dynamic and other translation must be static. When these two
translations are part of a group of translations, both the translations can be applied on a single packet as it
goes through the NAT device either from inside to outside or from outside to inside.

VRF Aware NAT

The VRF aware NAT feature enables a switch to understand an address space in a VRF (virtual routing and
forwarding instances) and to translate the packet. This allows the NAT feature to translate traffic in an
overlapping address space that is used between two VRFs.
Notes for VRF aware NAT:
• The VRF aware NAT feature is supported only on the Cisco Nexus 9300 platform switches.
• The VRF aware NAT feature is not supported on the Cisco Nexus 9300-EX, 9300-FX, 9300-FX2 and
9300-GX platform switches.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
334
Configuring Static and Dynamic NAT Translation
VRF Aware NAT

Note This is a NAT TCAM limitation for the Cisco Nexus 9300-EX and 9300-FX
platform switches. NAT TCAM is not VRF aware. NAT does not work with
overlapping IP addresses on Cisco Nexus 9300-EX, 9300-FX, 9300-FX2 and
9300-GX platform switches.

• Traffic flowing from one non-default-vrf to another non-default-vrf is not translated. (For example, vrfA
to vrfB.)
• For traffic flowing from a VRF to a global-VRF, a nat-outside configuration is not supported on a
non-default VRF interface.
• VRF aware NAT is supported by static and dynamic NAT configurations.
• When traffic is configured to flow from a non-default VRF (inside) to a default VRF (outside), the
match-in-vrf option of the ip nat command cannot be specified.
• When traffic is configured to flow from a non-default VRF (inside) to the same non-default VRF
(outside), the match-in-vrf option of the ip nat command must be specified.
The following is an example configuration:

Switch(config)# ip nat inside source {list <acl-name>} {pool <pool-name> [vrf

<vrf-name> [match-in-vrf]] [overload] | interface <globalAddrInterface> [vrf
<vrf-name> [match-in-vrf]] overload} [group <group-id> dynamic]

Switch(config)#ip nat outside source list <acl-name> pool <pool-name> [vrf <vrf-name>
[match-in-vrf]] [group <group-id> dynamic]}

• VRF aware NAT does not support fragmented packets.

• VRF aware NAT does not support application layer translations.
Therefore, Layer 4 and other embedded IPs are not translated and the following will fail:
• FTP
• ICMP failures
• IPSec
• HTTPS

• VRF aware NAT supports NAT or VACL on an interface. (However, both features cannot be supported
at the same time on an interface.)
• VRF aware NAT supports egress ACLs that are applied to the original packet, not on the NAT translated
packet.
• VRF aware NAT supports only the default VRF.
• VRF aware NAT does not provide MIB support.
• VRF aware NAT does not provide DCNM support.
• VRF aware NAT supports only a single global VDC.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
335
 Configuring Static and Dynamic NAT Translation
Guidelines and Limitations for Static NAT

• VRF aware NAT does not support the active/standby supervisor model.
• VRFs with overlapping subnets cannot go to a common destination without NAT. However, you can
achieve this functionality with inter-VRF NAT on dynamic NAT rule configuration. Static NAT
configuration is not supported for overlapping address.

Guidelines and Limitations for Static NAT

Static NAT has the following configuration guidelines and limitations:
• For Broadcom-based Cisco Nexus 9000 Series switches, if the route to your inside global address on the
translating device is reachable via the outside interface, packets for Network Address Translated flows
coming from outside to inside get software forwarded, duplicated, and looped in the network. For this
situation, you must enter the add-route CLI argument on the end of the NAT configuration for this flow.
For example, ip nat inside source static 192.168.1.1 172.16.1.1 add-route.
• The static NAT feature over vPC is not supported on Cisco Nexus 9300 platform switches.
• show commands with the internal keyword are not supported.
• The static NAT feature is supported on Cisco Nexus 9300 platform switches.
• The static NAT feature is supported on Cisco Nexus 9200 platform switches.
• On Cisco Nexus 9200 and 9300-EX platform switches, the add-route option is required for both inside
and outside policies.

Note NAT is not supported on Cisco Nexus 9500 platform switches.

• NAT supports up to 1024 translations which include both static and dynamic NAT.
• If the translated IP is part of the outside interface subnet, then use the ip proxy-arp command on the
NAT outside interface. If the add-route keyword is used, ip proxy-arp should be enabled.
• NAT and sFlow are not supported on the same port.
• The Cisco Nexus device supports NAT on the following interface types:
• Switch Virtual Interfaces (SVIs)
• Routed ports
• Layer 3 and Layer 3 subinterfaces.

• NAT is supported on the default Virtual Routing and Forwarding (VRF) table only.
• NAT is supported for IPv4 Unicast only.
• The Cisco Nexus device does not support the following:
• Software translation. All translations are done in the hardware.
• NAT and VXLAN routing.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
336
 Configuring Static and Dynamic NAT Translation
Restrictions for Dynamic NAT

• Application layer translation. Layer 4 and other embedded IPs are not translated, including FTP,
ICMP failures, IPSec, and HTTPs.
• NAT and VLAN Access Control Lists (VACLs) that are configured on an interface at the same
time.
• PAT translation of fragmented IP packets.
• NAT translation on software forwarded packets. For example, packets with IP-options are not NAT
translated.

• By default no TCAM entries are allocated for the NAT feature. You allocate the TCAM size for the NAT
feature by adjusting the TCAM size of other features. The TCAM can be allocated with the hardware
access-list tcam region nat tcam-size command.
• HSRP and VRRP are supported only on Static NAT.
• If an IP address is used for Static NAT or PAT translations, it cannot be used for any other purpose. For
example, it cannot be assigned to an interface.
• For Static NAT, the outside global IP address should be different from the outside interface IP address.
• NAT statistics are not available.
• When configuring a large number of translations (more than 100), it is faster to configure the translations
before configuring the NAT interfaces.
• For 7.0(3)I4(1) and later, NAT supports (non-disruptive) In Service Software Upgrade (ISSU).

Restrictions for Dynamic NAT

The following restrictions apply to dynamic Network Address Translation (NAT):
• For Broadcom-based Cisco Nexus 9000 Series switches, if the route to your inside global address on the
translating device is reachable via the outside interface, packets for Network Address Translated flows
coming from outside to inside get software forwarded, duplicated, and looped in the network. For this
situation, you must enter the add-route CLI argument on the end of the NAT configuration for this flow.
For example, ip nat inside source static 192.168.1.1 172.16.1.1 add-route.
• VRF aware NAT is not supported for overlapping inside/outside IP subnet addresses on Cisco Nexus
9200 and 9300-EX platform switches.
• show commands with the internal keyword are not supported.
• The dynamic NAT feature is supported on Cisco Nexus 9300 platform switches.
• The dynamic NAT feature is supported on Cisco Nexus 9200 platform switches.
• On Cisco Nexus 9200 and 9300-EX platform switches, the add-route option is required for both inside
and outside policies.
•
• VXLAN routing is not supported on Cisco Nexus devices.
• Fragmented packets are not supported.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
337
 Configuring Static and Dynamic NAT Translation
Guidelines and Limitations for Dynamic Twice NAT

• Application layer gateway (ALG) translations are not supported. ALG, also known as application-level
gateway, is an application that translates IP address information inside the payload of an application
packet.
• Egress ACLs are not applied to translated packets.
• Nondefault virtual routing and forwarding (VRF) instances are not supported.
• MIBs are not supported.
• Cisco Data Center Network Manager (DCNM) is not supported.
• Multiple global virtual device contexts (VDCs) are not supported on Cisco Nexus devices.
• Dynamic NAT translations are not synchronized with active and standby devices.
• Stateful NAT is not supported. However, NAT and Hot Standby Router Protocol (HSRP) can coexist.
• The timeout value for take up to the configured time-out + 119 seconds.
• Normally, ICMP NAT flows time out after the expiration of the configured sampling-timeout and
translation-timeout. However, when ICMP NAT flows present in the switch become idle, they time out
immediately after the expiration of the sampling-timeout configured.
• Beginning with Cisco NX-OS Release 7.0(3)I5(2), hardware programming is introduced for ICMP on
Cisco Nexus 9300 platform switches. Therefore, the ICMP entries consume the TCAM resources in the
hardware. Because ICMP is in the hardware, the maximum limit for NAT translation in Cisco Nexus
platform Series switches is changed to 1024. Maximum of 100 ICMP entries are allowed to make the
best usage of the resources.
• When creating a new translation on a Cisco Nexus 9000 Series switch, the flow is software forwarded
until the translation is programmed in the hardware, which might take a few seconds. During this period,
there is no translation entry for the inside global address. Therefore, returning traffic is dropped. To
overcome this limitation, create a loopback interface and give it an IP address that belongs to the NAT
pool.
• For dynamic NAT, pool overload and interface overload are not supported for the outside NAT.
• Because the NAT overload uses PBR (Policy-Based Routing), the maximum number of available next-hop
entries in the PBR table determines NAT scale. If the number of NAT inside interfaces are within the
range of available next-hops entries in the PBR table, the maximum NAT translation scale remains same.
Otherwise, the maximum number of supported translations may reduce. PBR and NAT-overload are not
mutually exclusive; they are mutually limiting.
• The Cisco Nexus devices does not support NAT and VLAN Access Control Lists (VACLs) that are
configured on an interface at the same time.

Guidelines and Limitations for Dynamic Twice NAT

For Broadcom-based Cisco Nexus 9000 Series switches, if the route to your inside global address on the
translating device is reachable via the outside interface, packets for Network Address Translated flows coming
from outside to inside get software forwarded, duplicated, and looped in the network. For this situation, you
must enter the add-route CLI argument on the end of the NAT configuration for this flow. For example, ip
nat inside source static 192.168.1.1 172.16.1.1 add-route.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
338
 Configuring Static and Dynamic NAT Translation
Configuring Static NAT

IP packets without TCP/UDP/ICMP headers are not translated with dynamic NAT.
In dynamic twice NAT, if dynamic NAT flows are not created before creating static NAT flows, dynamic
twice NAT flows are not created correctly.
When an empty ACL is created, the default rule of permit ip any any is configured. The NAT-ACL does
not match further ACL entries if the first ACL is blank.

Configuring Static NAT

Enabling Static NAT
Procedure

Command or Action Purpose

Step 1 switch# configure terminal Enters global configuration mode.

Step 2 switch(config)# feature nat Enables the static NAT feature on the device.

Step 3 switch(config)# copy running-config Saves the change persistently through reboots
startup-config and restarts by copying the running
configuration to the startup configuration.

Configuring Static NAT on an Interface

Procedure

Command or Action Purpose

Step 1 switch# configure terminal Enters global configuration mode.

Step 2 switch(config)# interface type slot/port Specifies an interface to configure, and enters
interface configuration mode.

Step 3 switch(config-if)# ip nat {inside | outside} Specifies the interface as inside or outside.
Note Only packets that arrive on a marked
interface can be translated.

Step 4 (Optional) switch(config)# copy Saves the change persistently through reboots
running-config startup-config and restarts by copying the running
configuration to the startup configuration.

Example
This example shows how to configure an interface with static NAT from the inside:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
339
 Configuring Static and Dynamic NAT Translation
Enabling Static NAT for an Inside Source Address

switch# configure terminal

switch(config)# interface ethernet 1/4
switch(config-if)# ip nat inside

Enabling Static NAT for an Inside Source Address

For inside source translation, the traffic flows from inside interface to the outside interface. NAT translates
the inside local IP address to the inside global IP address. On the return traffic, the destination inside global
IP address gets translated back to the inside local IP address.

Note When the Cisco Nexus device is configured to translate an inside source IP address (Src:ip1) to an outside
source IP address (newSrc:ip2), the Cisco Nexus device implicitly adds a translation for an outside destination
IP address (Dst: ip2) to an inside destination IP address (newDst: ip1).

Procedure

Command or Action Purpose

Step 1 switch# configure terminal Enters global configuration mode.

Step 2 switch(config)# ip nat inside source static Configures static NAT to translate the inside
local-ip-address global-ip-address [vrf local address to the inside global address or to
vrf-name] [match-in-vrf] [group group-id ] translate the opposite (the inside global traffic
to the inside local traffic). Specifying group
specifies the group to which this translation
belongs on the static twice NAT.
Note While performing twice NAT
configuration in Cisco Nexus 9000
Series switches, you cannot use the
same group ID across different
VRFs. A unique group ID should be
used for unique twice NAT rules.

Step 3 (Optional) switch(config)# copy Saves the change persistently through reboots
running-config startup-config and restarts by copying the running
configuration to the startup configuration.

Example
This example shows how to configure static NAT for an inside source address:
switch# configure terminal
switch(config)# ip nat inside source static 1.1.1.1 5.5.5.5
switch(config)# copy running-config startup-config

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
340
 Configuring Static and Dynamic NAT Translation
Enabling Static NAT for an Outside Source Address

Enabling Static NAT for an Outside Source Address

For outside source translation, the traffic flows from the outside interface to the inside interface. NAT translates
the outside global IP address to the outside local IP address. On the return traffic, the destination outside local
IP address gets translated back to outside global IP address.

Procedure

Command or Action Purpose

Step 1 switch# configure terminal Enters global configuration mode.

Step 2 switch(config)# ip nat outside source static Configures static NAT to translate the outside
outsideGlobalIP outsideLocalIP [vrf vrf-name global address to the outside local address or to
[match-in-vrf] [group group-id] [dynamic] translate the opposite (the outside local traffic
[add-route] ] to the outside global traffic). Specifying group
specifies the group to which this translation
belongs on the static twice NAT. When an
inside translation without ports is configured,
an implicit add route is performed. The original
add route functionality is an option while
configuring an outside translation.

Step 3 (Optional) switch(config)# copy Saves the change persistently through reboots
running-config startup-config and restarts by copying the running
configuration to the startup configuration.

Example
This example show how to configure static NAT for an outside source address:
switch# configure terminal
switch(config)# ip nat outside source static 2.2.2.2 6.6.6.6
switch(config)# copy running-config startup-config

Configuring Static PAT for an Inside Source Address

You can map services to specific inside hosts using Port Address Translation (PAT).

Procedure

Command or Action Purpose

Step 1 switch# configure terminal Enters global configuration mode.

Step 2 switch(config)# ip nat inside source static Maps static NAT to an inside local port to an
{inside-local-address inside-global-address | inside global port.
{tcp| udp} inside-local-address {local-tcp-port
| local-udp-port} inside-global-address
{global-tcp-port | global-udp-port}} {vrf
vrf-name {match-in-vrf} {group group-id} }

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
341
 Configuring Static and Dynamic NAT Translation
Configuring Static PAT for an Outside Source Address

Command or Action Purpose

Step 3 (Optional) switch(config)# copy Saves the change persistently through reboots
running-config startup-config and restarts by copying the running
configuration to the startup configuration.

Example
This example shows how to map UDP services to a specific inside source address and UDP port:
switch# configure terminal
switch(config)# ip nat inside source static udp 20.1.9.2 63 35.48.35.48 130
switch(config)# copy running-config startup-config

Configuring Static PAT for an Outside Source Address

You can map services to specific outside hosts using Port Address Translation (PAT).

Procedure

Command or Action Purpose

Step 1 switch# configure terminal Enters global configuration mode.

Step 2 switch(config)# ip nat outside source static Maps static NAT to an outside global port to
{outside-global-address outside-local-address an outside local port.
| {tcp | udp} outside-global-address
Specifying group specifies the group to which
{global-tcp-port | global-udp-port}
this translation belongs on the static twice NAT.
outside-local-address {global-tcp-port |
When an inside translation without ports is
global-udp-port}} {group group-id}
configured, an implicit add route is performed.
{add-route} {vrf vrf-name {match-in-vrf}}
The original add route functionality is an option
while configuration an outside translation.

Step 3 (Optional) switch(config)# copy Saves the change persistently through reboots
running-config startup-config and restarts by copying the running
configuration to the startup configuration.

Example
This example shows how to map TCP services to a specific outside source address and TCP port:
switch# configure terminal
switch(config)# ip nat outside source static tcp 20.1.9.2 63 35.48.35.48 130
switch(config)# copy running-config startup-config

Configuring Static Twice NAT

All translations within the same group are considered for creating static twice Network Address Translation
(NAT) rules.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
342
Configuring Static and Dynamic NAT Translation
Configuring Static Twice NAT

Procedure

Command or Action Purpose

Step 1 enable Enables privileged EXEC mode.
Example: • Enter your password if prompted.
switch> enable

Step 2 configure terminal Enters privileged EXEC mode.

Example:
switch# configure terminal

Step 3 ip nat inside source static Configures static twice NAT to translate an
inside-local-ip-address inside local IP address to the corresponding
inside-global-ip-address [group group-id] inside global IP address.
Example: • The group keyword determines the group
switch(config)# ip nat inside source to which a translation belongs.
static 10.1.1.1 192.168.34.4 group 4

Step 4 ip nat outside source static Configures static twice NAT to translate an
outside-global-ip-address outside global IP address to the corresponding
outside-local-ip-address [group group-id] outside local IP address.
[add-route]
• The group keyword determines the group
Example: to which a translation belongs.
switch(config)# ip nat outside source
static 209.165.201.1 10.3.2.42 group 4
add-route

Step 5 interface type number Configures an interface and enters interface

configuration mode.
Example:
switch(config)# interface ethernet 1/2

Step 6 ip address ip-address mask Sets a primary IP address for an interface.

Example:
switch(config-if)# ip address 10.2.4.1
255.255.255.0

Step 7 ip nat inside Connects the interface to an inside network,

which is subject to NAT.
Example:
switch(config-if)# ip nat inside

Step 8 exit Exits interface configuration mode and returns

to global configuration mode.
Example:
switch(config-if)# exit

Step 9 interface type number Configures an interface and enters interface

configuration mode.
Example:
switch(config)# interface ethernet 1/1

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
343
 Configuring Static and Dynamic NAT Translation
Configuration Example for Static NAT and PAT

Command or Action Purpose

Step 10 ip address ip-address mask Sets a primary IP address for an interface.
Example:
switch(config-if)# ip address 10.5.7.9
255.255.255.0

Step 11 ip nat outside Connects the interface to an outside network,

which is subject to NAT.
Example:
switch(config-if)# ip nat outside

Step 12 end Exits interface configuration mode and returns

to privileged EXEC mode.
Example:
switch(config-if)# end

Configuration Example for Static NAT and PAT

This example shows the configuration for static NAT:

ip nat inside source static 103.1.1.1 11.3.1.1

ip nat inside source static 139.1.1.1 11.39.1.1
ip nat inside source static 141.1.1.1 11.41.1.1
ip nat inside source static 149.1.1.1 95.1.1.1
ip nat inside source static 149.2.1.1 96.1.1.1
ip nat outside source static 95.3.1.1 95.4.1.1
ip nat outside source static 96.3.1.1 96.4.1.1
ip nat outside source static 102.1.2.1 51.1.2.1
ip nat outside source static 104.1.1.1 51.3.1.1
ip nat outside source static 140.1.1.1 51.40.1.1

This example shows the configuration for static PAT:

ip nat inside source static tcp 10.11.1.1 1 210.11.1.1 101

ip nat inside source static tcp 10.11.1.1 2 210.11.1.1 201
ip nat inside source static tcp 10.11.1.1 3 210.11.1.1 301
ip nat inside source static tcp 10.11.1.1 4 210.11.1.1 401
ip nat inside source static tcp 10.11.1.1 5 210.11.1.1 501
ip nat inside source static tcp 10.11.1.1 6 210.11.1.1 601
ip nat inside source static tcp 10.11.1.1 7 210.11.1.1 701
ip nat inside source static tcp 10.11.1.1 8 210.11.1.1 801
ip nat inside source static tcp 10.11.1.1 9 210.11.1.1 901
ip nat inside source static tcp 10.11.1.1 10 210.11.1.1 1001
ip nat inside source static tcp 10.11.1.1 11 210.11.1.1 1101
ip nat inside source static tcp 10.11.1.1 12 210.11.1.1 1201

Example: Configuring Static Twice NAT

The following example shows how to configure the inside source and outside source static twice
NAT configurations:
Switch> enable
Switch# configure terminal
Switch(config)# ip nat inside source static 10.1.1.1 192.168.34.4 group 4

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
344
 Configuring Static and Dynamic NAT Translation
Verifying the Static NAT Configuration

Switch(config)# ip nat outside source static 209.165.201.1 10.3.2.42 group 4

Switch(config)# interface ethernet 1/2
Switch(config-if)# ip address 10.2.4.1 255.255.255.0
Switch(config-if)# ip nat inside
switch(config-if)# exit
switch(config)# interface ethernet 1/1
switch(config-if)# ip address 10.5.7.9 255.255.255.0
switch(config-if)# ip nat outside
Switch(config-if)# end

Verifying the Static NAT Configuration

To display the static NAT configuration, perform this task:

Procedure

Command or Action Purpose

Step 1 switch# show ip nat translations Shows the translations for the inside global,
inside local, outside local, and outside global
IP addresses.

Example
This example shows how to display the static NAT configuration:

switch# sh ip nat translations

Pro Inside global Inside local Outside local Outside global

--- --- --- 51.3.1.1 104.1.1.1

--- --- --- 95.4.1.1 95.3.1.1
--- --- --- 96.4.1.1 96.3.1.1
--- --- --- 51.40.1.1 140.1.1.1
--- --- --- 51.42.1.1 142.1.2.1
--- --- --- 51.1.2.1 102.1.2.1
--- 11.1.1.1 101.1.1.1 --- ---
--- 11.3.1.1 103.1.1.1 --- ---
--- 11.39.1.1 139.1.1.1 --- ---
--- 11.41.1.1 141.1.1.1 --- ---
--- 95.1.1.1 149.1.1.1 --- ---
--- 96.1.1.1 149.2.1.1 --- ---
130.1.1.1:590 30.1.1.100:5000 --- ---
130.2.1.1:590 30.2.1.100:5000 --- ---
130.3.1.1:590 30.3.1.100:5000 --- ---
130.4.1.1:590 30.4.1.100:5000 --- ---
130.1.1.1:591 30.1.1.101:5000 --- ---

switch# sh ip nat translations verbose

Pro Inside global Inside local Outside local Outside global
any --- --- 22.1.1.3 22.1.1.2
Flags:0x200009 time-left(secs):-1 id:0 state:0x0 grp_id:10
any 11.1.1.130 11.1.1.3 --- ---
Flags:0x1 time-left(secs):-1 id:0 state:0x0 grp_id:0
any 11.1.1.133 11.1.1.33 --- ---
Flags:0x1 time-left(secs):-1 id:0 state:0x0 grp_id:10

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
345
 Configuring Static and Dynamic NAT Translation
Configuring Dynamic NAT

any 11.1.1.133 11.1.1.33 22.1.1.3 22.1.1.2

Flags:0x200009 time-left(secs):-1 id:0 state:0x0 grp_id:0
tcp 10.1.1.100:64490 10.1.1.2:0 20.1.1.2:0 20.1.1.2:0
Flags:0x82 time-left(secs):43192 id:31 state:0x3 grp_id:0 vrf: default
N9300-1#

Configuring Dynamic NAT

Configuring Dynamic Translation and Translation Timeouts
Procedure

Command or Action Purpose

Step 1 enable Enables privileged EXEC mode.
Example: • Enter your password if prompted.
Switch> enable

Step 2 configure terminal Enters global configuration mode.

Example:
Switch# configure terminal

Step 3 ip access-list access-list-name Defines an access list and enters access-list

configuration mode.
Example:
Switch(config)# ip access-list acl1

Step 4 permit protocol source source-wildcard any Sets conditions in an IP access list that permit
traffic matching the conditions.
Example:
Switch(config-acl)# permit ip
10.111.11.0/24 any

Step 5 deny protocol source source-wildcard any Sets conditions in an IP access list that deny
packets from entering a network.
Example:
Switch(config-acl)# deny udp
10.111.11.100/32 any

Step 6 exit Exits access-list configuration mode and

returns to global configuration mode.
Example:
Switch(config-acl)# exit

Step 7 ip nat inside source list access-list-name Establishes dynamic source translation by
interface type number [vrf vrf-name specifying the access list defined in Step 3.
[match-in-vrf] overload]
Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
346
Configuring Static and Dynamic NAT Translation
Configuring Dynamic Translation and Translation Timeouts

Command or Action Purpose

Switch(config)# ip nat inside source
list acl1 interface ethernet 1/1
overload

Step 8 interface type number Configures an interface and enters interface

configuration mode.
Example:
Switch(config)# interface ethernet 1/4

Step 9 ip address ip-address mask Sets a primary IP address for the interface.
Example:
Switch(config-if)# ip address
10.111.11.39 255.255.255.0

Step 10 ip nat inside Connects the interface to an inside network,

which is subject to NAT.
Example:
Switch(config-if)# ip nat inside

Step 11 exit Exits interface configuration mode and returns

to global configuration mode.
Example:
Switch(config-if)# exit

Step 12 interface type number Configures an interface and enters interface

configuration mode.
Example:
Switch(config)# interface ethernet 1/1

Step 13 ip address ip-address mask Sets a primary IP address for an interface.

Example:
Switch(config-if)# ip address
172.16.232.182 255.255.255.240

Step 14 ip nat outside Connects the interface to an outside network.

Example:
Switch(config-if)# ip nat outside

Step 15 exit Exits interface configuration mode and returns

to global configuration mode.
Example:
Switch(config-if)# exit

Step 16 ip nat translation max-entries Specifies the maximum number of dynamic

number-of-entries NAT translations. The number of entries can
be between 1 and 1023.
Example:
Switch(config)# ip nat translation
max-entries 300

Step 17 ip nat translation timeout seconds Specifies the timeout value for dynamic NAT
translations.
Example:

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
347
 Configuring Static and Dynamic NAT Translation
Configuring Dynamic NAT Pool

Command or Action Purpose

switch(config)# ip nat translation
timeout 13000

Step 18 end Exits global configuration mode and returns

to privileged EXEC mode.
Example:
Switch(config)# end

Configuring Dynamic NAT Pool

You can create a NAT pool by either defining the range of IP addresses in a single ip nat pool command
or by using the ip nat pool and address commands

Procedure

Command or Action Purpose

Step 1 switch# configure terminal Enters global configuration mode.

Step 2 switch(config)# feature nat Enables the NAT feature on the device.

Step 3 switch(config)# ip nat pool pool-name [startip Creates a NAT pool with a range of global IP
endip] {prefix prefix-length | netmask addresses. The IP addresses are filtered by using
network-mask} either a prefix length or a network mask.

Step 4 (Optional) switch(config-ipnat-pool)# address Specifies the range of global IP addresses if

startip endip they were not specified during creation of the
pool.

Step 5 (Optional) switch(config)# no ip nat pool Deletes the specified NAT pool.
pool-name

Example
This example shows how to create a NAT pool with a prefix length:
switch# configure terminal
switch(config)# ip nat pool pool1 30.1.1.1 30.1.1.2 prefix-length 24
switch(config)#

This example shows how to create a NAT pool with a network mask:
switch# configure terminal
switch(config)# ip nat pool pool5 20.1.1.1 20.1.1.5 netmask 255.0.255.0
switch(config)#

This example shows how to create a NAT pool and define the range of global IP addresses using the
ip nat pool and address commands:
switch# configure terminal
switch(config)# ip nat pool pool7 netmask 255.255.0.0

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
348
 Configuring Static and Dynamic NAT Translation
Configuring Source Lists

switch(config-ipnat-pool)# address 40.1.1.1 40.1.1.5

switch(config-ipnat-pool)#

This example shows how to delete a NAT pool:

switch# configure terminal
switch(config)# no ip nat pool pool4
switch(config)#

Configuring Source Lists

You can configure a source list of IP addresses for the inside interface and the outside interface.

Before you begin

Ensure that you configure a pool before configuring the source list for the pool.

Procedure

Command or Action Purpose

Step 1 switch# configure terminal Enters global configuration mode.

Step 2 (Optional) switch# ip nat inside source list Creates a NAT inside source list with pool with
list-name pool pool-name [overload] or without overloading.

Step 3 (Optional) switch# ip nat outside source list Creates a NAT outside source list with pool
list-name pool pool-name [add-route] without overloading.

Example
This example shows how to create a NAT inside source list with pool without overloading:
switch# configure terminal
switch(config)# ip nat inside source list list1 pool pool1
switch(config)#

This example shows how to create a NAT inside source list with pool with overloading:
switch# configure terminal
switch(config)# ip nat inside source list list2 pool pool2 overload
switch(config)#

This example shows how to create a NAT outside source list with pool without overloading:
switch# configure terminal
switch(config)# ip nat outside source list list3 pool pool3
switch(config)#

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
349
 Configuring Static and Dynamic NAT Translation
Configuring Dynamic Twice NAT for an Inside Source Address

Configuring Dynamic Twice NAT for an Inside Source Address

For an inside source address translation, the traffic flows from the inside interface to the outside interface.
You can configure dynamic twice NAT for an inside source address.

Before you begin

Ensure that you enable NAT on the switch.

Procedure

Command or Action Purpose

Step 1 switch# configure terminal Enters global configuration mode.

Step 2 switch(config)# ip nat outside source static Configures static NAT to translate an outside
outside-global-ip-address global address to an inside local address or to
outside-local-ip-address | [tcp | udp] translate inside local traffic to inside global
outside-global-ip-address outside-global-port traffic.
outside-local-ip-address outside-local-port
The group keyword determines the group to
[group group-id] [add-route] [dynamic]
which a translation belongs.

Step 3 switch(config)# ip nat inside source list Establishes dynamic source translation by
access-list-name [interface type slot/port creating a NAT inside source list with pool with
overload | pool pool-name overload] [group or without overloading.
group-id] dynamic]
The group keyword determines the group to
which a translation belongs.

Step 4 switch(config)# ip nat pool pool-name [startip Creates a NAT pool with a range of global IP
endip] {prefix prefix-length | netmask addresses. The IP addresses are filtered by using
network-mask} either a prefix length or a network mask.

Step 5 switch(config)# interface type slot/port Configures an interface and enters interface
configuration mode.

Step 6 switch(config-if)# ip nat outside Connects the interface to an outside network.

Step 7 switch(config-if)# exit Exits interface configuration mode and returns

to global configuration mode.

Step 8 switch(config)# interface type slot/port Configures an interface and enters interface
configuration mode.

Step 9 switch(config-if)# ip nat inside Connects the interface to an inside network,

which is subject to NAT.

Example
This example shows how to configure dynamic twice NAT for an inside source address:
switch# configure terminal
Enter configuration commands, one per line. End with CNTL/Z.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
350
 Configuring Static and Dynamic NAT Translation
Configuring Dynamic Twice NAT for an Outside Source Address

switch(config)# ip nat outside source static 2.2.2.2 4.4.4.4 group 20 dynamic

switch(config)# ip nat inside source list acl_1 pool pool_1 overload group 20 dynamic
switch(config)# ip nat pool pool_1 3.3.3.3 3.3.3.10 prefix-length 24
switch(config)# interface Ethernet1/8
switch(config-if)# ip nat outside
switch(config-if)# exit
switch(config)# interface Ethernet1/15
switch(config-if)# ip nat inside

Configuring Dynamic Twice NAT for an Outside Source Address

For an outside source address translation, the traffic flows from the outside interface to the inside interface.
You can configure dynamic twice NAT for an outside source address.

Before you begin

Ensure that you enable NAT on the switch.

Procedure

Command or Action Purpose

Step 1 switch# configure terminal Enters global configuration mode.

Step 2 switch(config)# ip nat inside source static Configures static NAT to translate an inside
inside-local-ip-address inside-global-ip-address global address to an inside local address or to
| [tcp | udp] inside-local-ip-address local-port translate inside local traffic to inside global
inside-global-ip-address global-port [group traffic.
group-id] [dynamic]
The group keyword determines the group to
which a translation belongs.

Step 3 switch(config)# ip nat outside source list Establishes dynamic source translation by
access-list-name pool pool-name [group creating a NAT outside source list with pool
group-id] dynamic [add-route] with or without overloading.

Step 4 switch(config)# ip nat pool pool-name [startip Creates a NAT pool with a range of global IP
endip] {prefix prefix-length | netmask addresses. The IP addresses are filtered by using
network-mask} either a prefix length or a network mask.

Step 5 switch(config)# interface type slot/port Configures an interface and enters interface
configuration mode.

Step 6 switch(config-if)# ip nat outside Connects the interface to an outside network.

Step 7 switch(config-if)# exit Exits interface configuration mode and returns

to global configuration mode.

Step 8 switch(config)# interface type slot/port Configures an interface and enters interface
configuration mode.

Step 9 switch(config-if)# ip nat inside Connects the interface to an inside network,

which is subject to NAT.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
351
 Configuring Static and Dynamic NAT Translation
Clearing Dynamic NAT Translations

Example
This example shows how to configure dynamic twice NAT for an outside source address:
switch# configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
switch(config)# ip nat inside source static 7.7.7.7 5.5.5.5 group 30 dynamic
switch(config)# ip nat outside source list acl_1 pool pool_1 group 30 dynamic
switch(config)# ip nat pool pool_2 4.4.4.4 4.4.4.10 prefix-length 24
switch(config)# interface Ethernet1/6
switch(config-if)# ip nat outside
switch(config-if)# exit
switch(config)# interface Ethernet1/11
switch(config-if)# ip nat inside

Clearing Dynamic NAT Translations

To clear dynamic translations, perform the following task:

Command Purpose
clear ip nat translation [ all | inside Deletes all or specific dynamic NAT translations.
global-ip-address local-ip-address [outside
local-ip-address global-ip-address] | outside
local-ip-address global-ip-address ]

Example
This example shows how to clear all dynamic translations:
switch# clear ip nat translation all

This example shows how to clear dynamic translations for inside and outside addresses:
switch# clear ip nat translation inside 2.2.2.2 4.4.4.4 outside 5.5.5.5 7.7.7.7

Verifying Dynamic NAT Configuration

To display dynamic NAT configuration, perform the following tasks:

Command Purpose
show ip nat translations Displays active Network Address Translation (NAT)
translations.
Displays additional information for each translation
table entry, including when an entry was created and
used.

show run nat Displays NAT configuration.

show ip nat max Displays active Network Address Translation (NAT)

maximum values.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
352
Configuring Static and Dynamic NAT Translation
Verifying Dynamic NAT Configuration

Command Purpose
show ip nat statistics Monitor NAT statistics.

Example
This example shows how to display IP NAT Max values:
switch# show ip nat max

IP NAT Max values

====================
Max Dyn Translations:80
Max all-host:0
No.Static:0
No.Dyn:1
No.Dyn-ICMP:1
====================
Switch(config)#

This example shows how to display NAT Statistics:

switch# show ip nat statistics

IP NAT Statistics
====================================================
Stats Collected since: Mon Feb 24 18:27:34 2020
----------------------------------------------------
Total active translations: 1
No.Static: 0
No.Dyn: 1
No.Dyn-ICMP: 1
----------------------------------------------------
Total expired Translations: 0
SYN timer expired: 0
FIN-RST timer expired: 0
Inactive timer expired: 0
----------------------------------------------------
Total Hits: 2 Total Misses: 2
In-Out Hits: 0 In-Out Misses: 2
Out-In Hits: 2 Out-In Misses: 0
----------------------------------------------------
Total SW Translated Packets: 2
In-Out SW Translated: 2
Out-In SW Translated: 0
----------------------------------------------------
Total SW Dropped Packets: 0
In-Out SW Dropped: 0
Out-In SW Dropped: 0

Address alloc. failure drop: 0

Port alloc. failure drop: 0
Dyn. Translation max limit drop: 0
ICMP max limit drop: 0
Allhost max limit drop: 0
----------------------------------------------------
Total TCP session established: 0
Total TCP session closed: 0
----------------------------------------------------
NAT Inside Interfaces: 1
Ethernet1/34

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
353
 Configuring Static and Dynamic NAT Translation
Verifying Dynamic NAT Configuration

NAT Outside Interfaces: 1

Ethernet1/32
----------------------------------------------------
Inside source list:
++++++++++++++++++++

Access list: T2
RefCount: 1
Pool: T2 Overload
Total addresses: 10
Allocated: 1 percentage: 10%
Missed: 0

Outside source list:

++++++++++++++++++++
----------------------------------------------------
====================================================
Switch(config)#
Switch(config)#

**No.Dyn-ICMP field is to display the no of icmp dynamic translations , its a subset of

"No.Dyn" field.

This example shows how to display running configuration for NAT:

switch# show run nat

!Command: show running-config nat

!Time: Wed Apr 23 11:17:43 2014

version 6.0(2)A3(1)
feature nat

ip nat inside source list list1 pool pool1

ip nat inside source list list2 pool pool2 overload
ip nat inside source list list7 pool pool7 overload
ip nat outside source list list3 pool pool3
ip nat pool pool1 30.1.1.1 30.1.1.2 prefix-length 24
ip nat pool pool2 10.1.1.1 10.1.1.2 netmask 255.0.255.0
ip nat pool pool3 30.1.1.1 30.1.1.8 prefix-length 24
ip nat pool pool5 20.1.1.1 20.1.1.5 netmask 255.0.255.0
ip nat pool pool7 netmask 255.255.0.0
address 40.1.1.1 40.1.1.5

This example shows how to display active NAT translations:

Inside pool with overload
switch# show ip nat translation
Pro Inside global Inside local Outside local Outside global
icmp 20.1.1.3:64762 10.1.1.2:133 20.1.1.1:0 20.1.1.1:0
icmp 20.1.1.3:64763 10.1.1.2:134 20.1.1.1:0 20.1.1.1:0

Outside pool without overload

switch# show ip nat translation
Pro Inside global Inside local Outside local Outside global
any --- --- 177.7.1.1:0 77.7.1.64:0

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
354
 Configuring Static and Dynamic NAT Translation
Example: Configuring Dynamic Translation and Translation Timeouts

any --- --- 40.146.1.1:0 40.46.1.64:0

any --- --- 10.4.146.1:0 10.4.46.64:0

Example: Configuring Dynamic Translation and Translation Timeouts

The following example shows how to configure dynamic overload Network Address Translation
(NAT) by specifying an access list:
Switch> enable
Switch# configure terminal
Switch(config)# ip access-list acl1
Switch(config-acl)# permit ip 10.111.11.0/24 any
Switch(config-acl)# deny udp 10.111.11.100/32 any
Switch(config-acl)# exit
Switch(config)# ip nat inside source list acl1 interface ethernet 1/1 overload
Switch(config)# interface ethernet 1/4
Switch(config-if)# ip address 10.111.11.39 255.255.255.0
Switch(config-if)# ip nat inside
Switch(config-if)# exit
Switch(config)# interface ethernet 1/1
Switch(config-if)# ip address 172.16.232.182 255.255.255.240
Switch(config-if)# ip nat outside
Switch(config-if)# exit
Switch(config)# ip nat translation max-entries 300
Switch(config)# ip nat translation timeout 13000
Switch(config)# end

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
355
 Configuring Static and Dynamic NAT Translation
Example: Configuring Dynamic Translation and Translation Timeouts

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
356
 APPENDIX A
Configuring Layer 2 Data Center Interconnect
This section contains an example of how to configure a Layer 2 Data Center Interconnect (DCI) with the use
of a Virtual Port-Channel (vPC).
• Overview, on page 357
• Example of Layer 2 Data Center Interconnect, on page 357

Overview
The purpose of a Data Center Interconnect (DCI) is to extend specific VLANs between different data centers.
DCI offers Layer 2 adjacency for servers and Network Attached Storage (NAS) devices that are separated by
large distances.
Beginning with Cisco NX-OS release 7.0(3)I2(2), Cisco Nexus 9000 series switches support DCI with FHRP
isolation. However DCI with FHRP isolation is not supported on Cisco Nexus 9500 switches with
N9K-X9636C-R and N9K-X9636Q-R line cards. Creating a single logical link between multiple sites with
vPC allows you to take advantage of the benefits of STP isolation using BPDU filtering across the DCI vPC
port-channel. With this configuration, Bridge Protocol Data Unit (BPDU) does not cross between data centers,
effectively isolating the STP fault domain between sites.

Note vPC is to interconnect a maximum of two data centers.

Example of Layer 2 Data Center Interconnect

The following is an example configuration of a Layer 2 Data Center Interconnect (DCI) with use of vPC. The
example allows for First Hop Redundancy Protocol (FHRP) isolation.

Note vPC and Hot Standby Routing Protocol (HSRP) have already been configured.

Note Link Aggregation Control Protocol (LACP) should be used on the vPC link, which acts as the DCI.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
357
 Configuring Layer 2 Data Center Interconnect
Configuring Layer 2 Data Center Interconnect

Figure 40: Dual Layer 2/Layer 3 POD Interconnect

In this example, the Layer 3 (L3) gateway is configured on the same vPC pair and acts as the DCI. In order
to isolate the Hot Standby Routing Protocol (HSRP), you must configure a Port Access Control List (PACL)
on the DCI port-channel and disable HSRP Gratuitous Address Resolution Protocols (ARPs) (GARPs) on the
Switched Virtual Interfaces (SVIs) for the VLANs that move across the DCI.

ip access-list DENY_HSRP_IP
10 deny udp any 224.0.0.2/32 eq 1985
20 deny udp any 224.0.0.102/32 eq 1985
30 permit ip any any

interface <DCI-Port-Channel>
ip port access-group DENY_HSRP_IP in

interface Vlan <x>

no ip arp gratuitous hsrp duplicate

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
358
 APPENDIX B
IETF RFCs supported by Cisco NX-OS Interfaces
This appendix lists the IETF RFCs for interfaces supported by Cisco NX-OS.
• IPv6 RFCs, on page 359

IPv6 RFCs
RFCs Title

RFC 1981 (7.0(3)I1(1) and later) Path MTU Discovery for IP version 6

RFC 2373 IP Version 6 Addressing Architecture

RFC 2374 An Aggregatable Global Unicast Address Format

RFC 2460 Internet Protocol, Version 6 (IPv6) Specification

RFC 2462 IPv6 Stateless Address Autoconfiguration

RFC 2464 Transmission of IPv6 Packets over Ethernet Networks

RFC 2467 Transmission of IPv6 Packets over FDDI Networks

RFC 2472 IP Version 6 over PPP

RFC 2492 IPv6 over ATM Networks

RFC 2590 Transmission of IPv6 Packets over Frame Relay Networks Specification

RFC 3021 Using 31-Bit Prefixes on IPv4 Point-to-Point Links

RFC 3152 Delegation of IP6.ARPA

RFC 3162 RADIUS and IPv6

RFC 3513 Internet Protocol Version 6 (IPv6) Addressing Architecture

RFC 3596 DNS Extensions to Support IP version 6

RFC 4193 Unique Local IPv6 Unicast Addresses

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
359
 IETF RFCs supported by Cisco NX-OS Interfaces
IETF RFCs supported by Cisco NX-OS Interfaces

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
360
 APPENDIX C
Configuration Limits for Cisco NX-OS Interfaces
The configuration limits are documented in the Cisco Nexus 9000 Series NX-OS Verified Scalability Guide.

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
361
 Configuration Limits for Cisco NX-OS Interfaces
Configuration Limits for Cisco NX-OS Interfaces

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
362
 INDEX

A E
address 348 ecmp load balancing 204
admin-shutdown 95 enable 343, 346
auto-recovery 232, 264, 267 encapsulation dot1Q 98, 100, 104, 106
autonomous-system 142 end 106, 348
errdisable detect cause 20, 35
errdisable detect cause acl-exception 35
B errdisable detect cause all 35
bandwidth 41, 181 errdisable detect cause link-flap 35
bfd 142, 143, 144 errdisable detect cause loopback 35
bfd authentication keyed-sha1 keyid 130, 131 errdisable recovery cause 20, 36
bfd echo 133 errdisable recovery cause all 36
bfd echo-interface loopback 129 errdisable recovery cause bpduguard 36
bfd interval 128, 130, 131, 151, 152 errdisable recovery cause failed-port-state 36
bfd per-link 131 errdisable recovery cause link-flap 36
bfd slow-timer 128, 132 errdisable recovery cause loopback 36
bfd startup-timer 129 errdisable recovery cause miscabling 36
broadcast 97 errdisable recovery cause psecure-violation 36
errdisable recovery cause security-violation 36
errdisable recovery cause storm-control 36
C errdisable recovery cause udld 36
channel-group 177, 179, 188 errdisable recovery cause vpc-peerlink 36
checkpoint 78 errdisable recovery interval 20, 37
clear counters interface 55, 87 ethernet 32
clear counters interface port-channel 206 Ethernet 11
clear ip nat translation 352
clear ip route 126 F
clear ipv6 route 126
clear l2protocol tunnel counters 315 feature bfd 128
clear lacp counters 207 feature eigrp 42
copy 29, 31 feature interface-vlan 82, 101
feature isis 106
feature lacp 187
D feature nat 339, 348
default interface 78 feature tunnel 286
delay 42, 181 feature vpc 247
delay restore 231, 233
deny 346 G
description 32, 184, 287
duplex 185 graceful consistency-check 257
duplex auto 185
duplex full 185
duplex half 185

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
IN-1
 INDEX

H ipv6 address 96, 98, 100, 101, 103, 112, 294

ipv6 address dhcp 93, 94
hardware access-list tcam region nat 337 ipv6 address use-link-local-only 94
how l2protocol tunnel summary 315 ipv6 nd mac-extract 112
hsrp bfd 146 ipv6 nd synchronize 228
hsrp bfd all-interfaces 146 isis bfd 144
isis bfd disable 150
I
L
iip nat inside source list 349
iip nat inside source static 351 l2protocol tunnel 311
include bfd 128 l2protocol tunnel cos 312
interface 32, 39, 43, 79, 110, 112, 130, 268, 339, 343, 347, 350, 351 l2protocol tunnel drop-threshold 313
interface breakout 11 l2protocol tunnel shutdown-threshold 313
interface ether 54 lacp graceful-convergence 169, 197
interface ethernet 33, 38, 40, 41, 45, 47, 69, 70, 75, 77, 96, 98, 104, 106, lacp max-bundle 190
114, 307, 310, 311, 313 lacp min-links 189
interface loopback 103, 105 lacp mode delay 200
interface overload 334 lacp port-priority 193
interface port-channel 75, 77, 79, 99, 131, 150, 151, 152, 175, 180, 181, lacp rate 191
182, 183, 184, 189, 190, 195, 196, 198, 199, 200, 202, 252, 258 lacp rate fast 192
interface tunnel 287, 289, 291, 293, 294, 297 lacp suspend-individual 197, 199
interface vlan 82, 101 lacp system-priority 192
interfaces-vlan 231, 233 link debounce link-up 47
ip 28 link debounce time 47
ip access-list 346 load- interval 87, 116, 206, 207
ip address 96, 98, 100, 101, 103, 105, 111, 151, 289, 297, 343, 344, 347 load-interval counters 54
ip address dhcp 93 loopback 104, 105, 107
ip arp synchronize 228
ip eigrp 142, 149
ip name-server 94
M
ip nat 335 mac-address ipv6-extract 112
ip nat inside 339, 343, 347, 350, 351 match-in-vrf 335
ip nat inside source list 346, 350 medium 97
ip nat inside source static 340, 341, 343 medium broadcast 97
ip nat outside 339, 344, 347, 350, 351 medium p2p 97, 104, 106
ip nat outside source list 349, 351 mgmt0 32
ip nat outside source static 341, 342, 343, 350 mtu 38, 40, 287, 292, 295
ip nat pool 334, 348, 350, 351
ip nat translation mas-entries 347
ip nat translation sampling-timeout 331, 332 N
ip nat translation timeout 347
negotiate auto 6, 27
ip ospf authentication 105
neighbor 141
ip ospf authentication-key 105
net 106
ip ospf bfd 143, 150, 152
ip ospf bfd disable 149
ip pim bfd 148 P
ip pim bfd-instance 148
ip pim pre-build-spt 230 p2p 97
ip pim spt-threshold infinity 229 peer-gateway 217, 254
ip pim use-shared-tree-only 229 peer-gateway exclude-vlan 217
ip route 94, 149 peer-keepalive destination 251
ip route static bfd 149 peer-switch 271, 272
ip router isis 107 permit 346
ip router ospf 105 permit ip any any 339
ip unnumbered 5, 104, 105, 106 port-channel load-balance 162, 186

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
IN-2
 INDEX

R show lacp counters 207

show lacp system-identifier 193
regex 28 show mac address-table 231
reload restore 264, 265 show port-channel capacity 276
role priority 262 show port-channel compatibility-parameters 160, 206
router bgp 141 show port-channel database 206
router eigrp 142 show port-channel load-balance 186, 206
router isis 106, 144 show port-channel summary 175, 188, 206
router ospf 143 show port-channel traffic 206
show port-channel usage 206
show run nat 352
S
show running config 97
sampling-timeout 332 show running-config 81, 87
show 97 show running-config all 39
show bfd 154 show running-config bfd 129, 130, 132, 133, 153
show bfd neighbors 154 show running-config bgp 141
show cdp all 54 show running-config hsrp 146
show cfs application 231 show running-config interface ethernet 87
show dot1q-tunnel 308, 315 show running-config interface port-channel 77, 80, 87, 191
show feature 128, 206, 248, 249, 275, 286 show running-config interface vlan 82, 87
show hardware feature-capability 210 show running-config l2pt 316
show hsrp detail 145 show running-config pim 148
show interface 32, 43, 54, 55, 69, 71, 72, 78, 177, 179 show running-config vpc 265, 267, 275
show interface brief 54, 86 show running-config vrrp 147
show interface capabilities 87 show spanning-tree 227
show interface counters 87, 207 show spanning-tree summary 271, 273
show interface counters detailed 87, 207 show startup-config bfd 154
show interface counters errors 87, 207 show startup-config interface vlan 82
show interface eth 33, 99 show udld 45, 54
show interface ethernet 33, 41, 42, 86, 115, 116 show udld global 54
show interface ethernet errors 116 show vlan 73, 75
show interface fec 13 show vpc brief 221, 227, 250, 253, 254, 257, 259, 264, 275
show interface loopback 103, 115, 116 show vpc consistency-parameters 220, 221, 258, 275
show interface port-channel 115, 116, 181, 183, 184, 185, 206 show vpc consistency-parameters global 258
show interface status err-disabled 20, 35, 36, 37, 54 show vpc consistency-parameters interface port-channel 258, 265, 267
show interface switchport 86 show vpc orphan-ports 268
show interface transceivers 27 show vpc peer-keepalive 276
show interface trunk 86 show vpc role 260, 261, 262, 276
show interface tunnel 7, 294, 298 show vpc statistics 251, 276
show interface vlan 101, 115, 117 show vrf 111, 297
show interfaces 98 show vrrp detail 146
show interfaces tunnel 288, 291, 293 shut 5
show ip copy 28 shut vPC domain 5
show ip eigrp 142 shutdown 20, 35, 43, 180, 182, 195, 196, 198, 199, 214
show ip interface brief 116 spanning-tree port type edge 307
show ip load-sharing 205 spanning-tree pseudo-information 272
show ip nat max 352 spanning-tree vlan 271
show ip nat statistics 353 speed 185
show ip nat translations 345, 352 speed 10 185
show ip ospf 143 speed 100 185
show ip route 116 speed 1000 185
show ip route static 149 speed auto 27, 185
show ipv6 icmp interface 112 speed-group 56
show isis 145 speed-group 10000 26
show l2protocol tunnel 315 static 330
show lacp 206 switchport 27, 58, 79, 97, 176, 307, 310, 311, 313

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
IN-3
 INDEX

switchport access vlan 69, 307 tunnel path-mtu discovery 296

switchport dot1q ethertype 310 tunnel path-mtu discovery age-timer 296
switchport host 70 tunnel path-mtu discovery min-mtu 296
switchport isolated 77 tunnel source 288, 289
switchport mode 64, 69, 72 tunnel ttl 287
switchport mode dot1q-tunnel 307, 311, 313 tunnel use-vrf 288, 294
switchport mode trunk 174, 177, 253
switchport trunk 177
switchport trunk allowed vlan 72, 74, 75, 177, 253
U
switchport trunk native 177 udld 45
switchport trunk native vlan 73 udld aggressive 45
system default interface-vlan autostate 81 udld message-time 45
system default switchport 58, 85 update-source 141
system default switchport shutdown 85
system jumbomtu 39
system-mac 260 V
system-priority 261 vlan 272
vlan dot1q tag native 304
T vpc 259
vpc domain 249, 251, 254, 257, 260, 261, 262, 263, 265, 267, 271, 272
terminal dont-ask 74 vPC domain 5
timeout 332 vpc orphan-ports suspend 237, 269
track 264 vpc peer-link 253
tunnel destination 288, 289 vrf context 149
tunnel mode 287, 291, 294 vrf member 111, 297
tunnel mode gre ip 287, 293, 294 vrrp 147
tunnel mode ipip 287, 289, 291, 294 vrrp bfd 147
tunnel mode ipv6ipv6 decapsulate-any 291

Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x
IN-4