#CLUS

Maximizing Network
Programmability and
Automation with
Open NX-OS
Nicolas Delecroix, Technical Marketing
Engineer, INSBU
ndelecro@cisco.com
BRKDCN-2025

#CLUS
Open NX-OS
• Open NX-OS represents the
programmability feature set of
NX-OS running on Nexus 9K.
• Every feature discussed in this
presentation is supported on every
Nexus 9K model.
• Some features are also supported on
Nexus 3K/5K/7K. This is summarized
at the end of the presentation.

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
Objectives
• Share with you the latest Open NX-OS programmability innovations.
• Share demos based on real-world use cases. We’ll use VXLAN
BGP EVPN as a common theme.
• Provide you with ways to get started in minutes.

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Agenda
• Extending NX-OS
• Guest Shell
• NX-SDK
• Automating NX-OS
• NX-API
• Ansible
• NETCONF and YANG
• YDK

• Monitoring NX-OS
• Streaming Telemetry

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 5
Cisco Webex Teams
Questions?
Use Cisco Webex Teams (formerly Cisco Spark)
to chat with the speaker after the session

How
1 Find this session in the Cisco Live Mobile App
2 Click “Join the Discussion”
3 Install Webex Teams or go directly to the team space
4 Enter messages/questions in the team space

Webex Teams will be moderated cs.co/ciscolivebot#BRKDCN-2025

by the speaker until June 18, 2018.

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
Extending NX-OS
Guest Shell
Securely Run Custom On-Box Linux Apps
$ dohost
NX-OS Guest Shell: Secure Linux Container 64
JSON Bit
CLI
Open-Source Your Custom Apps
bootflash: Apps (C, Python, Go…)

CentOS 7.0 rootfs

Network
#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
All Nexus 9K Front Panel Interfaces Exist as Linux
Network Devices
[root@guestshell ~]# ifconfig Eth1-42
Eth1-42: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 13.0.0.42 netmask 255.255.255.0 broadcast 13.0.0.255
ether 54:7f:ee:8e:27:bc txqueuelen 100 (Ethernet)
RX packets 3790 bytes 258373 (252.3 KiB)
RX errors 0 dropped 3553 overruns 0 frame 0
TX packets 772 bytes 201535 (196.8 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

Get data plane traffic in addition to control plane traffic:

monitor session 1
source interface Ethernet1/42 rx
destination interface sup-eth0
no shut
#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
 Guest Shell Real-World Customer Use Cases
• Monitoring agents distributed on • Intrusion Detection
every switch • DNSFlow agent to detect phishing
activity
• Open-source agents: OpenTSDB,
• Custom IDS agents
Ganglia, Nagios, etc.
Monitor both standard Linux • Custom control plane: Quagga
components (CPU, memory, BGP
interface counters), and NX-OS • Automatic configuration backup to
(routes, buffers,...) a private Git repository
• Custom agents: ECMP load
• tcpdump
balancing, PTP accuracy…
• Chef / Puppet agents • Use the Nexus 9K as a PXE server!
https://github.com/ndelecro/Nexus-9K-
Programmability/tree/master/Guest_Shell/PXE_Server
#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 11
 Guest Shell Import Export: Deployment at Scale
• New! Shipping from
7.0(3)I7(1) (September
2017)
• Define your custom
environment in the
Guest Shell, install
custom apps, etc…
• Then export it, and
import it on any other
switch

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
Guest Shell Export on the Development Switch
dev-switch# guestshell
[guestshell@guestshell ~]$ ls –l
total 6
-rw-rw-r-- 1 guestshell users 303 Apr 12 2018 buffer_monitoring.py
-rw-rw-r-- 1 guestshell users 946 Apr 12 2018 ptp_monitoring.py
[guestshell@guestshell ~]$ exit
logout

dev-switch#

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
Guest Shell Export on the Development Switch
dev-switch# guestshell
[guestshell@guestshell ~]$ ls –l
total 6
-rw-rw-r-- 1 guestshell users 303 Apr 12 2018 buffer_monitoring.py
-rw-rw-r-- 1 guestshell users 946 Apr 12 2018 ptp_monitoring.py
[guestshell@guestshell ~]$ exit
logout

dev-switch# guestshell export rootfs package bootflash:my_guest_shell

Access to the guest shell will be temporarily disabled while exporting it to
target destination. Are you sure you want to continue? (y/n) [n] y

2018 Apr 12 12:37:49 switch %VMAN-2-MOVE_STATE: Successfully exported virtual

service 'guestshell+' ; Destination set to bootflash:/my_guest_shell

dev-switch#

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
Guest Shell Export on the Development Switch
dev-switch# guestshell
[guestshell@guestshell ~]$ ls –l
total 6
-rw-rw-r-- 1 guestshell users 303 Apr 12 2018 buffer_monitoring.py
-rw-rw-r-- 1 guestshell users 946 Apr 12 2018 ptp_monitoring.py
[guestshell@guestshell ~]$ exit
logout

dev-switch# guestshell export rootfs package bootflash:my_guest_shell

Access to the guest shell will be temporarily disabled while exporting it to
target destination. Are you sure you want to continue? (y/n) [n] y

2018 Apr 12 12:37:49 switch %VMAN-2-MOVE_STATE: Successfully exported virtual

service 'guestshell+' ; Destination set to bootflash:/my_guest_shell

dev-switch# dir bootflash:my_guest_shell

262144000 Apr 12 12:37:48 2018 my_guest_shell

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 15
Guest Shell Import on the Production Switches
prod-switch# copy scp://dev-switch/my_guest_shell bootflash: vrf management
my_guest_shell 100% 250MB 12.5MB/s 00:20
Copy complete.

prod-switch# guestshell upgrade package bootflash:my_guest_shell

Your root filesystem will be deleted and replaced with an upgraded root
filesystem. Be sure to save your work. Are you sure you want to upgrade
the guest shell? (y/n) [n] y
Validating the provided rootfs

prod-switch#

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
Guest Shell Import on the Production Switches
prod-switch# copy scp://dev-switch/my_guest_shell bootflash: vrf management
my_guest_shell 100% 250MB 12.5MB/s 00:20
Copy complete.

prod-switch# guestshell upgrade package bootflash:my_guest_shell

Your root filesystem will be deleted and replaced with an upgraded root
filesystem. Be sure to save your work. Are you sure you want to upgrade
the guest shell? (y/n) [n] y
Validating the provided rootfs

prod-switch# guestshell
[guestshell@guestshell ~]$ ls –l
total 6
-rw-rw-r-- 1 guestshell users 303 Apr 12 2018 buffer.py
-rw-rw-r-- 1 guestshell users 946 Apr 12 2018 ptp_monitoring.py
[guestshell@guestshell ~]$

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 17
Agenda
• Extending NX-OS
• Guest Shell
• NX-SDK
• Automating NX-OS
• NX-API
• Ansible
• NETCONF and YANG
• YDK

• Monitoring NX-OS
• Streaming Telemetry

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
NX-SDK
Tighter Integration of Custom Linux Apps
with NX-OS
• NX-SDK: a simple, flexible and powerful set of APIs for
custom on-box applications to gain access to NX-OS infra
functionalities.
• Apps are still traditional Linux programs, they just use the NX-SDK
APIs and register with NX-OS.
• Apps run natively just like any other native NX-OS feature.
• Startup and management is handled by NX-OS, similar to the native
NX-OS features.

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
NX-SDK Features
• NX-OS 7.0(3)I6(1) – May • NX-OS 7.0(3)I7(3) – February
2017: 2018:
• Python and C++ support. • RIB APIs:
• Definition of custom CLIs: • Route lookup.
• Config and show commands. • Route events. Get notified upon
updates: route add, route remove,
• Callback handler with your code gets
next-hop change.
invoked when the CLI gets
executed. • Granularity for lookup and events
can be prefix, protocol, address
• Generation of custom syslogs. family, VRF.
• Streaming Telemetry support.

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
 Nexus 9K

Native NX-OS Applications

Custom Applications (L2, L3, Platform, etc.)

Linux
NX-SDK API Library
NX-OS
Much More…
Syslog, Event & Event
CLI Error history
HA Manager
DME, RIB, timers,
etc.

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
Any Linux NX-SDK Toolkit
Server Use the toolkit to write the app
Custom Application and package it offline
Code

Copy the app to the switch, and start it like a regular NX-OS feature

Nexus 9K

Native NX-OS Applications

Custom Applications (L2, L3, Platform, etc.)

Linux
NX-SDK API Library
NX-OS

Syslog, Event & Error Event Much More…

CLI HA Manager DME, RIB, timers, etc.
history
#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
NX-SDK App Benefits vs. Standalone Linux App
• For custom or existing on-box • Network-operators role:
monitoring agents: • Leverage NX-OS RBAC.
• Automated startup and • Familiar NX-OS environment.
management of the agent by NX-
OS. • Custom syslogs natively integrate
• Cleanly and more efficiently with your existing syslog
extend the agent for NX-OS monitoring systems.
monitoring by using APIs, instead
of CLI calls from the code. • Custom apps have their
configuration in the NX-OS
• Improved performance via event
running-config/startup-config:
notifications when accessing NX-
OS data based on triggers. one central repository.

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
Code: https://github.com/CiscoDevNet/NX-SDK/tree/master/examples
#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
CLI and Syslog APIs
• Gain access to NX-OS CLI
cli = sdk.getCliParser()
• Create a new custom CLI config command
cli.newConfigCmd(“threshold_cmd”,
“threshold <value>”)
cli.updateParam("<value>",
”Threshold value in Mbps",
nx_sdk_py.P_INTEGER)
• Add our command handler callback for custom CLI
myCmd = pyCmdHandler()
cliP.setCmdHandler(myCmd)

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
RIB APIs
• Gain access to NX-OS RIB
ribMgr = sdk.getRibMgr()

• Register for RIB callback handler, that will be invoked when a route event
happens
myRibCb = pyRibHandler()
ribMgr.setRibMgrHandler(myRibCb)

• Subscribe for route events

ribMgr.watchL3Route(“direct”)
ribMgr.watchL3Route(”bgp”, “65000”)
ribMgr.watchL3Route(“bgp”, “65000”, “vrf-42”, nx_sdk_py.AF_IPV4)

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
Demo: ECMP Load Monitoring with NX-SDK
 1. The user configures the link threshold via a custom CLI
10G link 2. Our app automatically detect ECMP bundles
3. If the bandwidth usage of a ECMP link member exceeds
4G flow the threshold, then display a custom NX-OS syslog

ECMP Bundle

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
 NX-SDK Visibility and Control
• Today NX-SDK is only available from the native Bash shell.
• See the memory footprint of all NX-SDK objects:

• Memory control is planned for a 2018 release.

• CPU control:
• Apps can be started with low (25% CPU), medium (50% CPU) & high (75% CPU) priorities.
Default is low.
• App priority can be changed using the API setAppPriority(nxos::prio_e prio).

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 30
#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 31
#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 32
Agenda
• Extending NX-OS
• Guest Shell
• NX-SDK
• Automating NX-OS
• NX-API
• Ansible
• NETCONF and YANG
• YDK

• Monitoring NX-OS
• Streaming Telemetry

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 33
Automating NX-OS
Telnet NX-API NETCONF RESTCONF gRPC
NETCONF NX-API SNMP
or REST Client Client Client
Client Client Client
SSH Client (YANG) (YANG) (YANG)

NETCONF SNMP NETCONF RESTCONF gRPC

Agent NGINX Server Agent Agent Agent Agent

CLI YANG Processor: YANG DME

Data Management Engine (DME)

Transaction Commit
Object Store Status: Success or Raise Fault

BGP VLAN LACP ACL QoS

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
NX-API
Automation via Telnet/SSH can be Challenging

• We typically use Expect scripts written in TCL.

• Combined with regular expressions to parse the CLI prompt.

• Those regexps can become complex.

• Those regexps might have to change if Cisco changes the

CLI output (we try not to!)
• We need to deal with CLI prompts and timeouts, too.

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
 {
"jsonrpc": "2.0",
"method": "cli",
CLI "params": {
"cmd": "show version",
Request "version": 1
Management },
Server }
"id": 1

{
"jsonrpc": "2.0",
"result": {
"body": {
"bios_cmpl_time": "03/02/2017",
JSON "bootflash_size": 7906304,
"kickstart_ver_str": "7.0(3)I7(3)",
Response "chassis_id": "Nexus 9508",
... Nexus 9K
#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 39
#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 40
NX-API Customer Use Case
• Top-3 hosting provider in Europe. • The placement of customers’
Infrastructure based on VXLAN. workload can be under any VTEP.
• Every tenant is able to use the • When a tenant is given new
whole 4K VLAN range for his/her servers for the first time, a robot
workloads. Packets egressing from uses NX-API to configure the
the server are already tagged with VTEPs:
a VLAN. Q-in-VNI is used to • Create a tenant-specific VRF and IP
preserve the customer VLAN. gateway for internet access.
• The network is fully automated • Create a tenant-specific L2VNI.
with NX-API. No manual CLI • Assign the L2VNI for every access
anymore, except for port under which the tenant
troubleshooting. workloads reside.

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 41
https://github.com/ndelecro/Nexus-9K-Programmability/tree/master/NX-API_CLI

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 42
 Get Started With Just Two Commands
Ready-to-use Docker container with pre-build environment and NX-API apps
ready to run:
[user@server ~]# docker run -it ndelecro/nexus9k-programmability:latest
Status: Downloaded newer image for docker.io/ndelecro/nexus9k-programmability:latest
root@a3d1f69d8067:~#

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
[user@server ~]# docker run -it ndelecro/nexus9k-programmability:latest
Status: Downloaded newer image for docker.io/ndelecro/nexus9k-programmability:latest
root@a3d1f69d8067:~# cd NX-API_CLI/VXLAN_BGP_EVPN/
root@a3d1f69d8067:~/NX-API_CLI/VXLAN_BGP_EVPN# ./1.Create_L2VNI.py vteps 42 42000 239.1.1.1 e1/41

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
[user@server ~]# docker run -it ndelecro/nexus9k-programmability:latest
Status: Downloaded newer image for docker.io/ndelecro/nexus9k-programmability:latest
root@a3d1f69d8067:~# cd NX-API_CLI/VXLAN_BGP_EVPN/
root@a3d1f69d8067:~/NX-API_CLI/VXLAN_BGP_EVPN# ./1.Create_L2VNI.py vteps 42 42000 239.1.1.1 e1/41
****** VTEP 93180-EX-1 ******
vlan 42
vn-segment 42000

int nve1
member vni 42000
mcast-group 239.1.1.1
suppress-arp

evpn
vni 42000 l2
rd auto
route-target import auto
route-target export auto

int e1/41
switchport access vlan 42

****** VTEP 93180-EX-2 ******

vlan 42
vn-segment 42000
...

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Preview: NX-API Rollback on Error
NX-OS 9.2(1) release (Q3CY18)

• When there’s a sequence of

commands, by default NX-API stops
upon getting the first error.
• But all the previous commands that 2000
went fine are still in the running entries
config.
total
• This can lead to inconsistent
configurations, especially for
elements that depend on limited
hardware resources.
• Example with a large ACL 

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 46
#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 47
#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 48
Agenda
• Extending NX-OS
• Guest Shell
• NX-SDK
• Automating NX-OS
• NX-API
• Ansible
• NETCONF and YANG
• YDK

• Monitoring NX-OS
• Streaming Telemetry

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 49
Ansible
 Overview
• Ansibles automates most DC assets with a
Management Server minimal learning curve.
• Both network and servers can be
Ansible Controller
managed.
Inventory Modules • Human-readable  very little scripting
skills required.
• Agent-less  easy to adopt.
Playbooks Config
• NX-OS: Ansible modules abstract the CLI.

• Advanced features:
Configure
• Variables • Events

• Conditionals • Loops
Switch or Server
#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
 • Inventory: target
Architecture systems for
automation.
Targets • Playbook: a series
of plays
Inventory (automation tasks).
• Modules:
accomplish specific
tasks in Ansible
(e.g. install
SSH packages,
Ansible configure NX-OS,
Playbook Python
Config etc.)
NX-API
• Ansible Config:
determines how
your Ansible setup
behaves (how many
Modules concurrent
connections, etc.)
#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
 Ansible Playbook = Sequence of Tasks to Execute
Example: Deploy NTP on all the Servers
Set of target devices (defined in a separate file)
[user@server ~]# cat vxlan.yml
---
- hosts: all sudo For all those target devices,
become: yes execute the tasks below
tasks:
- name: Ensure NTP is installed Arguments to
yum: name=ntp state=present the module
- name: Ensure NTP is running
service: name=ntpd state=started enabled=yes
[user@server ~]#

Ansible module that will do the actual work

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 53
Ansible Inventory
[user@server ~]# cat /etc/ansible/hosts [user@server ~]# cat
[nxos_vteps] /etc/ansible/group_vars/nxos_vteps
93180-EX-1 ---
93180-EX-2 ansible_connection: local
92160-1 nxapi_provider:
92160-2 username: admin
93180-FX-2 password: cisco
transport: nxapi
[spines]
spine-1
spine-2
Best practice: use Ansible Vault for
[older_routers] password encryption.
router-A
router-B

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 54
VXLAN BGP EVPN Automation
[user@server ~]# cat vxlan.yml
---
- name: Create L2VNI
hosts: nxos_vteps

tasks:
- name: Create VLAN and map to to VNI
nxos_vlan:
vlan_id: 2200
mapped_vni: 20200
provider: "{{ nxapi_provider }}”

- name: Add L2VNI to Overlay

nxos_vxlan_vtep_vni:
interface: nve1
vni: 20200
...

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 55
VXLAN BGP EVPN Automation
[user@server ~]# cat vxlan.yml [user@server ~]# cat /etc/ansible/hosts
--- [nxos_vteps]
- name: Create L2VNI 93180-EX-1
hosts: nxos_vteps 93180-EX-2
92160-1
tasks:
92160-2
- name: Create VLAN and map to to VNI
nxos_vlan:
93180-FX-2
vlan_id: 2200 ...
mapped_vni: 20200
provider: "{{ nxapi_provider }}” [user@server ~]# cat
/etc/ansible/group_vars/nxos_vteps
- name: Add L2VNI to Overlay ---
nxos_vxlan_vtep_vni: ansible_connection: local
interface: nve1 nxapi_provider:
vni: 20200 username: admin
... password: cisco
transport: nxapi

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 56
VXLAN BGP EVPN Automation
[user@server ~]# cat vxlan.yml
...
tasks:
- name: Create VLAN and map to to VNI
nxos_vlan: vlan 2200
vlan_id: 2200 vn-segment 20200
mapped_vni: 20200
provider: "{{ nxapi_provider }}”

- name: Add L2VNI to Overlay

nxos_vxlan_vtep_vni: interface nve1
interface: nve1 no shutdown
vni: 20200 host-reachability protocol bgp
multicast_group: 239.239.239.100 member vni 20200
suppress_arp: true suppress-arp
provider: "{{ nxapi_provider }}" mcast-group 239.239.239.100

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 57
VXLAN BGP EVPN Automation
[user@server ~]# cat vxlan.yml
...
- name: Add L2VNI to EVPN evpn
nxos_evpn_vni: vni 20200 l2
vni: 20200 rd auto
route_distinguisher: auto route-target import auto
route_target_both: auto route-target export auto
provider: "{{ nxapi_provider }}"

- name: Set the downlink to access

and assign the VLAN to it interface Ethernet1/41
nxos_l2_interface: switchport
name: Ethernet1/41 switchport access vlan 2200
mode: access
access_vlan: 2200
provider: "{{ nxapi_provider }}"

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 58
VXLAN BGP EVPN Automation
[user@server ~]# ansible-playbook vxlan_nxapi.yml
PLAY [Create L2VNI] ************************************************************

TASK [Create VLAN and map to to VNI] *******************************************

ok: [93180-FX-2]
changed: [92160-1]
ok: [92160-2]
ok: [93180-EX-1]
changed: [93180-EX-2]

TASK [Add L2VNI to Overlay] ****************************************************

changed: [92160-1]
...

PLAY RECAP *********************************************************************

92160-1 : ok=5 changed=4 unreachable=0 failed=0
92160-2 : ok=5 changed=3 unreachable=0 failed=0
93180-EX-1 : ok=5 changed=3 unreachable=0 failed=0
93180-EX-2 : ok=5 changed=4 unreachable=0 failed=0
93180-FX-2 : ok=5 changed=3 unreachable=0 failed=0

[user@server ~]# #CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 59
What’s the Cleanest Way to Handle This Error?
[user@server ~]# cat vxlan.yml Config prior to the playbook
... execution:
tasks:
- name: Create VLAN and map to to VNI
nxos_vlan: interface nve1
vlan_id: 2200 no shutdown
mapped_vni: 20200 host-reachability protocol bgp
admin_state: up member vni 20200
provider: "{{ nxapi_provider }}” suppress-arp
ingress-replication protocol bgp
- name: Add L2VNI to Overlay
nxos_vxlan_vtep_vni:
interface: nve1
vni: 20200
multicast_group: 239.239.239.100
suppress_arp: true
provider: "{{ nxapi_provider }}"

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 60
 [user@server ~]# cat rollback.yml
...

Checkpoint and Rollback! tasks:

- name: Create checkpoint
nxos_rollback:
checkpoint_file: backup.cfg
provider: "{{ nxapi_provider }}"

- name: VXLAN config

block:
- name: Create VLAN and map to the VNI
nxos_vlan:
vlan_id: 2200
mapped_vni: 20200
admin_state: up
provider: "{{ nxapi_provider }}"
- name: Add L2VNI to Overlay
nxos_vxlan_vtep_vni:
interface: nve1
vni: 20200
multicast_group: 239.239.239.100
provider: "{{ nxapi_provider }}"
rescue:
- name: Rollback to checkpoint
nxos_rollback:
rollback_to: backup.cfg
provider: "{{ nxapi_provider }}"
#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 61
[user@server ~]# ansible-playbook rollback.yml
TASK [Create checkpoint]
*************************************************************************************************
changed: [93180-FX-2]

TASK [Create VLAN and map to the VNI]

*************************************************************************************************
changed: [93180-FX-2]

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 62
[user@server ~]# ansible-playbook rollback.yml
TASK [Create checkpoint]
*************************************************************************************************
changed: [93180-FX-2]

TASK [Create VLAN and map to the VNI]

*************************************************************************************************
changed: [93180-FX-2]

TASK [Add L2VNI to Overlay]

*************************************************************************************************
fatal: [93180-FX-2]: FAILED! => {"changed": false, "clierror": "Cannot associate a multicast
group or vrf to an ingress replication vni.\n\n", "code": "400", "msg": "CLI execution error",
"output": [{"body": {}, "code": "200", "msg": "Success"}, {"body": {}, "code": "200", "msg":
"Success"}, {"body": {}, "code": "200", "msg": "Success"}, {"clierror": "Cannot associate a
multicast group or vrf to an ingress replication vni.\n\n", "code": "400", "msg": "CLI execution
error"}], "url": "http://93180-FX-2:80/ins"}

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 63
[user@server ~]# ansible-playbook rollback.yml
TASK [Create checkpoint]
*************************************************************************************************
changed: [93180-FX-2]

TASK [Create VLAN and map to the VNI]

*************************************************************************************************
changed: [93180-FX-2]

TASK [Add L2VNI to Overlay]

*************************************************************************************************
fatal: [93180-FX-2]: FAILED! => {"changed": false, "clierror": "Cannot associate a multicast
group or vrf to an ingress replication vni.\n\n", "code": "400", "msg": "CLI execution error",
"output": [{"body": {}, "code": "200", "msg": "Success"}, {"body": {}, "code": "200", "msg":
"Success"}, {"body": {}, "code": "200", "msg": "Success"}, {"clierror": "Cannot associate a
multicast group or vrf to an ingress replication vni.\n\n", "code": "400", "msg": "CLI execution
error"}], "url": "http://93180-FX-2:80/ins"}

TASK [Rollback to checkpoint]

*************************************************************************************************
changed: [93180-FX-2]

PLAY RECAP
*************************************************************************************************
93180-FX-2 : ok=5 changed=3 unreachable=0 failed=1

[user@server ~]# #CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 64
[user@server ~]# ansible-playbook rollback.yml
TASK [Create checkpoint]
*************************************************************************************************
changed: [93180-FX-2]

TASK [Create VLAN and map to the VNI]

*************************************************************************************************
changed: [93180-FX-2]

TASK [Add L2VNI to Overlay]

*************************************************************************************************
fatal: [93180-FX-2]: FAILED! => {"changed": false, "clierror": "Cannot associate a multicast
group or vrf to an ingress replication vni.\n\n", "code": "400", "msg": "CLI execution error",
"output": [{"body": {}, "code": "200", "msg": "Success"}, {"body": {}, "code": "200", "msg":
"Success"}, {"body": {}, "code": "200", "msg": "Success"}, {"clierror": "Cannot associate a
multicast group or vrf to an ingress replication vni.\n\n", "code": "400", "msg": "CLI execution
error"}], "url": "http://93180-FX-2:80/ins"}
93180-FX-2# sh run vlan
TASK [Rollback to checkpoint]
!Command: show running-config vlan
*************************************************************************************************
!Time: Mon Jun 4 13:39:42 2018
changed: [93180-FX-2]

PLAY RECAP version 7.0(3)I7(3)

vlan 1
*************************************************************************************************
93180-FX-2 : ok=5 changed=3 unreachable=0 failed=1
93180-FX-2#
[user@server ~]# #CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 65
 Apply a Patch Based on the NX-OS Version
---
- name: Patch
hosts: nxos_vteps_nxapi
vars:
my_patch: nxos.CSCve91311_n9000.rpm
my_ver: 7.0(3)I6(1)

tasks:
- name: Get the NX-OS version
nxos_facts:
gather_subset: "!interfaces"
provider: "{{ nxapi_provider }}"

- debug:
var: ansible_net_version

- name: Apply OSPF patch

nxos_smu:
pkg: "{{ my_patch }}"
provider: "{{ nxapi_provider }}"
when: ansible_net_version == "{{ my_ver }}"

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 66
 Apply a Patch Based on the NX-OS Version
--- TASK [Get the NX-OS version] ***************
- name: Patch ok: [93180-FX-2]
hosts: nxos_vteps_nxapi ok: [92160-1]
vars: ok: [93180-EX-1]
my_patch: nxos.CSCve91311_n9000.rpm
my_ver: 7.0(3)I6(1) TASK [debug] *******************************
ok: [93180-EX-1] => {
tasks: "ansible_net_version": "7.0(3)I6(1)"
- name: Get the NX-OS version }
nxos_facts: ok: [92160-1] => {
gather_subset: "!interfaces" "ansible_net_version": "7.0(3)I7(2)"
provider: "{{ nxapi_provider }}" }
ok: [93180-FX-2] => {
- debug: "ansible_net_version": "7.0(3)I7(3)"
var: ansible_net_version }

- name: Apply OSPF patch TASK [Apply patch] *************************

nxos_smu: skipping: [92160-1]
pkg: "{{ my_patch }}" skipping: [93180-FX-2]
provider: "{{ nxapi_provider }}" ok: [93180-EX-1]
when: ansible_net_version == "{{ my_ver }}"

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 67
 If the Patch Was Applied, Display the OSPF State
tasks: TASK [Apply OSPF patch] ********************
... skipping: [92160-1]
- name: Apply OSPF patch skipping: [93180-FX-2]
nxos_smu: ok: [93180-EX-1]
pkg: "{{ my_patch }}"
provider: "{{ nxapi_provider }}" RUNNING HANDLER [Get OSPF neighbors] *******
when: ansible_net_version == "{{ my_ver }}” ok: [93180-EX-1]
notify:
- Get OSPF neighbors RUNNING HANDLER [Display OSPF neighbors] ***
- Display OSPF neighbors ok: [93180-EX-1] => {
"ospf_neighbors": {
handlers: ...
- name: Get OSPF neighbors "addr": "10.0.0.2",
nxos_command: "drstate": " -",
commands: ["show ip ospf neighbors"] "intf": "Eth1/49",
provider: "{{ nxapi_provider }}" "priority": "1",
register: ospf_neighbors "rid": "100.100.100.100",
"state": "FULL",
- name: Display OSPF neighbors "uptime": "PT1M2S"
debug:
var: ospf_neighbors

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 68
 NX-OS Ansible Modules
Over 70 NX-OS Modules in Ansible 2.5

• AAA • Interfaces • Snapshot

• ACL • NTP • Static Routing

• BGP • NX-API • Upgrade

• Checkpoint / Rollback • NX-OS Facts • VLAN

• CLI • OSPF • vPC

Note: can be used with
Jinja2 templates • PIM • VRF

• HSRP / VRRP • Port-Channel / vPC • VXLAN Flood & Learn

• Patching • VXLAN BGP EVPN

• IGMP / IGMP Snooping

Open-source and officially supported by Cisco.

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
https://github.com/ndelecro/Nexus-9K-Programmability/tree/master/Ansible/2.5

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 70
 Get Started With Just Two Commands
Ready-to-use Docker container with Ansible installed and configured, and
NX-OS playbooks available:
[user@server ~]# docker run -it ndelecro/nexus9k-programmability:latest
Status: Downloaded newer image for docker.io/ndelecro/nexus9k-programmability:latest
root@a3d1f69d8067:~#

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
 Get Started With Just Two Commands
Ready-to-use Docker container with Ansible installed and configured, and
NX-OS playbooks available:
[user@server ~]# docker run -it ndelecro/nexus9k-programmability:latest
Status: Downloaded newer image for docker.io/ndelecro/nexus9k-programmability:latest
root@a3d1f69d8067:~# ansible-playbook ~/ansible/vxlan_nxapi.yml

PLAY [Create L2VNI]

*************************************************************************************************

TASK [Create VLAN and map to to VNI]

*************************************************************************************************
changed: [93180-FX-2]
changed: [93180-EX-1]

PLAY RECAP
*************************************************************************************************
93180-EX-1 : ok=1 changed=1 unreachable=0 failed=0
93180-FX-2 : ok=1 changed=1 unreachable=0 failed=0

root@a3d1f69d8067:~#
#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Agenda
• Extending NX-OS
• Guest Shell
• NX-SDK
• Automating NX-OS
• NX-API
• Ansible
• NETCONF and YANG
• YDK

• Monitoring NX-OS
• Streaming Telemetry

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 73
NETCONF and
YANG
 See Something Cool Here?
[user@server]$ python netconfclient.py ssh 93180-EX-2
Connecting to the NETCONF agent using the SSH protocol at 93180-EX-2:830.
User admin.
Using NETCONF version 1.1.
Response timeout value is 60 seconds.
Request exec count 1.
Connected to NETCONF agent. Waiting for <hello> message...

------------ Received from NETCONF agent ---Mon May 21 11:33:14 2018---------

User Access Verification
<?xml version="1.0" encoding="UTF-8"?>
<hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
<capabilities>
<capability>urn:ietf:params:netconf:base:1.0</capability>
<capability>urn:ietf:params:netconf:base:1.1</capability>
<capability>urn:ietf:params:netconf:capability:writable-running:1.0</capability>
<capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</capability>
<capability>urn:ietf:params:netconf:capability:candidate:1.0</capability>
<capability>urn:ietf:params:netconf:capability:validate:1.1</capability>
<capability>urn:ietf:params:netconf:capability:confirmed-commit:1.1</capability>
<capability>http://cisco.com/ns/yang/cisco-nx-os-device?revision=2018-02-12&amp;mod
<capability>http://openconfig.net/yang/bgp?revision=2016-06-06&amp;module=opencon
... #CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
 See Something Cool Here?
[user@server]$ python netconfclient.py ssh 93180-EX-2
Connecting to the NETCONF agent using the SSH protocol at 93180-EX-2:830.
User admin.
Using NETCONF version 1.1.
Response timeout value is 60 seconds.
Request exec count 1.
Connected to NETCONF agent. Waiting for <hello> message...

------------ Received from NETCONF agent ---Mon May 21 11:33:14 2018---------

User Access Verification
<?xml version="1.0" encoding="UTF-8"?>
<hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
<capabilities>
<capability>urn:ietf:params:netconf:base:1.0</capability>
<capability>urn:ietf:params:netconf:base:1.1</capability>
<capability>urn:ietf:params:netconf:capability:writable-running:1.0</capability>
<capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</capability>
<capability>urn:ietf:params:netconf:capability:candidate:1.0</capability>
<capability>urn:ietf:params:netconf:capability:validate:1.1</capability>
<capability>urn:ietf:params:netconf:capability:confirmed-commit:1.1</capability>
<capability>http://cisco.com/ns/yang/cisco-nx-os-device?revision=2018-02-12&amp;mod
<capability>http://openconfig.net/yang/bgp?revision=2016-06-06&amp;module=opencon
... #CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
New NETCONF Capabilities Support in 7.0(3)I6(1)
Greater Stability and Consistency

Before 7.0(3)I6(1): only the base capability is supported. Any changes get
directly applied to running config.
New capabilities since NX-OS 7.0(3)I6(1):

The candidate config temporarily holds any changes you make, without
changing the running configuration.

Validate the candidate config: the switch verifies the consistency of this
candidate config, SW and HW resources availability, etc.
#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 77
New NETCONF Capabilities Support in 7.0(3)I6(1)
Greater Stability and Consistency

• Commit: the candidate config becomes the actual running config.

• Confirmed-commit: the commit is final only when a confirm operation is
performed, before the confirm timeout expires.
• If not confirmed, it rolls back to the previous config state.
• Useful to make sure you don’t get locked out of the switch because of a
management IP or ACL change.

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 78
New NETCONF Capabilities Support in 7.0(3)I6(1)
Greater Stability and Consistency

• If an error occurs during the config processing, it will roll back to the
original state.
• “All-or-nothing” edit mode.
• Use case example: large ACL that could exceed the TCAM. If the TCAM
gets exhausted in the middle of the ACL programming, we don’t want to
end up with half of the ACL active. We would rather roll back to a clean
state.

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 79
#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 80
 A Simple YANG Model
YANG defines the basic blocks of syntax to define data in a tree hierarchy.

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 81
 A Simple YANG Model
YANG defines the basic blocks of syntax to define data in a tree hierarchy.

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 82
 A Simple YANG Model
YANG defines the basic blocks of syntax to define data in a tree hierarchy.

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 83
Who Defines YANG Models?
Model URL

IEEE https://github.com/YangModels/yang/tree/master/standard/ieee

IETF https://github.com/YangModels/yang/tree/master/standard/ietf

OpenConfig https://github.com/openconfig/public

Cisco https://github.com/YangModels/yang/tree/master/vendor/cisco

Juniper https://github.com/YangModels/yang/tree/master/vendor/juniper

More…

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 84
Source: https://pc.nanog.org/static/published/meetings/NANOG71/1535/20171004_Shaikh_Lightning_Talk_Openconfig_v1.pdf
#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 85
 NX-OS OpenConfig Support
Core Feature Set for Multi-Vendor Solution
Model Shipping NX-OS 9.2(1) – Q3CY18 End of 2018 Release – In Planning
ACL 1.0.0 1.0.0
BGP 2.1.0 (partial) 4.0.1 4.0.1
Interfaces 1.0.2 (partial) 2.0.0 2.0.0
Local Routing 1.0.0 (partial) 1.0.0 1.0.0
Network Instance (VRF) 0.8.1 0.8.1
OSPF 0.1.1 0.1.1
Platform including LC,
0.8.0 0.8.0
port, transceiver
Routing Policy 2.0.1 2.0.1 2.0.1
STP 0.2.0 0.2.0
System 0.3.0
RIB 0.2.0
VLAN 1.0.2 (partial) 1.0.2 2.0.0

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 86
 NX-OS Native YANG Support
For the Complete Feature-Set
• ACL • OSPF
• QoS • HSRP
• Segment Routing • Syslog
• VXLAN F&L and BGP EVPN • NTP
• ARP • Route Policy
• Interfaces • HSRP
• VLAN • L2 and L3 Tables
• IPv4 and IPv6 for interfaces • SNMP
• RBAC • MPLS
• Tunnels • Static Route
• VRRPv2 and v3 • Streaming Telemetry Configuration

Supported protocols for OpenConfig and Native models are NETCONF, RESTCONF, and gRPC.

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 87
How do we create YANG payloads?

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 88
#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 89
#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 90
Then send those payloads with the
usual NETCONF / RESTCONF
methods.

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 91
But Let’s Take a Step Back…
YANG CLI
<System> vlan 123
<bd-items> vn-segment 90001
<bd-items>
<BD-list>
<fabEncap>vlan-123</fabEncap>
<accEncap>vxlan-90001</accEncap>
</BD-list>
</bd-items>
</bd-items>
</System>

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 92
But Let’s Take a Step Back…
YANG CLI
<System> vlan 123
<bd-items> vn-segment 90001
<bd-items>
Interface for machines
<BD-list>
API = <fabEncap>vlan-123</fabEncap>
Application Programming Interface for humans
Interface
<accEncap>vxlan-90001</accEncap>
</BD-list>
</bd-items>
</bd-items>
</System>

It’s not efficient to have a human manually deal with YANG payloads!
#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 93
 Ansible Can Help
vxlan.yml playbook
---
- name: VXLAN NETCONF
hosts: netconf_vteps

tasks:
- name: VXLAN
netconf_config:
datastore: running
src: vxlan.xml

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
 Ansible Can Help
vxlan.yml playbook vxlan.xml payload
--- <config>
- name: VXLAN NETCONF <System xmlns="http://cisco.com/ns/...">
hosts: netconf_vteps <bd-items>
<bd-items>
tasks: <BD-list>
- name: VXLAN <fabEncap>vlan-123</fabEncap>
netconf_config: <accEncap>vxlan-90001</accEncap>
datastore: running <name>ansible</name>
src: vxlan.xml </BD-list>
</bd-items>
</bd-items>
</System>
</config>

Useful in heterogenous environments, where Ansible modules don’t exist for

all device types. #CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
[user@server ~]# ansible-playbook -vvv vxlan.yml
PLAYBOOK: vxlan.yml *******************************************************************
1 plays in vxlan.yml

PLAY [VXLAN NETCONF] ******************************************************************

TASK [VXLAN] **************************************************************************

changed: [93180-EX-2] => {
"changed": true,
"invocation": {
"module_args": {
"datastore": "running",
"host": "93180-EX-2",
"hostkey_verify": true,
"password": null,
"port": 830,
"src": "<config>\n <System xmlns=\"http://cisco.com/ns/yang/cisco-nx-os-device\">\n <bd-
items>\n <bd-items>\n <BD-list>\n <fabEncap>vlan-123</fabEncap>\n\t
<accEncap>vxlan-90001</accEncap>\n <name>ansible</name>\n </BD-list>\n </bd-
items>\n </bd-items>\n </System>\n</config>\n",
"username": null,
"xml": null
}
},
...
PLAY RECAP ****************************************************************************
93180-EX-2 : ok=1 changed=1 unreachable=0 failed=0
#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
[user@server ~]#
[user@server ~]# ansible-playbook -vvv vxlan.yml
PLAYBOOK: vxlan.yml *******************************************************************
1 plays in vxlan.yml

PLAY [VXLAN NETCONF] ******************************************************************

TASK [VXLAN] **************************************************************************

changed: [93180-EX-2] => {
"changed": true,
"invocation": {
"module_args": {
"datastore": "running",
"host": "93180-EX-2", 93180-EX-2# sh run vlan
"hostkey_verify": true,
"password": null,
...
"port": 830, vlan 123
name ansible
"src": "<config>\n <System xmlns=\"http://cisco.com/ns/yang/cisco-nx-os-device\">\n <bd-
items>\n <bd-items>\n <BD-list>\n vn-segment 90001
<fabEncap>vlan-123</fabEncap>\n\t
<accEncap>vxlan-90001</accEncap>\n <name>ansible</name>\n </BD-list>\n </bd-
items>\n </bd-items>\n </System>\n</config>\n",
"username": null,
"xml": null
}
},
...
PLAY RECAP ****************************************************************************
93180-EX-2 : ok=1 changed=1 unreachable=0 failed=0
#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
[user@server ~]#
 Ansible as a NETCONF Engine
vxlan_generic.yml playbook vxlan.j2 Jinja2 template
--- <config>
- name: VXLAN NETCONF GENERIC <System xmlns="http://cisco.com/ns/...">
hosts: netconf_vteps <bd-items>
<bd-items>
tasks: <BD-list>
- name: TEMPLATE
<fabEncap>vlan-{{vlan_id}}</fabEncap>
template:
<accEncap>vxlan-{{vxlan_id}}</accEncap>
src: vxlan.j2
dest: vxlan.config <name>{{vlan_name}}</name>
vars: </BD-list>
vlan_id: 42 </bd-items>
vxlan_id: 90042 </bd-items>
vlan_name: ansible jinja </System>
</config>
- name: VXLAN
netconf_config:
datastore: running
src: vxlan.config
#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
[user@server ~]# ansible-playbook -vvv vxlan_generic.yml

TASK [TEMPLATE] *************************************************************************************

changed: [93180-EX-2] => {
"changed": true,
"checksum": "344977b9e66258b703e7a1c02f35a5dc68a253a4",
"dest": "./vxlan.config",

root@c76f7dd9e1cf:~/ansible# cat vxlan.config

<config>
<System xmlns="http://cisco.com/ns/yang/cisco-nx-os-device">
<bd-items>
<bd-items>
<BD-list>
<fabEncap>vlan-42</fabEncap>
<accEncap>vxlan-90042</accEncap>
<name>ansible jinja</name>
</BD-list>
</bd-items>
</bd-items>
</System>
</config>
root@c76f7dd9e1cf:~/ansible#
#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
[user@server ~]# ansible-playbook -vvv vxlan_generic.yml

TASK [TEMPLATE] *************************************************************************************

changed: [93180-EX-2] => {
"changed": true,
"checksum": "344977b9e66258b703e7a1c02f35a5dc68a253a4",
"dest": "./vxlan.config",
...

TASK [VXLAN] ****************************************************************************************

changed: [93180-EX-2] => {
"changed": true,
"invocation": {
"module_args": {
"allow_agent": true,
"datastore": "running",
"host": "93180-EX-2",
...
"src": "<config>\n <System xmlns=\"http://cisco.com/ns/yang/cisco-nx-os-device\">\n
<bd-items>\n <bd-items>\n <BD-list>\n <fabEncap>vlan-42</fabEncap>\n\t
<accEncap>vxlan-90042</accEncap>\n <name>ansible jinja</name>\n </BD-list>\n
</bd-items>\n </bd-items>\n </System>\n</config>\n",
"username": null,
"xml": null
}
...
PLAY RECAP ******************************************************************************************
93180-EX-2 : ok=3 changed=2 unreachable=0 failed=0
[user@server ~]# #CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
[user@server ~]# ansible-playbook -vvv vxlan_generic.yml

TASK [TEMPLATE] *************************************************************************************

changed: [93180-EX-2] => {
"changed": true,
"checksum": "344977b9e66258b703e7a1c02f35a5dc68a253a4",
"dest": "./vxlan.config",
...

TASK [VXLAN] ****************************************************************************************

changed: [93180-EX-2] => {
"changed": true,
"invocation": {
"module_args": {
"allow_agent": true,
"datastore": "running",
"host": "93180-EX-2",
...
"src": "<config>\n <System xmlns=\"http://cisco.com/ns/yang/cisco-nx-os-device\">\n
<bd-items>\n <bd-items>\n <BD-list>\n <fabEncap>vlan-42</fabEncap>\n\t
<accEncap>vxlan-90042</accEncap>\n <name>ansible jinja</name>\n </BD-list>\n
</bd-items>\n </bd-items>\n </System>\n</config>\n",
"username": null,
"xml": null
}
...
PLAY RECAP ******************************************************************************************
The playbooks are available
93180-EX-2 in the docker
: ok=3 image
changed=2 ndelecro/nexus9k-programmability.
unreachable=0
#CLUS
failed=0
root@c76f7dd9e1cf:~/ansible# © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Agenda
• Extending NX-OS
• Guest Shell
• NX-SDK
• Automating NX-OS
• NX-API
• Ansible
• NETCONF and YANG
• YDK

• Monitoring NX-OS
• Streaming Telemetry

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 102
YDK
Beyond Payloads: Code APIs
• It can be complicated and time-
consuming to create YANG XML
payloads manually.
• It also doesn’t fit very well with
automation via scripting. We would
have to do text manipulation to insert
the dynamic data into the payload
(even though Jinja can help...)
• Let’s generate a library of classes
(code API) that follows exactly the
YANG model definition.
• Let’s also abstract the communication
to the device.
#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
YANG Development Kit

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 105
Telnet NX-API
NETCONF NX-API SNMP
or
Client Client
REST
Client
Client YDK APIs
SSH

NETCONF SNMP NETCONF RESTCONF gRPC

Agent NGINX Server Agent Agent Agent Agent

CLI YANG Processor: YANG DME

Data Management Engine (DME)

Transaction Commit
Object Store Status: Success or Raise Fault

BGP VLAN LACP ACL QoS

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Model-Driven Programmability Stack Model-Driven
Configuration

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 107
YDK Pre-Generated APIs Support
• Languages supported for all APIs below:
• Python (ydk-py)
• C++ (ydk-cpp)
• Go (ydk-go)

• APIs have been pre-generated for the

following YANG models:
• IETF
• OpenConfig
• IOS-XR
• IOS-XE
• NX-OS (in progress)

• Anyone can generate APIs via ydk-gen from

any valid YANG models.

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 108
Demo: VXLAN BGP EVPN Automation on
NX-OS with YDK
YDK “Hello World” Using OpenConfig
What happens when
crud.create() is invoked?
1. Validate that 65000 is a
valid AS number.
2. Convert the object to XML.
3. Build the netconf RPC for
an "edit-config”, and plug
the XML payload inside.
4. Send it to the switch.
5. Wait for the response from
the switch.
6. Process return code from
the switch: either end here,
or rollback.

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 110
YDK is the Ultimate Goal in Model-Driven Programmability

• The developer relies purely on APIs and this is independent of the

encoding and of the transport.
• There is no manual NETCONF transport manipulation. RPC, edit-
config, error handling… are all automated.
• There is no manual YANG XML payload manipulation.
• The developer focus is on the YDK object model and the application
logic.
• YDK APIs have been pre-generated for OpenConfig, IOS-XR, IOS-
XE, NX-OS models  key framework for network-as-code in
heterogenous multi-vendor environments.

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 111
 Comparison of YANG Frameworks
100
90
80
70
60
50
40
30
20
10
0
Automated Data model Transport NETCONF Error handling Scalability Coding skills
payload validation abstraction options required
generation

Manual YANG + Python ncclient Ansible netconf_config YDK

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 112
Agenda
• Extending NX-OS
• Guest Shell
• NX-SDK
• Automating NX-OS
• NX-API
• Ansible
• YANG and OpenConfig
• YDK

• Monitoring NX-OS
• Streaming Telemetry

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 113
Monitoring NX-OS with
Streaming Telemetry
Software Streaming Telemetry

Push, not pull Performance

Analytics-Ready Consumption

Data-Model Driven Granularity

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
 Telemetry Receiver

HTTP gRPC Nexus 9K

JSON Encoder GPB Encoder

Policy
Telemetry Engine Config

CLI Output Object Store

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 116
Telemetry Sources
CLI JSON Output as a Source

Only periodic, cannot be event-based.

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 118
Object Store as a Source
https://N9K_MGMT_IP/visore.html

System

Physical BGP ARP VRF

eth1/1, BGP router ARP VRF

eth1/2… ID, peers… entries instances

• Interval-based or event-based.
• Precise trigger definition.

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 119
 Stream Interface Counters

interface Ethernet1/41
switchport
switchport access vlan 42
no shutdown

telemetry
sensor-group 1
path sys/intf/phys-[eth1/41]-dbgIfIn depth 0
#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
 Stream ACL Counters Above a Given Threshold

ip access-list CLUS
statistics per-entry
10 permit ip 192.168.200.0/24 any

telemetry
sensor-group 1
path sys/acl/ipv4/name-CLUS/seq-10 depth 0 filter-condition gt(ipv4aclACE.packets,”10000”)
#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
 Stream a VLAN When It Goes Down

vlan 2202
name tenant-20000_Backup
vn-segment 20202

telemetry
sensor-group 1
path sys/bd/bd-[vlan-2202] depth 0 filter-condition and(updated(l2BD.operSt),eq(L2BD.operSt,”down”)

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Preview: Streaming Telemetry of YANG Models
NX-OS 9.2(1) release (Q3CY18)

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Collector Output

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Streaming Telemetry At Scale
Spine Spine Spine Spine

Leaf Leaf Leaf Leaf Leaf Leaf Leaf

• VXLAN EVPN leaf/spine topology.

• 2000 L2VNIs, 900 L3VNIs, 900
VRFs, 2900 VLANs, 250 NVE
Peers, 128 Multicast groups, 100K • Config equivalent to 22K specific
BGP EVPN routes. object paths, total data 14MB.

• GPB/gRPC with receiver on • Collection time + streaming time =

bare-metal server. 75 seconds.

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 125
Telemetry Collectors
Python Telemetry Collector
Receives JSON over HTTP

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
 Python Telemetry Collector
Receives JSON over HTTP

https://github.com/ndelecro/Nexus-9K-Programmability/blob/master/Streaming_Telemetry/HTTP_Transport/
#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
 ELK Telemetry Collector

http://docker.io/dockercisco/telemetryreceiver and http://docker.io/dockercisco/elklat

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 129
Preview: DCNM Telemetry Collector
DCNM 11 Release (Q3CY18)

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
 NX-SDK Integration with Streaming Telemetry
NX-OS 7.0(3)I7(3)
• In your NX-SDK app, define a custom “show” command that displays your data in a
JSON output.
• Configure the streaming telemetry sensor-group to call this CLI.

telemetry
destination-group 1
ip address 10.60.0.96 port 5000 protocol HTTP encoding JSON
sensor-group 1
path “show my_nx-sdk_app json” depth 0
subscription 1
dst-grp 1
snsr-grp 1 sample-interval 10000

• In a future release, NX-SDK will provide an API to inject streaming telemetry events
directly from the code. This will bring event-based telemetry support for NX-SDK apps.
#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Demo: NX-SDK Integration with Streaming
Telemetry
 1. User configures link threshold via new custom CLI
10G link 2. App automatically detect ECMP bundles
3. If the bandwidth usage of a ECMP link member exceeds
4G flow the threshold:
• Display a NX-OS syslog
• Create a streaming telemetry event

ECMP Bundle

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
 1. User configures link threshold via new custom CLI
10G link 2. App automatically detect ECMP bundles
3. If the bandwidth usage of a ECMP link member exceeds
4G flow the threshold:
• Display a NX-OS syslog
• Create a streaming telemetry event

Code and installation steps:

https://github.com/ndelecro/Nexus-9K-Programmability/tree/master/NX-SDK

ECMP Bundle

#CLUS © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public
Feature-Set Availability Across Nexus Switching Family
N3K N5K/N6K/7K N9K
Guest Shell Yes No Yes
NX-SDK Yes No Yes
NX-API CLI Yes Yes Yes
YANG Yes (*) No Yes
YDK Yes (*) No Yes
Ansible Yes Yes Yes
Streaming
Yes (*) No Yes
Telemetry
(*) On models equipped with 8GB+ of memory.

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 138
 + Goodies

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 139
 Resources
• N9K NX-OSv: https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/7-x/nx-
osv/configuration/guide/b_NX-OSv_9000/b_NX-OSv_chapter_01.html
https://software.cisco.com/download/release.html?mdfid=286312239&softwareid=282088129&rel
ease=7.0(3)I7(2)
• N9K NX-OSv in VIRL: https://learningnetwork.cisco.com/docs/DOC-31788
• DevNet sandbox: https://developer.cisco.com/site/sandbox/

• OpenConfig payloads and scripts: https://github.com/ndelecro/Nexus-9K-

Programmability/tree/master/OpenConfig
• NX-YANG payloads and scripts: https://github.com/ndelecro/Nexus-9K-
Programmability/tree/master/YANG/RESTCONF
• Ansible VXLAN BGP EVPN
• Playbooks: https://github.com/ndelecro/Nexus-9K-Programmability/tree/master/Ansible/VXLAN_BGP_EVPN
• Demo: https://youtu.be/E4Y_CQ_XQRk

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 140
Resources (continued)
• YDK: https://developer.cisco.com/site/ydk/
• Streaming Telemetry
• Sample HTTP receiver: https://github.com/ndelecro/Nexus-9K-
Programmability/blob/master/Streaming_Telemetry/HTTP_Transport/http_receiver
.py
• Sample GPB receiver: http://docker.io/dockercisco/telemetryreceiver
• Demo of manual receiver: https://www.youtu.be/CDK0VdLArnM
• Demo of Kibana receiver: https://www.youtu.be/s2_Xq0hEL7c
• Deployment guide for VXLAN EVPN: https://developer.cisco.com/site/nx-
os/docs/guides/telemetryvxlan/Telemetry-Deployment-VXLAN-EVPN.pdf

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 141
Complete your online session evaluation

Give us your feedback to be entered

into a Daily Survey Drawing.
Complete your session surveys through
the Cisco Live mobile app or on
www.CiscoLive.com/us.
Don’t forget: Cisco Live sessions will be available for viewing
on demand after the event at www.CiscoLive.com/Online.

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 142
Continue
your Demos in
the Cisco
Walk-in
self-paced
Meet the
engineer
Related
sessions
education campus labs 1:1
meetings

#CLUS BRKDCN-2025 © 2018 Cisco and/or its affiliates. All rights reserved. Cisco Public 143
Thank you

#CLUS
#CLUS