Configure and troubleshoot PTP in Nexus

3000
Contents
Introduction
PTP Configuration:
Verification:
Troubleshooting:

Introduction
Precision Time Protocol (PTP) is a distributed nanosecond accuracy timing synchronization protocol for packet networks. PTP’s accuracy comes from the
hardware support for PTP in the switch and server network interface cards (NICs). PTP uses multicast group 224.0.1.129 and UDP ports 319 and 320.
Current PTP standard is IEEE 1588-2008 Precision Time Protocol (PTP) Version 2 (PTPv2)

The scalability of PTP comes from the fact that this protocol has a concept of Boundary Clock (BC), which can be multiple physical/logical hops away from
the Grand Master (GM) clock and acts as a re-calibration point for the messages containing timestamps. The Nexus 3ks are widely deployed in Data Centres
(DCs) to act as BC and provide accurate timings to the servers connected to Local Area Network (LAN). The BC and the GM need to be reachable via Layer3
(ip connectivity) only. There can be several layers of BC between the GM and the PTP end-client. There can be multiple GMs to provide redundancy and the
Nexus 3K will select the best GM via Best Master Clock (BMC) algorithm.

Contributed by: Nishad Mohiuddeen

Edited by: Kumar Sridhar

PTP Configuration:
N3k(config)# feature ptp

This command enables PTP on the switch.

N3k(config)# ptp source <ip address>

This command specifies the source IP address for the PTP packets generated by the switch.

N3k(config)# interface Ethernet slot/port

N3k(config-if)# ptp

This command enables PTP on a port. The Cisco Nexus 3548 is a boundary clock, so it has both master and slave ports. There is no
configuration difference between a master port and a slave port. They are both configured with the “ptp” option and BMC algorithm will
determine whether the port is a PTP slave or master port.

N3k(config)# clock protocol ptp

This command configures the switch to use PTP to update the system calendar. This configuration keeps the clock of the switch synchronized
with PTP. Not enabling this command won’t prevent the switch from propagating the PTP clock on its master ports. However, the time source
will be the Nexus local clock.

N3k(config)# ptp priority1 <0-255>

N3k(config)# ptp priority2 <0-255>

Configure the priority values for the local (oscillator) clock. Lower numeric value indicates higher priority.

N3k(config)# no ptp grandmaster-capable

By default Nexus 3k is “ptp grandmaster-capable”, so disable this capability to get synched with the GM.
Optional parameters under the interface (configure to match parameter with GM):

N3k(config)# interface Ethernet slot/port

N3k(config-if)# ptp delay-request minimum interval 3

N3k(config-if)# ptp announce interval 2

N3k(config-if)# ptp sync interval 0

Verification:
N3k# show ptp clock

PTP Device Type: Boundary clock

Clock Identity : 00:62:ec:ff:fe:40:05:81

Clock Domain: 0

Number of PTP ports: 2

Priority1 : 1

Priority2 : 1

Clock Quality:

Class : 248

Accuracy : 254

Offset (log variance) : 65535

Offset From Master : 0

Mean Path Delay : 0

Steps removed : 0

Local clock time:Mon Jun 5 00:00:23 2017

N3k# show ptp parent

PTP PARENT PROPERTIES

Parent Clock:

Parent Clock Identity: 00:0c:ec:ff:fe:08:12:b1

Parent Port Number: 1

Observed Parent Offset (log variance): N/A

Observed Parent Clock Phase Change Rate: N/A

Grandmaster Clock:

Grandmaster Clock Identity: 00:0c:ec:ff:fe:08:12:b1

Grandmaster Clock Quality:

Class: 6 <<<<

Accuracy: 32
 Offset (log variance): 22752

Priority1: 128

Priority2: 128

N3k# show ptp brief

PTP port status

-----------------------

Port State

------- --------------

Eth1/5 Slave <<< towards GM

Eth1/24 Master <<< towards Host

Eth1/24 Listening <<< no Valid GM or Host detected

N3k#show run | in clock (to check clock-protocol)

N3k#show ptp counters all (to check the Tx and Rx of PTP messages like Sync, Announce, Delay_Req, Delay_Resp etc.)

N3k# show ptp counters interface ethernet 1/24

PTP Packet Counters of Interface Eth1/24:

----------------------------------------------------------------

Packet Type TX RX

---------------- -------------------- --------------------

Announce 558 4479

Sync 1773 8941

FollowUp 1754 8950

Delay Request 35 0

Delay Response 0 35

PDelay Request 0 0

PDelay Res 0 0

PDelay Followup 0 0

Management 0 0

----------------------------------------------------------------

N3k#show ptp corrections (to check PTP correction timestamps)

PTP past corrections

----------------------------------------------------------------------------------------------------------------

Slave Port SUP Time Correction(ns) MeanPath Delay(ns)

---------- ----------------------------------------------- ------------------ ------------------

Eth1/24 Mon Jun 5 05:48:45 2017 171026 -51 1806

Eth1/24 Mon Jun 5 05:48:47 2017 173329 -47 1806

Eth1/24 Mon Jun 5 05:48:48 2017 174047 86 1806

Eth1/24 Mon Jun 5 05:48:49 2017 175690 -55 1794

Eth1/24 Mon Jun 5 05:48:50 2017 235577 -6 1794

Eth1/24 Mon Jun 5 05:48:51 2017 178035 -44 1794

Eth1/24 Mon Jun 5 05:48:52 2017 178804 83 1794

Eth1/24 Mon Jun 5 05:48:53 2017 180371 35 1794

Eth1/24 Mon Jun 5 05:48:54 2017 181839 -48 1794

Eth1/24 Mon Jun 5 05:48:55 2017 183667 -42 1794

Eth1/24 Mon Jun 5 05:48:56 2017 184423 -5 1794

Eth1/24 Mon Jun 5 05:48:57 2017 186030 113 1794

Eth1/24 Mon Jun 5 05:48:58 2017 186653 -48 1794

Eth1/24 Mon Jun 5 05:48:59 2017 188298 -6 1794

Eth1/24 Mon Jun 5 05:49:00 2017 189000 -88 1794

<snip>

Troubleshooting:
1. Check if you can ping the GM from the PTP (unicast connectivity).

2. Check clock protocol, which should be PTP, via

N3k#show run | in clock

3. By default N3Ks are ‘grandmaster capable’. Need to disable it via

N3k# no ptp grandmaster-capable

4. Use show ptp brief command to check if port state is ‘Listening’.

5. Then do an ethanalyzer capture.

ethanalyzer local interface inbound-hi capture-filter "udp port 320" limit-captured-frames 0 <<< Announce and Follow up

ethanalyzer local interface inbound-hi capture-filter "udp port 319" limit-captured-frames 0 <<< Sync

From the PTP GM side there would be Announce and Sync messages. The client will send Delay_Req and GM will send Delay_Resp.

If no packets are captured in ethanalyzer then it may be a priority issue. If the Nexus 3k receives PTP packets with lower priority (or clock class in case
of equal priority) it will discard the packet and will not punt to CPU.

6. The best way is to do packet captures (to get the packet between GM and BC) to see why the BC is not synchronizing with the GM. From the capture we
can verify the PTP priority sent by GM by looking at the ‘Announce’ message coming from the GM. If the priority is lower (higher numeric value) than the
configured priority in the BC, then change the PTP priority of the BC to higher numeric value.

7. If the BC (Nexus3k) is synchronized with GM but the downstream hosts/servers are not, then look for ‘Delay_Req_Messsage’ from the host/server. If not
present, then there is something wrong with PTP daemon in host/server.