PCIe® Compliance Update

Manisha Nilange
PCI-SIG® Serial Enabling Group (SEG) Workgroup Co-chair
Intel Corporation
Disclaimer

• The information in this presentation refers to specifications still

in the development process. This presentation reflects the
current thinking of various PCI-SIG® workgroups, but all
material is subject to change before the specifications are
released.

2
Agenda

• Compliance Program Overview

• Compliance Workshop and Compliance Test Overview
• Electrical Testing
• Interoperability Testing
• Link and Transaction Layer Testing
• Lane Margining Testing
• Configuration Space Testing
• Retimer Testing

3
Compliance Program Overview
• PCIe® 5.0 Integrators List testing begun April 2022
• Tests for 2.5, 5, 8, 16, 32 GT/s maximum data rates
• The last PCIe 3.0 testing was carried out at the April 2022 workshop
• Devices must now be PCIe 4.0 or 5.0 (independent of highest data rate)
• July 2022 onwards workshops are PCIe 4.0 and 5.0 only
• PCIe 4.0 Integrators List testing for retimers started at July 2022 workshop
• PCI™, PCI-X™, PCIe 1.x retired as of Jan 1, 2013, PCIe 2.x retired Oct
20, 2017, PCIe 3.x retired starting July, 2022
• SFF-8639/PCI Express U.2™ Continues PCIe 3.0 Integrators List
• PCIe 4.0, PCIe 5.0 Compliance program under development
• PCI Express M.2™
• PCIe 4.0, PCIe 5.0 Compliance program under development

4
“PCIe 1.0, 2.0, 3.0, 4.0, 5.0???”
• What is the difference?
• These are different revisions of the specification
• They do not necessarily mean a device can support higher speeds
• The changes go beyond the speed capability and include ECNs and other updates
• We tend to use Gen 1, Gen 2, Gen 3, Gen 4, Gen 5 synonymously with
2.5GT/s, 5GT/s, 8GT/s, 16GT/s and 32GT/s but this is not strictly correct.
• A PCIe 1.0a or 1.1 device supports 2.5GT/s
• A PCIe 2.0 device must support 2.5GT/s and can support 5GT/s
• A PCIe 3.0 device must support 2.5GT/s and can support data rates up to 8GT/s
• A PCIe 4.0 device must support 2.5GT/s and can support data rates up to 16GT/s
• A PCIe 5.0 device must support 2.5GT/s and can support data rates up to 32GT/s

5
Compliance Workshop: Overview
• Usually announced 2 months in advance
• Registration closes 4 weeks prior to event
• No on-site registration!!!!
• Testing is done using the standard CEM and U.2 form factors
• Others under development
• Devices can use adapters for other form-factors
• Passing at a workshop is the only way to be listed on the PCI-SIG
Integrators List
• Pass all gold suite tests
• Pass interoperability testing
• PCI-SIG looking into enabling third party test labs for testing devices for
(N-1) Integrators List (e.g. PCIe 4.0)
• More details will be coming out in next few months

6
Current status of PCIe Generations

• PCIe 4.0 and 5.0 currently have active compliance programs

PCIe 1.0 (Compliance

Program Discontinued)
PCIe 2.0 (Compliance Program Discontinued)
PCIe 3.0 (Compliance Program Discontinued)
PCIe 4.0 (Active Compliance Program)
PCIe 5.0 (Active Compliance Program since Apr 2022)
PCIe 6.0 (Compliance Program Under Development)
2.5 GT/s 5 GT/s 8 GT/s 16 GT/s 32 GT/s 64GT/s

Rev 3.0 (2010)

Base Rev 1.0 (2002) Rev 2.0 (2006) Rev 4.0 (2017) Rev 5.0 (2019) Rev 6.0 v1.0 (2022)
Rev 3.1 (2014)

CEM Rev 1.0 (2002) Rev 2.0 (2007) Rev 3.0 (2013) Rev 4.0 (2019) Rev 5.0 (2021) Rev 6.0 v0.5 (2022)

Test Specs N/A Rev 2.0 (2008) Rev 3.0 (2013) Rev 4.0 (2018) Rev 5.0 v1.0 (2022) Rev 6.0 v0.5 (2022)

7
 The PCIe Compliance Workshop Timetable
WS108 WS109 WS110 WS111 WS112 WS114 WS115 Pre-FYI WS116 Pre-FYI WS117 FYI WS118 FYI WS119 WS120
Dec 2018 Apr 2019 Aug 2019 Oct 2019 Dec 2019 Jun 2020 Oct 2020 Feb 2021 Apr 2021 Jun 2021 July 2021 Sept 2021 Dec 2021 Jan 2022 April 2022 July 2022

PCIe 3.0/4.0 PCIe 5.0 PCIe 5.0 PCIe 3.0/4.0/5.0 PCIe 4.0/5.0
Compliance workshops Pre-FYI workshops FYI workshops Compliance Workshop Compliance Workshop

• Preliminary Workshop: This is also known as correlation workshop. Primary purpose is test specification development. These
workshops are used to ensure the Test Equipment, Test Solutions, test fixtures, test procedures and test software is ready for the
FYI workshops. It is also ensured there is vendor to vendor consistency in the reported results by testing a small set of devices
across all test equipment solutions.
• FYI Workshop: Vendors receive Pass/Fail results but no official Integrator’s List. 1-2 FYI workshops are run before official
compliance testing begins.
• Compliance Workshop: Test specification is complete and approved. Test procedures are approved. Devices are officially tested,
and passing devices are added to the Integrators List.

8
PCI-SIG Compliance Testing

• How does this fit into the layered model?

• Compliance testing is split out into sections:
• Configuration Space Application
• Using PCIECV tool from PCI-SIG®
• Link and Transaction Layer
• Transaction Layer testing Transaction
• Data Link Layer testing
• Testing Link Training Data Link
• Lane Margining Test
• Electrical Physical
• Link Equalization/De-emphasis testing Logical Sub Block

• Transmitter Signal Quality Electrical Sub Block

• Receiver Jitter Tolerance
• Note:
• BIOS Testing is no longer done for PCIe® 4.0 or later at any link speed.

9
PCI Express® Compliance Test Overview
Test name Notes
Transmitter Signal Quality
Transmitter Jitter Test Add-in Card only
Transmitter Preset
Transmitter initial Tx Equalization Add-in Card only
Electrical Transmitter Link Equalization (Response Time)
Receiver Link Equalization
PLL Bandwidth Add-in Card only
Reference Clock Jitter System only – new for 5.0 compliance
Lane Margining Tests Protocol Test (defined in PHY test spec)
Link Layer Tests Protocol Exerciser – Additional Tests for 5.0
Protocol
Transaction Layer Tests Protocol Exerciser – Additional Tests for 5.0
Configuration Space Tests (PCIeCV) Add-in Card only, Runs on PCI-SIG approved
Platform
Interoperability Test

10
Compliance Workshop: Interoperability Testing

• Systems get rooms

• Add-in Cards travel room to room
• Switches run both paths
• Retimers will run variants of both paths
• Goal: demonstrate interoperability between products
• 80% passing rate on interoperability required for eligibility for PCI-SIG
Integrators List
• Demonstrate that the link can train and operate at the highest common link
speed supported

11
Compliance Workshop: Electrical Testing
Test System Add-in Card Switch
Transmitter Signal Quality   
Transmitter Preset Test   
Transmitter Jitter Test*   
Reference Clock Test   
Receiver Link Equalization Test   
Transmitter Link Equalization   
Test
Initial Preset Test   
PLL/Loop Bandwidth Test   

*Transmitter Jitter Test requirement has been removed for the systems in PCIe 5.0 CEM compliance program.
Corresponding changes (ECR) to the CEM 5.0 specification and PCIe 5.0 PHY test specification are work in progress
in the respective workgroups
• For the products types indicated above, depending on the maximum data rate supported by the device and targeted
Integrator List (PCIe 4.0 or PCIe 5.0), applicable testing will be repeated for 8, 16 and 32 GT/s
12
Compliance Workshop: Electrical Testing

• Software and Hardware tools needed for 32 GT/s electrical testing

• SigTest Phoenix
• Clock Jitter Tool
• Transmitter Test Procedure/MOI (Specific to the test solution being used for testing)
• PLL BW Test Procedure/MOI
• TX/RX LEQ Test Procedure/MOI
• PCIe 5.0 PHY Test Specification
• CEM 5.0 fixture sets can be ordered from PCI-SIG
• Includes CLB, CBB, ISI Board
• Cable Set
• Adapters

13
Link and Transaction Layer

• What are the Link/Transaction tests?

• Link Layer Tests
• Transaction Layer Tests
• Logical PHY layer tests (Equalization Protocol)
• Logical PHY layer tests (Reserved Bits in TS)
• Lane Margining Test – Functional test
• Which conditions are being tested?
• Error handling at each layer
• Validation that the capability is implemented

14
Link and Transaction Layer

• Tests Require an Exerciser/PTC

• Tests are run on:
• Add-In Card
• Retimer
• Switch Upstream Port
• A subset of tests are run on Retimers
• Additional tests for retimers manually run (official testing for PCIe
4.0 retimers started in July 2022)
• Added Precoding Test (New for PCIe 5.0)
• Test 54-20 ECRC Test is now informational

15
Lane Margining Testing (LMT)

16
Lane Margining At Receiver
• Lane Margining Test is defined in the Electrical (PHY) Test Specification
• Lane Margining testing is FYI only for PCIe 4.0
• Tests are run, but results will not affect Integrators List
• Lane Margining tests are mandatory for PCIe 5.0
• For all Ports, Lane Margining for timing is required, while support of Lane
Margining for voltage is optional for PCIe 4.0 but required in PCIe 5.0
• Lane Margining feature at Receiver is mandatory for all Ports supporting
16.0 GT/s Data Rate or higher, including Retimers
• Lane Margining at Receiver enables system software to obtain the margin
information of a given Receiver while the Link is in L0 (Active) state
• The margin information includes both voltage and time, in either direction
from the current Receiver position

17
Lane Margining Testing (LMT)
• 3 flavors of testing
• Add In Card – No Device Driver Required
• These Add-in Cards can be tested using Link Transaction Test Suite or LMT software using a 32
GT/s platform
• Add In Card – Device Driver Required
• These Add-in Cards can be tested using LMT software using a 32 GT/s platform
• System Lane Margining Testing
• Needs LMT software installed on the system under test
• LMT Software test case in PCIe CV
• LMT Software donated by Intel

18
Lane Margining Test Tools

LMT in PCIe CV LMT donated by Intel

• Synopsys leveraged the PCIeCV 5.0 Test • Python based tool donated by Intel
Framework and created a new Test Case.
(TD_1_556)

• Both tools
• Are PCI-SIG approved tools used for PCIe 5.0 IL testing.
• Use PCI-SIG’s Hardware Access Driver. (Available for download from PCI-SIG website)
• Support Windows and Linux
• Can be used for testing Add-in Cards, Systems and Retimers

19
Lane Margining

• This test verifies that the Lane Margining feature is implemented on the device
• This test does not mandate a specific margin. It is left to the device vendor to validate
their device for margin requirements
• Test parameters like TX Preset and Channel loss are varied during the test to verify that
the device reports different margins with different channel conditions
• If a Retimer or Switch is present in the system, then the test needs to measure the
downstream facing Retimer port connected to the slot or Switch Downstream port

20
Configuration Space Verification (CV) Testing

21
Configuration Space Test Specification

22
Configuration Space Verification (CV)

• Goal of PCIeCV
• Verify that Configuration Space capability, Control and Status fields
adhere to the PCI Express Base Specification
• PCIeCV 5.0 tests all Device Types including Root Complex
• For PCIe 5.0 Integrators List, System CV testing is FYI only
• For PCIe 6.0 Integrators List, System CV testing is planned to be mandatory
• PCIeCV 5.0 runs under both Windows OS and Linux (specifically Ubuntu
18/20/22 LTS, Debian, Centos, command line only). Linux versions still
being correlated against Windows version.

23
Configuration Space Testing Using PCIeCV

• PCIeCV current revisions:

• For PCIe 4.0 – PCIeCV 4.0 v4.0.121
• For PCIe 5.0 – PCIeCV 5.0 v5.0.153
• Source code available to all PCI-SIG members with source code
sub-license agreement
• PCI-SIG members encouraged to contribute source code back for
CV development.
• PCI-SIG members encouraged to donate/loan DUTs to PCI-SIG for
CV development and regression testing (both Add-in Cards and
systems, Passing or Failing)

24
PCIeCV Tool

25
PCIeCV 4.0/5.0 Latest Status

• Known issues document exists in the members area (PCIe 5.0 CV known
issues, PCIe 4.0 CV known issues)
• U.2 tested with both CEM to Port A and Port B adapters
• See MOI/Test Procedure for details
• M.2 tested with CEM to M.2 adapter
• See MOI/Test Procedure for details

26
Approved Systems Used For PCIeCV Testing

• PCIeCV 4.0 Phase 2 Approved Motherboards

• AMD-based “x570” – ASUS, Gigabyte (several SKUs from both vendors)
• Intel-based “Ice Lake” – SuperMicro X12 family (several SKUs)

• PCIeCV 5.0 Phase 1 Approved Motherboard

• Intel-based Sapphire Rapids Reference Board

27
Configuration Space Testing with UniCV
• Currently, IL testing requires deployment of multiple test packages (PCIeCV 4.0, PCIeCV 5.0) on
distinct test platforms.
• UniCV is a new development effort intended to replace PCIeCV 4.0, PCIeCV 5.0 and all
subsequent PCIeCV releases.
• Single test package capable of validating DUT against any supported PCIe Base Specification on
single test platform.
• Permits determination of highest supported PCIe Base Specification at which DUT passes.

28
Hardware Access Driver

• PCI-SIG developed new 32-bit/64-bit Hardware Access Driver for Windows and Linux –
source code and test application available for all PCI-SIG members
• Same driver is used for LMT

29
Retimer Testing

30
Retimer Test Specification

31
PCIe 4.0 Retimer Compliance Tests
• Retimer Test Specification
• Covers testing stand-alone Retimer
• Retimer evaluation board in PCIe CEM form-factor required
• For interoperability testing (for down-stream port), Retimer vendors need to host a suite
with a system
• Retimer Electrical Testing
• For electrical testing, the upstream port is tested as Add-in Card and downstream port
is tested as a system
• If Slave Loopback is not supported by the Retimer, Add-in Card is required for
Loopback testing

32
PCIe 4.0 Retimer Compliance Tests
• Retimer Link-Transaction testing
• A subset of the Link/Transaction tests are run
• Testing reflects that Phase 2/3 EQ is directly negotiated with the Retimer
• Forwarding Mode Tests
• Logical & timing test performed when the Retimer under test is forwarding traffic between the
upstream & downstream ports
• Execution Mode Tests
• Logical & timing tests performed when the Retimer forms two separate electrical links between
the upstream port and the downstream port
• Some additional tests for retimers will be run manually
• See MOI for details

33
PCIe 4.0 Retimer Forwarding & Execution Setup

34
PCI Express 4.0 – Retimers

• Supported Topologies (PCI Express Base Specification 4.0, section 4.3):

35