Test & Measurement Planning for Performance

Ethernet Alliance

TEF 2021: The Road Ahead

Test & Measurement – Planning for Performance

2

The Ethernet AllianceGlobal Community of End Users, System Vendors, Component Suppliers & Academia

Our Mission

• To promote industry awareness, acceptance and

advancement of technology and products based on, or

dependent upon, both existing and emerging IEEE 802

Ethernet standards and their management.

• To accelerate industry adoption and remove barriers to

market entry by providing a cohesive, market-responsive,

industry voice.

• Provide resources to establish and demonstrate multi-

vendor interoperability.

Ethernet Alliance Strategy

3

● Facilitate interoperability testing

○ Industry Plug Fests supporting

member and technology initiatives

● Interoperability Assurance

○ PoE Certification Program

● Collaborative Interaction with

other Industry Organizations

○ Multiple SIGs, Applications and MSAs

○ Industry Consensus Building

● Global Outreach

○ Worldwide Membership

●Thought Leadership

○EA Hosts Technology Exploration

Forums (TEFs)

○Technology and Standards

incubation

●Promotion of Ethernet

○ Industry Analysts

○ Education

○ Marketing

■Trade shows & Panel Presentations

■White Papers, Blogs & Social Media

Expanding the Ethernet Ecosystem, Supporting Ethernet Development

TEST & MEASUREMENT

PLANNING FOR PERFORMANCEModerator:

David Rodgers, EA Events Chair and Teledyne LeCroy

Panelists:

John Calvin – Keysight Technologies,

Steve Rumsby – Spirent Communications

Francois Robitaille – EXFO

Testing on the Road Ahead

Basic Premise:Increased signaling speeds, and standards and interoperability conformance, and support combined with growing port counts, higher port densities, time/mission critical applications;

These are but a few of the market drivers impacting the equipment manufacturers, datacenter managers, and ultimately the Test & Measurement community.

The impact on hardware designs however requires specification and protocol awareness beginning with initial

design, through validation, and in the field after deployment.

2/2/20215

Universal T&M Considerations

Common to all stages of Development, Deployment, and Support

• What issue(s) are we trying to understand and correct?

• When and How does the issue manifest?

• Is the issue reproducible?

• Can root cause be definitively determined?

• What are the curative measures?

• Can you test the ‘fix’?

2/2/20216

Ethernet Test & Measurement Today

• The tool chest has remained stable and remarkably

unchanged up to the transition into PAM4 signaling

• Minor Imperfections, once ‘accepted’ as normal and

unremarkable are no longer “minor”

• Scopes, BERTs, PERTs, Traffic Generators and home-

grown utilities

• Trial and Error

• aka Plug and Pray

Bottom line: It’s been a “Roll your Own” Mentality

2/2/20217

Navigating the #NextEthernetEra

Ubiquitous deployment requires vendor interoperability

Mission Critical applications demand reliability and consistency

• Minor Imperfections no longer ‘accepted’ as normal and unremarkable

Integrating legacy and new Ethernet technologies creating new challenges

Exponential Storage Growth and Content Delivery demands require 5-9s of up time.

It’s imperative to know what’s “on the wire”• Testing no longer “ends” at the connector

2/2/20218

TEST & MEASUREMENT

PLANNING FOR PERFORMANCE

Design For Test with:

David. J Rodgers, Teledyne LeCroy

“Get this Panel Started…”

Design for Test

Design for Test

o From inception to EOL, consider the need for and access of test tools in product designs

Ethernet Standards Evolving at Breakneck Pace

• 25GbE to 100GbE, now 50GbE to 200&400GbE “and Beyond!”

• Soon, 100GbE to 800GbE to ???

• Automotive, Industrial, Commercial

Ethernet Fabrics Fueling Storage Explosion

o Speed and Optimization meeting QOS Expectations

• iSCSI, FCoE, NVMe-oF, NFS, IBXoE, FCIP, iSER, iWARP, RoCE, Routable RoCE (v2), and so on, and so on…

Standards beget Interoperability?

o Interpretation and implementation differences abound

10

This Photo s

licensed under

CC BY-NC-ND

https://eyeonasia.wordpress.com/2010/06/02/sci-fi-revoltech-011-toy-story-buzz-lightyear-action-figure/

https://creativecommons.org/licenses/by-nc-nd/3.0/

“You Can’t Test What You Can’t Measure!”

11

• Hardware Test tools must change to keep up with market and

technology demands

• New Speeds adding new complexities

– NRZ vs PAM4 signaling

• There is a “protocol” to the Phy

– Auto-negotiation

– Link Training

– FEC

• No two vendors implementations are identical

Test and Measurement – The Bell Weather

T&M vendors; We’re On the Leading Edge!

• Partners in Pain of “being first”

• Traditional – Signal Integrity Tools

• New to the Scene – Protocol Specific Tools

Purpose Built Protocol Tools!

o Compliment to, not replacement for Traditional Tools

• Optimized for the Fabric/Device under test

o Becoming essential in all HW/SW test environments

The Goal; Testing must be “standardized”

and repeatable

o Interop PlugFests, 3rd party testing services

12

Key Interoperability Challenges

13

• Identifying Participants

– Characterizing Functionality of All Ecosystem Players

• Determining Root Cause

– Eliminate “finger pointing”

• Crafting the Solution

• Remediation Validation

– Test the fix

• Timely Resolution!

Our Test & Measurement Panel

14

Steve Rumsby:Where the Frame Hits

the Road

John Calvin:Sub-Atomic Phy

Guy

David J Rodgers: Wrangling the

Talent!

Francois Robitaille: Shining a light on Optics

TEST & MEASUREMENT



John Calvin, Keysight Technologies

“Validating Methods for Emerging 106Gbps

Electrical and Optical Specifications relating

to IEEE P802.3cu/P802.3ck”

The state of 100G and early Phy validation

www.ethernetalliance.org 16

IEEE 802.3ck (100G, 200G, 400Gb/s

Electrical) entering into a state of

“Technical Completion” by mid March 2021

In a pivotal point where, early silicon is being

evaluated and comparisons against simulation

are getting a lot of scrutiny.

IEEE 802.3cu (100Gb,400Gb/s SMF Optical) in it’s D3.2 state

Measurement advances and challenges reasonably well managed

through contemporary optical test solutions and existing TDECQ

centric test techniques.

17

100GEL Test Equipment EQ Considerations

Tx Emphasis: Five-tap transversal filter (FIR): (85.8.3.3.6) Transfer function between the transmit function and TP2 Tx

coefficients are calculated based on pulse response. (This is key given coefficients limits in Rx section, and no FIR in the RX

system. Its Important to set Tx first as part of the overall link analysis)

Rx EQ: What’s new in 100G system is the incorporation of a 4 tap Decision Feedback Equalizer (DFE) into the RX block

behind the three pole, 2 gain-stage CTLE. As the DFE is non-linear it makes for some interesting optimization terms and as

multiple “minima” can be observed , exhaustive CTLE + DFE solution space optimization in test instrumentation is needed.

DFE limits are constrained to spec limits underscoring the importance of Tx FIR configuration ahead of time. 4 Tap Rx DFE

optimization is performed inline with Mueller-Muller phase alignment from the pulse response.

6.8dB 6.8dB14dB

36dB bump2bump

4.1dB 4.1dBi.e.: 400GBASE-KR4

FEC Analysis at 100G

Measurement technology and it’s limits


The 802.3ck task force has been steadily advancing the 100Gb/s spec

• The heavy reliance on simulation tools and the process of transitioning into physical layer validation has led to some

interesting discoveries that are key to measurement accuracy and repeatability.

IEEE 802.3bs/cd (IEEE 802.3-2018)

• Baud Rate: 26.5625 GBd

• EOJ Max: 19 mUI = 715 fs

IEEE 802.3ck Draft 1.3



# of mUI is the same, but absolute time is reduced by 50%.

IEEE 802.3ck Draft 1.4



There has been some spec relief as well as measurement

methodology changes in the last 1.4 Draft to recognize and work

around a number of practical issues.


o Contemporary lab instrumentation has an EOJ “measurement floor” of ~9mUI (170fs).

o Spec limits of working 100G silicon at 19mUI (358fs) have a jitter floor nominally 2X higher than the highest precision

instrumentation available, which when measurement repeatability is taken into account leaves zero margin against

the 19mUI specs.

o Contributing factors in this are pattern sub-harmonics and PLL behavior but the net stack-up of this a very tight set of

specifications at 100G, with work arounds in test pattern selection and relaxed spec limits.

Actual DUT EOJ

• EOJ Mean: 12.7mUI (240fs)

EOJ

# o

f M

ea

sure

me

nts

• EOJ Mean: 8.75mUI (170fs)

# o

f M

ea

sure

me

nts

EOJ

Precision Instrumentation EOJ “floor”

802.3ck /OIF-CEI 4.1/5.0

Electrical test flow• Reference: IEEE P802.3ck/OIF-CEI

5.0 400 GEL Interfaces Task Force

working group

• It’s critical to start this Rx equalization

cycle with a properly configured set of

Tx coefficients. If this step is skipped

the DFE will go to it’s spec limits and Rx

optimization will not converge.

• The 100GEL Task Forces have

balanced the pros and cons of

equalizers from a power, noise and

cost sensitivity standpoint.

• A DFE’s unique ability to improve SNR

and its effective use in reflection

mitigation makes it a key addition to

the 100GEL projects link performance.

Reference IEEE: 802.3ck Chip-to-Module TP1a/TP4

Compliance Test Measurement Methodology

(li_3ck_02_1119)

• Equalization: Introduction of a non-linear equalizer has made

interpreting measurement data challenging.

• Limits to equalization: Coefficient space restrictions and careful

adherence to methodology does not offer the end all in optimized

signals. It’s not supposed to. The instrumentation and tool

providers hands are intentionally tied here.

• Chasing the physical origins of FEC failures: A new field of

measurement science and is increasingly a need with early

characterization of 100GEL systems.

• Instrument precision, bandwidth and noise performance:

Particularly relevant to successful layer 1 characterization.

100GEL Test and Interoperability Summary

TEST & MEASUREMENT



Steve Rumsby, Spirent Communications

“Recommended Design Practices for the

Next Generation Ethernet Rate”

24Spirent Promise Assured 24Proprietary and Confidential

Ethernet Alliance TEF 2021

Testing Considerations on the Path to 800G

By Steve Rumsby

Spirent Communications

Proprietary and Confidential 25Spirent Promise Assured 25

Testing Considerations on the Path to 800GIntroduction

• Design practices that worked for current generation Ethernet must be extended and advanced to launch high quality next-gen Ethernet solutions

• 802.3ck doubles the underlying Physical Layer Electrical Lane rate that introduces several challenges that must be addressed during product design and testing.

• Let’s review best practices and key considerations for enabling successful 800G deployments.


Key Testing Considerations on the Path to 800G

Layer 2-7 is important and must be tested atop a robust Layer 1 Physical Layer interface

• Why is testing Layer 2-7 important?

‒ Testing Layer 2-7 is necessary to ensure that your products or network can deliver the expected Quality

of Service (QoS). Frame loss and high latency will negatively impact your customers.

‒ e.g., Next-generation networking relies on low latency, robust, reliable networking

• Why is it important to ensure robust Layer 1 first?

‒ A robust Layer 1 layer is critical because it is the foundation that all upper layers rely on

‒ Uncorrected bit errors at Layer 1 will turn into packet loss or protocol errors at the higher layers, which

can cause increased latency or data loss depending on your protocol. In any case, it will decrease the

throughput and reliability of your network.

Building on a Solid Foundation


Technological Challenges of 800G

What are the key technological challenges on the path to

wide-scale, robust, reliable 800G deployments?

• 800G leverages 802.3ck's 100Gbps Electrical Lanes (GEL), that is DOUBLE the Symbol rate of 400G's 50GEL technology

• Electrical Lane Symbol rate doubling doubles spectral content & halves symbol time

• Electrical design is more challenging than ever (higher signal spectral content results in design aspects impacting performance in new ways),

e.g. PCB stack-up and small feature impact

• Halving symbol time puts an even higher burden on clock and power supply sub-system design, analysis, and measurement

• Initial deployments of 800G leverage current 400G's RS 544/514 FEC

28Spirent Promise Assured

How to address & manage technological challenges

in the design?

• Switches, gearboxes, re-timers, and connectors are all

critical and high profile, BUT don’t forget about other

aspects. The electrical lane signal chain is only as

strong as its weakest link.

• C2M (Channel to Module) Channel Analysis via 3D

simulation tools

• Consider using RF design methodologies, both in design

analysis & verification

The Value of TestingEnabling Successful Deployments

29Spirent Promise Assured

How to address & manage technological challenges

during 800G interoperability testing?

• Ensure access to all key Physical Layer statistics, including:

- Receiver Signal to Noise Ratio to characterize lane health

- FEC Statistics to predict likelihood of uncorrectable errors

- PAM4 receive histogram

• Interpret the data

- Consider all PHY Stats available, apply expert judgment to them leveraging off retrospective experiences

- e.g., leverage receiver statistics to predict problems before they happen

• Collaborate to make the best use of the data

- Consider interoperability testing as opportunity for partnership & collaboration

• After interoperability testing, consider stressing link via adversarial testing

- e.g., symbol error insertion to validate FEC robustness

Overcoming Interoperability Challenges


Testing Considerations

How to address & manage technological challenges during 800G Layer 2-7 testing?

• Continue to use Layer 1 statistics to predict if you are going to receive uncorrected code words

- e.g., if the Symbol Error Histogram shows a high number of symbol errors per codeword, any minor event

could cause uncorrectable code words.

• Receive Signal to Noise Ratio (SNR) can also be used to predict average error rates

• System Events such as powering up optics in other ports or traffic changes through the switching chip seem

minor, but they can significantly impact L1 error rate (may cause dips in supply voltage or spikes in temperature).

Capturing & tracking Layer 1 error rate during these events will help ensure a stable system.

Why is this important?

• Time, temperature, part-to-part variations can cause drifting in Electrical Lane performance, which may impact

performance

Layer 2-7 is important and must be tested atop a robust Layer 1 Physical Layer interface


Key TakeawaysTesting Considerations on the Path to 800G

1. Achieving consistent and reliable 802.3ck Electrical Lane technology makes the Layer 1

Physical layer more challenging than ever

2. Electrical design analysis & measurement methods must evolve to meet these new challenges

3. A robust Layer 1 enables critical Layer 2-7 testing

4. After establishing a robust Layer 1, continue to monitor Layer 1 statistics to predict issues that

could impact Layer 2-7 performance

Spirent® Communications, Inc. and its related company names, branding, product names and logos referenced herein,

and more specifically “Spirent” are either registered trademarks or pending registration within relevant national laws.

Spirent® Communications, Inc. and its related company names, branding, product names and logos referenced herein,

and more specifically “Spirent” are either registered trademarks or pending registration within relevant national laws.

TEST & MEASUREMENT



Francois Robitaille, EXFO

“Full Compliance Validation of Next-Gen

Transceivers”

François RobitailleDirector, NEMs

PLANNING FOR PERFORMANCE:Challenges and best practices ahead for Ethernet test and measurement

Optical

transceivers

today

© 2020 EXFO Inc. 20200503 35

NZR

PAM4

CFP

QSFP

OSFP

SFP

Formfactors

SFP+

10G

QSFP28

100G/100G

CFP

100G

CFP2

100G

SFP28

25GSFP56

50G

CFP4

100G

OSFP

400G

QSFP+

40G

QSFP56

200GQSFP56-DD

400G

CFP8

100G

SFP56-DD

50G/100G

© 2020 EXFO Inc. 20200503 36

From NRZ to PAM4

1 10 000 0 0 0

1

0 0 000 1

1 010

1 1 1 1

NRZ

PAM4

Level

0

1

0123

1 1 1 1

Bandwidth

Bandwidth/2

Eye diagram

Upper eye

Middle eye

Lower eye

© 2020 EXFO Inc. 20200503 37

Transceivers moving to Silicon Photonics

Source: Silicon Photonics 2020 report. Yole Développement.

Silicon photonic 2019-2025 market forecast by application

© 2020 EXFO Inc. 20200503 38

PIC-dominant technology for high-speed communications driven

by optical transceivers for telecommunications and data centers.

PIC: a disruptive technology for transceivers

• Miniaturization → increase bandwidth density

• Lower power consumption

• Better thermal management

Commercialized units

• 400G QSFP-DD DR4

• 100G QSFP28 PSM4

• 100G CWDM4 QSFP28

PIC*-based transceiver: present & future

*Photonic integrated circuit (PIC)

© 2020 EXFO Inc. 20200503 39

Transceivers: from design to troubleshooting

Compliance specifications and industry

standards

during packaging and final verification

Wafer and die

manufacturing

Functional test for

quality control

Parametric test

during production

Commissioning and

troubleshooting

Wafer and

die test

© 2020 EXFO Inc. 20200503 40

OPTICAL ACTIVE COMPONENTSOptical spectrum analyzer (OSA)

OPTICAL PASSIVE COMPONENTSSwept laser technique

Hybrid

InP

Si

PIC waferOSA

PIC waferPassive

component tester

Tunable

laser

Parametric

test

• Average optical power

• Wavelength verification

• Side mode suppression ratio

© 2020 EXFO Inc. 20200503 41

Transmitter optical subassembly (TOSA)

Receiver optical subassembly (ROSA)

Functional

test

© 2020 EXFO Inc. 20200503 42

• Bit error rate (BER)

• Optical modulation amplitude (OMA)

• Extinction ratio (ER)

• Tx stress by reflectance

• Receiver sensitivity

© 2020 EXFO Inc. 20200503 43

What is so different?

Discrete components that

are tested individually

ROSA

TOSA

Optical

interfaceElectrical

interface

Integrated silicon

photonics based

© 2020 EXFO Inc. 20200503 44

PIC manufacturing costs

70-80% of PIC-based devices

costs come from packaging

vs. 20-25% for silicon

electronics

Identifying issues at the wafer levelCan save thousands $$$

Requires integration from

design to commission

PIC

© 2020 EXFO Inc. 20200503 46

Coherence through the entire process

• Type of tests

• Instruments selected

• Data format

• Data storage

Functional test for

quality control

Parametric test

during production

Commissioning and

troubleshooting

Wafer and die

manufacturing

© 2020 EXFO Inc. 20200503 47

Management through

the entire cycle

Test data

Functional test for

quality control

Parametric test

during production

Commissioning and

troubleshooting

Wafer and die

manufacturing

AI for optimization

Design

to test

Think ahead of time, during design

phase, about testing strategy

Make sure to implement test points and

capabilities at the wafer and die level

Define which data will be collected,

how it will be used and by whom

© 2020 EXFO Inc. 20200503 48

1

2

3

Q & A

50

John Calvin

Keysight Technologies

Steve RumsbySpirent Communications

Francois RobitailleEXFO

David RodgersTeledyne LeCroy

If you have any questions or comments, please email [email protected]

For our TEF 2021 on-demand content go to

www.ethernetalliance.org


mailto:[email protected]

mailto:[email protected]

Test & Measurement Planning for Performance

Documents

Transcript of Test & Measurement Planning for Performance