External Use

TM

Implement IEEE® 1588v2 on

QorIQ Communications Platforms

FTF-NET-F0071

A P R . 2 0 1 4

Xie Xiaobo | Software Engineer

TM

External Use 1

Agenda

• Introduction of IEEE1588

• Basics of Precision Time Protocol

• Hardware: IEEE 1588 hardware assist block

• Software: Device driver and application

• Test setup and results

• Summary

• References

TM

External Use 2

IEEE 1588 Introduction

TM

External Use 3

What is the IEEE Std 1588?

• IEEE Std 1588 — Standard for a precision clock synchronization

protocol for networked measurement and control

− The standard defines a Precision Time Protocol (PTP) designed to

synchronize real-time clocks in a distributed system

− Intended for local area networks

− Targeted accuracy of microsecond to sub-microsecond with easy

configuration and fast convergence between components

− IEEE Std 1588-2002 (Version 1) approved September 2002 and

published November 2002

− IEEE Std 1588-2008 (Version 2) approved March 2008 and published

August 2008

Available from the IEEE 1588 web site

(http://www.nist.gov/el/isd/ieee/ieee1588.cfm)

TM

External Use 4

NTP GPS TTP IEEE Std 1588

Target Uses Autonomous systems

dispersed over a wide

area. Time information

passed via messages on

the Internet.

Autonomous systems

dispersed over a wide

area. Time information

passed via satellite.

Tightly integrated , closed

systems usually connected

via a bus or specialized

TDMA network

Groups of relatively

stable components,

locally networked (a

few subnets),

cooperating on a set of

well defined tasks

Target Accuracy under 1ms possible, 1-

10ms typical in LAN,

<100ms over the Internet

Sub microsecond

Sub microsecond

Sub microsecond (±

50ns typ)

Synchronization

Resolution Time Minutes to hours < Minute

Resource

Requirements Moderate network and

compute footprint

Moderate compute

footprint

Moderate compute footprint Small network and

compute footprint

Latency Correction Yes Yes Configured Yes

Update Interval Variable, but normally

seconds

Approximately every

second

Milliseconds Approximately every 2

seconds

Hardware Required No Yes Yes Yes, to achieve

greatest accuracy

Protocol Comparison

TM

External Use 5

Major motivation for version 2

• New application areas with new requirements: Telecom, IEEE

802.1as, power industry.

• New requirements in original applications

− Higher accuracy

− Varied update rates

− Linear topology (in addition to hierarchical)

− Rapid reconfiguration after network changes

− Fault tolerance

TM

External Use 6

Main differences between V1 & V2

• Shorter synchronization frames to save network bandwidth.

• Transparent clocks to avoid exponential error propagation in

cascaded networks.

• End-to-End and Peer-to-Peer TC.

• Wider choice of ‘Sync’ (timing) and ‘Announce’ (M-S hierarchy)

update rates.

• Profiles to accommodate different market requirements.

• ……

TM

External Use 7

IEEE 1588

Precision Time Protocol Basics

TM

External Use 8

IEEE Std 1588 Synchronization Message Sequence

TM

External Use 9

IEEE Std 1588 PTP Non-Management Message Types

SYNC messages

• Master sends an estimate of

the sending time

• When received by a slave

clock, the receipt time is

noted

FOLLOW_UP Messages

• Always associated with the

preceding Sync message

• Contain the ‘precise sending time’

of SYNC message measured

close to the physical layer of the

network

DELAY_RESP Messages

• Always associated with a

preceding Delay_Req message

from a specific slave clock

• Contain the receipt time of the

associated Delay_Req message

DELAY_REQ messages

• Issued by clock nodes in the

‘Slave’ state

• When received by the master

clock the receipt time is noted

TM

External Use 10

Protocol Stack Delay Problem

PTP

UDP

IP

MAC

PHY

Master Clock

PTP

UDP

IP

MAC

PHY

Slave Clock

Milliseconds

of delay and

variation

introduced

by protocol

stack

Network

Hardware time-stamping removes protocol

stack delay

Milliseconds

of delay and

variation

introduced

by protocol

stack

IEEE Std 1588 PTP Code

Network protocol

stack & OS

PHY

Timestamp

generation /

message

detection

MII /

GMII

TM

External Use 11

Boundary Clock

PTP

UDP

IP

MAC

PHY

(Slave)

PTP

UDP

IP

MAC

PHY

(Master)

Switch / Router with

Boundary Clock

PTP

UDP

IP

MAC

PHY

Master Clock

PTP

UDP

IP

MAC

PHY

Slave Clock

Network 1

Synchronization across multiple network/subnets

Network 2

TM

External Use 12

Peer-To-Peer Transparent Clock

• Forwards and Corrects Sync and Follow Up messages only.

• Correction achieved by addition of bridge residence time plus the peer to

peer link delay into a Correction filed within the header of message.

• Need insert a picture for TC

TM

External Use 13

IEEE 1588 Hardware Assist

Timer logic

TM

External Use 14

IEEE 1588 Hardware Assist Clock

Time-Stamp

Nominal

Clock

External

Clock Ref

Platform

Clock Ref

32-bit Accum Carry

Bypass Enable

32-bit

Addend

64-bit Counter

TMR_CNTH/L

TCLK_PERIOD

+

+

RTC Clock

eTSEC Tx

Clock

Software

Tip – ADDEND

is modified to

fine tune the

slave clock

Tip – TMR_CNTL/H

are modified when

difference between

master and slave is

huge

TM

External Use 15

IEEE Std 1588 Hardware Assist Clock

Phase/Frequency Adjustment

• Four choices of input clock

• Addend and accumulator provides a digital fractional divider

• Provision for bypassing the divider logic

• ADDEND = 232 ÷ FreqDivRatio

− If input clock = 150MHz & desired nominal clock = 100MHz

− ADDEND = 232 ÷ (150/100) = 0xAAAA_AAAA

Nominal

Clock

External

Clock Ref

Platform

Clock Ref

32-bit Accum Carry

Bypass Enable

32-bit

Addend

64-bit Counter

TMR_CNTH/L

TCLK_PERIOD

+ +

RTC Clock

eTSEC Tx

Clock

TM

External Use 16

Selection of ADDEND and TCLK_PERIOD

• The 64-bit counter increments by TCLK_PERIOD on every pulse of “nominal clock”

• To represent time in nanoseconds, TCLK_PERIOD should be equal to reciprocal of frequency of “nominal clock”

− It is recommended to have TCLK_PERIOD as integral factor of 109

• Example:

− Few integral factors of 109 : 2, 4, 8, 10, 16, 20…

− For Input clock = 333MHz, recommended choices for nominal clock are 100MHz, 125MHz, 200MHz and 250MHz

Nominal

Clock

External

Clock Ref

Platform

Clock Ref

32-bit Accum Carry

Bypass Enable

32-bit

Addend

64-bit Counter

TMR_CNTH/L

TCLK_PERIOD

+

+

RTC Clock

eTSEC Tx

Clock

TM

External Use 17

Working Example of ADDEND

• Example assumes the accumulator overflows after “9”, therefore ADDEND= 9 / (150/100) = 6

• The resultant clock may not have 50% duty cycle or uniform period

6

+

Input Clock = 150MHz

Nominal Clock = 100MHz

Addend

= 9 / 1.5

= 6

Accumulator

Carry after 9

6 2 8 4 0

TM

External Use 18

Using External VCXO Clock

• Systems which require to synchronize an external clock may

choose to bypass internal fine-tuning

• The counter runs on the input clock

• SPI or IIC can be used to fine-tune VCXO

Nominal

Clock

External

Clock Ref

Platform

Clock Ref

Carry

Bypass

Enable

Accum

Addend

Counter

TMR_CNTH/L

TCLK_PERIOD

+ RTC

Clock

eTSEC Tx

Clock

VCXO

IIC/SPI

DAC

+

Tip – External clock needs to be tuned

TM

External Use 19

Timer Extensions

Nominal

Clock

External

Clock Ref Platform

Clock Ref 32-bit Accum

Carry

Bypass Enable

32-bit

Addend

+

RTC Clock

eTSEC Tx

Clock

64-bit Counter

TMR_CNTH/L

TCLK_PERIOD

+

64 –bit ALARM

≥

32-bit FIPER

TCLK_PERIOD

−

Prescaler GCLK

PULSE_OUTn

ALARM_OUTn

TM

External Use 20

Timer Extensions (continued)

• ALARM_OUT is generated when counter is

equal to or greater than ALARM

− can be used to trigger the periodic pulse

generator

• With FIPER as the initial value, a down-

counter decrements by TCLK_PERIOD on

every pulse of nominal clock

• A pulse is generated when the down counter

reaches zero or less than TCLK_PERIOD

− Generates periodic pulse with a width of one

period of the pre-scaled output clock

− Down counter is reloaded; the process repeats

• GCLK outputs pre-scaled output clock

Nominal

Clock

64-bit Counter

TMR_CNTH/L

TCLK_PERIOD

+

64 –bit ALARM

≥

32-bit FIPER

TCLK_PERIOD

−

Prescaler GCLK

PULSE_OUTn

ALARM_OUTn

TM

External Use 21

Synchronizing PULSE_OUT with TMR_CNT

• FIPER should be programmed to an integer multiple of TCLK_PERIOD value to ensure a period pulse being generated correctly

• To generate PPS signal:

− FIPER = <109 nanoseconds> - TCLK_PERIOD

• To align PPS signal with TMR_CNT:

− Program ALARM to a value which is a whole number of seconds, and greater than the present TMR_CNT

ALARM = (floor{(TMR_CNT/109) + n})*109

Example: For TMR_CNT = 5.3s and n=2, ALARM = 7s

− Set TMR_CTRL[FS] to trigger FIPER by ALARM

Tip – Only ALARM1 can trigger FIPER1.

Nominal

Clock

64-bit Counter

TMR_CNTH/L

TCLK_PERIOD

+

64 –bit ALARM

≥

32-bit FIPER

TCLK_PERIOD

−

PULSE_OUTn

ALARM_OUTn

TM

External Use 22

FIPER Re-Alignment Upon Changing TMR_CNT

• Should user need to change the value

of TMR_CNT, the following procedure

should be followed to realign the PPS

signal:

− Calculate the new value of ALARM

− Write new values to TMR_CNTL/H

− Write calculated values to ALARM1L/H

− Re-write FIPER to reset the down counter

− Set TMR_CTRL[FS]

Nominal

Clock 64-bit Counter

TMR_CNTH/L

TCLK_PERIOD

+

64 –bit ALARM

≥

32-bit FIPER

TCLK_PERIOD

−

PULSE_OUTn

ALARM_OUTn

TM

External Use 23

Accessing Timer Registers

• TMR_CNT_L should be read first to get correct 64-bit TMR_CNT_H/L counter values

− Reads from the TMR_CNT_L register copies the entire 64-bit clock time into shadow registers

• TMR_CNT_L should be written first

− Contents of the shadow registers are copied into the TMR_CNT_L and TMR_CNT_H registers following a write into the TMR_CNT_H register

• Writing the TMR_ALARMn_L register deactivates the alarm event

• Writing the TMR_ALARMn_L followed by the TMR_ALARMn_H register rearms the alarm function with the new compare value

• Writing new value to FIPER register resets the down counter used in PULSE_OUT generation

ATTENTION – Above recommendations should be strictly followed.

Any violation may result in unpredictable results.

TM

External Use 24

IEEE 1588 Hardware Assist

Time-Stamping logic

TM

External Use 25

Time-Stamping Logic

• There are three different time-stamp capture triggers

− Reception of a packet

− Transmission of a packet

− On the positive or negative edge of the external trigger

TM

External Use 26

Time-Stamping on Ethernet Frame Reception

• For eTSEC:

• On detection of SFD, the value of TMR_CNT_H/L is copied to

TMR_RXTS_H/L if RCTRL[TS] in eTSEC is set to 1

• In addition, the time-stamp is inserted into the packet data buffer as

padding alignment bytes if

− TMR_CTRL[RTPE] is set to 1 AND

− RCTRL[PAL] (receive pad alignment length) is set to a value greater

than or equal to 8

• eTSEC indicates reception of PTP packet to CPU

• CPU reads time-stamp from RxBuffer or TMR_RXTS_H/L

TM

External Use 27

Time-Stamping on Ethernet Frame Reception

• For dTSEC:

• When enabled by setting RCTRL[RTSE] in dTSEC, every incoming

packet will be accompanied with an 8-byte time-stamp

• The BMI will extract the timestamp and copy it to the timestamp

field within the IC.

• The whole frame together with timestamp is copied into external

buffers and FD is enqueued to indicate reception of PTP packet to

CPU

• CPU reads time-stamp from Frame Descriptor

TM

External Use 28

Time-Stamping while Ethernet Frame Transmission

• Foe eTSEC:

• eTSEC supports two-step clock

− The time-stamp of frame being transmitted is stored in registers or frame

control buffer

− The follow-up packet carries the actual time-stamp of previous packet

• eTSEC supports selective time-stamping for Tx packets using

TxFCB[PTP]

• In dTSEC, setting TCTRL[TTSE] to 1 ensures that all the packets

will be time-stamped during transmission

• The packet ID and time-stamp are stored in the TMR_TXTS1-2_ID

and TMR_TXTS1-2_H/L registers

TM

External Use 29

Time-Stamping in FCB while Ethernet Frame Transmission

• To get time-stamps of transmit packets on FCB, following

requirements should be met:

− TMR_CTRL[RTPE], TxBD[TOE] and TxFCB[PTP] should be set to 1

− A minimum of two TxBDs are used per packet

The first points to the start of the 8 byte TxFCB

The second points to the start of frame data

− The TxFCB, and at least the first 16 bytes of the TxPAL, must be located

in contiguous memory locations

• The time-stamp is written to memory location TxBD[Data Buffer

Pointer]+ 0x10

TM

External Use 30

Time-Stamping while Ethernet Frame Transmission

• For dTSEC:

• When enabled by setting TCTRL[TTSE], every requested transmit

packet will cause the return of a time-stamp value from the dTSEC

• The BMI receives the actual timestamp after the frame is

transmitted. In the TX confirmation phase, The BMI writes the

timestamp into the timestamp field within the IC of the sent frame

and issues DMA request to copy IC portion into external memory

• dTSEC also supports two-step clock

TM

External Use 31

Time-Stamping on TRIG_IN

• The polarity of TRIG_IN signal can be chosen using

TMR_CTRL[ETEPn]

• TMR_TEVENT[ETSn] is set if external trigger is received

• TMR_ETTS1–2_H/L stores the time-stamp

Attention – P1010 has 16 pairs of TMR_ETTSn_H/L registers

TM

External Use 32

IEEE 1588 Hardware Assist

Interrupts, Registers and Signals

TM

External Use 33

Share Hardware Assist IEEE 1588 Block

• Generally QorIQ have more than one eTSEC/dTSEC(s)

• There is single IEEE1588 block shared among all eTSEC(s) in a

device

• Every FrameManager has its own instance of 1588 hardware assist

block

− All dTSECs corresponding to an FM share the 1588 hardware assist

• However, there are some registers and interrupts dedicated per

eTSEC/dTSEC

Tip – Since the common 1588 time-stamping registers exist within the eTSEC1 memory space, the eTSEC1 controller must remain enabled in order to use 1588 time-stamping for any Ethernet port.

TM

External Use 34

Hardware Assist IEEE 1588 Interrupts

• The Interrupt controller has different interrupt numbers associated

with 1588 interrupts of different eTSECs

• Interrupts generated on transmission or reception of Ethernet

packet are dedicated per-eTSEC

− These interrupts are indicated by TMR_PEVENT

• Interrupts generated by ALARM, FIPER and external trigger

(TRIG_IN) are registered to “eTSEC1 1588 timer”

− These events are shown by TMR_TEVENT

Internal Interrupt Number Interrupt Source

52 eTSEC1 1588 timer

53 eTSEC2 1588 timer

54 eTSEC3 1588 timer

TM

External Use 35

Hardware Assist IEEE 1588 Registers

• All the registers of Hardware Assist IEEE 1588 are valid only on

eTSEC1 memory region except registers listed below

− TMR_TXTS1–2_ID : Transmit Time Stamp Identification Register

− TMR_TXTS1–2_H/L : Transmit Time Stamp Register

− TMR_RXTS_H/L : Receive Time Stamp Register

− TMR_PEVENT: Timer PTP Packet Event Register

− TMR_PEMASK : Timer Event Mask Register

− TMR_STAT : Timer Status Register

Attention – Access to any other register of “Hardware Assist IEEE 1588” from memory region other than eTSEC1 is illegal

TM

External Use 36

Hardware Assist IEEE 1588 Registers (Cont.)

• For dTSEC:

• All the registers of Hardware Assist IEEE 1588 are valid on FMan’s 1588 timer module memory region except registers listed below which resides in the dTSEC’s memory region:

− TMR_CTRL – is not the same to the TMR_CTRL register in the 1588 timer module

− TMR_PEVENT - Time-stamp event register

− TMR_PEMASK - Timer event mask register

• Comparing to eTSEC:

− No TMR_TXTS1–2_ID Register

− No TMR_TXTS1–2_H/L Register

− No TMR_RXTS_H/L Register

The BMI get the timestamp to put into the IC timestamp field

TM

External Use 37

Hardware Assist IEEE 1588 Signals

• TSEC_1588_CLK_IN: One of the four choices of input clocks

• TSEC_1588_CLK_OUT: Output of pre-scalar

• TSEC_1588_TRIG_IN[1:n] : External trigger input

• TSEC_1588_PULSE_OUT[1:n] : Output of FIPER

• TSEC_1588_ALARM_OUT[1:n]: Output of ALARM

Attention –

1. The number of TSEC_1588_TRIG_IN, TSEC_1588_PULSE_OUT

and TSEC_1588_ALARM_OUT may vary from device to device

2. There might be some variation in the name of the signal from device

to device

TM

External Use 38

IEEE 1588 Software

Device Driver and Application

TM

External Use 39

Initialization Routine

• Get the frequency of the input clock

− Calculate as explained earlier and feed results in ADDEND and

TCLK_PERIOD

• Write desired value to TMR_PRSC and FIPER

• Calculate and feed ALARM register

• Set TMR_CTRL[FS] to trigger FIPER with ALARM

• Choose input clock using TMR_CLK[CKSEL]

• Start timer by setting TMR_CTRL[TE] to 1

• Initialize rest of the registers for time-stamps and interrupts are

required

TM

External Use 40

IXXAT Software Block Diagram

• The application interacts with the

network layer to send/receive PTP

messages

• Using 1588 APIs, the application

gets the time-stamps of packets

sent or received

• Based on time-stamps, it decides to

tune the clock using 1588 APIs

Hardware

Kernel

1588

Timer

APIs

Network

Layer

Application Layer

Clock Servo

Mechanism

Messaging

Unit

TM

External Use 41

IOCTL command for IXXAT Software

• GET_RX_TIMESTAMP : To read time-stamp of packet received

• GET_TX_TIMESTAMP : To read time-stamp of packet transmitted

• GET_CNT : Read value of TMR_CMT

• SET_CNT : Write new value of TMR_CNT along with reinitializing

FIPER and ALARM

• ADJ_ADDEND : Write new data to ADDEND

• GET_ADDEND : Read ADDEND

TM

External Use 42

Open Source Software Block Diagram

• The ioctl interface is implemented as socket ioctl.

- Command: SIOCSHWTSTAMP

• The 1588 message transfer by UDP protocol.

• The Clock PTP driver is character driver

TM

External Use 43

PTP Clock driver Interface

• PTP Clock driver register themselves by presenting a 'struct

ptp_clock_info' to the registration method - ptp_clock_register().

struct ptp_clock_info ptp_gianfar_caps = {

.owner = THIS_MODULE,

.name = "gianfar clock",

.adjfreq = ptp_gianfar_adjfreq,

.adjtime = ptp_gianfar_adjtime,

.gettime = ptp_gianfar_gettime,

.settime = ptp_gianfar_settime,

.enable = ptp_gianfar_enable,

};

• A character device is created for each registered clock. User space can

use an open file descriptor from the character device as a POSIX clock id

and may call clock_gettime, clock_settime, and clock_adjtime.

TM

External Use 44

Timestamp Processing in Open Source

• A shared time stamp structure – struct skb_shared_hwtstamps.

struct skb_shared_hwtstamps {

ktime_t hwtstamp;

ktime_t syststamp; /* hwtstamp transformed to system time base */

};

• Time stamps for received packets was stored in the skb. Get a pointer to

the shared time stamp structure by calling skb_hwtstamps(). Then set the

time stamps in the structure skb_shared_hwtstamps.

• For outgoing packets, skb_hwtstamp_tx() clones the original skb and adds

the timestamp to structure skb_shared_hwtstamps. The cloned skb with

the send timestamp attached is looped back to the socket's error queue.

• PTPd get the RX timestamp by calling recvmsg() and the TX timestamp by

calling recvmsg(flags=MSG_ERRQUEUE).

TM

External Use 45

PTPd Block Diagram

TM

External Use 46

IEEE 1588 Test

Setup and Result

TM

External Use 47

IEEE 1588 Std Protocol on QorIQ Platform

-200

-150

-100

-50

0

50

100

150

200

1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93 97 101 105 109 113 117 121 125 129 133 137

Plot the “Offset from Master” Raw Data

PTP stack evaluation version - will stop after 4 hours

-0,085857248; 0,000000000; 0,000000000; 0,000000000; 2152;

-0,000002152;-0,000002152; 0,000000000; 0,000000000; 3132;

-0,000002056;-0,000002056; 0,000000000; 0,000000000; 2960;

-0,000000856;-0,000000856; 0,000000000; 0,000000000; 2268;

0,000000264; 0,000000216; 0,000000808;-0,000000048; 1792;

0,000000608; 0,000000560; 0,000000808;-0,000000048; 1648;

0,000000448; 0,000000400; 0,000000808;-0,000000048; 1760;

0,000000112; 0,000000064; 0,000000808;-0,000000048; 1944;

-0,000000128;-0,000000176; 0,000000808;-0,000000048; 2032;

IEEE Std 1588 Master (TWR-P1025)

IXXAT/PTPd IEEE Std 1588 application S/W IEEE Std 1588 Slave (TWR-P1025)

IXXAT/PTPd IEEE Std 1588 application S/W

TM

External Use 48

IXXAT Application Testing Result

TM

External Use 49

PTPd Application Testing Result

Time [Sec]

[nS

ec]

TM

External Use 50

Summary

Using IEEE 1588 Hardware Assist logic, sub-50ns

synchronization can be achieved over the network

Hardware support for 1588 is available in all the

devices of QorIQ family

Being used for industrial, telecom and consumer

(audio-video sync) applications

Synchronized pulses and alarm functionality

available in QorIQ family

TM

External Use 51

References

Reference Manuals of QorIQ devices

AN3423: Application note on IEEE1588

http://www.nist.gov/el/isd/ieee/ieee1588.cfm

TM

© 2014 Freescale Semiconductor, Inc. | External Use

www.Freescale.com