FPGA firmware of DC5 FEE

Outline

List of issue

• Data loss issue• Command error issue (DCM to FEM)• Command lost issue (PC with USB connection to GANDALF)

Module Connection

Signal and Clock rate

— Optical Tx and Rx 3.1104 Gbps— Master CLK 38.88 MHz (was 155.52MHz)— Command 38.88 Mbps (was 155.52MHz)— FLT (First level Trigger) Pulse sync to 38.88MHz CLK — Data 155.52 Mbps

Command & FLTGANDALF DCM FEM

Optical fiberEthernet

cable

Master CLK

Data

FEM

……

Optical Tx

Optical Rx

x1 x8 x20

Totally1 GANDALF8 DCM160 FEM

Clock Structure

GANDALF

DCM FEM

Optical Tx

Optical RxXilinx

Transceiver

OSC155.52M

Rx clock155.52M

commandDeformater PLL1

PLL2

155.52M

FEM Ctrl Logic

77.76M

FEM2DCMtransmit

155.52MB

TDC

Clock 38.88M

233.28M

233.28M 90°

2M CMAD setting

• Deformater is a VHDL code from T. Grussenmeyer.— Master 38.88MHz CLK is generated and phase adjusted by RX CLK and

command from GANDALF.• Two TDC CLKs will be 233.28MHz = 38.88MHz x 6. (248MHz now)• 155.52MB is a phase adjustable CLK for data output.

First Level Trigger

• In DCM, First level trigger is generated according to a specific command/data pattern from GANDALF.

• A FLT pulse is distributed to FEMs sync to master CLK.

GANDALFOptical Tx

Optical RxXilinx

Transceiver

Rx clock155.52M

commandDeformater

Clock 38.88M

FLT(first level trigger)

TCS Reset signal

• TCS reset will be distributed to FEMs by a dedicated command sync to master CLK.– Not fully implemented yet.• A comment to FEMs to reset TDC counters in the

current implemetation.

– Need inputs about TCS Reset command/pattern from GANDALF.

Trig Time

FEM TDC Block Diagram

CMADx16 inputs TDC

x16

FLT trigger TDC

x1TriggerLogic

Flag

Reset

TriggerMatch

Trig FlagEventFIFO(512 x 32bit)

DCM & FEM(8b/10b)Link logic

BufferCtrl

Logic

x16 Time

x16 Flag

x16 Reset

Cycling buffer

(512 hits)

Trig Flag

Write point

# of TDC hits

0100110110Serial data

Commandhandler

It will be4096x32 bit.

Trig Time

Data

TDC counter

• TDC value for each hit is 16 bit.– MSB 14 bit is from a counter by 233MHz CLK.– LSB 2 bit is determined by a four bit pattern

latched with 233MHz CLK and 233MHz 90o CLK.

14 bits 2 bits

233.28M

233.28M 90°

14 bits ≈ 70.3us

Trigger Match

• Time resolution– TDC time = 16 bit (1ns lsb)– Trigger Latency = 12 bit (4ns lsb)– Matching Window = 12 bit (4ns lsb)

• (Ttrig–Tlatency–Twindow) < Thit < (Ttrig–Tlatency+Twindow)• Matching process stops at

– 16 matched hits.– No more TDC hit left (Max hits for matching process is 255).– 4 unmatched hits after last matched hit.– All conditions will be adjusted according to the final noise level.

DCM block diagram

010011Serial data

DCM & FEM(8b/10b)

Transmitter

x20

DCM to FEMCommand

FIFO

x20

FEM to DCMDataFIFO

(512 x 32bit)

x20

Commandhandler

GANDALFDCM

Link logic

Transceiver 010011Serial data

TCS info

command

FLT(first level trigger)

FEM to DCMFrame flag

FIFO

x20data

packing

dataDCM & FEM

(8b/10b)Receiver

x20

010011Serial data

DCM pack FEM data procedure

Idle

Any FEM framevalid

No

Wait 4 system clock

Yes

Timeoutor

all FEM valid

No

Send S-link begin mark

Yes

Send S-link header

Scan all FEM FIFO

framevalid

Readout FEM data

EOF wordNo

All FIFO scan over

Yes

Yes

No

Send S-link end mark

Yes

Power on reset

No

Data Loss issue(DCM to GANDALF)

• Previous version of DCM FPGA design, the same state machine controls both command flow and data flow.– When a command arrives in DCM, the data packing and

transmission will be interrupted and caused the data loss.

• The latest DCM FPGA design is modified to have independent control for:– Command flow (GANDALF DCM FEMs)– Data flow (FEMs DCM GANDALF)

Latest Data transmission Test(with new DCM firmware)

2. DCM generates100k trigger in one sec.

GANDALF DCM FEM

5. Use counter check validdata frame number from FEM

Optical fibre

x1 x1 x1

Count mode

USB Ethernet cable

1. Pass command Trigger_on to DCM

3. FEM send 1 data frameto DCM per trigger

4. Packing data frame andpass to GANDALF

6. Pass data frame to PC7. Save data into fileand use program analysis

• Test result– DCM did received 100k data frames from FEM.– The data stored in PC lost about 1.5% data frame.

Command Error issue(DCM to FEM)

• A timing issue, long operational logic path due to– 8b/10b encoding– Multiplexing of commands and fill pattern

• The latest DCM FPGA design add pipeline/FIFO to reduce logic path– Under test

Command lost issue(PC with USB connection to GANDALF)

• When PC is taking data from GANDALF and sending commands to GANDALF at the same time, – DCM receives commands with error or loses commands.

• Try to stop trigger first before sending commands.

Summary

Backup

Command lost (DCM to FEM)

DCM to FEMCommand FIFO

• Timing issue when do 8b10b encoder– Original structure

8b10bencoder

control

command

Idle(K28.5) Trig func

Mode controlstatus

0101001100

ERROR

control

Controller + Serializer

Fix Command lost (DCM to FEM)

• New version

DCM to FEMCMD FIFO

CMD CMD / idleSelector

32b to 8bFIFO

8b10bencoder

Serializer

Trig modefunc

Mode select with command for DCM0101001100

FLT pulse signal