CMS June 20031

TriDAS UpdateDrew Baden

University of Marylandhttp://www.physics.umd.edu/hep/HTR/hcal_june_2003.pdf

USCMS HCALUSCMS HCALUSCMS HCALUSCMS HCAL

CMS June 20032

HTR StatusHTR StatusHTR StatusHTR Status

• Rev 1 run Summer 2002 testbeam Board worked well – all functional requirements met Big concern on mechanical issues for production

o Had a difficult experience with previous board manufacturing

• Rev 2 produced March 2003 Board production changes:

o New assembler, in-house X-ray, DFM review, QCo Gold plated (Rev 1 was white-tin) for better QC

Changes to HTR:o Change from Virtex 1000E FBGA (1.00mm) to Virtex2 3000 BGA (1.27mm)o Added stiffenerso Moved all SLB/TPG output to front-panel daughterboardso Modified Rx refclk scheme (the usual TTC/refclk clocking concerns)

Full 48 channel capability (Rev 1 was “half HTR”) As of this date, no issues – this board is functioning well

CMS June 20033

Dual-LC O-to-E

VME

Deserializers

Xilinx XC2V3000-4

Sti

ffen

er

s

6 SLBs

TTC mezzanine

HTR Rev 3 (cont)HTR Rev 3 (cont)HTR Rev 3 (cont)HTR Rev 3 (cont)

CMS June 20034

HTR Rev 3HTR Rev 3HTR Rev 3HTR Rev 3

• 30 boards delivered April 21 Checkout consisted of

o All systems except connectivity to SLB

o Fiber links checked out at 1.7Gbaud bit rate (1.6Gbaud is CMS requirement)Frame clock up to 2.0Gbaud bit rate and it stays synchronizedNo BER yet…will do a lab measurement soon12 boards x 16 links ~200 links(~5% of total) with no problemsUsed both onboard crystal oscillator and external clock for REFCLK

Minor adjustments will be needed for front panels, stiffeners, etc.

Will battle test these boards this yearo May synchronous testbeam just completed

o July testbeam

o Vertical Slice tests to commence in the fall

CMS June 20035

Clocks and SynchronizationClocks and SynchronizationClocks and SynchronizationClocks and Synchronization

• Clocking considerations can be divided into 2 parts: Deserializers REFCLK: stability critical (80MHz frame rate)

o Stability: must have a very low jitter – 30 to 40ps pkpk spec

o Frequency: TI TLK2501 spec is 100ppm (8kHz) to lock Measured ~350ppm (30kHz) needed to establish link

LHC variation expected to be few kHz

Once link is established, just needs to be stable (it’s a REFCLK!!!)

o Phase: relationship to LHC clock totally irrelevant

Phase critical clock for pipeline synchronizationo Must be in phase with LHC clock

o Jitter spec is very lose – this clock is used inside FPGA for sequential logic

CMS June 20036

HCAL Clock FanoutHCAL Clock FanoutHCAL Clock FanoutHCAL Clock Fanout

• HTR clocks provided by a single 9U VME board Chris Tully/Jeremy Mans from Princeton Has fiber TTC input

• Signals fanned out over Cat6 twisted pair: TTC stream

o To be used by each HTR and by DCC to decode commands & L1A

BC0o To be used by SLBs to synchronize TPGs

“40MHz” clocko To be used by FPGA and SLBs to maintain pipeline

Comes from QPLL

“80MHz” clean clocko To be used for deserializer REFCLK

Comes from QPLL

CMS June 20037

Clock DistributionClock DistributionClock DistributionClock Distribution

HTR

TTC fiber

TTC

CLK80

BC0

CLK40

distributionto 6 SLBs

and to 2 Xilinx

Brdcst<7:0>,BrcstStr, L1A

O/E

BC0

TTCTTC

TTC

FPGA

..

..

TestPoints forRxCLKand RxBC0

..

..

..

..

80.18 MHz

..

..

..

..

TTCrx

to Ref_CLK of SERDES(TLK2501)

CLK40

CLK80

Princeton FanoutBoard

TTCrx

QPLL

HTRHTR

Cat6Eor Cat7Cable

CMS June 20038

TTC receiver - TTCumdTTC receiver - TTCumdTTC receiver - TTCumdTTC receiver - TTCumd

• General purpose TTC receiver board (TTCumd)

TTCrx ASIC and associated PMC connectors

• Will be used to receive TTC signal by HTR, DCC, and clock fanout boards• No signal receivers!

Copper/fiber receivers must be on the motherboard

Signal driven through TTC connectors

• Tested successfully by Maryland, Princeton, BU groups

Princeton Fanout Card

CMS June 20039

May Testbeam SetupMay Testbeam SetupMay Testbeam SetupMay Testbeam Setup

• Lack of a QPLL or decent equivalent – had to improvise: Front-end used commercial Cypress PLL

o Matches GOL 100ps pkpk jitter spec

Fanout card o No clean 80MHz REFCLK, so provided 2 alternatives:

2xLHC clock from crystal oscillator High quality clock from HP signal and pulse generators Jumper selectable on mezzanine cards

o No clean 40MHz system clock Just used 40MHz output from TTCrx chip anyway

CMS June 200310

May Testbeam ExperienceMay Testbeam ExperienceMay Testbeam ExperienceMay Testbeam Experience

• To establish link: FE

o TTC 40MHz clock cleaned up by Cypress “roboclock” chip (Cy7B993)

o FE reset signal to GOL

Fanout cardo Fanout from onboard 80.1576MHz crystal oscillator for REFCLK

o Fanout TTC 40MHz clock for system clock

HTRo TLK2501 link circuitry always enabled

Result: Fiber 1.6GHz link established oko No problem locking – worked every time.

CMS June 200311

Testbeam Experience (cont)Testbeam Experience (cont)Testbeam Experience (cont)Testbeam Experience (cont)

• Link stability - VERY PRILIMINARY, STILL STUDYING Ran 10hr test on 48 fibers

o 3 x 1015 bitso 20% failed to maintain linko During synchronized beam running, sent reset between spills to ensure link

Similar tests at Maryland using TI eval board showed no link errors, similar number of bits sent

• Curret plan Study FE →HTR link at FNAL this month

o FNAL test stand setup this week Investigate noise characteristics of H2 environment

o H2 is clearly different than FNAL, Maryland and BU experience Review of HTR and Fanout card

o Will learn what we need to do from the above

• Best guess All tests in US indicate solid link, but experience in H2 disagree Probably some kind of new noise component – figure out and correct.

CMS June 200312

HCAL TPGHCAL TPGHCAL TPGHCAL TPG

• Nothing new since May Electronics Week• TPG under development…

Preliminary FPGA code for TPGs doneo LUT for linearization (downloadable), 0.5GeV steps, 255Gev max ET

o E to ET and sums over as many as 7 channelsNot implemented in code yet…TBD

o Muon window in Eo BCID filter algorithm TBD from testbeamso Compression LUTs for output to SLBs

Utilization is ~50% of Virtex2 3000o We are confident this chip will be sufficient

Simulation effort under way…

• Latency issue See below – we are working on this…

CMS June 200313

HTR TPG CommissioningHTR TPG CommissioningHTR TPG CommissioningHTR TPG Commissioning

• 2 Xilinx FPGAs per HTR 3 SLBs per Xilinx Each mounted on triPMC

connectors

• Will test internal connectivity to SLBs at UMD For signals and for

localbus connections

• Need a scheme to test HTR/RCT connectivity Not just electrical! Also

includes data integrity

SLB

SLB

SLB

SLB

SLB

SLB

Xilinx

Xilinx

RCT

CMS June 200314

HTR/RCT TestingHTR/RCT TestingHTR/RCT TestingHTR/RCT Testing

• Will build PMC tester card to mount onto HTR Host to 1 or more Wisconsin

RCT Vitesse receiver boards

• Will run the signals from this RCT tester card back into Xilinx Using 1 HTR, both FPGAs –

one source, one sink – to test sending data from HTR to RCT

• Will try to engineer 3-SLB test to test single Xilinx → SLB →RCT

SLB Xilinx

Xilinx

RCT

CMS June 200315

HTR ProductionHTR ProductionHTR ProductionHTR Production• Full contingent of HTRs: 260 boards

Includes 10% spares, 20% spares for parts

• Full production will begin after: Testbeam demonstrates I/O works under battle conditions Successful testing of the 6 SLB daughter card functions Understanding of how to meet latency issues

o We are still some clock ticks short, but firmware is still very immature for the TPG part of the HTR (see slides below)

• Best guess: fall 2003 There is no reason to hurry other than to finish with the R&D part

of the project Current board design will be final, perhaps some layout

adjustments based on conclusion of testbeam effort

CMS June 200316

Overall Commissioning ScheduleOverall Commissioning ScheduleOverall Commissioning ScheduleOverall Commissioning Schedule

• Summer 2003 testbeam Repeat previous test w/production prototype boards

• Fall 2003 Slice tests HCAL will join as schedule allows

• 2003/2004 HCAL burn-in Continue with firmware development/integration as needed

• 2004/2005 Vertical Slice and magnet test We will be ready All HCAL TriDas production cards involved

• October 05 beneficial occupancy of USC Installation of all racks, crates, and cards We do not anticipate any hardware integration

o Should be all firmware / timing / troubleshooting

CMS June 200317

TPG LatencyTPG LatencyTPG LatencyTPG Latency

“Minimizing the trigger latency”

Item LatencyTOF .5

HCAL Optics 1

FE (CCA+QIE) 8-9

GOL 2

Fiber Tx to HTRs 18

Deserializer 2-3

HTR Alignment 6

HTR TPG path 5-10

SLB 3

TPG Cables 4

TOTAL 50 - 57

• Current total 50 – 57 clocks Very rough guesses

o Many numbers have not been measured

• Optimizations: Fiber cables need to be 90m? HTR firmware needs optimization Deserializer random latency fix TPG cables changed to 15m will save

1 tick Others…main efforts over next 6

months

CMS June 200318

TPG PathTPG PathTPG PathTPG Path

ETcomp

7 10

Muon bit

SumConsecutive

Time-samples9

TP

8

QIE-dataINPUT

LUTLineariz.

and Et

ET[9:0]

2

Compression

LUT

2Muon LUT

1

Delay to synchronize

with BCID

10

L1 Filter

10

Sum in ET

PeakDetection

TP_Bypass

1

0

2 2

2

Mask &Reset

“NO-SHOWER” LUTtake care of cases where showers can leak into a cell and incorrectly set the muon bit.

BCID

BCID avoids to flag as a muon the tail of a

more energetic event