The LCLS Timing & Event System - An Introduction – John Dusatko / Accelerator Controls
description
Transcript of The LCLS Timing & Event System - An Introduction – John Dusatko / Accelerator Controls
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA1
The LCLS Timing & Event System- An Introduction –
John Dusatko / Accelerator Controls
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA2
Outline
1. Introduction to LCLS2. Some background on SLAC Timing3. The SLAC Linac Timing System4. The LCLS Timing System5. Issues
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA3
Introduction
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA4
LCLS Introduction
The Linac Coherent Light Source is an X-ray FEL based on the SLAC Linac:
1.0nC, 14GeV e- are passed thru an undulator, a Self Amplifying Stimulated Emission process produces 1.5 Angstrom X-Rays.
LCLS is an addition to the existing SLAC Linac: it uses the last 1/3 of the machine►This is important to note because we have to integrate
the New LCLS Timing System with the Existing Linac (SLC) Timing System.
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA5
The LCLS – Schematic View The LCLS – Schematic View (ignoring photon beamline)(ignoring photon beamline)
Single bunch, 1-nC charge, 1.2-Single bunch, 1-nC charge, 1.2-m m sliceslice emittance, 120-Hz repetition rate… emittance, 120-Hz repetition rate…
(RF phase: (RF phase: rfrf = 0 is at accelerating crest)= 0 is at accelerating crest)
SLAC linac tunnelSLAC linac tunnel research yardresearch yard
Linac-0Linac-0L L =6 m=6 m
Linac-1Linac-1L L 9 m9 m
rf rf 25°25°
Linac-2Linac-2L L 330 m330 mrf rf 41°41°
Linac-3Linac-3L L 550 m550 mrf rf 10°10°
BC-1BC-1L L 6 m6 m
RR5656 39 mm39 mm
BC-2BC-2L L 22 m22 m
RR5656 25 mm25 mm LTULTUL L =275 m=275 mRR56 56 0 0
DL-1DL-1L L 12 m12 mRR56 56 0 0
undulatorundulatorL L =130 m=130 m
6 MeV6 MeVz z 0.83 mm 0.83 mm 0.05 %0.05 %
135 MeV135 MeVz z 0.83 mm 0.83 mm 0.10 %0.10 %
250 MeV250 MeVz z 0.19 mm 0.19 mm 1.6 %1.6 %
4.54 GeV4.54 GeVz z 0.022 mm 0.022 mm 0.71 %0.71 %
14.1 GeV14.1 GeVz z 0.022 mm 0.022 mm 0.01 %0.01 %
...existing linac...existing linac
newnew
rfrfgungun
21-1b21-1b21-1d21-1d XX
Linac-Linac-XXL L =0.6 m=0.6 mrfrf= =
21-3b21-3b24-6d24-6d
25-1a25-1a30-8c30-8c
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA6
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA7
LCLS Timing – Some Definitions
The LCLS Timing System can be viewed as consisting of three parts:
Part 1: ‘Standard’ Accelerator Timing10ps Triggers for Acceleration and Diagnostics
Part 2: S-Band Timing2856MHz LCLS RF Phase Reference Distribution
Part 3: Ultra-Precise Timing10fs Synchronization for Experiments (LBL System)
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA8
LCLS Timing – Some Definitions
The LCLS Timing System can be viewed as consisting of three levels:
Part 1: ‘Standard’ Accelerator Timing10ps Triggers for Acceleration and Diagnostics
‘Triggers’ are signals from the timing system used by HW to accelerate & measure the beam
Part 2: S-Band Timing2865MHz RF Phase Reference Distribution
Part 3: Ultra-Precise Timing10fs Synchronization for Experiments (LBL System)
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA9
LCLS Timing – Performance RequirementsLCLS Timing System Requirements
Maximum trigger rate: 360 Hz (120Hz)
Clock frequency: 119 MHz
Clock precision: 20 ps
Delay Coarse step size: 8.4 ns ± 20 ps
Delay range; >1 sec
Fine step size: 20 ps
Max timing jitter w.r.t. clock; 2 ps (10ps) rms
Differential error (skew), location to location:
8 ns
Long term stability: 20 ps
Signal Level: TTL
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA10
Requirements Comparison
Timing Reqmts for earlier SLAC systems:Original Linac:
(~1968)
- Resolution: 50 nSec
- Jitter: 15 nSec
- Main Trigger Line
Waveform:
+ / - 400 Volts
PEP – II:
(~1998)
- Resolution: 2.1 nSec
- Jitter: 20 pSec
- NIM Level
Waveform:
0 to –0.7 V into 50 Ohms
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA11
Background
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA12
SLAC’s Timing Systems
In order to explain the New LCLS timing system, we first need to understand how the old SLAC timing system works – i.e. how we got from there to hereThe SLAC Accelerator complex consists of several machines: Linac, Damping Rings, Stanford Linear Collider, PEP-II, FFTB, NLCTA / each with its own timing sub-system►The overall timing system consists of incremental add-
ons to the original system►Design Challenge for LCLS Timing System was that it
had to know about and work with the existing system
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA13
(pre-LCLS) SLAC Accelerator Complex(Lots of Pieces)
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA14
SLAC Linac Timing System
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA15
Old SLAC Timing System
We’ll talk a little about the existing SLAC Linac Timing System:The Linac is a Pulsed Machine (get a packet of beam per pulse) runs at a max of 360HzThree Main Timing Signals:
476MHz Master Accelerator Clock (runs down 2mile Heliax Main Drive Line cable)360Hz Fiducial Trigger (used to ‘tell’ devices when the beam bunch is present) / encoded onto the 476MHz master clock128-Bit PNET (Pattern Network) Digital Broadcast
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA16
Some More Details
Why 360Hz?Original design rate of the Linac / derived from the 3-phase, 60Hz AC power line frequency: want to trigger devices (Klystrons, etc.) consistently so as to not create huge transients on the Power Line Sync’d to 476MHz
PNET BroadcastA special computer called the Master Pattern Generator (triggered by the 360Hz fiducial) broadcasts a 128-bit digital message containing information (conditions, rate, charge, etc.) about the beam This is used by the Trigger Generator computers to set up triggers and their delaysSent over SLAC coax cable TV network
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA17
How The 360Hz is GeneratedThe Sequence Generator
creates a 360Hz signal as well as 6 Timeslot pulses (used for
further synchronization)
Source: SLAC Blue Book
c.1962
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA18
Timing HW at Head-End of Linac
For Phase Stabilization
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA19
The 360Hz Signal is Amplitude Modulated onto the 476MHz Accelerator Clock and propagated down the Main Drive Line.
The AM process is not ideal and some FM occurs; in addition, the signal gets more dispersed as it heads down the 2 mile MDL
Generation of The Linac Timing Signal
(AM Modulator)
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA20
Linac RF Phase Reference DistributionThis slide is to give you an idea of how the Linac Phase Reference is used
Each sector (30 total) taps off the MDL, to extract the RF clock
This is just the Phase Ref, trigger generation is accomplished by a different set of HW
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA21
Old Timing System (CAMAC based) generates triggers by combining the RF Clock, 360Hz Fiducial and PNET Data
Timing CAMAC Crate
PNET Data on Serial Link
476MHz is divided/4 to get 119Mhz + fiducial by another system / This is because the older technology HW could not run at 476MHz
The Programmable Delay Unit (PDU) Module generates the triggers. It contains digital counters that get set with delay values from PNET and started when the Fiducial Pulse comes along
Trigger Generation
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA22
The LCLS Timing System
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA23
Finally – The LCLS Timing System
Old CAMAC System is no longer viable for new Systems (performance limited, obsolete)Seek to implement a new Timing System that has similar functionality, better performance, and can be laid atop the old system, working alongside itIn addition, LCLS has have its own master oscillator (PLL sync’d with Linac MO) and local phase reference distribution system at S20
►LCLS System is VME based, using High-Speed digital serial links to send Clock, Trigger and Data all on one optical Fiber to timing clients
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA24
LCLS RF Front End LCLS Master Osc – slaved to Linac MO / Lower Phase noise req’d by LCLS
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA25
DEV
LCLS Timing/Event System Architecture
~ Linac main drive line
Sync/Div
SLCMPG
PNET
119 MHz 360 Hz
SLCevents
LCLSevents
PNET
P
PDU
EVR
OC
TTL-NIMconvert.
DigitizerLLRFBPMsToroidsCamerasWire Scanner
SLC klystrons
TTL
SLC Trigs
FAN
OC
Low Level RF
EPICS Network
Precision<10 ps
fiber distribution
LCLS Timing System components are in RED
*MicroResearch
*
*
EVG
LCLS MasterOscillator
476MHzLinac
Master Osc
System is based around the EVent Generator and EVent
Receiver
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA26
The Event System
Based on Commercial Hardware (MicroResearch Finland)Which was based on a design from ANL-APS Timing System
Designed Around Xilinx Virtex-II FPGAsUses FPGA’s internal High-Speed 2.38 Gb/s Serial xcvr, which connects to a fiber optical transceiverFPGA logic implements all of the timing functionsUses 119MHz clock from Linac which get multiplied up to by FPGA internal PLL to 2.38GHzEVG sends out 8-bit event code to EVRs along with clock and trigger information over one fiberEVR Receives event code & with its associative memory, generates a trigger with a delay set by digital counters
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA27
The Event System
EVR
EVG EVR
Data Frame:2 Bytes @ 119MHz
8-bitDistributed Databus /
Data Buffer Bus
119MHz RF Clock
360Hz Fiducial
128-bit PNET Pattern
MPS Signals
From SLC Timing System
HWTriggers
PNETEPICS TSMPS Data
EVR
EVR
Event System Architecture
One EVG Per System
[Note: there can be multiple EVGs in one system, arranged in a daisy-chain fashion with priority encoded data tranmission]
Fiber Fanout(s)
8-bit
Eve
nt C
od
eP
NE
TE
PIC
S T
ime
stam
p
Timing Event Code Byte►Upon RX’ing a 360Hz Fid, the EVG
sends out a stream of serial Data to the EVRs over a fiber link. The serial stream consists of 16-bit words sent every 8.4ns.
►Each word contains an Event Code byte and some trigger setup data
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA28
Event Generator ►EVG contains a RAM that gets loaded with event codes, based on PNET data. 360Hz Fiducial causes the RAM to get sent out over the Serial fiber link to the EVRs.
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA29
Event Receiver
►EVR contains another RAM that looks for matches of event codes to its contents. If match, it starts a counter running that generates a trigger
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA30
EVG Hardware
Event Generator
VME-64x Module: Sits in Master Timing Crate with VME PNET receiver and Master Timing CPU.
Receives 119MHz reference and 360Hz master timing fiducial from SLC timing system.
Receives PNET pattern from SLC system.
Broadcasts timing system data in the form of a high-speed 25Gb/s serial data stream to the EVRs over an optical fiber.
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA31
Event ReceiverComes in two flavors: VME and PMC.
Receives 2.5Gb/s serial datastream from EVR and generates triggers based on values of the event codes.
Also receives and stores PNET timing pattern, EPICS timestamp and other data and stores them in an internal data buffer.
Can output 14 total pulsed-output triggers and several more level-type
Triggers are output via a rear transition module (not shown)
Trigger delay, width, duration and polarity are fully programmable
Trigger signal level format is TTL VME Version has 10ps jitter PMC Version has 25ps jitter
EVR Hardware
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA32
System SW
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA33
What Happens during one 2.8mS machine interval:
F3
0
Fiducial
B0
Acq Trigger
1023
BeamKly Standby
Record processing (event, interrupt)
Fiducial Event Received
Event Timestamp, pattern records, and BSA ready
Receive pattern for 3 pulses ahead
Hardware Triggers
5000.3
18100
Triggering Event Codes
StartKly Accel
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA34
Performance
9 ps rms jitter
EVR jitter w.r.t. fiducial
The Event System Trigger Jitter was measured using an Agilent Infinium 54845A Digital Oscilloscope in Jitter Histogram Mode / data was collected for 30 minutes
The EVR output (shown) was measured against the system input trigger (360Hz fiducial)
The Actual jitter performance is much better after subtracting off the intrinsic jitter of the scope:
Actual Jitter:JITTERsystem = [ (JITTERsys_meas)2 – (JITTERscope)2 ]1/2
Event System Jitter:JITTEREVR =[ (9.7472ps)2 – (6.3717ps)2 ]1/2
= 7.3763 ps
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA35
Issues
Current LCLS Timing System has growing pains (integration w/old system, SW, HW)Use of commercial HW doesn’t quite fit our needs / having to modify EVG & EVRSystem scalability / with an eye towards eventually replacing the old SLAC timing system with this one (we think it will work..)Phase Drift on Fiber (where Russell’s Laser System will help us)
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA36
Issues – Fiber Phase Drift
Temperature-Dependent Phase DelayThe Fiber-Optic cable will experience temperature-
induced optical transport and length variations which will affect the timing. What should we expect?
Calculation (warning hand-waving guesstimate!):Delay Coefficient: 30ps/C/Km (source J. Frisch ILC timing notes)
Worst Case (max length / largest temp variation): Length: Sector 20 FEH (~2.2Km)Temperature Variation: 32F – 100F (0C – 40C) (typ
winter/summer)
ps
KmCKmCps
2204
2.240//30
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA37
Temperature-Dependent Phase Delay (cont.) How will this affect the timing system?
Badly, but its not a 1:1 correlation between the trigger timing and the fiber phase delay
Why? Because the fiber contains a 20-bit serial digital data stream (w/embedded clock) that gets recovered and decoded at the receiver. So its not straightforward to understand the effect
Here’s a guess: 20-bit datastream @ 2.38Gb/s 1 bit cell = 420ps With = 2204ps 5 to 6 bit slots get shifted. This will affect
when the Event Code gets de-serialized and loaded in the memory to produce a trigger
The trigger could jump around by 8.4ns (119Mhz clk cycle) increments
> This is one of the LCLS applications that needs the Stabilized Laser Distribution System
John Dusatko
USPAS Fundamentals of Timing & Synchronization [email protected]
January 25, 2008 / Santa Rosa, CA38
End of Talk