Post on 04-Jan-2016
description
Xiangming Sun 1PXL Sensor and RDO review – 06/23/2010
STARSTAR
XIANGMING SUN
LAWRENCE BERKELEY NATIONAL LAB
Firmware and Software Architecture for PIXEL
L. Greiner,
H. Matis
T. Stezelberger
M. Szelezniak
C. Vu
H. Wieman
… …
Xiangming Sun 2PXL Sensor and RDO review – 06/23/2010
STARSTAR Outline
• Requirements for event readout mode• Event readout mode• Other firmware functions• Firmware and readout software structure• Summary
Xiangming Sun 3PXL Sensor and RDO review – 06/23/2010
STARSTARChip set
sensor output frequency
Mimostar2 analog 50MHz
Mimostar3 analog 50MHz
Phase1/2 4 LVDS pairs and analog
160MHz
Ultimate 2 LVDS pairs 160MHz
Xiangming Sun 4PXL Sensor and RDO review – 06/23/2010
STARSTAR
Individual sensor test - A single sensor is connected. Probe test - The sensors are put on probe station and the power supply is controlled. Infrastructure Test Board (ITB) test - 10 sensors are connected in parallel, long cables are used. Beam test - 4 sensers are built into a telescope. The trigger is included. Phase-2 sector test - More than 1 sector is connected. The trigger is included. Final version - 40 sensors are connected. Long cables are used. The trigger is included.
Firmware structure is designed to handle all these tests. The set of firmware modules for the various tests will be very similar with only minor task dependant differences.
Test set
Xiangming Sun 5PXL Sensor and RDO review – 06/23/2010
STARSTAR Rrequirement for event readout mode
40 sensors per readout board( Each sensor has 2 LVDS pairs. 80 LVDS pairs in 160MHz).
Sensor digitizes signal and records hit address in on-chip memory, then transfer hit address serially.
Trigger is STAR DAQ1000(about 1KHz).
DAQ DDL assumed throughput is 80MB/s on average. (DDL link rated for 200MB/sec).
requirement :mass terminition board
mother board
xilinx V5 PRO
SIU/DDL
DAQ PC
Xiangming Sun 6PXL Sensor and RDO review – 06/23/2010
STARSTARData is buffered in FPGA
STAR and DAQ policy: 1 trigger per event
DDL peak speed(2Gb/s) is lower than data generation peak speed (12.8Gb/s).
TPC can buffer 8 triggers when DDL is not available. PIXEL should buffer ≥8 triggers
frame1frame1 frame2frame2 frame3frame3
Trigger time Event data
Xiangming Sun 7PXL Sensor and RDO review – 06/23/2010
STARSTAREvent format
7
For sector
header
ender
Data blockFor ladder1
Data blockFor ladder2
Data blockFor ladder3
Data blockFor ladder4
Header token
Trigger info
system status
temperature
Firmware version
Hardware ID
…….10 words reserved
Block start Data length
…Hit address
end token
sensor ID is encoded in four unused bits
Xiangming Sun 8PXL Sensor and RDO review – 06/23/2010
STARSTAR
Ladder#1 Ladder#2 Ladder#3 Ladder#4
Event Builder #1
(head token added)
Event Builder #2
Event Builder #3
Event Builder #4(end token
added)
Event fifo
SIU
Event mode Readout structure for one sector
Xiangming Sun 9PXL Sensor and RDO review – 06/23/2010
STARSTAREvent Builder
Serial to parallel conv Serial to parallel conv… 10 sensors
data memory
Trigger time memory
start frame memory
Builder state machine
To event fifo
circular bufferSensor 1
output1 output2
Xiangming Sun 10PXL Sensor and RDO review – 06/23/2010
STARSTARSerial to parallel converter
Serial to parallel conv Serial to parallel conv
… 10 sensors
datamemory
Serial-to-parallel converter1, convert 160MHz 1 bit data to 10MHz 16bit parallel data.2, add sensor ID into the data.
Data memory scans Serial-to-parallel converter one by one every 10 ns (100MHz) . It starts from sensor 1 and returns to sensor 1 after reaching sensor 10.
From sensor1From sensor1
From sensor2From sensor2
From sensor…From sensor…
From sensor10From sensor10
From sensor…From sensor…
From sensor1From sensor1
Sensor 1
output1 output2
From sensor3From sensor3
Xiangming Sun 11PXL Sensor and RDO review – 06/23/2010
STARSTARCircular buffers
datamemory
Trigger time memory
start frame memory
frame counter when trigger comes
index of start frame memory
index of data memory for the first frame
index of data memory for the second frame
64K words
512 words
512 words
trigger token and buffer full
frame1
frame2
Xiangming Sun 12PXL Sensor and RDO review – 06/23/2010
STARSTARData check out
datamemory
Trigger time memory
start frame memory
frame1
frame2
get trigger timeand start frame index
get buffer index for 2 frames
Check out region where hit address bigger than trigger time for frame1 and smaller for frame2
Release unused memory.
Xiangming Sun 13PXL Sensor and RDO review – 06/23/2010
STARSTARBuffering capability
• 19K words/trigger (684kb).( buffer length in chip is assumed to be 500 words and 150 word for inner and outer ladder from simulation ) . Current FPGA has 3Mb for event mode. XC5VLX330 has 8Mb.
• 256 continous triggers can be stored (data memory may be full already)• bottleneck is DDL. 39MB/s for trigger DAQ1000
device Total room Room used usage
sensor 2000word 500 / 150 25% / 7.5%
FPGA data 8Mb 684kb*2 17%
FPGA trigger 512 word 2word*2 1%
DDL 80MB/s 39MB/s 48%
In normal condition, memory and DDL usage
10 triggers can be buffered when the DDL link is busy.
Xiangming Sun 14PXL Sensor and RDO review – 06/23/2010
STARSTAROther firmware functions
• JTAG function (all sensors)
• Analog full frame readout (Mimostar3 and Phase1)
• Digital full frame readout (Phase1 and Ultimate)
• I2C device control (Phase1 and Ultimate)
• System monitoring (Phase1 and Ultimate)
Xiangming Sun 15PXL Sensor and RDO review – 06/23/2010
STARSTARFirmware structure
15
System control can access every module
DDL/USB PC
sensorsensor
Xiangming Sun 16PXL Sensor and RDO review – 06/23/2010
STARSTAR JTAG function (all sensors)
JTAG is used for initializing the sensor.
JTAG is done by the FPGA in Virtex5 development board.
We reserve 5 FPGA pins as JTAG pins which connect to sensor’s JTAG pin serially. The JTAG data is sent from PC to FPGA SRAM. The JTAG module in FPGA will read the data from FPGA SRAM and transmit the data to sensor by JTAG pins.
sensorsensor
Xiangming Sun 17PXL Sensor and RDO review – 06/23/2010
STARSTARAnalog full frame readout
analog signal is digitized in ADCADS5272
sensor
ADS5272300MHz, IOdelay is used
data is buffered in SRAM
Xiangming Sun 18PXL Sensor and RDO review – 06/23/2010
STARSTAR IO delay for digital data alignment
1, set sensor to pattern mode, a single pulse is sent from sensor for each row.2, scan IO delay setting from 0 to 63. readout data in event mode.3, search for the pulse in data taken. plot the relation between pulse address and IO delay setting.4, find the transition region in above plot. set IO delay by shifting a half clock.
Digital signals pass 7 meters before arriving FPGA.How the digital signals from all sensors are aligned?
Scan steps0 63
NN+1
half clock
pulse
1 clock cycle
Transition region
Iodelay set
Xiangming Sun 19PXL Sensor and RDO review – 06/23/2010
STARSTAR Digital full frame readout
sensorsensor
Xiangming Sun 20PXL Sensor and RDO review – 06/23/2010
STARSTARI2C voltage control and measurement
Xiangming Sun 21PXL Sensor and RDO review – 06/23/2010
STARSTARSystem monitoring
We have a separate data channel to monitor the system in physics run.
This channel includes a information builder module in FPGA, USB data channel and a monitor PC. If something wrong happens, we can easily see it from monitor PC.
sensorsensor
Xiangming Sun 22PXL Sensor and RDO review – 06/23/2010
STARSTAREvent mode
Xiangming Sun 23PXL Sensor and RDO review – 06/23/2010
STARSTARReadout software structure
command: generate commands in windows and linuxusb.exe :upload commands to FEE in windowsdownload_data_block_to_FEE: upload commands to FEE in windows getdata:get frame data in windows and linuxrorc_receive: get event data in linuxIO delay package: tune IOdelay for each digital channelI2C package: control I2C device
Design Goal:portable between Linux and Windowsevery test can be scripted
Xiangming Sun 24PXL Sensor and RDO review – 06/23/2010
STARSTARSystem Control
command Hex file Hex file
usb.exedownload_data_block_to_FEE0x0402fffd0x1d82ff3f0x1502ffcf0x2642ffff0x2642fdff0x2202feff0x0c03fff00x1547ffff0x1547ffff0x1547ffff0x1547ffdf0x0cc7ffff0x0cc7ffff………….
LinuxPC
LinuxPC
rorc_receive getdata
WindowsPC
WindowsPC
Xiangming Sun 25PXL Sensor and RDO review – 06/23/2010
STARSTARSummary
• The design meets the requirements as stated.
• The full set of expected operation modes are addressed.
Xiangming Sun 26PXL Sensor and RDO review – 06/23/2010
STARSTAR
backup
Xiangming Sun 27PXL Sensor and RDO review – 06/23/2010
STARSTARI2C voltage control