D. Seckel

July 29, 2004

IceTop DAQ

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Outline

• II = {tasks for InIce DAQ}

• IT = {tasks for IceTop DAQ}

• A = IT II

• B = IT dt

• Critical = A B

• Status(C)

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

IceCube DAQ Architecture

IceTop DataHandler

II

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

IceTop Science

• Calibration: x, E, (bundles)• Validation*: tag 5% of atmospheric-• Veto: 1 km2 sr, E > 100 TeV• Cosmic ray & particle physics

* In-Ice trigger (IceTop data sub-threshold)

IT

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Event typesIT

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

IceTop Station SchematicIT

To DAQ

IceCubeDrill Hole

HG HG LGLG

Junction box

25 m

Two Ice Tanks

3.6 m2 x 90 cm

Two DOMs: 10” PMTHigh Gain w/station coincidence: 1 p.e. resolLow Gain: 1 resol

10 m

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Data Types & Rates

• Single “Tank hits” (Muons + e > E1 = 30 [tbd] MeV)– status (1), time (5), fit parameters(8) – 14 bytes – 2500 Hz * 14 B = 35 KBps/DOM

• Coincident Tanks: “Station hits” (2 x e > E2 = 30 [tbd] MeV)– Check for local coincidence (two tanks)– Rshowers ~ 50-100 Hz– Runcorrelated coincidence ~ 1.25 Hz (For 2500 Hz, 100 ns) – Mostly simple fits < 1 KBps/DOM

• Waveforms– 95% consistent with impulsive event: FX data only– 5% not impulsive, return compressed waveform: CWF ~ 100 B

• rate 125 Hz + Scaled selection of minimum bias events ~ 10 Hz• 200 Hz * 100 B = 20 KBps/DOM

• Total: 35+20 = 55 < 100

IT

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Requirements

Air Showers threshold 300 TeV Detector threshold and tank spacing Two Tanks in CoincidenceShower Trigger

Energy Resolution for 300 TeV-1 EeV Dynamic Range 1 - 100,000 pe LG+HG DOMsOverlapping dynamic range

Calibration methods HV change for calibration (TBD)Resolve 1 mu LG+HGOptical uniformity of tanks (flashers ?)Calibration operating modes

Pointing Resolution 1 deg @ PeV Time Resolution 10 ns DOM thermal stabilityOptical properties of tankDt in FX

Stable operation for 10 yrs Detector stability Room at bottom end of dynamic rangeTank thermal/mechanical stabilityRedundant DOM operation

Live time > TBD% Sustainable data rates 100 KBps communication per DOMFX on 95% of waveforms/many 2nd req.

Fast hit processing Smart ATWD selection/two thresholds

Surface Tag 5% for in-ice muons Detector threshold and tank spacingIdentify shower vs background Two Tanks in CoincidenceLook back into Data Buffer

Horizontal showers E > 10 (TBD) PeV Projected tank area Horiz. shower triggerKeep muon data 2 GB buffer

Two types of "hits"

Science Requirements Derived Requirements IceTop 's

IT

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

IceTop DAQ components

IceTopData

Handler

IceTopData

Handler

HG DOM

LG DOMTank 1

LG DOM

HG DOMTank 2

Station 1

Station 2

Station 80

DOM Hubs (10)

IceTopData

Handler(IDH)

Shower Trigger

.

.

.

.

GlobalTrigger

InIceDATA

InIceTrig.Gen.

On line

Hor. Sh. Trigger

CommonEvent Builder

DAQControl

MonitoringDOMs (320)

100 kB/s

32 MB/s

10 Hz

Priority/calib. Trig.

IT

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

IceTop DAQ Development

Pre-DAQ Design & Test• FPGA• IDH & Triggers • Review Components• Deliver algorithms/data formats/implementations

IceTop DAQ v1 • Initial Design• Interface with In-Ice FPGA• Implement and bench test• SW – inc. IDH & Triggers • Test at UD-station• Assessment

8-31-2004

• Requirements • Acquire Test Data• Digital design• Design Firmware• Bench test

FP

GA

• Acquire DAQ Simulation• SPE, muon, shower• High Gain, Low Gain• Coincidence logic

UD

T

est}

• 4 months

IceTop DAQ v1.1• Monitoring• Calibration

5-31-2005

7-31-2005

IceTop DAQ v2• Design Review• Build• Test at UD-station• Station test at Pole• Array test at Pole

2-28-2006

In-Ice/IceTop Integrated DAQ• CONUS tests• Tests at Pole• Acceptance Milestone

6-30-2006

PY-03 Pole-DAQ• Requirements• Modify In-Ice DAQ• Test at UD-station• Review (9-1-04)• Install & Test• Deliver

1-31-2005

Iterate

Annually

IT

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

A = IT II

• Hardware

• Logic– DOM

• FPGA• DOMApp

– Counting House• IDH• IceTop Triggers• DOMHub/Global Trigger/Event Builder

• Testing

A

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Hardware

• DOM– Thermal control (see Tilav)– Cables, penetrators, connectors

• DOM components– PMT, Delay, MB, … - all ok– Flasher – ok for yr 1

• Counting House– DOM Hub: configuration changes– IDH/Triggers … Standard components.

Performance/capacity may drive some changes.

A

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

FPGA

• Build on InIce implementation

• Data volume reduction– Feature extraction– Coincidence

• Special modes (e.g. single pe)

• Separate version for Low Gain DOMs– Focus on FADC

A

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

IceTop FPGA

Date Task Resources

3/2002 Proof of concept 2001 test tank

Altera dev. tools

11/2002 Practical Studies String 18 Data

2-5/2004 Digital Design UD: Bench test DOM

UD: Test station

Pole: 03/04 Tanks

5-8/2004 Implementation Altera dev. tools

UD: bench & station

9/2004 -1/2005

Integration w/in-ice FPGA & DOM App

Development: support from DAQ (1.3.3). Testing: at UD

Rate + Bandwidth : Advanced feature recognition

Rate + Livetime 2-level trigger, fast ATWD decision

10 ns + synch. trigger Advanced feature recognition

Projection of muon waveform ontofour basis functions & reconstruction.

Projection of string-18 SPE waveform w/pedestal & reconstruction.

Shower vs Background e+ e– : 2-tank coincidence

A

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

FX algorithm (based on string 18)

• Determine status (ATWD, co-in, trigger, …)• Capture waveform

– Subtract ATWD pedestal– Subtract trigger pedestal– Shift peak (t)– Normalize (A = i wi

2

• Project onto 3-basis functions• Check quality• Return(t0, t, status, ci, A)

– 4,1,1,6,2 - 14 bytes

A

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

FPGA flow I

WaveformBuffer

D1Start

Ping-Pong

AcquireI

AcquireII

Busy

Inc. Counter

Get Time

D2 ?

StartAcquire

Waveformbuffer

Launch ATWD

Wait for ATWD

Read ATWD-0

Stop

Disc Normal CalibD1 AS 0.2 speD2 x AS x1 AS

Launch Coin.

Read ATWD-1

Saturated?

Read ATWD-1

A

Busy Should initiate “BUSY” event, coincidence logic

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

FPGA flow II

CheckCoincidence

FX ?

GetWF

FeatureExtraction

Coin + >AS?

> A

PrioritySend Full

Report

Send FXReport

Send FXReport

Send FullReport

Coin + >AS?

A

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Calibration

• Discriminator

• Pedestals– ATWD (A/B, 0/1/2)– Trigger condition (noise)

• Basis functions built on top of– Single pe– Impulsive event

• ATWD– Baseline: included in pedestal and basis functions– Abs gain: calibrate to spe at ref HV– Rel gain: common signals in ATWD-0/1/2

• Low Gain– Overlapping dynamic range

A

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Reconstruction

• Apply process

• Amplitude from c1

• Time correction from c2, 2= 1’(t)

A

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

IDH & Trigger Functionality

• IceTop Data Handler (from Hubs to …)– Science data

• Timing summaries to triggers

• Respond to Event Builder requests

• Manage IceTop Data Buffer

– Monitor stream– Timing corrections

• Triggers– Shower trigger: 4x station hits– Horizontal shower trigger: N x muons (single tanks)– Priority/Calibration trigger(s).

• Control and service functions

A

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Horz. Shower Trigger

Shower Trigger

Priority/Calibration Trigger

Time Correction (320)

CommonEvent Builder

GlobalTrigger

Hubs

Process Monitor Data (320)

Pre-process Science Data (320)

IceTopData

Buffers(320)

IceTop DB Access

Monitoring

Online

In-Ice Trigger

DAQControl

IDH and Trigger

Time splice (merge)

Sortby type

A

Merged,Time sorted,

event list

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

IDH Requirements

• Pre-trigger– IceCube standard Time Calibration– Throughput capability > 32 MB/s– Time ordered merge – Sorted output streams to triggers

• Switching capability to time slice– Store data

• Buffer– Capacity: 60 sec = 2 GB– Organized for efficient access

• Post-trigger– Query handler for EB access– Buffer management

A

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Trigger Requirements

• Shower trigger– 4-fold: Time coincidence 5 s (TBD) – Reduced threshold mode: space and time

coincidence (desired)

• Horizontal shower trigger– 15-fold (TBD) Time only– 5-fold (TBD) Shower front coincidence (desired)

• Priority (calibration) trigger– Additional processing (TBD)– Define data request for use by EB

A

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Testing

• DOM qualification– Expanded temperature range– Reserve time for IceTop Needs

(TBD: Implement in Year-2)

• DAQ testing– Participate in integration– IceTop specific tests not defined

A

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

B = IT dt

• Verify functionality

• Calibration – CPU triggers– Reduced threshold data– Single muon data

• Showers– 4 station coincidence– Reconstruction– Dynamic range

B

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

“Priority” Data

• Method– DOM: throw status (TBD)– Surface DAQ:

• Catch status (IDH)• Throw IceTop calibration trigger (ICT)

– Online: Catch calibration trigger. Route event to IceTop calibration module

• Types– Diagnostic mode– Pedestal - cpu mode (part of monitoring?)– Gain (change threshold/HV for spe mode)– Flasher (tbd)– External coincidence (tbd)

B

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Single Muons

• Single muons are a key to calibration– Physics of source– Tank response– PMT + electronics

• Difficulties– Variation in trajectory– Shower background

• Filters– Feature recognition (Software Filter)– Shower tag (Firmware/Software - Anti)– External muon trigger (HW - True)

B

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Single candidatesB

3286844099 , Bubble : hv 1475 , thr 550 , raw True , ch 3, evs 1,30

020406080100120150200250300350

020406080100120150200250300350

020406080100120135140145150155160165

020406080100120140145150155160165

020406080100120135140145150155160165

020406080100120140145150155160165

020406080100120140145150155160165

020406080100120140145150155160165

020406080100120140145150155160165

020406080100120140145150155160165

020406080100120135140145150155160165

020406080100120150200250300350400

020406080100120130140150160

020406080100120140145150155160165

020406080100120135140145150155160165

020406080100120150200250300350400450

020406080100120140145150155160165

020406080100120135140145150155160165

020406080100120135140145150155160165

020406080100120100200300400500

020406080100120140160180200

020406080100120135140145150155160165

020406080100120125150175200225250275300

020406080100120135140145150155160165

020406080100120140150160170180190

020406080100120140145150155160165

020406080100120135140145150155160165

020406080100120135140145150155160165

020406080100120140145150155160165

020406080100120140145150155160165

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Muon calibration

• Baseline: InIce DAQ + analysis. Plenty of data.

• Apply coincidence filter to eliminate large showers.

• External “true”. Various possibilities. (Attach to preinstalled after-pulsing LED?) Useful to define geometry. Engineering effort would be required.

B

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Showers: 4 station coincidence

• Coincident data from 4 stations– f = Probability that DOM is awake– tw = Coincidence window (d/c)– Livetime:

= t (2 f – f2)4

– Coincidence rates True: t = 10 Hz (t) (Ethr/0.3 PeV)-2 (Area/km2) False: f = (t)tw)3

• Use of Baseline InIce DAQ wo/coin f = 0.1, t = 10-3, f1/yr (tw = 1 s), t10/day

• Options– Use local coincidence– Synchronize livetime

• coordinated enable/disable: t = f– Raise threshold to 2

• Estimated shower threshold 5 Pev, rate 5/hr

B

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Year-1 Summary

• Goals– Capture coincident IceTop data within InIce triggers– Single muon calibration– Shower reconstruction

• IDH/Shower trigger/Priority trigger - V1.0 needed

• InIce FPGA should be adequate– Main concern: loss of livetime– FX not critical for single muons– Coincidence desireable but

Acceptable shower rates with raised threshold– Need support for priority trigger

• Horizontal shower trigger not critical

B

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Current Activity

• IDH/triggers– IDH/Splicer performance tests– Development of IDH/trigger templates

for integration (Aug)

• FPGA– FX algorithm definition – Data format definition (Aug)

• Dynamic range studies

C

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Gallery: single pe eventC

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Gallery: ATWD-0 waveforms

spe mu

C

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Gallery: Collection of spe-sC

gi

bi

ti

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Gallery: arrival times

(7+) x 3.3 = 25 ns

C

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Gallery: Baseline jitter

Magnitude of sample 100

C

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Gallery: ATWD0 ped/spe

20 40 60 80 100 120

180

185

190

195

200

205

20 40 60 80 100 120-0.1

00.10.20.30.40.50.6

C

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Gallery: ensemble stats

-0.5 0 0.5 1tpk

510152025

count

-5 0 5 10b

510152025

count

0 20406080100120sample

-40-20

020

fw

50100150200250300350g

51015202530

count

C

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Gallery: ATWD2-spe

0 20 40 60 80 100 120

-4

-2

0

2

C

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Gallery: ATWD2 ped/spe?C

20 40 60 80100120

140145150155160165

20 40 60 80100120-0.6-0.4-0.2

00.20.4

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Gallery: g0 vs g2

C

0 50 100 150 200 250 300 350g0

0

5

10

15

20g 2

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

IDH/Splicer efforts

• 1 sec FX sim data– 320 DOMs - 2000 events/channel– 13 bytes/event: 6 time, status, 6 data– No non-FX events. No time calibration/monitoring– Separate stream for each DOM– Output analysis is trivialized

• Results– 11.6 sec (2.4 GHz P4)– 6.5 (3 GHz P4+, RH enterprise)– 3.5 “optimized” splicer

• Optimizations– Reduce object creation/garbage collection– Eliminate objects in favor of primitive data-types– Reduce data handling: Adjust references/leave buffer static

C

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Report on IDH/Trigger Templates

• Reviewing string processor/state machine template

• Adapt for IDH & IceTop trigger(s).

• Prepare for “Data-flow” DAQ – Aug/Sept.

C

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Dynamic range

• IDD plan– 1 … ~105 pe– HG+LG DOMs w/overlapping range

• “Requirements” – Resolution of 1 pe 1:30 ?– Necessary range 105 pe is aggressive– Overlap region include 1 ?

• Effective range of DOM ?– Linearity few V– Noise < mV ?– ATWD .125 mV – 8V

C

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Summary slide

• IceTop DAQ: InIce +– FPGA designed for reduced data volume

• Feature extraction• Modified coincidence logic• Support for “priority data”

– Counting house• IceTop Data Handler• IceTop Triggers

• Year 1:– Goals

• Capture coincident IceTop data within InIce triggers• Single muon calibration• Shower reconstruction

– Development• IDH/Shower trigger/Priority trigger V1.0 needed• FPGA changes probably not critical

S

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

End

• Spare slides follow

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Thermal issues

• 1 slide with discussion

• 1 slide with data

A

content

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

Common Requirements

• Conform to In-Ice Standards– Control (state machine)– Monitoring & Logging– Data flow architecture

• Streams• Heartbeats• Latency requirements

• Multi-processor capable (TBD)

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

IceTop DAQ effort

• Support from LBNL/UWisc– Build on IceCube SW engineering

• DOM-FPGA FX/coincidence– UD faculty effort– Lab development bench– Data from SP/UD stations

• IDH/Trigger – UD faculty– 1 CIS grad student

D. Seckel

IceTopReview

DAQ

July 292004

Delaware

IceTop Station Schematic

Two Ice Tanks

3.6 m2 x 90 cm

Two DOMs: 10” PMTHigh Gain w/station coincidence: 1 p.e. resolLow Gain: 1 resol

To DAQ

IceCubeDrill Hole

10 m

HG HG LGLG

IT