Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

20
Brent Carlson EVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator

Transcript of Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Page 1: Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Brent Carlson EVLA System PDR (Correlator V2)December 4-5, 2001

1

Correlator

Page 2: Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Brent Carlson EVLA System PDR (Correlator V2)December 4-5, 2001

2

Outline

• Requirements

• Architecture

• Technology

• Software

• Budget

• Schedule

• Installation

Page 3: Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Brent Carlson EVLA System PDR (Correlator V2)December 4-5, 2001

3

Requirements

• 16 GHz bandwidth (8 x 2 GHz bands).

• 16,384 spectral channels/baseline (wideband), 0.25 million (narrower w/recirculation).

• 16 independently tunable digital sub-bands/baseband + N.B. radar filter.

• Flexible: tradeoff B.W. for freq. channels.

• 2 banks of 1000 phase bins/baseline.

• High performance, flexible dumping.

• Very long baseline capable (>10k km baselines).

• 1/16th sample digital delay tracking.

• Baseband and sub-band multi-beaming…on the same data.

• Simultaneous 1 GHz B.W. phased output on multiple sub-arrays.

Page 4: Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Brent Carlson EVLA System PDR (Correlator V2)December 4-5, 2001

4

RequirementsDescription “Dream” Correlator Spec. Planned deliverable

No. of antennas 36 32, expandableBandwidth 4 x 2 x 2 GHz (16 GHz) 4 x 2 x 2 GHz (16 GHz)Freq. Resolution few Hz ... 10’s MHz 1 Hz ... 2 MHzNo. of independently tunable IF pairs at least 4, prefer 8 64, with digital sub-bandsNo. of frequency channels 1000 (full polarization per IF pair),

8000 total1024 W.B. (more w. recirc)

16384 W.B. total (more w. recirc)Frequency channel flexibility split flexibly among IFs,

select subset for writing

split flexibly among IFs and sub-bands,

select subset for writingFlexibility Frequency resolution: factors of 2

Flexible tradeoffs (#baselines, B.W.,#channels, pol’n, time res’n)

Interf. sub-arrays: 4 independentPhased sub-arrays: 4 independent

OkCan’t tradeoff baselines

at full bandwidthInterf. sub-array: unlimited.

Phased sub-arrays: 5Integration times 0.1 sec (less with tradeoffs) 0.011 sec (less with tradeoffs)Total data rates few tens of Mvis/sec several Gvis/secAutocorrelations all stokes parameters W.B. all stokes, SNR loss

S.B. all stokes, no SNR lossRFI as many channels as possible

106 dynamic range

automatic flagging

gating

Ok: 16,384...262,1444-bit sampling standard, up to 8-

bit sampling avail (d.r. depends onnoise+RFI)

post-corr. interference excision +facilitates post-corr. cancellation.

from antenna, external signalPulsar phase binning up to 1000 2 x 1000, min 15 sec eachPhase Cal at least auto-spectra minimum 1 pCal extractor/IFDelay tracking 1/16th sample, 250 km baseline digital 1/32nd sample, 104 km+ bl

Page 5: Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Brent Carlson EVLA System PDR (Correlator V2)December 4-5, 2001

5

Architecture

• FIR filter banks followed by complex XF correlator:– “Stitch” sub-bands together after correlation to yield wideband cross-corr.

– Use small LO offsets in antenna to keep fringe rotators “wet”: fringe stopping, anti-aliasing, artifact decorrelation, digital sub-sample delay tracking, VLBI.

– Each poly-phase FIR independently programmable for flexibility…scientific req’t.

• Three main modules:

– Station Board (2 x 2 GHz).

– Baseline Board (64 baselines ea).

– Phasing Board (48 stations, 2 sub-bands, 5 sub-arrays).

• Plus some 4 small interconnect modules…expandable, flexible.

Page 6: Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Brent Carlson EVLA System PDR (Correlator V2)December 4-5, 2001

6

Architecture

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MDR-80

MDR-80

MDR-80

MDR-80

MDR-80

MDR-80

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

MD

R-8

0

4-S tationC om plex Mixer

+ 1st S tage

Adder

48-S

tati

on

Dat

a E

ntr

y C

on

nec

tors

-48 VDC to1.8V, 2.5VPOWERSUPPLY

Po

we

rC

on

ne

cto

r

MC BInterfaceModule

Fibre W DMDem odulators

Sub-band Distributor Backplane

BaselineEntryBackplane

StationDataFanoutBoard

PhasingBoardEntryBackplane

Network Switches

Host/Control

Com puters;Archiving

Antennas:R eceivers,Sam plers,F ibre W D M

FOTS

PC/CPCI

TIMECODEGeneratorBox (TGB)

R eference C lock

R eference Tim e Tick

TIMECODE(s) + CLOCK(s)

Finalsynchronization

and VLBI recorderinterface

VLBIRecorder(s)

Phased outputfeedback

synchronization

To phased-VLAStation Board

input(s)

B aseline B oard

PC/CPCI

PC/CPCI

-48 V D C to1.8V , 2.5VPO W ERSU PPLY

MC BInterfaceModuleE

the

rne

t

Fibre-OpticReceiverM odule

DELAY

DELAY

FIRFilterBank

FIRFilterBank

Sw

itch

Sw

itch

Tim ing

S tation Board

Retim ing/Drivers

8 'X

' Rec

ircu

latio

n C

ontr

olle

rs

8 'Y ' Recirculation Controllers

8 x 8Correlator Chip

and LTA ControllerMatrix

MC BInterfaceModuleE

the

rne

t

-48 V D C to1.8V , 2.5VPO W ERSU PPLY

SERIALI/F(s)

4-S tationC om plex Mixer

+ 1st S tage

Adder

4-S tationC om plex Mixer

+ 1st S tage

Adder

4-S tationC om plex Mixer

+ 1st S tage

Adder

4-S tationC om plex Mixer

+ 1st S tage

Adder

4-S tationC om plex Mixer

+ 1st S tage

Adder

4-S tationC om plex Mixer

+ 1st S tage

Adder

4-S tationC om plex Mixer

+ 1st S tage

Adder

4-S tationC om plex Mixer

+ 1st S tage

Adder

4-S tationC om plex Mixer

+ 1st S tage

Adder

(5)Sub-array2nd-Stage

Adders

FIRFilterBank

Sw

itch

P has ing B oard

M aster S lave 1 S lave 2 S lave 3

W idebandAutocorrelator

200

pin

type

E20

0 pi

n ty

pe E

200

pin

type

E20

0 pi

n ty

pe E

200

pin

type

E20

0 pi

n ty

pe E

4-S tationC om plex Mixer

+ 1st S tage

Adder

4-S tationC om plex Mixer

+ 1st S tage

Adder

PC/CPCI

Page 7: Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Brent Carlson EVLA System PDR (Correlator V2)December 4-5, 2001

7

Architecture

-48 V DC to1.8V , 2.5VPOW ERSUPPLY

MC BInterfaceModuleE

the

rne

t

Fibre-OpticReceiverModule

DELAY

DELAY

FIRFilterBank

FIRFilterBank

Sw

itch

Sw

itch

Tim ing

S tation Board

W idebandAutocorrelator

Page 8: Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Brent Carlson EVLA System PDR (Correlator V2)December 4-5, 2001

8

Architecture

B aseline B oard

8 'X

' Rec

ircu

latio

n C

on

tro

llers

8 'Y ' Recirculation Controllers

8 x 8Correlator Chip

and LTA ControllerMatrix

MC BInterfaceModuleE

the

rne

t

-48 V DC to1.8V , 2.5VPOW ERSUPPLY

SERIALI/F(s)

Page 9: Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Brent Carlson EVLA System PDR (Correlator V2)December 4-5, 2001

9

Architecture

4-StationC omplex Mixer

+ 1st S tage

Adder

48-S

tati

on

Dat

a E

ntr

y C

on

nec

tors

-48 VDC to1.8V, 2.5VPOWERSUPPLY

Po

we

rC

on

nec

tor

MC BInterfaceModule

4-StationC omplex Mixer

+ 1st S tage

Adder

4-StationC omplex Mixer

+ 1st S tage

Adder

4-StationC omplex Mixer

+ 1st S tage

Adder

4-StationC omplex Mixer

+ 1st S tage

Adder

4-StationC omplex Mixer

+ 1st S tage

Adder

4-StationC omplex Mixer

+ 1st S tage

Adder

4-StationC omplex Mixer

+ 1st S tage

Adder

4-StationC omplex Mixer

+ 1st S tage

Adder

4-StationC omplex Mixer

+ 1st S tage

Adder

(5)Sub-array2nd-Stage

Adders

FIRFilterBank

Sw

itch

P has ing B oard

4-StationC omplex Mixer

+ 1st S tage

Adder

4-StationC omplex Mixer

+ 1st S tage

Adder

Page 10: Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Brent Carlson EVLA System PDR (Correlator V2)December 4-5, 2001

10

Architecture

• Future upgrade possibilities:– “maxed-out” on bandwidth.– replace Baseline Board with “Moore” lags…

(more channels and/or more bandwidth in bandwidth/number of antennas tradeoff.)

– keep existing software and hardware infrastructure…painless upgrade.

Page 11: Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Brent Carlson EVLA System PDR (Correlator V2)December 4-5, 2001

11

Technology

• 256 MHz system clock rate:– FPGA and gate array tech. supports this.

– Development tools (Mentor/Cadence) well-equipped for this speed/complexity.

– Can de-scope to 128 MHz on PCB (with 256 MHz interconnects) if absolutely necessary.

• FIR filter:– Prototype in FPGA (power, $$). Convert to 0.18 m gate array (AMIS).

Should be able to get 1024 taps. ($200k NRE, $50 ea, 10k qty). Claim that they use 1/5th the number of gates for same function as Xilinx.

• 2048-tap power estimate: 20 nW/MHz/gate * 300k gates * 256 MHz * 1.0 (switching fraction) = 1.5W

Page 12: Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Brent Carlson EVLA System PDR (Correlator V2)December 4-5, 2001

12

Technology

• Correlator chip:– Original plan: develop full-custom 0.18 m standard cell from

scratch ($900k NRE + $700k production).

– Current plan: prototype with scaled-down (fewer lags) FPGA, convert to gate array or standard cell afterwards. Pushes back technology freeze to latest possible date to take advantage of improvements…

• Power: 20 nW/MHz/gate * 0.5 million (hi-speed switching) gates * 256 MHz * 0.75 transition fraction = 1.9W (2.5W with 1.0 transition fraction)

Page 13: Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Brent Carlson EVLA System PDR (Correlator V2)December 4-5, 2001

13

Software

• Use hierarchical approach (mirrors AMCS):– 1 SCC and 1 BCC (perhaps one platform).– Use NRAO MIBs on boards.

• Backend software/configs…

Page 14: Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Brent Carlson EVLA System PDR (Correlator V2)December 4-5, 2001

14

Software

PhB B0

S B 0 S B 1 S B 2 SB 15 S BCa l S Bra d

P hB B1

P hB B2

P hB B3

P hB B4

P hB B5

P hB B6

PhB B7

slo t_ facto rs lo t_num ber

B B str(a ...b )

S B _LOfshiftS RC _coord

B B _wid th

fi lte r_parm s

b inning_parm sint_ tim e

po la riza tiongatingpulsar_parm sphasing_parm s

de lay_parm s

RF I_contro l

da ta_parm s

P C al_parm s

sta ts_parm s

copy_contro l

autocorr_parm s

S RC _coord

array_center

array_center

antenna_IDantenna_coords

B B str[16 ](a ...b )

wb_autocorr_parm swb_sta ts_parm s

B B LO[16]B B sideband[16 ]

S ub-band C ontro lP aram eters

P hysical B aseB andC ontro l P aram eters

A "station input": shares T IM E C O D E , D U M P TR IG etc .

Station Board Station Board Station Board Station Board

Noise-ca lfi lte r

Radarfi lte r

B B b its[16 ]B B cod ing [16 ]

requant_parm s

antenna_coords

recirc_parm s

S ub-bandfilte rs (16)

num _spec_chansnum _po l

Ind ic a tes w ha t w e w an tthe c o rre la to r to do w ithth is s ub-band , bu t no t howit is done .

physica l_S B out

Page 15: Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Brent Carlson EVLA System PDR (Correlator V2)December 4-5, 2001

15

Software

SBC0

SBC1

SBC2

SBC15

C C C 0 C C C 1 C C C 2 C C C 3

CC

Q#1

C C C 12 C C C 13 C C C 14 C C C 15

CC

Q#4

X -inputY-input

Sub-bandCorrelators

B aseline C orre lator U n it (C orre la tor C h ip )

sta rt_ lagend_ lag

X /Y_dum p_se lect

X -inputY-input

sta rt_ lagend_ lag

X /Y_dum p_se lect

X-input, Y -input: P hB B 0...P hB B 7 , o r "ad jacent"

start_lag, end_lag : full range o f lags this C C C acquires.

X/Y_dum p_select : X o r Y

CCC0128-com p lex lag "C orre la to rC hip C e ll" .

S BC0 S ub-band co rre la to r.

N O TE S :

Page 16: Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Brent Carlson EVLA System PDR (Correlator V2)December 4-5, 2001

16

Software

LocalResourceManager

LocalS tation

R esourceTable

ConfigurationManager

ErrorMessageManager

ModelGenerator

DataManager

Tim eManager

Low-Lev elStatus and

DebugManager

FilterC oeffic ientG eneratorProgram

ISRModelServ erTask

Mo d e l q ue ue

ConfigurationTask

DataHandler

Task

D ata q ue ue

FPGA BootServ erTask

DataReadout

Task

ErrorMessage

Task

Tim eClientTask

S/WMonitor

Task

Low-Lev el

Status andDebugTask

StationBoard

MonitorTask

Delay& PhaseControl

Task

DUMPTRIGControl

Task

C o nf ig uratio nq ue ue

C onfigurationTable

H /W

H /W

H /W

H /W

H /W

H /W

ConfigurationServ er

H /W

OSKernel

Station Board MIB (128)Station Control Computer (1)VCI-SCC

TGB MIB

1 0 m sINT

S E M(d atare ad y)

d ata

d ata

d atad ata

E rro rm e s s .

E rro rm e s s .

E rro rm e s s . tim e

c o nf igd ata

c o nf igd ata

c o nf igd atac o nf ig

d ata

c o nf igd ata

c o nf igd ata

c o nf igd ata

tim e

E rro rm e s s . m o d e l

re q .

c o nf igd ata

c o nf igd ata

line arm o d e ls

line arm o d e ls

c o nf igd ata

m o d e l p o lyno m ials

ac k.

E rro rm e s s .

E rro rm e s s .

trig g e rre q .

S E M(trig g e r)

m o d e lre q .

m o d e l+d e laym e as .

re q .m e as .

d e lay m e as .

E rro rm e s s .

vo lt + te m p +s tatus re advo lt+ te m p

m e s s ag e s

E rro rm e s s .

E rro rm e s s .

E rro rm e s s .

E rro rm e s s .

tas ks tatus

tas ks tatus

write

re ad

b o o td ata

ac k.

tim e

c o nf ig re q + d ata

ac k.tim e

tim e s e ttingre q ./ac k.

s tatus /func tio nre q .

ac k, d ata

E rro rm e s s .

F P G A b o o t re q + d ata

ac k.

m o d e l g e nre q ue s t

ac k.

array/o b s e rvatio ntim e

F IT S d ata f rag m e nts

re s o urc ed ata

re s o urc ed ata

E rro rm e s s .

s tatus /func tio n re q ue s t

ac k, d ata

Correlator Station Control Data Flow Diagram

wid e b andauto c o rre latio ns

re s ync

re s o urc e d ata

ac k.

c o nf igre q ue s t

ac k.

H /W

re q uantize rthre s ho ld s

BaselineProcessing

PCs

s tatec nts

Page 17: Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Brent Carlson EVLA System PDR (Correlator V2)December 4-5, 2001

17

Budget

EVLA Correlator, 32 StationsTotal Cost Breakdown ($12.17 million)

NRE+Miscl Dev.19% ($1.9M)

Contingencies13% ($1.64M)

Labour19% ($2.4M)

System Comps14% ($1.8M)

H/W Modules35% ($4.5M)

Could shave ~$1.3 million off this budget with cheaper cables and AMIS gate array for FIR (& corr chip).

Page 18: Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Brent Carlson EVLA System PDR (Correlator V2)December 4-5, 2001

18

Schedule

Date MilestoneNov. 2/2001 Conceptual Design Review (CoDR). Design frozen.Q1, 2002 New personnel in place. Design tools in place. Training and design work

begins.Q1, 2004 Critical User Manuals in place. Device driver code can be written.Q2, 2004 Preliminary Design Review (PDR). Designs ready for prototype fabrication.Q1, 2005 Prototype test at the VLA starts.Q2, 2005 Prototype test at the VLA complete.Q3, 2005 Critical design review (CDR). Prototype testing complete. Ready for

procurement of production components and full production.Q2, 2006 Production model test and burn-in, system integration and test in Penticton, and

rack and cable installation begins at the VLA.Q4, 2006 Begin full installation at the VLA. Earliest possible start of installed correlator

testing.Q2, 2007 Earliest possible “beta” science data. (Middle of full installation schedule.)Q1, 2008 Correlator commissioning. Correlator fully on-line for observing. Continuing

debug support available.Q1, 2009 End of project. End of NRC debug support. Full handover to NRAO complete.

Page 19: Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Brent Carlson EVLA System PDR (Correlator V2)December 4-5, 2001

19

Installation

Station Racks (12)

48 VDC P lant

48-Station Rack Layout: 1 Floor; 2 sub-racks per 7 ft rack (Nov. 20/2001)

3 Racks =2 sub-bandcorrelators

44 ft

44.5 ft

Air

Con

ditio

ner

Baseline Racks (24)

Air

Con

ditio

ner

Air

Con

ditio

ner

Air

Con

ditio

ner

Max cable length=36 ft (11 m)

Page 20: Brent CarlsonEVLA System PDR (Correlator V2) December 4-5, 2001 1 Correlator.

Brent Carlson EVLA System PDR (Correlator V2)December 4-5, 2001

20

Summary

• Wideband, high-performance, flexible.

• Expandable, re-configurable.

• Painless upgrade path.

• (010k+ km baselines).

• $10 - $12 million (32 stations; another ~$6 million for 48 stations).

• Start installation ~2006.

• First “beta” science ~2007.

• Completion 2008-2009.