New Technologies for Future VLBI Correlators B. Carlson High Res. Astronomy, June 8-12, 2003 Socorro...

New Technologies for Future VLBI Correlators

B. Carlson

N ational R esearch C ounci lC anada

C onsei l national de recherchesC anada High Res. Astronomy, June 8-12, 2003

Socorro NM

B. Carlson, 2003-Jun 8-12 Socorro NM

High Res. Astronomy - New Technologies for Future VLBI Correlators

2

Outline

• Key architecture and design issues.

• Review of correlator groups/developments.

• EVLA Correlator for VLBI.

• Design challenges.

• Design methodologies.

• Design tools.

• “Vision for the Future”



3

Key Issues• Architecture: determined by requirements

– BBC configurations: are parallel processing methods required?

– Bandwidth: directly affects digital clock rate, choice of technology, cost.

– Number of antennas: small…large N, may affect flexibility, signal processing choices/tradeoffs.

– Number of spectral channels. Flexibility requirements determine how spectral channels are allocated to bandwidth.



4

Key Issues• Architecture cont’d:

– Dynamic range: affects number of bits used at various stages. Can have a major impact on the choice of architecture.

– Baseline, real-time/fiber delays, recording based: affects delay tracking hardware complexity/cost.

– Flexibility: specific antennas/BBC configurations OR wide range of different antenna handling abilities?

– Number of bits in correlator: can affect where digital filtering (if any) is performed (i.e. at antenna or at correlator).



5

Key Issues

• Architecture cont’d:

– Delay and phase tracking at the antenna: simplifies correlator design; more complex H/W and S/W at antenna; “got one chance to get it right”; some say YES, some say NO.



6

Key Issues• H/W design issues: determined by architecture and speed

requirements– Low, medium, high-speed. Transmission lines? Propagation delays

on circuit boards important? Signal integrity? Power dissipation?

– Station-to-baseline signal distribution: critical at high speeds, easier at medium to low speeds. May or may not be issue.

– H/W or Software correlator: determined by bandwidth, number of antennas, number of channels, duty cycle.

– Delay and phase tracking: dynamic range, precision requirements, drive how these are handled.



7

Key Issues

• S/W design issues: determined by architecture and flexibility requirements.

– “Observing modes”-based design?

– Goal/science-oriented design?

– Data handling performance requirements: how fast does data have to be dumped? How many channels/baseline?



8

Correlator Groups + Recent Developments

• XF Correlators

– MkIV/JIVE/… (Haystack/European cons.) – 1k…4k channels/baseline; 2 bits; 64 MHz clock rate.

– Specifically built for VLBI.



9


• FX Correlators

– ATA (SETI Institute/UC Berkeley) – ~1k chans/baseline; 4 bits; 350 antennas(!) (60k baselines); 100(?) MHz clock.

– ATNF research into ALMA (J. Bunton)

– Japanese ALMA 2GC proposal – ~>10k(?) chans/baseline; 4 bits; 64 antennas; 125(?) MHz clock



10


• FX Correlators cont’d

• (All) use poly-phase FIR/FFT filterbank for superior performance over previous generation (VLBA, Mitaka) FX correlators.

• FX design may limit flexibility in handling BBs, and assignment of channels to multiple spectral regions of interest.

• Rely on re-quantization to 4 bits after FFT to deal with station-to-baseline bandwidth problem…could limit dynamic range depending on how this is done.

• Not particularly built for VLBI, but could handle VLBI with front-end delay/phase compensation.

• Possibly the optimum solution for large-N correlation.



11

• Hybrid Correlators (digital filter…XF correlation)

– EVLA “WIDAR” – NRC Canada – 16k…4M chans/baseline; 4/8-bit; 40+ antennas; high dynamic range (50-60 dB); 256 MHz clock

– U.S. ALMA – NRAO – recently proposed to be upgraded to hybrid capability – up to 32k chans/baseline; 2/4-bit; 64 antennas; 125 MHz clock.

– European ALMA 2GC proposal – ASTRON et. al. – 16k…32k(?) chans/baseline; 2/4-bit; 64 antennas; 125 MHz clock

• (All) perform digital filtering followed by XF correlation.• Various dynamic range capabilities.• Various flexibilities, “ways of doing things”.• EVLA correlator is only one with VLBI capability “out of the box”…U.S. ALMA could with

front-end phase tracking…European ALMA has fundamental ability if so chosen…




12


• Software Correlators– Australia (S. Tingay et. al.) – Swinburne University of Technology

parallel supercomputer: 180, 2 GHz PCs. (XF?); a few antennas; modest bandwidth; low duty cycle.

– NRAO (T. Morgan) – feasibility study using the Cray X1 supercomputer. May have advantages over PC cluster S/W correlators because of “matrix math unit” that allows bit-level operations. FX. Too expensive for EVLA, but for smaller-scale VLBI correlator…?

– LOFAR – ASTRON/NRL/Haystack – FFT (spatial) beamforming or FX correlator?. 32 MHz. Mixture of H/W and S/W elements?



13

EVLA Correlator for VLBI—Overview

• Bandwidth, flexibility, and signal processing method makes it suitable/useable for VLBI.

• Uses VLBI-standard sample rates.

• Can handle antennas with a mixture of bandwidths w/ or w/o delay/phase tracking at antennas.

• Recording-based or real-time VLBI (fiber/VSI interfaces).



14

EVLA Correlator for VLBI—Overview• Sub-band multi-beaming for multi-source observing/in-beam

phase referencing.

• Tradeoff bandwidth for number of antennas/beams w/o rewiring the correlator.

• 16k chans/baseline…4M chans/baseline w/ recirculation.

• 1, 2, 3, 4, 8-bit capability. High-speed, 2x1000 phase bins

• Digital phased-EVLA output.



15

EVLA Correlator—Station Brd Processing

• Flexible BBC handling: 2 BBCs @ 2 GHz each…32 BBCs <=128 MHz each.

• 0.25 second delay buffer using inexpensive SDRAM…complex design.

• All-digital +/- 1/16th sample delay tracking (wideband…narrowband)

• 32 independent digital filters…bandwidth, beam, location in BB. Probably implement in gate array.

• Phase-cal extraction, quantizer statistics.

• 2 Pulsar timers for gating/binning.

• Data path switching.

• VSI input for connection to new-generation VLBI recorders.

• VSI output for recording sub-band data (EVLA Phase-II reqs)…could use board as filterbank board at antenna if desired.



16

EVLA Correlator—Baseline Brd Processing

• 64, 2048 complex-lag correlator chips: 16 cells @ 128 lags/cell; fringe-stopping; high performance readout for recirculation + phase binning.– Corr. chip design complete: 4 Mgates; est 2.7 W PD; est $400k NRE,

$90/chip in 10k qty, $50/chip in 25k qty.

• Recirculation by up to a factor of 256 for up to 256k chans/correlation (4M chans/baseline).

• High performance Long-Term Accumulator (LTA).

• Output UDP/IP frames containing data…for switching to backend.



17

EVLA Correlator Configurations

• NRC will deliver a 32-station correlator (16 GHz/station)– For EVLA Phase-I this leaves 5 “spares”: 5 stations @ 16 GHz; 10

stations at 4 GHz (4k chans/baseline); 20 stations @ 1 GHz (1k chans/baseline).

– This extra capacity could be used for the VLBA (+some control S/W).

• EVLA Phase-II– NRAO will upgrade to 40 stations.

– 27 EVLA + 10 NMA…leaves 3 spares (3 stations…6 stations…12 stations)



18

S.I. S.I. S.I. S.I. S.I. S.I. S.I. S.I.

EVLA

NMA

EVLA EVLA EVLA EVLA EVLA EVLA EVLA


EVLA EVLA EVLA EVLA EVLA EVLA EVLA EVLA


EVLA EVLA EVLA EVLA EVLA EVLA EVLA EVLA

S.I. S.I. S.I.

EVLA EVLA EVLA

S.I.

NMA NMA NMA NMA NMA NMA NMA NMA NMA

VLBA

12 dedicated Mark 5playback units @ 1GHz

each, or 6 playback units @4 GHz each.

VLBA VLBA VLBA VLBA VLBA VLBA VLBA

VLBA-site sub-banding and recording

S.I. S.I.

Up to 24 additional Mark 5playback units up to 4 GHzeach when all/part of EVLA is in4 GHz mode.

10 Mark 5 units (record/playback)

S.I.

VSI

S.I.

VSI

S.I.

VSI

S.I.

VSI

S.I.

VSI

S.I.

VSI

S.I.

VSI

S.I.

VSI

S.I.

VSI

S.I.

VSI

S.I.

VSI

S.I.

VSI

S.I.

VSI

S.I.

VSI

S.I.

VSI

S.I.

VSI

S.I.

VSI

S.I.

VSI

LEGEND:

EVLA

NMA

VLBA

S.I.

S.I.

VSI

S.I.

VSI

EVLA Antenna: 16 GHz tot. BW, 8 BBCs

NMA Antenna: 16 GHz tot. BW, 8 BBCs

VLBA Antenna: up to 4 GHz tot. BW

Correlator “Station Input”: 4 Station Boardseach one with fiber and VSI inputs

Correlator “Station Input": 4 Station Boardswith VSI input and outputs

1 Station Board: used as filterbank with 1 ormore VSI output connectors

Mark 5 recorder c/w VSI interface

1 of 2 optical selection switch (4 GHz BW, 3 fibers)

4 GHz BW fiber connection (3 fibers)

1:2 optical splitter (4 GHz BW, 3 fibers)

EVLA/NMA/VLBA-to-Correlator Connection Architecture

Additional 1 of 2 optical selection switch (4 GHz BW,3 fibers) required if dynamically operating in 8-bitmode, both polarizations

1 or 2 VSI connections



19

Design Challenges

• Speed– Most if not all signals require transmission lines.

– Power dissipation is everything. • can affect choice of technology: FPGA or ASIC + feature size.

• can limit functionality: “what can be put on a chip”.

• Complexity– Astronomers want “everything”.

– Requires increasingly complex H/W functions for performance.

• Moore’s Law yeah, but with increasing speed and complexity, everything gets more difficult for engineers.



20



21

5 n 1 0 n 1 5 n 2 0 n 2 5 n 3 0 n 3 5 n 4 0 n 4 5 n

-0 .5

0 .0

0 .5

1 .0

1 .5

2 .0

2 .5

V o lts



22

Design Methodologies

• To cope with increasing speed and complexity:– HDL/RTL design for chip logic (“code that describes hardware”)

• Incremental code and test…hierarchical design.• Much faster than schematic-based design.• Code can be written in technology-independent fashion to target the

“latest and greatest” chips. Code can be re-used…• “Testbenches” for testing can take as long to write as the logic function

itself!

– Signal integrity and timing analysis through the development cycle.– High performance ($$), constraint-based PAR (Place And Route)

tools both for chips and circuit boards.– Post-PAR simulation, to verify signal integrity and timing margins…

pretty darn sure its going to work before PCB fabricated.



23

Design Tools

• We are using “Mentor Graphics”– FPGA Advantage: complete suite of tools for chip HDL+graphical

hierarchical design, simulation, and “synthesis”.

– “Expedition” PCB tools:• “Signal Vision” for transmission line “what-if” analysis.

• Schematic capture…constraints entry.

• PCB constraint-based PAR: unbelievable capability.

• Post-PAR signal integrity analysis, timing margin analysis.

• Thermal analysis.

– Cost: about $250k for the EVLA correlator team; ~$40k/yr maintenance…already finding it is well worth the cost.



24

“Vision for the Future”• Maximum bandwidth possible.

• Moderate size N.

• ~10k or so channels/baseline.

• Disk-based and eventually fiber-connected VLBI.

• Flexibility, able to “absorb” outputs from variety of antennas (GBT? DSN? EVN? LAR?)

• Delay and phase tracking at the correlator (my choice, my guess).

• EVLA and VLBA merge into one instrument, into one correlator.

• EVLA correlator suited to meet all of these requirements for the foreseeable future…

New Technologies for Future VLBI Correlators B. Carlson High Res. Astronomy, June 8-12, 2003 Socorro...

Documents

Transcript of New Technologies for Future VLBI Correlators B. Carlson High Res. Astronomy, June 8-12, 2003 Socorro...