Download - EGEE NA4 Lofar

Transcript
Page 1: EGEE NA4 Lofar

EGEE: Lofar

Lofar Information System DesignOmegaCEN

Kapteyn Institute TARGET- Computing Center

University Groningen

Orsay, 10 June 2008

Lofar Information System DesignOmegaCEN

Kapteyn Institute TARGET- Computing Center

University Groningen

Orsay, 10 June 2008

Fokke Dijkstra(most slides by Edwin Valentijn)

Fokke Dijkstra(most slides by Edwin Valentijn)

EGEE NA4 LofarEGEE NA4 Lofar

Page 2: EGEE NA4 Lofar

EGEE: Lofar

LOFAR in shortLOFAR in short• Large distributed radio telescope

– In NL 2008: initial roll out 20 stations – In NL 2009: 18 central (24 Gbps) + 18 remote (2 Gbps)

phased array antenna stations each ~4 soccerfields size– Full scale aperture synthesis array, extends 100 km

• Two main bands– High Band ~ 7,500 tiles 120 – 240 MHz – Low Band ~ 7,500 dipoles 20 – 80 MHz

• Fibre network, Software Correlator• Run as a broad common-user observatory

– Making the transition now from “project” to “observatory”

• Large distributed radio telescope– In NL 2008: initial roll out 20 stations – In NL 2009: 18 central (24 Gbps) + 18 remote (2 Gbps)

phased array antenna stations each ~4 soccerfields size– Full scale aperture synthesis array, extends 100 km

• Two main bands– High Band ~ 7,500 tiles 120 – 240 MHz – Low Band ~ 7,500 dipoles 20 – 80 MHz

• Fibre network, Software Correlator• Run as a broad common-user observatory

– Making the transition now from “project” to “observatory”

Page 3: EGEE NA4 Lofar

E-LOFAR ParticipantsE-LOFAR Participants

• GLOW:-- Effelsberg station delivered-- Garching, Potsdam, Tautenburg stations ordered--Jülich station federal funding proposal submitted

• LOFAR-UK:– Chilbolton, Cambridge, Jodrell, Edinburgh– 1 station funded, site to be chosen--3 more stations

• Sweden:– Onsala station funded

• + Poland, Italy, Austria, Bulgaria, Ireland, France

• GLOW:-- Effelsberg station delivered-- Garching, Potsdam, Tautenburg stations ordered--Jülich station federal funding proposal submitted

• LOFAR-UK:– Chilbolton, Cambridge, Jodrell, Edinburgh– 1 station funded, site to be chosen--3 more stations

• Sweden:– Onsala station funded

• + Poland, Italy, Austria, Bulgaria, Ireland, France

EGEE: Lofar

Page 4: EGEE NA4 Lofar

Wires in the fieldWires in the field

EGEE: Lofar

Page 5: EGEE NA4 Lofar

EGEE: Lofar

Station based processing

Station based processing

Input data rate: ~ 460 Gbps

Output data rate: ~ 2 Gbps

Processing capacity: ~ 1.5 Tmul/s

Storage capacity: 96 Gbyte

–Receiver

–Filter

–Beamformer

– Bu

ffer

–Receiver

–Filter

–Beamformer

– Bu

ffer

–Receiver

–Filter

–Beamformer

– Bu

ffer

–Receiver

–Filter

–Beamformer

– Bu

ffer

–10 GbE

–1 GbE

–Used for 1 beam & station cross correlation

Page 6: EGEE NA4 Lofar

Lofar: Data StreamLofar: Data Stream

Gb

E s

witc

h

Gb

Esw

itch

Gb

Es

witc

hG

bEsw

itch

Gb

Esw

itch

Gb

Esw

itch

Auxiliary processingsection

Sto

rag

e sy

ste

m

Off-line processing/user software

Off-lineStorageOn-Line

Long term Archiving& Data Export

Gb

Esw

itch

On-Line pipeline processingBlue Gene/L

WAN

EGEE: Lofar

Page 7: EGEE NA4 Lofar

EGEE: Lofar

Lofar StorageLofar Storage

Page 8: EGEE NA4 Lofar

EGEE: Lofar

LoWISE designLoWISE design

Page 9: EGEE NA4 Lofar

LoWise- conceptsLoWise- concepts• All Tier 1s on an equal footing• Centrally operated GPFS distributed filesystem• File retrieval

– Db viewer (web)– Prompt (db query)– Pathname – data server– gLite – Grid ftp

• Single distributed db- with all Meta and source data– Valid --Quality -- ready for publication flags

• Visibility of data– Owner –Project –Archive --EURO-VO

• Authentication / Assigned resources– AstroWISE Oracle system

• All Tier 1s on an equal footing• Centrally operated GPFS distributed filesystem• File retrieval

– Db viewer (web)– Prompt (db query)– Pathname – data server– gLite – Grid ftp

• Single distributed db- with all Meta and source data– Valid --Quality -- ready for publication flags

• Visibility of data– Owner –Project –Archive --EURO-VO

• Authentication / Assigned resources– AstroWISE Oracle system

EGEE: Lofar

Page 10: EGEE NA4 Lofar

Processing - optionsProcessing - options

• Tier 0 – CP• Tier 1 – re-processing

– Local clusters– EGEE GRID

• Tier 2– On Tier 1– Local anarchy

• Tier 0 – CP• Tier 1 – re-processing

– Local clusters– EGEE GRID

• Tier 2– On Tier 1– Local anarchy

EGEE: Lofar

Page 11: EGEE NA4 Lofar

EGEE: Lofar

AstroWISE-UpAstroWISE-Up

Page 12: EGEE NA4 Lofar

EURO-VO: GRID and VO

–VST - Virtual Survey Telescope–VST - Virtual Survey Telescope

Page 13: EGEE NA4 Lofar

TARGET diagram TARGET diagram

EGEE: Lofar

Extreme data lineage!

Page 14: EGEE NA4 Lofar

Astro-WISE: ServicesAstro-WISE: Services

EGEE: Lofar

Page 15: EGEE NA4 Lofar

Astro-WISE: Grid Computing

Astro-WISE: Grid Computing

–selection of Target in Astro-Wise

–storage element defined a priori –EGEE computing EGEE: Lofar

Page 16: EGEE NA4 Lofar

LoWise and Euro-VOLoWise and Euro-VO

EGEE: Lofar

Page 17: EGEE NA4 Lofar

NA4 areas of workNA4 areas of work

• Improve integration with EGEE– Authentication

• Use of Grid certificates through whole chain• Portable client tools

– Computing• Portability of software (Python)• Coherent and fast execution of subtasks

– Storage• SRM interfaces to Lofar archive• StoRM ?

• Improve integration with EGEE– Authentication

• Use of Grid certificates through whole chain• Portable client tools

– Computing• Portability of software (Python)• Coherent and fast execution of subtasks

– Storage• SRM interfaces to Lofar archive• StoRM ?

EGEE: Lofar