The Australian Virtual Observatory
description
Transcript of The Australian Virtual Observatory
![Page 1: The Australian Virtual Observatory](https://reader035.fdocuments.net/reader035/viewer/2022062723/56813eca550346895da93438/html5/thumbnails/1.jpg)
The Australian Virtual Observatory
e-Science MeetingSchool of Physics, March 2003
David Barnes
![Page 2: The Australian Virtual Observatory](https://reader035.fdocuments.net/reader035/viewer/2022062723/56813eca550346895da93438/html5/thumbnails/2.jpg)
What is a Virtual Observatory?
• A Virtual Observatory (VO) is a distributed, uniform interface to the data archives of the world’s major astronomical observatories.
• A VO is explored with advanced data mining and visualisation tools which exploit the unified interface to enable cross-correlation and combined processing of distributed and diverse datasets.
• VOs will rely on, and provide motivation for, the development of national and international computational and data grids.
![Page 3: The Australian Virtual Observatory](https://reader035.fdocuments.net/reader035/viewer/2022062723/56813eca550346895da93438/html5/thumbnails/3.jpg)
Scientific motivation• Understanding of astrophysical processes depends
on multi-wavelength observations and input from theoretical models.
• As telescopes and instruments grow in complexity, surveys generate massive databases which require increasing expertise to comprehend.
• Theoretical modeling codes are growing in sophistication to consume available compute time.
• Major advances in astrophysics will be enabled by transparently cross-matching, cross-correlating and inter-processing otherwise disparate data.
![Page 4: The Australian Virtual Observatory](https://reader035.fdocuments.net/reader035/viewer/2022062723/56813eca550346895da93438/html5/thumbnails/4.jpg)
Aus-VO in 2003
• “Phase A” funded AUD 260K by a 2003 ARC grant:– The University of Melbourne– The University of Sydney– CSIRO Australia Telescope National Facility– Anglo-Australian Observatory
• Funded common format on-line archive projects:– HIPASS: HI spectral line and 1.4-GHz continuum survey– SUMSS: 843 MHz continuum survey– ATCA archive: spectral line and radio continuum images– 2dFGRS: optical spectra of >200K southern galaxies
![Page 5: The Australian Virtual Observatory](https://reader035.fdocuments.net/reader035/viewer/2022062723/56813eca550346895da93438/html5/thumbnails/5.jpg)
www.aus-vo.org
![Page 6: The Australian Virtual Observatory](https://reader035.fdocuments.net/reader035/viewer/2022062723/56813eca550346895da93438/html5/thumbnails/6.jpg)
www.aus-vo.org/twiki
![Page 7: The Australian Virtual Observatory](https://reader035.fdocuments.net/reader035/viewer/2022062723/56813eca550346895da93438/html5/thumbnails/7.jpg)
Melbourne
AdelaideCanberra
Sydney
Parkes?
Swinburne
Data CPU?
CPU?
CPU?
CPU?
Data
Data
Data
ATNF/AAO
Theory?
HIPASSGemini?
ATCAMSO
2dFGRSRAVE
SUMSS
CPU
Theory
GrangeNet
... thinking about the Aus-VO Grid, having data nodes and compute nodes...
GrangeNet: Grid and Next Generation Network – a 10 Gbit backbone
APACCPU
VPACCPU
Theory
![Page 8: The Australian Virtual Observatory](https://reader035.fdocuments.net/reader035/viewer/2022062723/56813eca550346895da93438/html5/thumbnails/8.jpg)
VO Interface & Portal
• Agreement with AstroGrid (UK e-Science project) to be testers for their data publication and portal creation code.
• Collecting the necessary resources and intend to have an AstroGrid-based portal serving HIPASS catalogue data for demonstration at IAU General Assembly in July 2003.
![Page 9: The Australian Virtual Observatory](https://reader035.fdocuments.net/reader035/viewer/2022062723/56813eca550346895da93438/html5/thumbnails/9.jpg)
The MACHO Grid!
• MACHO: 8-yr lightcurves for >18 million stars
• ANU, APAC and MSO have the data on mass store, and are working on a VOTable XML description of the data (metadata).
• Agreement with San Diego Supercomputer Center to install a storage resource broker (SRB) at ANU, with a view to making the MACHO data available on an international Grid.
![Page 10: The Australian Virtual Observatory](https://reader035.fdocuments.net/reader035/viewer/2022062723/56813eca550346895da93438/html5/thumbnails/10.jpg)
Grid-based Visualisation• ATNF will build a Java
PixelCanvas so that AIPS++ visualisation applications can be deployed as Web-Service and Grid- Service Java Applets
• AIPS++ is modern, OpenSource software for reducing (radio) astronomy data, 1.6M lines of code.
![Page 11: The Australian Virtual Observatory](https://reader035.fdocuments.net/reader035/viewer/2022062723/56813eca550346895da93438/html5/thumbnails/11.jpg)
Grid-based Volume Rendering• Agreement between Melbourne and AstroGrid to develop our
existing distributed-data volume rendering code into a fully-fledged Grid-Service.
• Challenge is to interactively render a multi-GB cube at the IAU GA 2003, using GridFTP to transfer the data volume from a remote data warehouse to a remote rendering cluster.
Time to render 512x512 view of 1024x1024x1024 volume (seconds)
1
10
100
1000
0 10 20 30 40
number of nodes
![Page 12: The Australian Virtual Observatory](https://reader035.fdocuments.net/reader035/viewer/2022062723/56813eca550346895da93438/html5/thumbnails/12.jpg)
DataGrids for Aus-VO
• Australian archives range from ~10 GB to ~10 TB in processed (reduced) size.
• providing just the processed images and spectra on-line requires a distributed, high-bandwidth network of data servers – that is, a DataGrid.
• users may want some simple operations such as smoothing or filtering, applied at the data server. This is a Virtual DataGrid.
![Page 13: The Australian Virtual Observatory](https://reader035.fdocuments.net/reader035/viewer/2022062723/56813eca550346895da93438/html5/thumbnails/13.jpg)
ComputeGrids for Aus-VO
• More complex operations may be applied requiring significant processing:– source detection and parameterisation– reprocessing of raw or intermediate data
products with new calibration algorithms– combined processing of raw, intermediate or
"final product" data from different archives
• These operations require a distributed, high-bandwidth network of computational nodes – that is, a ComputeGrid.