AstroGrid (UK Space Conference 2008)
-
Upload
a-rocketeer -
Category
Technology
-
view
941 -
download
0
Transcript of AstroGrid (UK Space Conference 2008)
Jonathan Tedds
UK Space Conf Mar 08
Jonathan Tedds
University of Leicester/AstroGrid/EuroVO DCA science team
Ground Based Observatories
Copyright RASC Calgary Centre
Jonathan Tedds
UK Space Conf Mar 08
Space Observatories
Copyright NASA
Copyright ESA
Copyright ESA/D. Ducros
Future Skymapping Projects
LSST
PanStarrs
VST
WISEVISTA
The need for a VO
� Data Rate� all sky/1-band/16-bit/0.1"
==> 100TB
� Today : 20 TB/yr
� 2008 : 100TB/yr (VISTA)
Jonathan Tedds
UK Space Conf Mar 08
2008 : 100TB/yr (VISTA)
� 2015 : 5PB/yr (LSST)
� 2020 : 100PB/yr (SKA)
� All-sky monitors….WASP, PanStarrs, LSST
� We will have Petabytes of data by 2010
Data Archives
Jonathan Tedds
UK Space Conf Mar 08
Jonathan Tedds
UK Space Conf Mar 08
Jonathan Tedds
UK Space Conf Mar 08
Jonathan Tedds
UK Space Conf Mar 08
What are Virtual Observatories (VOs)?
“The VO vision can be summed up as the desire to make all archives speak the
same language – all searchable and analyzable by the same tools, all data
sources accessible through a common interface, all data held in distributed
databases that appear as one.”
Jonathan Tedds
UK Space Conf Mar 08
September 2003
Professor Andy Lawrence (AstroGrid)
Institute for Astronomy
Royal Observatory Edinburgh
Creating a Virtual Observatory - An
International Effort
The International Virtual Observatory Alliance
"facilitate the international coordination and
Jonathan Tedds
UK Space Conf Mar 08
"facilitate the international coordination and
collaboration necessary for the development and
deployment of the tools, systems and organizational
structures necessary to enable the international
utilization of astronomical archives as an integrated
and interoperating virtual observatory."
As a result of the IVOA we have
� A common Resource Metadata
� A common data querying language� Astronomical Data Query Language (ADQL)
Jonathan Tedds
UK Space Conf Mar 08
� Astronomical Data Query Language (ADQL)
� Based on Structured Query Language (SQL)
� Common table format� Virtual Observatory Table (.vot)
� …and much more in the pipeline
Advantages of VO data access
� Accessing multiwavelength data easier:
� Do not need to access different interfaces, register once only
Jonathan Tedds
UK Space Conf Mar 08
Advantages of VO data access
� Accessing multiwavelength data easier:
� Do not need to access different interfaces, register once only
� Data access from one single entry point
Able to build workflows i.e. pieces of code which run on
Jonathan Tedds
UK Space Conf Mar 08
� Able to build workflows i.e. pieces of code which run on the server and are reusable
http://www.astrogrid.org (April 2008 1st public release)
Jonathan Tedds
UK Space Conf Mar 08
VO Explorer:
� AstroGrid front end, part of VOdesktop
� Resource-centric
� Select a search-space
� Search for resources
� Filter these resources
View selected resources
Jonathan Tedds
UK Space Conf Mar 08
� View selected resources
� Use the selection
� Invoke it
� Save/Bookmark/Tag it
� Export it
Access existing X-ray cluster catalogues
� Browse through keyword metadata
Jonathan Tedds
UK Space Conf Mar 08
� Browse through keyword metadata
Find
Resources!
�� If data in a simple DB can If data in a simple DB can publish to VO in ~day!publish to VO in ~day!
�� We can help you: June 23We can help you: June 23--27, 27, 2008 2008 data publishingdata publishing workshop workshop at at ESO, ESO, GarchingGarching
content +resources
�� For catalogues, imagesFor catalogues, images, spectra, , spectra, transient & theoretical archivestransient & theoretical archives
�� CensusCensus of European Data of European Data Centres & ArchivesCentres & Archives
�� I am UK contact: I am UK contact: [email protected]@star.le.ac.uk
technicalsystem
operations +support
Description
Filter
Query on position
Publishing VOEvent Data
� First point of VG
� Subpoint 1
� Subpoint 2
� Sub sub point 2.1
� Second point of VG
� Etc…
Advantages of VO data access
� Accessing multiwavelength data easier:
� Do not need to access different interfaces, register once only
� Data access from one single entry point
Able to build workflows i.e. pieces of code which run on
Jonathan Tedds
UK Space Conf Mar 08
� Able to build workflows i.e. pieces of code which run on the server and are reusable
� Tools to analyse the returned data
� Different tools which share standards on how to interoperate
•Graphical tool
•Direct SSAP queries to VO or receive from VO via PLASTIC
•Display, compare, modify and analyse spectra
•Read in, display & match multiple spectra in multiple formats
•NDF, FITS, TEXT, NDX…
••Spectral coordinate transforms between multiple input spectra as Spectral coordinate transforms between multiple input spectra as
described in FITS WCS Paper III (described in FITS WCS Paper III (GreisanGreisan+)+)
••WavelengthWavelength
••FrequencyFrequency
••EnergyEnergy
••VelocityVelocity
••Data value unit conversion as per FITS WCS Paper I (Data value unit conversion as per FITS WCS Paper I (GreisanGreisan+)+)
••Flux conversion per unit frequency to/from per unit wavelengthFlux conversion per unit frequency to/from per unit wavelength
••Convert between standards of restConvert between standards of rest
••TopocentricTopocentric, heliocentric, dynamic & kinematic local, heliocentric, dynamic & kinematic local
=> need for IVOA data standards & interoperabilityneed for IVOA data standards & interoperability
More advanced query: Access all extended 2XMM
X-ray sources
� Make own X-ray cluster candidate catalogue
� Filter large multiwavelength archives remotely:
Jonathan Tedds
UK Space Conf Mar 08
� Filter large multiwavelength archives remotely:
� Create manageable science subsets
� 247K detections → 192K unique sources
� 520 sq.deg. / 330 sq.degs excluding overlap
� Full catalogue product set available
Aug07Aug07: THE 2XMM X: THE 2XMM X--RAY CATALOGUERAY CATALOGUE
Jonathan Tedds
UK Space Conf Mar 08
� Full catalogue product set available
� World’s largest ever X-ray catalogue
Find
Columns
Filter columns
Use
Query
Help!
Help!
Correlation with SDSS
Jonathan Tedds
UK Space Conf Mar 08
Spectroscopic z = 0.223 (Lamer+, in prep)
Command Line Scripting� Connect to the AstroGrid system from your own Python scripts
(IDL+ to come)
#!/usr/bin/python# Given an input list od objects, queries the 2MASS catalogue returning a # VOTable for each object.
from astrogrid import ConeSearchfrom astrogrid.utils import broadcast, read_votable
# Read input list of ra, decs from VOTablevot=read_votable('sample.vot')id = vot['id']
Jonathan Tedds
UK Space Conf Mar 08
id = vot['id']ra = map(float, vot['ra'])dec = map(float, vot['dec'])
# We are going to query 2MASS at RoEivorn = 'ivo://wfau.roe.ac.uk/twomass-dsa/cone'cone = ConeSearch(ivorn)
nsrc = len(ra) # Number of sourcesradius=20./3600.0 # Search radius in degrees
print 'Starting Query: %d objects' % nsrc
for i in range(nsrc):res = cone.execute(ra[i], dec[i], radius)# Uncomment the following line if you want to send the output to TOPCAT# broadcast(res)print 'Writing 2mass_%s.vot' % id[i]open('2mass_%s.vot' % id[i], 'w').write(res)
Ast
roG
rid
Re
dsh
ift
Ma
ker
Ast
roG
rid
Re
dsh
ift
Ma
ker
Jonathan Tedds
Let’s exchange data and expertise� Explore VO
� Find� Filter� Correlate� Science!
Jonathan Tedds
UK Space Conf Mar 08
� Science!
� Upload to VO� ingest your survey(s) to DB� Publish to VO registry� AstroGrid & EuroVO Data Centre Alliance are here to help� Science use!
http://www.astrogrid.org (April 2008 1st public release)
Jonathan Tedds
UK Space Conf Mar 08