Jonathan Tedds

UK Space Conf Mar 08

Jonathan Tedds

University of Leicester/AstroGrid/EuroVO DCA science team

jat@star.le.ac.uk

Ground Based Observatories

Copyright RASC Calgary Centre

Jonathan Tedds

UK Space Conf Mar 08

Space Observatories

Copyright NASA

Copyright ESA

Copyright ESA/D. Ducros

Future Skymapping Projects

LSST

PanStarrs

VST

WISEVISTA

The need for a VO

� Data Rate� all sky/1-band/16-bit/0.1"

==> 100TB

� Today : 20 TB/yr

� 2008 : 100TB/yr (VISTA)

Jonathan Tedds

UK Space Conf Mar 08

2008 : 100TB/yr (VISTA)

� 2015 : 5PB/yr (LSST)

� 2020 : 100PB/yr (SKA)

� All-sky monitors….WASP, PanStarrs, LSST

� We will have Petabytes of data by 2010

Data Archives

Jonathan Tedds

UK Space Conf Mar 08

Jonathan Tedds

UK Space Conf Mar 08

Jonathan Tedds

UK Space Conf Mar 08

Jonathan Tedds

UK Space Conf Mar 08

What are Virtual Observatories (VOs)?

“The VO vision can be summed up as the desire to make all archives speak the

same language – all searchable and analyzable by the same tools, all data

sources accessible through a common interface, all data held in distributed

databases that appear as one.”

Jonathan Tedds

UK Space Conf Mar 08

September 2003

Professor Andy Lawrence (AstroGrid)

Institute for Astronomy

Royal Observatory Edinburgh

Creating a Virtual Observatory - An

International Effort

The International Virtual Observatory Alliance

"facilitate the international coordination and

Jonathan Tedds

UK Space Conf Mar 08

"facilitate the international coordination and

collaboration necessary for the development and

deployment of the tools, systems and organizational

structures necessary to enable the international

utilization of astronomical archives as an integrated

and interoperating virtual observatory."

As a result of the IVOA we have

� A common Resource Metadata

� A common data querying language� Astronomical Data Query Language (ADQL)

Jonathan Tedds

UK Space Conf Mar 08

� Astronomical Data Query Language (ADQL)

� Based on Structured Query Language (SQL)

� Common table format� Virtual Observatory Table (.vot)

� …and much more in the pipeline

Advantages of VO data access

� Accessing multiwavelength data easier:

� Do not need to access different interfaces, register once only

Jonathan Tedds

UK Space Conf Mar 08

Advantages of VO data access

� Accessing multiwavelength data easier:

� Do not need to access different interfaces, register once only

� Data access from one single entry point

Able to build workflows i.e. pieces of code which run on

Jonathan Tedds

UK Space Conf Mar 08

� Able to build workflows i.e. pieces of code which run on the server and are reusable

http://www.astrogrid.org (April 2008 1st public release)

Jonathan Tedds

UK Space Conf Mar 08

VO Explorer:

� AstroGrid front end, part of VOdesktop

� Resource-centric

� Select a search-space

� Search for resources

� Filter these resources

View selected resources

Jonathan Tedds

UK Space Conf Mar 08

� View selected resources

� Use the selection

� Invoke it

� Save/Bookmark/Tag it

� Export it

Access existing X-ray cluster catalogues

� Browse through keyword metadata

Jonathan Tedds

UK Space Conf Mar 08

� Browse through keyword metadata

Find

Resources!

�� If data in a simple DB can If data in a simple DB can publish to VO in ~day!publish to VO in ~day!

�� We can help you: June 23We can help you: June 23--27, 27, 2008 2008 data publishingdata publishing workshop workshop at at ESO, ESO, GarchingGarching

content +resources

�� For catalogues, imagesFor catalogues, images, spectra, , spectra, transient & theoretical archivestransient & theoretical archives

�� CensusCensus of European Data of European Data Centres & ArchivesCentres & Archives

�� I am UK contact: I am UK contact: jat@star.le.ac.ukjat@star.le.ac.uk

technicalsystem

operations +support

Description

Filter

Query on position

Publishing VOEvent Data

� First point of VG

� Subpoint 1

� Subpoint 2

� Sub sub point 2.1

� Second point of VG

� Etc…

Advantages of VO data access

� Accessing multiwavelength data easier:

� Do not need to access different interfaces, register once only

� Data access from one single entry point

Able to build workflows i.e. pieces of code which run on

Jonathan Tedds

UK Space Conf Mar 08

� Able to build workflows i.e. pieces of code which run on the server and are reusable

� Tools to analyse the returned data

� Different tools which share standards on how to interoperate

•Graphical tool

•Direct SSAP queries to VO or receive from VO via PLASTIC

•Display, compare, modify and analyse spectra

•Read in, display & match multiple spectra in multiple formats

•NDF, FITS, TEXT, NDX…

••Spectral coordinate transforms between multiple input spectra as Spectral coordinate transforms between multiple input spectra as

described in FITS WCS Paper III (described in FITS WCS Paper III (GreisanGreisan+)+)

••WavelengthWavelength

••FrequencyFrequency

••EnergyEnergy

••VelocityVelocity

••Data value unit conversion as per FITS WCS Paper I (Data value unit conversion as per FITS WCS Paper I (GreisanGreisan+)+)

••Flux conversion per unit frequency to/from per unit wavelengthFlux conversion per unit frequency to/from per unit wavelength

••Convert between standards of restConvert between standards of rest

••TopocentricTopocentric, heliocentric, dynamic & kinematic local, heliocentric, dynamic & kinematic local

=> need for IVOA data standards & interoperabilityneed for IVOA data standards & interoperability

More advanced query: Access all extended 2XMM

X-ray sources

� Make own X-ray cluster candidate catalogue

� Filter large multiwavelength archives remotely:

Jonathan Tedds

UK Space Conf Mar 08

� Filter large multiwavelength archives remotely:

� Create manageable science subsets

� 247K detections → 192K unique sources

� 520 sq.deg. / 330 sq.degs excluding overlap

� Full catalogue product set available

Aug07Aug07: THE 2XMM X: THE 2XMM X--RAY CATALOGUERAY CATALOGUE

Jonathan Tedds

UK Space Conf Mar 08

� Full catalogue product set available

� World’s largest ever X-ray catalogue

Find

Columns

Filter columns

Use

Query

Help!

Help!

Correlation with SDSS

Jonathan Tedds

UK Space Conf Mar 08

Spectroscopic z = 0.223 (Lamer+, in prep)

Command Line Scripting� Connect to the AstroGrid system from your own Python scripts

(IDL+ to come)

#!/usr/bin/python# Given an input list od objects, queries the 2MASS catalogue returning a # VOTable for each object.

from astrogrid import ConeSearchfrom astrogrid.utils import broadcast, read_votable

# Read input list of ra, decs from VOTablevot=read_votable('sample.vot')id = vot['id']

Jonathan Tedds

UK Space Conf Mar 08

id = vot['id']ra = map(float, vot['ra'])dec = map(float, vot['dec'])

# We are going to query 2MASS at RoEivorn = 'ivo://wfau.roe.ac.uk/twomass-dsa/cone'cone = ConeSearch(ivorn)

nsrc = len(ra) # Number of sourcesradius=20./3600.0 # Search radius in degrees

print 'Starting Query: %d objects' % nsrc

for i in range(nsrc):res = cone.execute(ra[i], dec[i], radius)# Uncomment the following line if you want to send the output to TOPCAT# broadcast(res)print 'Writing 2mass_%s.vot' % id[i]open('2mass_%s.vot' % id[i], 'w').write(res)

Ast

roG

rid

Re

dsh

ift

Ma

ker

Ast

roG

rid

Re

dsh

ift

Ma

ker

Jonathan Tedds

Let’s exchange data and expertise� Explore VO

� Find� Filter� Correlate� Science!

Jonathan Tedds

UK Space Conf Mar 08

� Science!

� Upload to VO� ingest your survey(s) to DB� Publish to VO registry� AstroGrid & EuroVO Data Centre Alliance are here to help� Science use!

http://www.astrogrid.org (April 2008 1st public release)

Jonathan Tedds

UK Space Conf Mar 08