Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from...
-
Upload
rachel-alders -
Category
Documents
-
view
216 -
download
2
Transcript of Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from...
![Page 1: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/1.jpg)
www.g-vo.org
The Virtual ObservatoryThe Virtual Observatoryin Germany and abroadin Germany and abroad
A status report with examples/demos from GAVO
and other national VOs
![Page 2: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/2.jpg)
www.g-vo.org
OverviewOverview
• The Virtual Observatory– Motivation– Approach
• The IVOA– Standards– Examples, demos
private demos are always possible !
• GAVO– Project– Results– Outlook
• Theory in the VO: – The Millennium Simulation database
![Page 3: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/3.jpg)
www.g-vo.org
Astronomical data sets are Astronomical data sets are large, and getting largerlarge, and getting larger
Simulations as well(courtesy Simon White)
(courtesy Alex Szalay)(courtesy Alex Szalay)
![Page 4: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/4.jpg)
www.g-vo.org
Most data is (or could be) onlineMost data is (or could be) online
Google(http://www.google.com/Top/
Science/Astronomy/Data_Archives/)
MPE (http://www.mpe.mpg.de/AstR/ )
![Page 5: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/5.jpg)
www.g-vo.org
OpportunitiesOpportunities
• So, the Internet is (or could be) the world’s best telescope:– It has data on every part of the sky– In every measured spectral band:
optical, x-ray, radio..– As deep as the best instruments (2
years ago).– It is up when you are up– It’s a smart telescope: links objects
and data to literature on them– It even contains truly virtual data
(courtesy Alex Szalay)(courtesy Alex Szalay)
![Page 6: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/6.jpg)
www.g-vo.org
![Page 7: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/7.jpg)
www.g-vo.org
ProblemsProblems
• Data formats not uniform• Interfaces not uniform• Data sets very large• Data set far away (makes them larger)• How to find data of interest?• Computers have difficulty
understanding web pages
![Page 8: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/8.jpg)
www.g-vo.org
Promises (/hopes) of the VO.Promises (/hopes) of the VO.
• To make results of astronomical research, more readily available to the community– data and applications
• To facilitate their– communication, reuse, comparison,
combination , checking
![Page 9: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/9.jpg)
www.g-vo.org
How ?How ?
Online availability of datasets and applications
• Standardisation of – publication and discovery mechanisms– description/meta-data (FITS++)– selection mechanisms– form(at)s of transmitted data– data access protocols
• Why standardisation?
![Page 10: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/10.jpg)
www.g-vo.org
Babylonian confusionBabylonian confusion
SDSS
ROSAT
2MASS
FIRST GAIA
N x M
![Page 11: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/11.jpg)
www.g-vo.org
Why standardisationWhy standardisation
![Page 12: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/12.jpg)
www.g-vo.org
We need a common We need a common languagelanguage
ROSATFIRST GAIA
SDSS2MASS
N + M
![Page 13: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/13.jpg)
www.g-vo.org
Standardise on:Standardise on:
• Discovery• Data description (“meta-data”) • Data formats• Protocols• (Web) Application Interfaces• Query language
![Page 14: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/14.jpg)
www.g-vo.org
Why (standardised) Why (standardised) protocolsprotocols??
• VO service oriented.• Data sets too large to move :
– You can FTP 1 MB in 1 sec (optimistically)– You can FTP 1 GB in min (= 1 $/GB)– You can FTP 1 TB in 2 days and 1K$– You can FTP 1 PB in 3 years and 1M$(numbers from Alex Szalay)
– SFTP even slower
• Remote filtering required to reduce size data sets
• “Move analysis to data”: services• Allow programmatic access to services
![Page 15: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/15.jpg)
www.g-vo.org
![Page 16: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/16.jpg)
www.g-vo.org
VO’s esperantoVO’s esperanto
![Page 17: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/17.jpg)
www.g-vo.org
WarningWarning
• VO can not (and does not aim to) be everything to everyone
• Users will have to be able to visit the underlying data in all gory detail: provenance
• Even then standardisation helps• Agreement is hard to come by (see FITS)• Problems are hard ! • VO is still partially a research project.
![Page 18: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/18.jpg)
www.g-vo.org
The International Virtual The International Virtual Observatory Alliance (IVOA)Observatory Alliance (IVOA)
Facilitate the international coordination and collaboration necessary for the development and deployment of the tools, systems and organizational structures necessary to enable the international utilization of astronomical archives as an integrated and interoperating virtual observatory.
![Page 19: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/19.jpg)
www.g-vo.org
Alliance of national VOsAlliance of national VOs
![Page 20: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/20.jpg)
www.g-vo.org
Working and Interest GroupsWorking and Interest Groups
• WGs– Standards and Process: how the IVOA works– Resource registry: where to register and discover
resources– Data Modeling: how to describe data sets– Semantics: how to understand one another– VOTable: standard format for tabular data sets– Data Access Layer: very simple data access services– VO Query Language: more sophisticated data access– Grid and web services: programmatic accessibility– Applications: stand alone, and together– VOEvent: astronomical telegrams in XML
• IGs– Theory: virtual observations for virtual universes– Open GridForum Astro-RG: VObs and virtual organisations– Data Curation and Preservation: how not to loose your data
![Page 21: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/21.jpg)
www.g-vo.org
Publication standard:Publication standard:Resource RegistryResource Registry
• Implementations:– Searchable Registry at STScI – NCSA Publishing Registry – ESAVO Full Registry
• Client tools:– VOExplorer (AstroGrid)– Splat, SpecView, Aladin
![Page 22: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/22.jpg)
www.g-vo.org
Messaging standard: Messaging standard: VOTableVOTable
• http://www.ivoa.net/twiki/bin/view/IVOA/IvoaVOTable
• XML format for tabular data• Column definitions included with data• Binary streams and FITS included• Understood by many client tools (see applications
set)– TOPCAT– VOPlot– Aladin– Many others
![Page 23: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/23.jpg)
www.g-vo.org
VOTable exampleVOTable example
![Page 24: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/24.jpg)
www.g-vo.org
Data Access Protocols IData Access Protocols I
• Simple protocols for discovering and retrieving data sets
• Generally HTP-GET based:– Base url:
http://www.g-vo.org/ssa?action=doQuery&archiveid=ssa.service.cdfs
– Query parameters:POS=53.161,-27.832&SIZE=0.2&BAND=0/&TIME=,&FORMAT=ALL
• Returns (heavily) annotated list of spectra with access URL for retrieving them.
![Page 25: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/25.jpg)
www.g-vo.org
Data Access Protocols IIData Access Protocols II
• Examples from GAVO, for more usage examples see tool demos– Simple cone search (source catalogues)
• ROSAT source catalogues and photon event lists)
– Simple Image Access• ROSAT pointed observations and All-Sky
Survey fields
– Simple Spectral Access• Chandra Deep Field South optical follow up
![Page 26: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/26.jpg)
www.g-vo.org
Registry + DAL protocols:Registry + DAL protocols:InteroperabilityInteroperability
Standard services, once registered,
can be found by client tools …
![Page 27: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/27.jpg)
www.g-vo.org
InteroperabilityInteroperability
….and executedtogether (too
many ROSAT results to show all here!)
![Page 28: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/28.jpg)
www.g-vo.org
InteroperabilityInteroperability
… and shown together
![Page 29: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/29.jpg)
www.g-vo.org
Applications, toolsApplications, tools
• VO aware tools:– Images: Aladin– Source lists and tables: TOPCAT, VOPlot– Spectra: Splat, SpecView, VOSpec– 3D: VisIVO
• Application interoperability: Plastic– Messaging standard– Tying TOPCAT to Aladin to Splat to …– Uses VOTable to send data from one app to
another – All on your desktop
![Page 30: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/30.jpg)
www.g-vo.org
GAVOGAVO
![Page 31: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/31.jpg)
www.g-vo.org
GAVO: VO in GermanyGAVO: VO in Germany
• BMBF funded• In 2nd phase of financing• Institutes/people
– AIP: M. Steinmetz (Co-I), I. Nickelt– ARI-ZAH: PI J. Wambsganss (PI),
M. Demleitner, U. Stampa– MPE: W. Voges (Co-I), J.Kim, G.Lemson– Uni Tϋbingen: T. Rauch– TUM-Informatik: B. Gufler
![Page 32: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/32.jpg)
www.g-vo.org
GAVO tasksGAVO tasks
• Investigate archiving and IVOA compatible publication
• Investigate data mining techniques• Participate in -, and represent IVOA efforts• Special emphasis on theory in the VO• GAVO-expertise center and archive @ARI• Support for custom services• Info/help on all things VO for Germany
(Please come by !)
![Page 33: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/33.jpg)
www.g-vo.org
GAVO services, GAVO services, standard and customstandard and custom
• SCS and SIAP for ROSAT results (J.Zuther)• Cross matcher tool (used by S. Komossa)• SSA for CDFS spectra (G. Szokoly)• Publication A. Finoguenov’s A3266 data.• SSA for TMAP synthetic spectra (T. Rauch,
Tubingen)• Various virtual telescopes (Planck@MPA, K.Dolag,
J. Blaizot)• RASS source classifier (Class-X team)• Cluster finder (P. Schuecker et al)
– Now in AstroGrid-D (A. Carlson)
![Page 34: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/34.jpg)
www.g-vo.org
GAVO current and planned GAVO current and planned projectsprojects
• GAVO@ARI– Archive for small science results– Feedback on all things GAVO– Hopefully long term
• Collaborative tool– Automated upload and annotation of
data– Wrap with IVOA protocols– Pre-publication collaboration capabilities
• Simulation registry
![Page 35: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/35.jpg)
www.g-vo.org
Theory in the VOTheory in the VO
![Page 36: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/36.jpg)
www.g-vo.org
Observations in the VOObservations in the VO
• Most VO efforts concentrate on observational data sets– simple observables: photons detected at a certain
time from a certain area on the sky– long history of archiving– pre-existing standards (FITS)– valuable over long time (digitising 80 yr old
plates)
• Standards observationally biased– common sky: cone search, SIAP, region– common objects: XMatch– data models: characterisation of
sky/time/energy(/no polarisation yet)
![Page 37: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/37.jpg)
www.g-vo.org
Theory in the VO: issuesTheory in the VO: issues
• Simulations not so simple– complex observables– no standardisation (not even HDF5)– archiving ad hoc, for local use
• Moore’s law makes useful lifetime relatively short: few years later can do better
• Current IVOA standards somewhat irrelevant– no common sky– no common objects– requires data models for content, physics, code
![Page 38: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/38.jpg)
www.g-vo.org
The The MillenniumMillennium database database + web server+ web server
• Post-processing results only• SQLServer database• Web application
– portal: http://www.mpa-garching.mpg.de/millennium/
– public DB access: http://www.g-vo.org/Millennium– private access: http://www.g-vo.org/MyMillennium
• Access methods– browser with plotting capabilities through VOPlot
applet– wget + IDL or R– TOPCAT plugin
![Page 39: Www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.](https://reader036.fdocuments.net/reader036/viewer/2022081602/551b88df550346a10a8b556f/html5/thumbnails/39.jpg)
www.g-vo.org
Usage statisticsUsage statistics