Stéphane ErardBaptiste Cecconi Pierre Le SidanerFrançois AndrieuAngelo Pio Rossi Mikhail MininPierre FerniqueChiara MarmoVincent GénotNicolas AndréMichel Gangloff

VESPA: recent progress & shopping list

stephane.erard@obspm.fr

PADC / Observatoire de Paris

Jacobs Univ, Bremen

CDS, StrasbourgGEOPS, Univ Paris SudIRAP, CDPP

Nick AchilleosKevin BensonBernard SchmittMaria Teresa CapriaBenoît CarryRicardo HuesoAnni MäättänenEhouarn MillourOndrej SantolìkJan SoucekManuel ScherfTarek Al-UbaidiWilliam ThuillotAnn Carine Vandaele

Interop meeting 2018, Victoria28/5/2018

UCL, London

IPAG, GrenobleIAPS, RomeOCA, Nice + IMCCE EHU/PVU, BilbaoLATMOSLMD, ParisIAS, Prague

IWF, Graz

IMCCEIASB-BIRA, Brussels

+ Contributions from

the community

2015-19

Varietyofdata/specifici7esfortheSolarSystem

- Perimeter of data to be accessed• Ground-based: moving objects on dark sky • Space-borne (PSA/PDS), including HR imaging in various coord frames• Time series, Parameter lists, Atmospheric / 3D, Volume…• Variations with time (secular, seasonal, local time)• Experimental / lab support data• Simulations on demand

- Physical data access:Main issue: existing PDS3 (space borne) archives - no software!FITS (sometimes)CDF (plasma physics)+ funny formats: netCDF, HDF5, TIFF, GeoTIFF, data streams… 

TOPCAT

Cassis

ExoplanetsTitan atm

BDIP

APIS

…

VESPAclient

SSODnetname resolver

VEx / Mag

PSA/ESA

DARTS/Jaxa

EPN-TAP

PDAP

IVOA

OV-Paris planet

… Aladin

VOspec

SpecView

GIS…

OGC

EPN-TAP services

LineList

Dust particles

catalogue

IVOA databases

IVOA registries

…

IVOA databases

IVOA tools:SAMP / VOtable

PDAP services

Skybot

…

OVParis registry

AVirtualObservatoryinPlanetaryScienceBuiltonastronomicalVOdevelopments+previousEuropeanprograms:IMPEx,HELIO,VAMDC…+interfaceswith:PDS,GIS/OGC,etc…

h"p://vespa.obspm.fr

Whatever data service

TOPCAT

Cassis

ExoplanetsTitan atm

BDIP

APIS

…

VESPAclient

SSODnetname resolver

VEx / Mag

PSA/ESA

DARTS/Jaxa

EPN-TAP

PDAP

IVOA

OV-Paris planet

… Aladin

VOspec

SpecView

GIS…

OGC

EPN-TAP services

LineList

Dust particles

catalogue

IVOA databases

IVOA registries

…

IVOA databases

IVOA tools:SAMP / VOtable

PDAP services

Skybot

…

OVParis registry

Simulations on demand

TOPCAT

CASSIS

ExoplanetsTitan atm

PVOL

APIS

SsODNetname resolver

VEx / Mag

PSA/ESA

DARTS/JAXA

…

Aladin

VOspec

SpecView

QGIS

EPN-TAP data services from teams

LineList

Solar services

IVOA databases

IVOA registries

3Dview

IVOA databases

IVOA tools:SAMP / VOtable

PDAP services

Skybot

AMDA db

IVOA or ObsParis registry

Query page Result page

Observatory & spacecraft list

EPN-ping services

Cesium

Mizar

APERICubes

WMS / GIS i/f

Miriade

Cosmographia

VESPA main search interface

…

PDS nodes

PDS4 services

PDS4-EPN-TAP bridge?

Other TAP clients (TapHandle…)

MATISSE

Computation & Inversion

services

AMDA

PDS tools

Aladin footprints

EPN-TAP library or

TAP

SSHADE db

PlanetServer

EPN-TAP access

GDL: PDS3 access & computing

SAMP

AVirtualObservatoryinPlanetaryScienceConnecIngmorepieces…

EPN-TAP

PDAP

IVOA

OGC

VESPADAL:

DataModel:EPNCore(EPN-TAPparameters)-SimilartoObsCore,butwiderscope-Describealldataaxes:spa7al,spectral,7me,viewinggeometry

=> IAU body-related coord systems needed in STC, to be reviewed

-Someparametersareen7relynewe.g.,incidence/emergenceangles-Includedescrip7onofdatainfilese.g.,reflectancemeasurements

-SomeparametersreusedfromObsCore,withspecifici7esEx:s_region-mustuseE-handedframes(IAUstandardonplanets)=> We need at least a generic E-handed frame in STC strings to implement the IAU standard! Call it BODY, or BODY-FIXED, or E-HANDED?

=> POLYGON('BODY-FIXED', 150, -30, 30, -30… etc

Lunar crater catalogue on Kaguya HiPS

CoordinatesareprovidedineastwardconvenGon(IAU)s_regionmustusethesameconvenGon

Mars crater catalogue on MOLA HiPS

CRISM footprint on THEMIS HiPS, Mars

VESPADAL:

MainDataAccessProtocol:EPN-TAP2.0-TAP+setofEPNCoreparameters—similartoObsTAP,widerscope-Datacanbeintegratedinthetableorinlinkedfiles(onefile/tableentry)

-RequireUtype(EPNCore/EPN-TAPdocinprogress)UCDs-usedbysometoolstounderstanddatacorrectly(spectro,…)=>FirstcontribuGontoUCDlistdiscussedthisweek(viewingangles,mainreflectancequanGGes…)

-Needtoiden7fyobserva7ons/simula7ons/experimentaldata-Usehashlistsformul7valuedfields(UCDs,targets,instruments…)=>addi7onsrequiredinADQLtosupportsearchfunc7onsivo_hashlist_hasRegTAPfuncGontohandlelistsofvaluesNeedtocombineivo_hashlist_haswithLIKEe.g.,toextractUCDsfromlists

Ex from TNOsarecool(trans-Neptunian objects properties)multiple alt_target_namebib_referencefor each granule

Ex from MPC (asteroid list)multiplemeasurement_typeprovided in the table

VESPADAL:

Registry:-Ini7alissuesortedoutDec2017(ivo-idrelated)-EPN-TAPcurrentlyinv2.0,v2.1indevelopment—willco-existProtocolversioncanberetrievedfromregistry

-VESPAclientwillsooncombineresultsfrommanyservicesNeedtotrackscredits&originofdata=>Needtoretrievecredits,bibreference,boilerplate,etc…fromregistry=>AddinparamorfieldofTAPoutputVOtable—tobeusedbytheclients

VESPADAL:

OtherDataAccessProtocols-(EPN-)TAPcoversmostneeds—e.g.,asortofbasicplanetarySIAP-PDAP(IPDA)supportedbyVESPAclient—seemsstalled-VOevents—needtobeabletotellthattheeventoccursatanamedplanetoraspacecraW(e.g.,"Jupiter","Juno")intheWHEREsecGon

-Connectswithnon-VOprotocols:das2stream,OGC(WMS,WCS…)(seeBapGste'spresentaGonfortheformer)

-Needamethodtocallsimula7onsandcomparewithobserva7ons-nowexperimen7ngwithdatalink(seePierreLS'spresentaGon)-WilluseSLAPinthenearfuturetohandlebothsolidphaseandgasspectroscopy(tosupportidenGficaGonofspecies)

VESPA/EPN-TAPclients:

Mainportal:http://vespa.obspm.fr-Currentversionqueriesallservices,thenlistsindividualresults-Nextversionwillintegratequeries&serviceresultspages-Afutureversionwillintegrateindividualresultsfromallservices

OnlydisplaysselectedEPN-TAPservicesfromtheregistry<=qualitycheckisexpectedfromtheportal(requirementfromEuroplanetandCNRS/INSUinFrance)

CurrentVESPAinterface

NewVESPAinterface(indev)

VESPA/otherEPN-TAPclients:

EPN-TAPlibraries-Javaversionin3Dview,CASSIS-JavaScriptversiontobeimplementedinAMDA

GenericTAPclients(TAPhandle,TOPCAT,Aladin…)-CompleteaccesstoEPN-TAPservices,butnoextrasupport(conversions,etc…).DatalinksupportwelcomeinTOPCAT4.6!

Astropy&PyVOforVESPA-WorkingonJupytertutorials

DataservicesconnectedviaEPN-TAP

Currently:39interoperabledataservicesconnected,from13insGtutes(~15indev)EncompassmanyaspectsofSolarSystemstudies:-surfaces-smallbodies/satellites/rings/dynamics-atmospheres-magnetospheres/radioobservaGons-solarphysics/planetaryplasmas-exoplanets-solidspectroscopy/experimentalreferencedataSeveralhigh-qualityamateurservicespreselected(PVOL,RadioJove…)MostservicesimplementedonDaCHS+someonTAPlibrary(ESA/PSA…)

Veryefficientcall/workshopproceduretoinvolvethecommunity

Future/possibledataservicesraisingissues

-PSA(ESAspacemissionsarchive)10millionsfiles,70experiments—serviceopen,buttable~emptyDatadescripGonneedtobeextractedfromheadersanddocs-discussiontomorrowinSSIGII

-Viziercatalogues(solarsystemrelated)DescripGonschemetobeidenGfied—on-goingatCDS

-Telescopic/ground-basedarchives?e.g.otherESAarchives(Herschel…)e.g.CADC,ESOinterfaces;)

Toolsconnectedto/usedbyVESPA

-Standard/exis7ngtoolsassociatedtoVESPA/Europlanet:Aladin(CDS/CNRS),CASSIS(IRAP/CNRS)-SeePierreF'spresentaIonBothincludePlanetaryScienceupdatesfromVESPA

3Dview(CNES/IRAP/GFI):plotalongs/ctrajectoriesMATISSE(ASI/IAPS):3DvisuforsomePDS3data,supportsshapemodels

-NewtoolsdevelopedinVESPA:PlanetaryCesiumViewer(GEOPS/CNRS)APERICubes(somePDSspectralcubes,ObsParis)-seeRenaud'spresentaIonQGISSAMPplug-in(VO-GISbridge,JacobsUniv)ImageJSAMPplug-in&improvedfitssupport(ObsParis,inprogress)

-OtherusefultoolsinVESPAcontext:TOPCAT(Bristol),SPLAT-VO(Heidelberg),Autoplot(Iowa),Mizar(CNES)+possiblyVOspec(ESA),DS9(SAO),Cosmographia(JPL)

Toolsconnectedto/usedbyVESPA

ToolsareconnectedthoughSAMPNeedforaddi7onalSAMPmessages:-ForPDSfiles,possiblyseveralones(PDS3vs4,detachedlabels…)-geoJSON-das2stream-etc…

Func7onsneeded(butnotfoundyet):-sta7s7ctools-basicarithme7ctools(averagingonthefly,inparGcularvectors/profiles)-imagesegmenta7on

FavoriteVESPAtools

CRISM cubes on MOLA HiPS

Lunar crater catalogue on Kaguya HiPS

ReliesonIAUplanetarycoordinateframes(fits'WCS)

Aladin (CDS/CNRS):

Georeferenced images + objects superpositions

45 planetary HiPS (USGS maps)

Newfunc7onsinVESPA MulGresoluGonmapsinTOPCAT(Bristol Univ)

Startfromatableofsparseobserva7ons(lon/lat)VIRTIS/RoseWaon67P

Virtis/Rosetta coverage of 67P for a selected mission phase, scale = 7

Integrate/averageonhealpixcells

Virtis/Rosetta coverage, scale = 6

Modifyresolu7on/scaleonthefly Virtis/Rosetta coverage, scale = 5

NeedastdformatusableinAladin

UpdatedVESPAtool

CASSIS v5 (IRAP/CNRS)- Auto converts spectral axis & flux- Support data in flux & various types of reflectance (scaling)- Now supports échelle spectra

VIRTIS-H / Rosetta spectrum (8 separated orders)

PDS spectral library SNC meteoritesvs M4ast telescopic spectra of Vesta

IRTF (flux) / Solar flux * coeff vs La Silla (reflectance) spectra of Jupiter

NewVESPAtools:VO-GISconnec7on

QGIS (open source):

SAMP plug-in installed

- receives data from VESPA and TAP clients

- Provides visualization to EPN-TAP services which distribute WMS products

(=> planetary mapping)

GEOfits format / GDAL update

- WCS (small) extension to support planetary surfaces

- georeferenced fits files can be projected

- geoTIFF files converted to geofits for use in VO tools

CRISM cube coverage / plane on MOLA topography & THEMIS daytime blend

NewVESPAtool

Planetary Cesium Viewer (GEOPS/CNRS)- Quick multiresolution 3D visu - Supports elliptic shapes- Annotation/validation tool- SAMP implementation

Robbins Mars craters database in PCV

NewtoolsinVESPA ImageJ (open source):

- SAMP connection installed (input only)

- Provides support for unusual data types (TIFF…) & format conversion- Provides image processing functions in the VO

Historical image of Jupiter from BDIP (tiff format) in ImageJ

Nextsteps

•Moredataservices!-includingexternalcontributors•Processanswersfrommul7pleservices•FinalizeVO-GISinterface-includinggeofits(ESSpaperbeingreviewed)•Developinterfacewithlabspectroscopyservices(mineralsandicesspectra,bandlists,etc)-now4servicesimplemented

•BridgeEPNCoreandPDS4•Connectsimula7onservices,usewithrelateddataservicesrequiresdifferentprotocols•Formalizedocs/standards=>submittoIPDA&IVOAandrefine/completetutorials

Nextsteps,2

•EuroplanetGatewayproposal,submiqedtoEUcall,March2018ApplicaGonofFAIRprinciples,consolidaGon,sustainability,collaboraGons

•Europlanet-RInextbidtobesubmiqed,March2019Willfocusmore(=also)onknowledgeextracGon,machine,learning,etcWillassociatenonbeneficiaryteams/insGtutes

searchinterfaceh,p://vespa.obspm.frwebsite(w/tutos)h,p://www.europlanet-vespa.euwikih,ps://voparis-confluence.obspm.fr/githubh,ps://github.com/epn-vespa