1D. Heather (22/10/2009)

ESAC / ESTEC

PSA status and interactions with PDS

26th March 2010

Dave Heather

ESA-ESTEC, dheather@rssd.esa.int

2D. Heather (22/10/2009)

Scope1. Historical perspective

2. PSA general properties / services

3. Current mission archive status

4. PSA and PDS collaboration / interaction and issues

5. IPDA and interoperability

3D. Heather (22/10/2009)

Definition and Purpose

The Planetary Science Archive (PSA) is the initiative, the setup, the process and the implementation to preserve data from ESA’s spacecraft to planetary bodies, as well as supplementary information acquired in laboratories or ground-based observatories.

The prime objectives of the PSA are: to support the experimenter teams in the preparation for the spacecraft and ground-based long-term archives to enable and ensure the (long-term) preservation of these archives distribution of scientific useful data to the world wide scientific community provision of supplementary data services aiming to maximize the usage of planetary mission data and ease the scientific data analysis.

The PSA is the primary resource for all scientific and engineering data from the European Space Agency’s planetary missions.

http://www.rssd.esa.int/psa

4D. Heather (22/10/2009)

Mars Express Science Management Plan

The Science Operations Centre will prepare the final Mars Express Scientific Data Archive (MESDA) within one year of the receipt of the complete data sets from the individual Mars Express Orbiter and Lander Module science investigations. Based on current technology and IMEWG recommendations on standards for scientific data of future missions to Mars, the archive would be distributed as a set of CDROMS based on the NASA Planetary Data System.

ESA/C(2002)148: Rosetta ESA/NASA MoU

“ARTICLE 10 - RIGHTS IN AND DISTRIBUTION OF SCIENTIFIC DATA

… Following the period defined in 10.1 above, all scientific and ancillary Rosetta spacecraft data records will be deposited with the ESA Data Library, and NASA’s Small Body Node of the U.S. Planetary Data System (PDS) using PDS format. NASA will place the science data in a public archive in the Small Body Node of the U.S. Planetary Data System …”

Rosetta Science Management Plan (ESA/SPC(94)37, page 28):

“6. Provide the reduced and calibrated scientific data sets from his/her instrument in useable form to the RSOC for inclusion in the Rosetta Science Data Archive.”

Page 42, RSOC reponsibilities:

“the preparation of guidelines for science data archiving and - supported by the PI team - to create the Rosetta Data Archive.”

Page 42: “5.4 Rosetta Scientific Data Archive

… After this period, the scientific data products from the mission have to be submitted to RSOC in a reduced and calibrated form such that they can be used by the scientific community. RSOC will prepare the Rosetta Scientific Data Archive within one year of the receipt of the complete data sets from the individual Rosetta science investigations. Based on current technology, the archive would be distributed as a set of CD-ROMs. …”

Historical Perspective GIOTTO Science Data Archive – initiated before we had a legal basis in the

Agency. NASA participated via the International Halley Watch plus the MoUs signed for NASA support (DSN, Navigation).

ROSETTA Science Management Plan ROSETTA ESA/NASA Memorandum of Understanding Mars Express and Venus Express Science Management Plans Bebi-Colombo Science Management Plan

Giotto Mission

The Giotto spacecraft was launched on 2 July 1985 to study comet 1P/Halley. Giotto encountered comet 1P/Halley on 13 March 1986 and came within 596 km of the nucleus. Giotto encountered comet 26P/Grigg-Skjellerup on 10 July 1992.

Giotto instrument data are archived in co-operation with NASA’s PDS Small Bodies Node and were distributed on a set of CDROMs (“International Halley Watch Campaign”)

A close collaboration with the PDS Small Bodies node was established and maintained up to today.

BepiColombo, Science Management Plan

Provide the scientific data (raw data, calibrated data, and higher level data), including relevant calibration products, to the BepiColombo ESA-JAXA archive in a format that will be agreed with the ESA SOC for application by the general science community.

The ESA-JAXA science data archive, which will be compatible with the Planetary Data System (PDS), will be based on and part of the Planetary Science Archive (PSA) developed for Smart-1, Mars Express and Rosetta.

5D. Heather (22/10/2009)

Data Rights and Duties ESA Convention (SP-1271) Resolution on the Rules concerning Information, Data and

Intellectual Property

ESA Convention, Article V

In carrying out its activities under Article V, the Agency shall ensure that any scientific results shall be published or otherwise made widely available after prior use by the scientists responsible for the experiments. The resulting reduced data shall be the property of the Agency.

ESA/C/CLV/Rules 5 (Final)

“OWNERSHIP, ACCESS, USE AND DISSEMINATION OF RAW AND CALIBRATED DATA RESULTING FROM A PROGRAMME OR ACTIVITY OF THE AGENCY

1. OWNERSHIP OF RAW AND CALIBRATED DATA

The Agency shall be the owner of all raw and calibrated data directly resulting from a payload flown in the context of an Agency’s programme or activity (excluding any data which are required for the control of the payload itself) including when the payload is furnished by a Provider.”

6D. Heather (22/10/2009)

Planetary Science Archive• Available since March 2004:

http://www.rssd.esa.int/PSA • Active development

• Datasets received from PI teams Needs to be Peer-reviewed by

independent team Needs to be validated before being

ingested into the PSA (check PDS format compliance)

• Around 13TB of data on hard disks

• Classical User Interface• Map based interface for MEX• Dataset Browser (not searchable)

7D. Heather (22/10/2009)

PSA : one archive, several missions

Mars Express

Rosetta

Smart-1

GiottoHuygens

VenusExpress

8D. Heather (22/10/2009)

PSA-DH Personnel and Mission Contacts

9D. Heather (22/10/2009)

ESAC Science Archives Team

Pedro Osuna*ESA

Neil CheekSERCO

Ignacio LeonINSA

Jesus Salgado*INSA

Data Access Group

Monica FernandezINSA

Inaki Ortiz*SERCO

User Interface Group

Christophe ArvisetESA

2 FTEs for PSA

10D. Heather (22/10/2009)

End Users, Services and Functionalities Offered PSA Scientific Advisory Group

Helpdesk, Notification

Data Query and Retrieval

PDS Standard Support (currently only for Version 3!!)

Coordination and chair for Peer Reviews

Design, Production and Delivery of auxiliary data SPICE conversion Software consultancy and support ESOC data long-term archive preparation Science planning information, long-term archive preparation

11D. Heather (22/10/2009)

PSA Advanced Interface

PSA supports 4 different views:-Query-Latest Results-Shopping Basket-Login/Register-(Request Monitor to come)

Selection of Result Display Options

Query Panel are ANDed when querying the database

12D. Heather (22/10/2009)

PSA UI – The Map Interface

Select Base Map Type (MOLA / VIKING…)

Zoom function / day/night projection

Select Instrument / Detector

Long / Lat information

Search Button

Access to ‘standard’ UI

3. Click on desired footprint

1. Click and drag to select your area of interest

2. Press ‘Search’ and wait for footprints

4. Download directly or view label / docs etc.

5. Transfer at any time to the standard interface for advanced searches. Your search parameters will be remembered.

6. Return to the map browser, you can view day/night boundaries and sub-solar point for selected footprint.

13D. Heather (22/10/2009)

PSA Browser Interface Opening Screen

Click on the instrument that interests you

Select the data set you wish to look at

Browse through the directories to locate the files you want

Right click the product you want to save etc. to bring up the menu

Left click to view directly (where possible)

14D. Heather (22/10/2009)

Data Producers, Services and Functionality Offered Data Archive Workshops on request (internal and external)

SPICE Workshops (on request – in co-ordination with NAIF)

Individual Archive Consultancy from early phase

Coordination of Mission Data Archive Working Groups (DAWG)

Coordination and support of Mission geometrical parameter information

Dataset Support/Development Tools Product generation from telemetry to L2 (GDP) – used for C1 Dataset validation Dataset ingestion

15D. Heather (22/10/2009)

PSA Data Producer Services • We offer a PSA Volume Verification Tool (PVV)

Data producer » Can verify the structure of the whole data set» Can verify the content of the ODL language» Can verify the references, catalogue structure, etc

Database» Can easily ingest data sets (standard delivery within one week)

• PVV includes Semi-automatic update of the tool Direct connection to the PSA database (network based)

» Always the latest dictionary Support of former dictionary versions

• PVV does not include A guarantee of PDS compliance!

• PVS: qualitative validation now operational!!

16D. Heather (22/10/2009)

PSA Management Services

Statistics Tool (PST) Currently does NOT account for FTData set Browser access.

Notification Management

Data Set and User Administration (PAT)

Dictionary Administration (PDT)

17D. Heather (22/10/2009)

Current Status, Available Datasets GIOTTO: data from comet Halley and Gripp-Skellerup

Comet Halley ground-based observations (Halley-Watch)

Comet Wirtanen ground-based observations

Mars Express: instrument and auxiliary data available (~11Tb) HRSC Radiometric, Map-Projected (to July 2009) and DTM (to Dec

2006) data. ASPERA NPI and ELS data up to June 2009. OMEGA, SPICAM, MARSIS and MaRS data all available and

deliveries continue. PFS data only on FTP. Calibrated data from several instruments now coming in. Working group on Mars upper atmosphere

18D. Heather (22/10/2009)

Current Status, Available Datasets Huygens: instrument and housekeeping data available for all

instruments New GCMS data has just been sent DISR re-ingestion planned with improved ‘processing’

Venus Express: first data sets released Spring 2009 VMC, SPICAV-SOIR, MAG available and regular deliveries. First SPICAV data now available. VIRTIS still closing out review, but available via ftp.

Rosetta: first data release still in preparation, pending PDS/PSA agreement PDS collaborations ongoing Validation is a major issue. Steins fly-by joint PSA / PDS review last October last year. Very

successful.

19D. Heather (22/10/2009)

Current Status, Available Datasets SPICE: Rosetta, MEX and VEX data sets released. New versions will

be ingested after Easter.

Chandrayaan-1: data pipeline running at ESAC for CIXS and SIR2.

SMART-1: AMIE, SIR, D-CIXS and XSM data being prepared for release in May.

BepiColombo: data handling and archive support now taking shape. New approach for ESA.

Updated PSA WWW area – continuous improvements: Tools Documentation FTP Access http://www.rssd.esa.int/PSA

20D. Heather (22/10/2009)

PSA and PDS Interactions• It was decided at an early stage that PSA data would comply with

PDS Standards: To maximise the cross-compatibility of ESA and NASA data Existing and well established standard in the community

• We have good contact and excellent support from several of the PDS Nodes: MEX: Geosciences / PPI. MOU between ESA and NASA Rosetta: Small Bodies Node (primary). International mission. VEX: Atmospheres Node. Interoperability. Review support. Ancillary data: NAIF. SPICE used for all ESA planetary missions.

21D. Heather (22/10/2009)

PSA and PDS collaboration Rosetta issues

Rosetta first review took place Feb 2007 but data are still not available Full ingestion (and hence compliance) required at PDS How does one ensure full compliance?

VTOOL? LVTOOL? TBTOOL? PVV? Others?

REVIEW?

FULL / OPEN PSA/PDS INVOLVEMENT IN ARCHIVE DEVELOPMENT CYCLE?

ALL OF THE ABOVE…

MORE!?

22D. Heather (22/10/2009)

PSA and PDS Interaction -> IPDA / PDS4• Development of our validation tool has highlighted several

inconsistencies in the PDS Standards and the way in which various experts interpret them » IPDA

• PDS and PSA are drawing upon the lessons learned on both sides of this working relationship in order to refine and streamline the Standards.

• These ‘lessons learned’ are being used by the IPDA to help develop a refined set of IPDA recommendations for archiving that will allow for international archive interoperability and avoid similar issues in future.

• Some issues also input to PDS2010 (PDS4) development.

• Involved in PDS2010 System Review and in IPDA assessment of PDS4

23D. Heather (22/10/2009)

IPDA - VEX Interoperability VEX - First interoperability studies between PDS and PSA

Access to data remotely Full search functionality Transparent access to data on various servers Planned for PDS data access on VEX

24D. Heather (22/10/2009)

Open Issues

“Difficulty” to control Dataset delivery schedule Affects PSA-DH schedule Affects PSA-DT schedule

Incomplete deliveries to the scientific community as calibrated data deliveries are not available in time by some experimenter teams on the on-going planetary missions Still not fully resolved

Insufficient resources within some of the experimenter teams To be solved with new concept in Solar System Science Operations

Division - but can only be done partially. Issue was addressed with delegations (SPC) on a number of occasions

Parallel activities in member states – we try to coordinate and focus resources

25D. Heather (22/10/2009)

Open Issues

Standard evolution. With IPDA (all international partners), and maintaining our well established links with PDS, we follow PDS-Standard Evolution.

Peer Review handling – data are sometimes slow to be released through the formal procedure Requires time and significant resources to perform review Delay in DS ingestion into PSA

Data set ingestion turnaround Manual process

Potential Remedy actions Planning & monitoring through Dataset Spreadsheet Try automate ingestion process? Review Peer-review concept? Immediately release the incoming Datasets as “provisional”?