Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory 1 Synthesis of LoI's F. Mignard OCA/Cassiopée.

Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory1

Synthesis of LoI's

F. Mignard

OCA/Cassiopée


Main StatisticsMain Statistics

•162 answers received and validated

29 data processing in general

7 for calibration

10 double and multiple stars

16 photometry

10 planetary systems

13 radial velocity

8 relativity and reference frame

5 science alerts

12 simulations

17 solar system objects

20 source classification

17 variable stars


Major Coordinated answers Major Coordinated answers

• A set from Cambridge to lead the photometric processing

• A combined answer from Cambridge and Geneva for the science alert

•One answers from Geneva coordinating the involvement of ISDC

• One series from ARI to cover the first look, bright stars analysis

• Two coordinated letters from CNES covering support for simulation, major

involvement in the data analysis, system architecture

• One global answer from Italy for a quality and validation system

• An international answer coordinated by Meudon for the RVS analysis

•One global answer from the SSWG to carry out the analysis for solar system

objects


Answers for the Variable stars


Synthesis of answersSynthesis of answers

•Number of answers Collective : 6

Individual : 11 but some may have considered a personal reply unecessary

•Areas covered Detection of variability

Analysis of light curves

Period determination

Classification

Follow-up

Science alert

•Areas marginally covered Simulation

overall management


# institute Contact FTE Areas of interest

99 Geneva Obs. G. Burki Commitment of Geneva Obs.

45 Inst. Tech. Lausanne S. Morgenthaler 1 Statistical modelling, classification

98 Geneva Obs. L. Eyer 2 to 10 ISDC involvement, study of variability

22 Cambridge G. Gilmore 1-2 Detection and analysis of variables

146 OHP M. Boer 4 Follow-up, science alert

165 Univ. Cathol. Louvain C. Aerts 5 ? Automated classification of lightcurves

Answers to LoI : collective answers Answers to LoI : collective answers


# Name institute FTE/yr Areas of interest

2 E. Milone Calgary 2-4 EB : modeling, simulation

13 L. Szabados Budapest 2 Variable stars analysis

15 Gomez de Castro Madrid ? Data mining, mathematical modeling

77 J. Fabregat Valencia 0.5 O-B Star classification

87 P; Mathias Nice 0.2 Light curves of Gamma Dor

111 P. de Laverny Nice 0.5 Long period Miras

113 T. Zwitter Lublijana 0.5 EB and Spectro binaries

116 A. Prsa Lublijana 0.5 Period determination, EB

145 M. Chadid Nice 0.1 Pulsating stars

149 E. Poretti Milan 0.2 Pulsating stars

177 G. Gilmore Cambridge 1-2 Light curves and planetary transits

Answers to LoI : individual answers Answers to LoI : individual answers


# Name institute FTE/yr Areas of interest

18 G. Gilmore Cambridge 5-7 S/W for the photometric processing

51 A. Brown Leiden 2-3 Core photometric processing

94 J. Carme Barcelone 2 Photometric simulation, calibration

144 A. Vidal Madjar Paris (IAP) 3 Periodic variables, micro-lensing,

planetary transits

30 C. Bailer-Jones Heidelberg 5.5 Stellar classification

Related proposalsRelated proposals


Organisation of the Data

Processing

F. Mignard

From GST and DACC inputs


Objectives and challengesObjectives and challenges

•Gaia will observe over 5 years with three instruments

•Gaia will produce ~ 200 TB of raw data

At the lowest level they are primarily CCD counts

•The DP must cope with

A large treatment both in volume and computation

A complex and iterative treatment

Interconnected data and tasks

•It is a community task, neither organised, nor funded by ESA

•If must be ready by the launch time


Objectives and challengesObjectives and challenges

•It requires specific expertises and commitment :

Overall analysis of a complex project

Professional S/W developers

Quality control

Strong commitment of teams over ~ 10 to 15 years

Ability to work with large team under pressure with tight schedule

•The basic organisation does not exist yet

This is very different from the WG structure

The level of coordination is much more demanding

Science will not be the primary activity in the DP

The organisation must have clearly defined responsibilities


Overall ConceptsOverall Concepts

•There are three obvious main steps in the DP

The initial data treatment of the CCD images up to centroiding

The astrometric, photometric and spectroscopic iterative solution

The special non iterative processes

•The global structure should be

Must be as simple as possible

Clear hierarchy with non overlapping responsibilities

GST must have the full visibility and retain control of the DP

•The current workplan

Global system called Gaia Data Analysis Consortium (DAC)

Central concept around Coordination Units (CU)

Within each CU there are Development Units (DU)

The lower level is the Work Package (WP)


Coordination UnitsCoordination Units

•Small in number ( < 10)

•Each must be in charge of a significant fraction of the DP

•Interfaces betweens CU's are limited

•Each CU coordinates the developments of many tasks

It defines the structure in DU

It follows the development, testing and documentation of the S/W

It defines and implement the interfaces between the DUs

•A CU is placed under the responsibility of a manager and his

deputy

They manage their CU

They provide contact and interfaces with the other CUs


Identified Cus (provisional)Identified Cus (provisional)

•C1 : Overall system architecture and central DB

•C2 : Data simulation ( three levels)

•C3 : Astrometric core processing

•C4 : Object processing

•C5 : Photometric reduction

•C6 : Spectroscopic reduction

•C7 : [Scientific exploration and Catalog access]


Identified Cus leaders and Identified Cus leaders and deputiesdeputies

•C1 : Overall system architecture W. O'Mullane, U. Lammers

•C2 : Data simulation X. Luri, C. Babusiaux, F. Mignard,

•C3 : Astrometric core processing U. Bastian, M. Lattanzi, J. Torra

•C4 : Object processing D. Pourbaix, C. Bailer Jones, P.

Tanga

•C5 : Photometric reduction F. van Leeuwen, A. Brown, C.

Jordi

•C6 : Spectroscopic reduction D. Katz, M. Cropper, U. Munari


Developpement UnitsDeveloppement Units

•Responsibility for the development of S/W

Conceptual development

Algorithm testing on simulated data

Preparation of the implementation of the operational versions

Process the Gaia data and provide outputs for the DB

•A DU should be based on a small number of well connected groups

Typically 5 people in each place

•Examples of Dus :

PSF calibration, RV determination, orbit determination, taxonomic

classification


Architecture and H/WArchitecture and H/W

•Overall architecture not yet defined

•Several approaches possible :

All computational H/W and DB at one place serious funding problem

All core processing at one place + distributed shell tasks data transfer

Centralised data base and core tasks ; separate location for integration

of shell tasks and centralised or distributed execution

Local data bases with shell tasks processing and coordination units


Astro-1: 20 TB

Astro-2: 20 TB

RVS : 30 TB

MBP : 25 TB

50 GB/day

11h mean GS visibility per day

5 Mbps downlink rate

MOC

TMArchive

GAIA

primary

Data

Base

Initial treatementCalibrations

First lookSystem determination (GIS)

C3

C2

Simulation

Data Streams

Images

Objects

C5Photometry

C6Spectro

C4Object

Photo DB Spectro DB Object DB

MBPBBP

VariabilityAlerts

Calibrations

SGISRVAP

DMSSolar System

PlanetsGalaxies

QSOs

Class. DB

APAge

Global Class.


Astro-1: 20 TB

Astro-2: 20 TB

RVS : 30 TB

MBP : 25 TB

50 GB/day

11h mean GS visibility per day

5 Mbps downlink rate

MOC

TMArchive

GAIA

primary

Data

Base

Initial treatementCalibrations

First lookSystem determination (GIS)

C3

Simulation

C2

Data Streams

Images

Objects Photo DB

C5Photometry

Spectro DB

C6Spectro

Object DB

C4Object

MBPBBP

VariabilityAlerts

Calibrations

SGISRVAP

DMSSolar System

PlanetsGalaxies

QSOs

Class. DB

APAge

Global Class.

CambridgeGenève

CNES

ESAC, UB , ISDC ?

UB,CNES


How to form the Consortium How to form the Consortium ??

•Setting of a DACC (Data Analysis Coordination Committee)

•Must establish the structure of the consortium :

Breakdown into CU compatible with data flow

List of DUs within each CU

Examine the funding issues, data flow problem, system architecture

Identify institutes, teams or individuals for the WPs

•DACC established for one year

Will be replaced by the DACE (Data Analysis Consortium Executive)

•An AO will be issued to formalise the structure

no competitive AO is foreseen


Composition of the DACCComposition of the DACC

•Chair : F. Mignard

•Co-chair : C. Bailer-Jones

•CU managers :

W. O'Mullane, X. Luri, U. Bastian, D. Pourbaix, F. Van Leeuwen, D. Katz

•Project scientists : M. Perryman

•+ Experts : A. Brown, U. Lammers, L. Lindegren, J. Torra


Pyramidal structurePyramidal structure

•GST (includes all the CU managers)

•Data Analysis Consortium Executive (DACE)

Consists of the CU managers

•Coordination Units

Manager, deputy manager, S/W engineers, DUs

•Development Units

Leader, includes several work packages

•Work Packages

From individual to small groups within one location


Next StepsNext Steps

•List of work-packages of each CU

Compilation document in preparation

•Construct the data flow constraints

Functional analysis

List of processes and dependencies

•CU managers build their architecture of DUs

Should reflect the answers to the LoI

Large contact with the community needed

• Next meeting of the DACC : 6-7/10 in Heidelberg


MOC: Telemetry ArchiveHousekeeping, Orbit Files

Operations Reports

Astrometry+ Data Base

(sample tasks in GDB)

Object Tasks& Classification

+ Data Base

Spectroscopy+ Data Base

(sample tasks TBD)

Photometry+ Data Base (position and flux information)

(sample tasks in GDB)

DecodingDecompression

Pass Archive(flat files, t order)

Spatial Ordering(HTM)

Cross Matching(A/P/S)

INTEGRATOR

Raw DataBase

[Oracle] Update LSF centroidPosition + Flux

Element AstromElement Photom

Match Table

Source (+RV)Attitude

CalibrationGlobal

Gaia Main Data Base

VN VN-1

Source (+RV)Attitude

CalibrationGlobal

Element AstromElement Photom

Match Table

SAGC VN-1

AstrometricGlobal Iterative

Solution

SourceAttitude

CalibrationGlobal

SpectroscopicGlobal Iterative

Solution

Calibration

Cross CorrelationCCD/FoV/Mission

ASPC results communicated via ID + parameters + data base flags + other flags

120 618

V0 (ground) V1V2

GIS + shell

Integrator

Timeline

months

8-1 weeks

Gaia Data Flow, V1.1, 20-6-05 (MP, WOM, UL, LL, JdB +UB, FvL)

Construct and applyPre-pass

Catalogue (VN-1)

1 iterationearlier

Solar system objects

Radial velocitiesCalibrated spectra

Metallicity, Rotation

Photometric Calibration Model

Source Updating(secondary stars)

AC new

SGC VN-1

Image parametersInitial cross-matchingOn-ground att. 50 mas

Cross Matching

First Look

Science Alerts

Real-Time Attitude(~6 arcsec)

PSF/LSF calibrationCCD calibrationA/P/S calibration

Heidelberg

ASM imaging(disturbing stars)

Reference Frame

Photometric CalibrationParameters

Photometric CalibrationStars

Visual binariesAstrometric binaries

Planets

Source classificationAstrophysical parameters

Calibration Monitoring Reduced photometric data

Epoch photometry

Constant stars

Accumulating statisticsVariability analysis

Period type?

No: Unsolved variables

Period search

Period type?

Yes: Periodic variables

Alert filter

Unexpected change inbrightness/classification

Alerts: SNe, NEO

ESAC

ESAC

Cambridge ?

CNES? CNES?

ESAC

Lund, Barcelona Cambridge, Leiden, Geneva Meudon, MSSL

ESAC

Heidelberg, Bruxelles, Paris

Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory 1 Synthesis of LoI's F. Mignard OCA/Cassiopée.

Documents

Transcript of Gaia, VSWG/SWG 06-08/07/05 Geneva Observatory 1 Synthesis of LoI's F. Mignard OCA/Cassiopée.