[email protected] SAG Meeting – Postdam – November 2006 - The XCATDB: run by...

[email protected]

SAG Meeting – Postdam – November 2006 - The XCATDB: run by Saadahttp://amwdb.u-strasbg.fr/jacds

OAS Contribution

• The ACDS

• The new XCATDB based on Saada

Laurent MICHELChristian MOTCHFrançois Xavier PINEAU

Observatoire Astronomique de Strasbourg (Fr)Saada: http://astro.u-strasbg.fr/websaadaXCATDB: http://amwdb.u-starsbg.fr/jacds

[email protected]


The ACDS

• Delivered Products

– FITS catalogue extractions• X-ray source counterparts• Archival sources in the XMM field of view

– Graphical products (PDF)• Finding charts • Rosat Images• Catalogue plots

– HTML product• Browsing pages for ACDS data with many links to CDS resources

[email protected]


The ACDS

Vizier at CFA(Harvard USA)

Vizier at Cambridge(UK)

Vizier at CADC(Victoria Canada)

ACDS:• 2 Sparcs 400 Mhz• 1Gb RAM• 8 Agents in //

Vizier at CDS(Strasbourg France)

Simbad(Strasbourg France)

NED(Caltech USA)

Aladin(Strasbourg France)

Pipeline(Leicester)

• Correlation computing• A lot of remote server accesses• A few computation

• File product generation• Some remote server accesses (Aladin)• A lot of computation

• Correlation computing• A lot of remote server accesses• A few computation

• File product generation• Some remote server accesses (Aladin)• A lot of computation

[email protected]


The ACDS

20%

72%

8%

Cross CorrelationGraphical Products Other Tasks

• Actual Throughput (8 process in //)– 3300 sec /dataset

– (8*3600*24)/3300 = 209 datasets/day

• Most part of the time is pure CPU (graphical prd)– IDL, Distiller….

– Easy to improve

• new hardware

[email protected]


The ACDS

• Operation issues

– Support the pipeline throughput (but not more)• > 2200 datasets processed since October 13th

– 157 pending this morning due to a 2 days Aladin shutdown• No need to improve the hardware

– CDS database upgrades• Vizier is being moved on PSQL without serious consequences for us• Major Simbad to be release late 2006

– New error definition (no effect for standard users)

– External failures• Remote server shutdown• Corrupted products returned (cache miss-synchronization after crash)

– Internal failures• Empty Rosat images/wrong PPS index• Hardware failure

– Spare available/never used• Miss-configuration• Holidays

[email protected]


The ACDS

[email protected]


The XCATDB

• The XCATDB is not a data repository but a tool to make science

• N to M persistent relationship– Difficult to implement in an efficient way for complex queries– Possibility of doing more than with dynamic cross-matches

• Data-mining feature

X Source 1 Arch Source 1

Arch Source 2

Arch Source 3

Arch Source 4

Arch Source 5

X Source 2

X Source 3

[email protected]


Saada in a few Words

• Saada: An Astronomical Database generator– Making automatic the process

of building a database

– Hosting heterogeneous datasets

– Highlighting scientific content

– Publishing personal data into the VO

• Databases are installed on local machines– Any Linux/Windows/Mac box

– Tomcat 5.xx

– PostrgreSQL 8.xx

[email protected]


The XCATDB

• Based on Saada– Queries expressed in SaadaQL– Data loader

• How to build quickly millions of correlation links• How to restore archival source uniqueness

– GUI• Layout inspired from the 1XMM XCatDB• Architecture based on native Saada GUI

• Prototype based on RC2 data open for the community– http://amwdb.u-strasbg.fr/jacds

Source X

Source X

Source X

Arch Source 1

Arch Source 2

Arch Source 1

Arch Source 1

Arch Source 2

Source X

Source X

Source X

[email protected]


2nd Release of the XMM Catalogue

150,000 X-ray sourcesIn 2800 observations

1,400,000 archival sources

1,000,000 links

Implemented with Saada relationshipsSpecific Module

Native Saada Data-loader

Native Saada Data-loader

Demo

[email protected]


A simplified example: Looking for High Mass X-ray Binaries

Hardness ratio range for X-ray binaries

Hardness ratio limit for stars

HR3 = (F2-4.5keV – F1-2keV) / (F2-4.5keV + F1-2keV)HR4 = (F4.5-12keV– F2-4.5keV) / (F4.5-12keV + F2-4.5keV)

HR3 = (F2-4.5keV – F1-2keV) / (F2-4.5keV + F1-2keV)HR4 = (F4.5-12keV– F2-4.5keV) / (F4.5-12keV + F2-4.5keV)

Demo

< 1

.5ke

V

kT =

5 –

50k

eV

[email protected]


HR Condition

With bright counterparts

Without WR counterparts

[email protected]


Using UCDs in queries

• Selecting X-Rays sources by correlation patterns on 200 classes of X-Ray sources requires to handle a lot of meta-data

select oidsaada from Arch_9037AEntry where ( _Flux > 1e-13 )select oidsaada from Arch_7181AEntry where ( _Flux2 > 1e-16 ) select oidsaada from Arch_9032AEntry where ( _FX > 1e-16 ) select oidsaada from Arch_9015AEntry where ( _fX > 1e-16 ) select oidsaada from Arch_9031AEntry where ( _Fx > 1e-13 )………….

[phot.flux;em.X-ray] > 1e-16 [W/m2]

• Solution: Expressing queries using UCDs and Units

Demo

[email protected]


(att1>x1 and att1<x2) or (att2>x1 and att2<x2) or (att1>x1 and att2<x2) or (att2>x1 and att1<x2)

UCD Issue

[UCD]>x1 and [UCD]<x2

Usually makes no sense

Solution: Using unambiguous operators: [UCD]In(x1,x2)

(att1>x1 or att2>x1) and (att1<x2 or att2<x2)

[email protected]


UCD Issue (continued)

• Query language issue: using a simple syntax– One operator for one UCD

• Query execution– Editable execution plan

• UCD attribution: a sensible issue– Using multi-words UCDs as often as possible– Limit the columns accessible per UCDs– Take care with units

• Existence• Name: known by the converter

• Result presentation: another sensible issue– The client must understand how resources have been selected and accessed

[email protected]


XCATDB: Status and Prospects

• Data loader– Correlation setup works fine– Other products can be loaded with the native data-loader

• GUI– HTML form improvements– Display of constrained values

• Interoperability– External Xmatch in SaadaQL (F.X. Pineau)– Connection with ESAC resources (XSA, VOSpec)– Connections with CDS clients (VO portals)– VO protocols

[email protected]


Thank You for your Attention

[email protected] SAG Meeting – Postdam – November 2006 - The XCATDB: run by...

Documents

Transcript of [email protected] SAG Meeting – Postdam – November 2006 - The XCATDB: run by...