● STILTS– Command-line tools for table/VOTable manipulation

● TOPCAT VO features– VO features in the GUI tabular data exploration tool

● SPLAT-VO– SSAP functionality in GUI spectral analysis tool

Starlink VO Applications Software

http://www.starlink.ac.uk/

STILTS

● Command-line tools for table manipulation● VOTable structure-sensitive tools

– votcopy: DATA encoding conversion– votlint: VOTable document validation

● Generic table tools:– tcopy: format conversion– tpipe: table pipeline processing

● Current release is beta

STIL Tool Set

http://www.starlink.ac.uk/stilts/

tcopy● Converts tables between formats● Streaming operation where possible● Extensible to new formats

VOTable

FITS

SQL query

ASCII

CSV

...

VOTable

FITS

SQL table

ASCII

HTML

LaTeX

Mirage

...

tcopy

tcopy usage scenarios

● The VO is a heterogeneous environment

● Convert from legacy formats to FITS/VOTable

● Adapt VOTable data to formats suitable for legacy tools

● Populate SQL databases from FITS files

– ... or vice versa● Provide choice of output format for table query output

– put tcopy as a layer between standard output and what gets delivered to the user

– easy to write an HTML form providing output format options

– or a web service parameter naming chosen format

Interoperability!

tpipe● Multi-purpose scriptable table pipeline● Uses powerful extensible expression language

VOTableFITS

ASCIICSV

...

file

stream

SQL query

select rows

new column

delete column

rearrange cols

sort rows

sample rows

head/tail

VOTableFITSASCIICSV...

file

stream

new SQL table

calculate statistics

show metadata

display in TOPCAT

input outputprocess

tpipe expression language

● Algebraic expressions in terms of existing columns– Used for row selection, defining new synthetic columns

– Refer to existing columns by column name or index

– Normal arithmetic/relational operators plus useful functions● trigonometry, string handling, coordinate conversion, conditionals, ...

– Null value handling supported

● Compiled to Java bytecode at runtime (uses JEL)– Fast execution

– Full power of Java language

● Extensible by supplying own classes at runtime

tpipe examples● Display first few rows of table

tpipe survey1.xml -head 5

● Select rows on algebraic expressiontpipe cat.fits -select 'BMAG < 21.2 && startsWith(NAME,”cyg”)' -write catsub.fits

● Add new calculated columnstpipe -ifmt csv data.csv -addcol MAG_MEAN “0.5 * (IMAG + VMAG)” -addcol IV_COLOUR “VMAG - IMAG”

● Convert columns from sexagesimal to degreestpipe -ifmt csv acat.csv -addcol RA '“radiansToDegrees(hmsToRadians($1,$2,$3))”' -addcol DEC '“radiansToDegrees(dmsToRadians($4,$5,$6))”' -delcols '“$1,$2,$3,$4,$5,$6”'

● Calculate and display statistics on sample of large tabletpipe USNO-B.FITS -every 1000000 -stats

● Display the thousand highest redshifts in TOPCATtpipe 2QZ.fits.gz -keepcols “NAME RA DEC Z1” -sort Z1 -tail 1000 -topcat

tmatch

● ... maybe one day

TOPCAT

● View/analyse/edit/write tabular data– Manipulate table data and metadata

– Plot, match, sort, calculate, select, ...

– Powerful and extensible expression language

– Multi-format, including VOTable (all encodings) & SQL

– Large tables (10^6 rows x 10^2 columns)

http://www.starlink.ac.uk/topcat/

Tool for OPerations on Catalogues And Tables

TOPCAT VO features● Load/Save dialogues

– Filestore browser

– Cone search

– SIAP query

– Registry query

● Activation actions– Display images/spectra corresponding to table rows

Filestore Browser● Looks like standard file chooser

● Remote filespaces are available in the same way as local filesystem

● Log in/out to view remote filespaces

● Both load and save are available

● Pluggable framework for filespace implementations

– Currently MySpace and SRB supported

– Not hard to add new ones

● Available as separate library component for use in other applications

Cone Search● Get cone search services from registry

● Perform a query on one of the services returned

● Open results as a table

● SIAP and Registry queries work in a similar way

Activation actions● Can configure some action when table row is activated

● Activation performed by:

– clicking on row in table display

– clicking on point in plot

● Actions include:

– display cutout image from service (SoG)

– display spectrum from service (SPLAT)

– ... or anything you like

TOPCAT as service client● Query registry for SIAP/Cone search services

● Perform a query on one of the services returned

● Plot/view results and select a point/row

● Retrieve resulting data file and display image/spectrum

...but it's a bit clunky

Starlink SPLAT for the VO

SPLAT features

• Not just a VO client, also an analysis tool with extensive graphics features, printing, zooming, scrolling etc.

• Source code available under GPL.

• Available as a desktop or webstart application (Linux, Windows, Mac OS X and Solaris).

• http://www.starlink.ac.uk/splat-vo

New SSAP Browser

● Various interactive features: object name lookup, RA and Dec in natural units (hh/ddd:mm:ss.s). Simple double click to load a spectrum.

Double click download

Starlink AST• All coordinate and units transforms provided by AST.

• Spectral coordinates cover FITS-WCS paper III: Wavelength, Frequency, Energy and Velocity

• Flux units, based only on interpretation of standard FITS-WCS paper I descriptions:

Jy, W/m^2/Hz, W/m^2/Angstrom, W/cm^2/um, erg/cm^2/s/Hz, erg/cm^2/s/Angstrom

Composite plot of VO data

3C273 all frequencies

Summary● STILTS

– tcopy - table format conversion

– tpipe - table pipeline processor

● TOPCAT VO features– local/remote filestore browser

– passable SIAP/Cone search service client

● SPLAT-VO– Spectral analysis program, SSAP client, sophisticated

coordinate handling

● Other items not mentioned here● The future is uncertain...