Identification of Important VO Spectral Services benefiting from deployment on the GRID
Theory, Grid and VO
description
Transcript of Theory, Grid and VO
![Page 1: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/1.jpg)
09.04.2008
Theory, Grid and VO
Matthias Steinmetz (AIP)
![Page 2: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/2.jpg)
09.04.2008
applicationData
analysis
Super Computer
Super computer
Telescopes
Data Archive
DataArchive
PC Cluster
PC Cluster Data analysis
User
MIDDLEWARE
Resources are "virtualized", i.e. can not be identified
individually
Characteristics of a Grid: network of IT-Ressourcen
![Page 3: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/3.jpg)
3VODCA: Theory, Grid and VO09.04.2008
VO and Grid
What is the dividing line between VO and Grid? Not well defined Example UK:
AstroGrid covers VO and Grid aspects Example Germany
GAVO: application layerAstroGrid-D: middle ware
![Page 4: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/4.jpg)
4VODCA: Theory, Grid and VO09.04.2008
Theory and Grid
Benefits of the Grid Logistics (resource monitoring, scheduler/broker,
virtual organizations, …) Virtual Surveys (Millenium simulation) Enterprise computing (access to supercomputers
via grids, e.g. DEISA) Cloud computing (“task farming”) Volunteer computing (@home model) Visualization
![Page 5: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/5.jpg)
5VODCA: Theory, Grid and VO09.04.2008
StellarIS: resource monitoring
Grid-Ressource-Map
Basiert auf GoogleMap
![Page 6: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/6.jpg)
6VODCA: Theory, Grid and VO09.04.2008
Stellaris: Job monitoring
Webinterface for simple job monitoring
Time table forsubmitted
Jobs
Minutes
hours
days
![Page 7: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/7.jpg)
7VODCA: Theory, Grid and VO09.04.2008
Analysing Cosmological Simulations in the Virtual Observatory:Designing and Mining the Millennium Simulation Database
Gerard Lemson German Astrophysical Virtual
Observatory
ARI, Heidelberg
MPE, Garching bei München
![Page 8: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/8.jpg)
8VODCA: Theory, Grid and VO09.04.2008
Time evolution: merger trees
![Page 9: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/9.jpg)
9VODCA: Theory, Grid and VO09.04.2008
![Page 10: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/10.jpg)
10VODCA: Theory, Grid and VO09.04.2008
Merger trees :select prog.* from galaxies des , galaxies prog where des.galaxyId = 0 and prog.galaxyId between des.galaxyId and des.lastProgenitorId
Branching points :select descendantId from galaxies des where descendantId != -1 group by descendantId having count(*) > 1
![Page 11: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/11.jpg)
11VODCA: Theory, Grid and VO09.04.2008
![Page 12: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/12.jpg)
12VODCA: Theory, Grid and VO09.04.2008
Usage statistics
Up since August 2006 (astro-ph/...) ~210 registered users > 4.4 million queries ~ 35 billion rows (since March 2007)
# queries/day # rows/day # secs/day
![Page 13: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/13.jpg)
13VODCA: Theory, Grid and VO09.04.2008
Enterprise Computing:Mare Nostrum Simulations at HLRZ
WMAP3 parameters, w=0.8Testrun using the grid: 10243+10243 particles
![Page 14: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/14.jpg)
14VODCA: Theory, Grid and VO09.04.2008
NBODY6++ UseCase
Computer simulation of dense stellar systems
Example: globular clusters
Gravitational Star-Star interactionComplexity N2 (N: number of stars)
![Page 15: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/15.jpg)
15VODCA: Theory, Grid and VO09.04.2008
Enterprise ComputersEnterprise Computers
Recent Development: GPU – Graphics Cards
GeForce 8800 GTX (NVIDIA)Using CUDA LibrarySpecial Interfaces and API from GRACE project ported.
Berczik et al. 2008Spurzem et al. 2008
![Page 16: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/16.jpg)
16VODCA: Theory, Grid and VO09.04.2008
Cloud Computing: UseCase Dynamo
Visualization of results
Rechner 1 Rechner 2
Rechner 3 Rechner 4
2D-Darstellungder
Magnetfeldstärke auf der
Sternoberfläche
Querschnittdurchden
Stern
![Page 17: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/17.jpg)
17VODCA: Theory, Grid and VO09.04.2008
<jsdl:JobDefinition xmlns="http://www.gacg-grid.de/namespaces/job-mgmt/2006/08/jsdl" xmlns:jsdl="http://schemas.ggf.org/jsdl/2005/11/jsdl" xmlns:jsdl-posix="http://schemas.ggf.org/jsdl/2005/11/jsdl-posix" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<jsdl:JobDescription> <jsdl:JobIdentification> <jsdl:JobName> Sample Dynamo run </jsdl:JobName> <jsdl:Description>
Use Case Dynamo </jsdl:Description> <jsdl:JobProject> n/a </jsdl:JobProject> </jsdl:JobIdentification><jsdl:Resources> <jsdl:FileSystem name="HOME"> <jsdl:Description> User's home directory </jsdl:Description> </jsdl:FileSystem> </jsdl:Resources>
[…]
<?xml version="1.0" encoding="UTF-8"?>
<job> <executable>test.x</executable> <directory>/${GLOBUS_USER_HOME}/
dynamo</directory> <stdout>test.out</stdout> <maxWallTime>100</maxWallTime> <maxMemory>1</maxMemory> <fileStageIn> […]
Dynamo JDSL und RSL
![Page 18: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/18.jpg)
18VODCA: Theory, Grid and VO09.04.2008
Volunteer Computing: GEO600 / LIGO
Laser Interferometer Gravitational Wave Observatory
![Page 19: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/19.jpg)
19VODCA: Theory, Grid and VO09.04.2008
GEO600/LIGO
Network von 4 Detectors Hanford (2000m side length) USALivingston (4000m side length) USAGEO600 ( 600m side length) GermanyVirgo (3000m side length) Italy
Pathfinder for LISA, Satellite mission with 3 detectorsside length: 5*109 m!
![Page 20: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/20.jpg)
20VODCA: Theory, Grid and VO09.04.2008
Gravitationional waves: Data analysis via the Grid
Data analysis via small data packages, “embarrassingly parallel”.
Einstein@Home is, like SETI@Home, suitable to exploit idle cycles on work stations.
Einstein@Home is an ideal, simple Grid application, supporting many operation system.
Checkpointing and Recovery allows a very accurate control of CPU-Requirements and walltime.
Automatic software deployment job submission and job management, a good scalability of grid application can be obtained
Current workload: 30000 CPU h per week
![Page 21: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/21.jpg)
21VODCA: Theory, Grid and VO09.04.2008
GEO600 – Resource Integration
user friendly User-Management via VOMRS
Resource information via MDS und StellarIS
Grid Service Monitoring
automatic job submission on D-Grid resources
Job monitoring und job management using a
Laptop
data management on Astrogrid-D Storage
Cluster
![Page 22: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/22.jpg)
22VODCA: Theory, Grid and VO09.04.2008
Visualization of a galaxy merger
Simulation: two galaxies on collision orbit Visualization: 2D-projections of 3D snapshots
ZIB AIP
Submit Execution
ProC SubmitWorkflow
Video Workflow A
stop
exit
exit
start
GT4 submit
GT4 submit
Video Workflow B
ARI ...
![Page 23: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/23.jpg)
23VODCA: Theory, Grid and VO09.04.2008
Grid-Visualization
Submission Host: ZIB ProC + Master workflow
Submission of video workflows Display of videos
Execution Hosts: AIP + ZAH PiCo + Video workflow
Projection to 2D Color coding
Future: Graphics rendering at LRZ, graphics output on local host
![Page 24: Theory, Grid and VO](https://reader030.fdocuments.net/reader030/viewer/2022032710/56814011550346895dab5304/html5/thumbnails/24.jpg)
24VODCA: Theory, Grid and VO09.04.2008
Theory and Grid
Benefits of the Grid Logistics (resource monitoring, scheduler/broker,
virtual organizations, …) Virtual Surveys (Millenium simulation) Enterprise computing (access to supercomputers
via grids, e.g. DEISA) Cloud computing (“task farming”) Volunteer computing (@home model) Visualization