Projects using Cactus Gabrielle Allen [email protected] Cactus Retreat Baton Rouge, April 2004.
-
Upload
bryan-miller -
Category
Documents
-
view
214 -
download
0
Transcript of Projects using Cactus Gabrielle Allen [email protected] Cactus Retreat Baton Rouge, April 2004.
![Page 2: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/2.jpg)
Cactus Projects @ AEI/LSUCactus Projects @ AEI/LSU
User Support Applications Research and Development
Support for more numerical models (Saturday)
Large scale computing Frameworks Visualization Data models and formats Grid computing
![Page 3: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/3.jpg)
Community DevelopmentCommunity Development Adaptive Mesh Refinement
PAGH (Wash U) Carpet (Albert Einstein Institute)
Application Performance Modeling University of Chicago
Data Formats and Visualization Lawrence Berkeley Lab Konrad Zuse Zentrum Albert Einstein Institute
Optimization and Performance NCSA, Intel, Cray, Lawrence Berkeley Lab, Absoft
User GUIs Wash U
![Page 4: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/4.jpg)
InteroperabilityInteroperability
Pending proposal to NSF ITR program “Hypercode: Interoperable Infrastructure Initiative”
Addressing interoperability for general computational infrastructures, Cactus, Chombo, Paramesh and CCA-based frameworks
Focused around applications: Numerical Relativity Computational Astrophysics Coastal Modeling Climate Modeling Cosmology Computational Fluid Dynamics
![Page 5: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/5.jpg)
InteroperabilityInteroperability
![Page 6: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/6.jpg)
InteroperabilityInteroperability
Develop common mechanisms and abstractions to enable different simulation codes to use modules and data interchangeably
Partners: LSU LBL (John Shalf, Phil Colella, Julian Borill) U. Maryland (Kevin Olsen, Joan Centrella) U. Indiana (Denis Gannon) NCSA (Greg Daues)
![Page 7: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/7.jpg)
InteroperabilityInteroperability
Main developments for Cactus: Incorporate other AMR drivers into Cactus
– Chombo, Paramesh Incorporate other elliptic solvers into Cactus Develop a community toolkit for CFD Add new features to Cactus
– Adaptivity– Dynamic component loading
Develop common data model– New visualization tools available
![Page 8: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/8.jpg)
VisualizationVisualization
Ongoing visualization projects at AEI, LSU, LBL with Cactus
Pending DST-NSF proposal with the computer science department and C-DAC to build visualization infrastructure to allow data to be analyzed and visualized on the fly Additions to Cactus I/O infrastructure Thorns for visualization Web-based visualization tools
![Page 9: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/9.jpg)
Large Scale ComputingLarge Scale Computing
NSF Software Technologies for High End Computing (being written)
Incorporate fault tolerant MPI in Cactus driver layer University of Tennessee
Develop HTTPD thorn into an interactive, real time parallel debugger
Detect and exploit memory and network connection hierarchy from processor cache, through node layout on clusters of SMPs, to cluster interconnections on the Grid
Performance monitoring and adaption using e.g. PAPI library
![Page 10: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/10.jpg)
Grid ComputingGrid Computing
… infrastructure enabling the integrated, collaborative use of high-end computers,
networks, databases, scientific instruments owned and managed by multiple
organizations …
… applications often involve large amounts of data and/or computing,
secure resource sharing across organizational boundaries, not
easily handled by today’s Internet and Web infrastructures …
“resource sharing and coordinated problem solving in dynamic, multi-institutional virtual organizations”
![Page 11: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/11.jpg)
Remote Monitoring/Steering: Remote Monitoring/Steering: HTTPD Thorn which allows
simulation to any to act as its own web server
Connect to simulation from any browser anywhere … collaborate
Monitor run: parameters, basic visualization, ...
Change steerable parameters Running example at www.
CactusCode.org Wireless remote viz, monitoring
and steering
![Page 12: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/12.jpg)
User Portal (ASC and GridSphere)User Portal (ASC and GridSphere)
Collaboration focal point for a virtual organization
Interact, share data Start jobs on remote
resources Move/browse files Track and monitor
announced jobs Access to new Grid
technologies www.ascportal.org www.gridsphere.org
![Page 13: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/13.jpg)
Portal Server
SMS Server
Mail Server
“TestBed”
Running
Appli cations
Notification (Announce)Notification (Announce)
![Page 14: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/14.jpg)
OpenDX, Amira, …
HDF5
GridFTP VFD
Stream VFD
Visualization Tools
GridFTP
Remote Data Server
IOStreamedHDF5
Simulation
Hyperslabbing, Downsampling
Remote Data and Visualization (GriKSL)Remote Data and Visualization (GriKSL)
![Page 15: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/15.jpg)
Dynamic Adaptive Distributed ComputationDynamic Adaptive Distributed Computation
SDSC IBM SP1024 procs5x12x17 =1020
NCSA Origin Array256+128+1285x12x(4+2+2) =480
OC-12 line
(But only 2.5MB/sec)
GigE:100MB/sec
17
12
5
4 2
12
5
2
These experiments: Einstein Equations (but could be
any Cactus application)Achieved:
First runs: 15% scaling With new techniques: 70-85%
scaling, ~ 250GF
“Gordon Bell Prize” (Supercomputing 2001,
Denver)
Dynamic Adaptation: Number of ghostzones, compression, …
![Page 16: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/16.jpg)
New Grid ScenariosNew Grid Scenarios Intelligent Parameter Surveys, Monte Carlo Dynamic Migration: faster/cheaper/bigger
machine Multiple Universe: create clone to investigate
steered parameter Automatic Component Loading (Needs of process
change) Automatic Convergence Testing Look Ahead Spawn Independent/Asynchronous Tasks Routine Profiling: best machine/queue,
parameters Dynamic Load Balancing: inhomogeneous loads,
multiple grids
![Page 17: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/17.jpg)
GridLabGridLab
Cactus experiments with grid computing on the E-Grid Cactus Worm: thorns which allowed simulations
to migrate themselves from machine to machine Spawning: sending (asynchronous) calculations in
analysis thorns to different machines We wrote the GridLab proposal to be able to do
these, and other scenarios, in a better way
![Page 18: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/18.jpg)
GridLab ProjectGridLab Project
http://www.gridlab.org EU Funded ($5M) by 5th Framework
January 2002-December 2004): Many partners in Europe and US
PSNC (Poland), AEI & ZIB (Germany), VU (Netherlands), MASARYK (Czech), SZTAKI (Hungary), ISUFI (Italy), Cardiff (UK), NTUA (Greece), Chicago, ISI & Wisconsin (US), Sun/Compaq
LSU now a collaborating partner
![Page 19: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/19.jpg)
GridLabGridLab
Developing an easy-to-use, flexible, generic and modular Grid Application Toolkit (GAT), enabling applications to make innovative use of global computing resources
Focused on two principles: co-development of infrastructure with real
applications and user communities (Badly needed in grid computing!!)
dynamic use of grids, with self-aware simulations adapting to their changing environment.
![Page 20: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/20.jpg)
GridLabGridLab
12 Work Packages covering:Grid PortalsMobile Users
Different Grid ServicesApplications(Development) Test Bed
Grid Application Toolkit (GAT)
![Page 21: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/21.jpg)
Grid Application ToolkitGrid Application Toolkit
Need a layer between applications and grid infrastructure:
Higher level than existing grid APIs, hide complexity, abstract grid functionality through application oriented APIs
Insulate against rapid evolution of grid infrastructure and state of grid deployment
Choose between different grid infrastructures Make it possible for application developers to use
and develop for the grid independent of the state of deployment of the grid infrastructure
![Page 22: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/22.jpg)
Monitoring
Resource Management
InformationSecurity
DataManagement
GLOBUS
ApplicationManager
Logging
NotificationMigration
Profiling
SOAP WSDL Corba OGSA Other
Other GridInfrastructure?
Application
“Is there a better resource I could be using?”
![Page 23: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/23.jpg)
Application
“Is there a better resource I could be using?”
GAT_FindResource( )
The Grid
![Page 24: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/24.jpg)
The Same Application … The Same Application …
Application
GAT
Application
GAT
Application
GAT
Laptop The GridSuper Computer
No network! Firewall issues!
![Page 25: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/25.jpg)
GAT: Grid Application ToolkitGAT: Grid Application Toolkit
Standard API and Toolkit for developing portable Grid applications independently of the underlying Grid infrastructure and available services
Implements the GAT-API Used by applications (different languages)
GAT Adaptors Connect to capabilities/services
GAT Engine Provides the function bindings for the GAT-API
http://www.gridlab.org/software/GAT
![Page 26: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/26.jpg)
GAT ArchitectureGAT Architecture
![Page 27: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/27.jpg)
Cactus/GAT IntegrationCactus/GAT Integration
GATLibrary
Cactus Flesh
Thorn
CGATThorn
Thorn
Thorn
Thorn
Thorn
Physics and Computational Infrastructure
Modules
Cactus GAT wrappers Additional
functionalityBuild system
GridLab Service
GridLab Service
Grid ScenarioThorn
Grid ScenarioThorn
![Page 28: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/28.jpg)
TFM
TFM TFM TFM TFM
Task Farming on the GridTask Farming on the Grid
TFM implementedin Cactus
GAT used for starting remote TFMs
Designed for the Grid
Tasks can be anything
![Page 29: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/29.jpg)
Grid-Black HolesGrid-Black Holes
Task farm small Cactus black hole simulations across testbed
Parameter survey: black hole corotation parameter
Results steer a large production black hole simulation
![Page 30: Projects using Cactus Gabrielle Allen gallen@cct.lsu.edu Cactus Retreat Baton Rouge, April 2004.](https://reader030.fdocuments.net/reader030/viewer/2022032709/56649eb45503460f94bbbae8/html5/thumbnails/30.jpg)
Physicist has new idea !
S1 S2
P1
P2
S1S2
P2P1
SBrill Wave
Dynamic Grid ComputingDynamic Grid Computing
Found a horizon,try out excision
Look forhorizon
Calculate/OutputGrav. Waves
Calculate/OutputInvariants
Find bestresources
Free CPUs!!
NCSA
SDSC
RZG
LRZ
Archive data
SDSC
Add more resources
Clone job with steered
parameter
Queue time over, find new machine
Archive to LIGOpublic database