Army High Performance Computing Research Center Vipin Kumar Director, AHPCRC Overview of Research...

Army High Performance Computing Research Center

Vipin KumarDirector, AHPCRC

Overview of Research Activities

Overview of AHPCRC Research Activities

AHPCRC

High performance computing (HPC) plays a key role in bringing about a transformation from the heavily mechanized, slow to deploy forces, to an objective force.

Mission: Leading edge HPC research in support of the Army’s science and

technology goals for the Objective Force and Army Transformation.

Transfer of technology to the Army via direct interactions with Army scientists and through an Infrastructure Support program.

Education and outreach programs that include the Summer Institute for undergraduate students and a series of international/national workshops and conferences.

Forging of synergistic relationships among partner institutions via strong, collaborative research.


AHPCRC Team

Clark Atlanta University (CAU)

Florida Agricultural and Mechanical University (FAMU)

Howard University (HU)

Jackson State University (JSU)

University of Minnesota (UM)

University of North Dakota (UND)

NetworkCS (NCS)

AHPCRC – Current Research


Electromagnetic Signature Modeling

Projectile Target Interaction

Virtual Computing Environment

Chem/Bio Defense & Environmental

Modeling

ET & Computational

Algorithms

Atmospheric Science X XChemical/Biological Defense XEnvironmental Quality Modeling XComputational Fluid Dynamics X XEnergetic Materials XComputational Mechanics XSignature Modeling XAlgorithms for PDE X XAlgorithms for Transient/Dynamic Simulations

X X X

ET in support of NCC XET in support of ICR X X

Research Program


Portfolio: ELECTROMAGNETIC SIGNATURE MODELING IN THE SYNTHETIC BATTLEFIELD

Objectives: Develop HPC modeling and analysis techniques for low observable technologies to obtain an improved understanding of the entire spectrum of signatures of combat systems in a realistic battlefield environment.

Computational modeling of radio frequency (RF) and infrared (IR) signatures, and the effects of coatings on these signatures.

Understanding the effect of atmospheric on gathering and detecting acoustic and near infrared (IR) signatures.


Portfolio: ELECTROMAGNETIC SIGNATURE MODELING IN THE SYNTHETIC BATTLEFIELD

Projects:

RCS Modeling of Multilayered Material Structures/Coatings.

Solution of Electromagnetic Field Equations Using Basis Sets in Time and Space and Surface Interactions.

Signature Modeling Applications of an Innovative Objective Analysis Technique.

Fine Scale Modeling and Product Generation for Visualization of Atmospheric Dynamics and Phenomena.

Algorithms For Signal and Image Processing, and ATR.


Radar Cross Section (RCS) Modeling of Multilayered Material Structures/Coatings

Objectives Determine the radar signatures of military hardware, and the use of various methods to control them, including the shaping of the geometry to reduce electromagnetic signatures, and the use of coatings or shields which are absorptive or which scatter incoming signals to reduce these returns.

MethodologyThe program is focusing on the formulation of a boundary integral approach for the solution of 3-D electromagnetic scattering and on the development of a corresponding code.


Atmospheric Effects on Signature Modeling

Extend current generalized response function framework to multiple dimensions Theoretical framework Testing and Validation

Develop variationally-based filter Theoretical framework Evaluation

Develop software that applies the new filter Initial (preliminary) incorporation of the new filter into Local Analysis and Prediction System

(LAPS)


Portfolio: PROJECTILE TARGET INTERACTION

Objectives: Multi-disciplinary HPC modeling and simulations technology for projectile-armor/anti-armor target interaction (including blast-target interaction) in support of Future Combat Systems (FCS). Computational fluid dynamics and

aerothermodynamics for external and internal flows in anti-tank projectiles and missiles.

Structural dynamics, contact-impact and penetration. Energetic materials.


Portfolio: PROJECTILE TARGET INTERACTION

Projects:

Computational Structural Mechanics and Dynamics. Discontinuous Galerkin Method In Linear/Nonlinear Structural Mechanics. Developments Of Stable, Locking Free Finite Elements for Nonlinear Structural Problems Without Hourglass

Control. Characterization, Aging, and Dynamics of Energetic Materials. Energetic Nano-particle Architecture and Properties. Modeling and Simulation of Nano-particle Growth in Turbulent Flows. Mining Computational Chemistry Data Sets. Complex Configuration Aerodynamics with Time-Dependant Interactive Geometries. Computational Analysis of Transitional and Turbulent High-Speed Flows. Numerical Simulation of Turbulent Reacting Flows. Scalable Parallel Algorithms for Partitioning Multi-physics and Multi-phase Computations. Molecular Dynamic Simulation of Atomic-Scale Friction and Wear for Micro Electro Mechanical Systems (MEMS)

Applications.


Synthesis and Properties of NanoEnergetic Materials

Current Modeling

• 20-40 Atoms (quantum mechanics)

• 104-105 Atoms (molecular dynamics)

• Simple idealized interactions

• Limited Spatial and Time scales

• Perfect Crystals, 2-Dimensional

Objective

Monte-Carlo based simulation and classical molecular dynamics (MD) approaches to characterize the evolution of surface to volume ratio and properties of energetic nanoparticles.

Model includes nanoparticle coagulation and finite coalescence, as well as energy release and non-isothermal effects resulting from coalescence.

Surface energy and kinetic rates for caolescence determined from MD calculations.

Computational Approach

Develop a hierarchical computational approach to characterize the thermophysical properties of nanoparticles and their manufacture in areas of interest to the Army, including: -- Predicting the properties of nanoparticles for

application in energetic materials -- Predicting the morphology and architecture

of nanoparticles grown from vapor -- Augmenting the experimental program of the

Army funded Center for Nanoenergetics Research (CNER)

Current Modeling• Build a model capable of modeling the morphological evolution of aluminum nanoparticles using bulk property data.

• Develop a simulation capability for Aluminum nanoparticles using an in-house code or with “Temperature Accelerated Dynamics.”

• Conduct detailed comparisons to experimental results whenever feasible.


Modeling and Simulation of Nanoparticle Growth inTurbulent Flows

Instantaneous concentration of 4nm diameter particlesTurbulence/transport effects significant downstream

Goal: Develop an analytical and algorithmic framework to facilitate the prediction of nanoparticle formation and growth in turbulent flows.

Zero-dimensional simulations Laminar flow approximations Approaches: moment methods, direct simulation, and

discrete / sectional methods


Numerical Simulation of Turbulent, High-Speed Flows

Three-dimensional round jet of propane issuing into air.Non-reacting (hydrodynamic and hydrochemical interactions only)

Goal: Develop "subgrid-scale" SGS closures for turbulent reacting flows, and to implement these closures in flows representative of those encountered in projectile-target interactions.

Reynolds-averaged Navier-Stokes Simulation Computationally affordable Closures model wide range of length scales

Large Eddy Simulation Capture “large” scale - Model “small” scale Computationally affordable?


Parallel Software Developments

Critical stages Scalable parallel graph partitioning – Mapping and load balancing Sub-domain integration – Domain decomposition techniques Time integration operators

Scalable computations - arbitrary large problems and processors Congruent amongst sub-domains Reduced solution times Numerical scalability – no degradation in

convergence of numerical algorithms. Non-linear solvers and linear solvers

Parallel - ability to deliver speedups in terascale range Memory utilization

REduced Complexity In Programming Environment – [RECIPE] paradigm Few lines of code - unified and integrated

implementation, increased functionality, arbitrary increase in order of accuracy of time integrators

Code optimization – reduced development efforts

Cray T3E3D bi-directional torus

interconnection network

SGI Originbristled fat hypercube

interconnection network


Portfolio: VIRTUAL COMPUTING ENVIRONMENT FOR FUTURE COMBAT SYSTEMS

Objectives: Develop a virtual computing environment for Future Combat Systems which includes a synthetic battlefield and the creation of a multidisciplinary computing environment for virtual HPC design.

Scene Generation Visualization


Portfolio: VIRTUAL COMPUTING ENVIRONMENT FOR FUTURE COMBAT SYSTEMS

Projects:

Visualizing Spatial/Temporal Data for Battlefield Visualization.

High Performance Geographic Information Systems for Battlefield Visualization in Future Combat Systems. The Virtual Data Grid: An Infrastructure to Support Distributed Data-centric Applications.

Rapid Physical Prototyping in Support of Battlefield Visualization and Signature Modeling.

Innovative HPC Design and Analysis Approaches for Flexible Multi-body Structures.


High Performance Geographic Information Systems (GIS) for Battlefield Visualization in

Future Combat Systems

HPGIS

National Assets, e.g. Maps

Sensor Network

Shooters NetworkMaps are as important to soldiers as guns

Example Usage of Geographic Info. Systems (GIS) in Battlefield :• Rescue of pilots after their planes went down (recently in Kosovo)• Precision targeting e.g. avoid civilian casualities (e.g. friendly embassies)• Logistics of Troop movements, avoid friendly fires


RPP is used to create physical prototypes of 3D solids from their digital representations, using a “3D printer” attached to a workstation.

RPP “prints” the 3D model as a stack of 2D layers, using a technique called Layered Manufacturing.Goal: Create, on demand, physical scale models of enemy terrains and assets to help mission planners and field commanders develop and evaluate different combat strategies.

Model Acquisition•CAD Software•CT Scans•Laser Scanning•3D Photography

Computer-AidedProcess Planning•File repair•Model orientation•Slicing•Support creation

Model Building via

Layered Manufacturing

LAN orInternet

LAN orInternet

Postprocessing•Remove supports•Improve finish•Inspect model

Rapid Physical Prototyping


The Virtual Data Grid: An Infrastructure to Support Distributed Data-centric Applications

VDG is a persistent data network designed to support military applications both in the field and at the Army labs

Key Features: Reliable, Efficient, Security, Heterogeneity

connections may be intermittent

ET metacomputingportfolio

data can be pushed or pulled into VDG

VDG API for client access

VDG API for data producers

hpc simulations can be launched byVDG clients to “produce” VDG data

VDG server

VDG server

VDG server

sensor networks national assets Internet sources hpc simulations

battlefield simulation battlefield visualization

wired or wireless access

VDG user

information grid/sensor grid/shooter grid


Portfolio: CHEM-BIO DEFENSE AND ENVIRONMENTAL MODELING

Objectives: An understanding of protein interactions with toxins and

pathogens at the atomistic level to help counteract chem-bio threats.

Modeling of the adsorption, transport, diffusion, and dispersion of chemical and biological agents within, across, or into a variety of media, i.e., the atmosphere, water, soil, clothing, building materials, and vegetation

Millard et al. (1999) Biochemistry 38, 7032-9

E199

Bound Sarin

H440 E327

F288F290

G119

G118

A201


Portfolio: CHEM-BIO DEFENSE AND ENVIRONMENTAL MODELING

Projects:

Chemical/Biological Defense.

Wireless GIS-Based High Performance Simulation of Dispersions. Chemical-Biological Applications of Mesoscale Atmospheric Modeling. Mechanics of Colloidal Transport in Contaminant Dispersion. Evaluation of the Behavior of Chemical and Biological Agents in the Soil and Deep Subsurface

Environments Air Quality, Dispersion and Atmospheric Radiative Properties. Environmental Quality Modeling.


Finding Reactive Sites in Proteins

Objective Develop effective countermeasures to protect personnel from nerve agents such as sarin or the biological snake toxin, FAS2

Computational ApproachThe target of many nerve agents is the enzyme, acetylcholinesterase (AChE). Molecular dynamics tools are used to study the molecular motions that occur inside the structure of the target, when it reacts with chemical “nerve agents,” or is inhibited by FAS2.

Millard et al. (1999) Biochemistry 38, 7032-9

E199

Bound Sarin

H440 E327

F288F290

G119

G118

A201

Nerve AgentNerve AgentSarinSarin--AChE AChE ComplexComplex

Snake ToxinSnake ToxinFAS2-FAS2-AChEAChE Complex Complex

Kryger et al. Sussman (2000) Acta Cryst D56, 1385-9

FAS2HumanAChE

Current ModelingApproach uses a combination of experimental and theoretical techniques. The above figures are derived from X-ray crystal structures. The structures are then computationally modeled to develop an understanding of the process by which the nerve agents bind to the enzyme.


Geometry Representation Compatible Geometry Representation for Automatic

Mesh Generation Accurate Flow Solver Boundary Condition + Initial Condition

Approach

GIS-Based High Performance Simulation of Dispersion


CFD Modeling of Contaminant Dispersion in Urban Environments

Governing equations:

Incompressible N-S equations

Boussinesq approximation for

LES model for turbulent dissipation

Numerical method:

Standard predictor-corrector method

Solve Poisson equation for pressure

Use grid masking for solid objects:

Enforce zero flux through surface

Modify equations within object

Solve pressure equation in full domain

Inexpensive, flexible


Mechanics of Colloidal Transport in Contaminant Dispersion

Problem Statement: Contaminants adsorb to colloidal

particles (e.g. clay fragments in groundwater, aerosol/droplet inclusions).

Colloids are transported greater distances than contaminants dissolved in the fluid or gas phase.

Governing Equations: Flow is resolved on the length scale of

typical pore spaces using 3-D Navier-Stokes equations.

Transport is modeled by the microscale convection-diffusion equation. No tunable parameters.


Chem-Bio Applications of MM5 Address the added

complications of land-surface and land-cover at finer resolutions Improve model adaptation to time

varying land-surface conditions through GIS LULC extraction

Identify model enhancements required that involve physics of flow of water and transport of heat within variably-saturated, variably-frozen soils

Implement MM5 on T3E with a grid spacing of 1-km & evaluate the results of the surface condition and boundary layer packages Evaluation of model reliability

Assess methods to incorporate chemical-biological dispersion models implicitly with MM5 (with CAU)

Chemical-Biological Applications of Mesoscale Atmospheric Modeling


Portfolio: ENABLING TECHNOLOGY AND COMPUTATIONAL ALGORITHMS

Objectives: Conduct basic research in enabling technologies and computational algorithms in support of the Army's scientific and technology goals of the Objective Force and Army transformation Data mining algorithms for discovery of useful patterns in massive

data sets. Software infrastructure for metacomputing systems. Mesh partitioning / domain decomposition. Basic research on the development of new computational

algorithms in support of interdisciplinary computational research.


Portfolio: ENABLING TECHNOLOGY AND COMPUTATIONAL ALGORITHMS

Projects:

Data Mining Techniques for Large Data Sets. Scheduling and Resource Management in Metacomputing Systems. Computational Algorithms for Time Dependent Problems: Unified Mathematical Framework, Design

and Implementation Aspects. New Time Dependent Algorithms For Quantum Mechanics/Molecular Mechanics. Hybrid Structured/Unstructured Implicit Methods for Complex High-Speed Flows. PSE Method Development for Projectile Transition and Turbulence. Precise Contact-Algorithms for Computational Mechanics. Scalable High Performance Computing Environment for Large-Scale Simulation of Free Surface Flows. Development of Numerical Algorithms for 3D, Viscoelastic, Fluid Flows with Moving Boundaries for

Novel Lightweight Materials.


Data Mining for Homeland Defense

Objectives: Information fusion from

diverse data sources including intelligence, agencies, law enforcement, profile …

Data mining on this information base to uncover latent models and patterns


Data Mining for Network Intrusion Detection

Objective: Develop techniques to detect and identify attacks against computers, networks, and the information stored therein.

Misuse Detection - Predictive models Mining needle in a haystack – models must be able to handle skewed class

distribution, i.e., class of interest is much smaller than other classes. Learning from data streams - intrusions are sequences of events

Anomaly and Outlier Detection Able to detect novel attacks through outlier detection schemes Detect deviations from “normal” behavior as anomalies


Scheduling and Resource Management in Metacomputing Systems

Metacomputing /Grid technology is a fundamental building block for distributed HPC

interconnect disparate resources, data sources, and computations virtualization of computing high performance opportunities

single applications: remote supercomputing, resource aggregation multiple applications: high throughput

Scheduling is needed to realize this HPC potential


Technology Transfer - Collaboration with Army Scientists

AHPCRC has a strong record of joint work with Army scientists and engineers, including numerous scientific papers, workshops, reciprocal short- and long-term visits and software development:

State-of-the-art software developed by AHPCRC researchers has been incorporated into many Army and DoD codes. Such transfers of software are the result of significant interaction with Army scientists.

Many students formerly supported by AHPCRC research programs are now employed as scientists at Army laboratories and various DoD organizations

The Infrastructure Support staff plays an instrumental role in technology transfer, by collaborating in the development of software and applying the latest software developments to specific Army applications.


Technology Transfer - Software

Software and tools developed at AHPCRC are being used at the Army and throughout the DoD. Geometric modeling Automatic mesh generation Visualization Mesh partitioning Sparse linear systems solvers Process modeling and simulation Structural dynamics

Example: METIS and ParMETIS graph partitioning libraries are used world-wide for partitioning unstructured and adaptive graphs and have been incorporated in numerous DoD codes such as CTH/PCTH, Paradyn, ParaAble, ADH, MPMC-NET, ICM-TOXI, and FEMWATER.

Material Deformations


Summer Institute

Summer Institutes are conducted to encourage students, particularly minority and female students, to pursue studies and careers in HPC and defense-critical technology areas:

Over 200 students since 1991 Emphasize recruiting from partner

schools High success rate

Majority of them have entered Master and Ph.D. programs (many at AHPCRC partner institutions)

Many placed in internships or have pursued technical careers (ARL, Cray Research, Dupont, Intel Corp, GTE, GM, CEWES, NASA, Dept. of Energy, Lucent Technologies, Dow Chemical, Cigna, Argonne National Laboratory, LLNL, 3M, IBM, TRW, Citibank, Oracle Corp., Lockheed Martin, Konica, Digi International, Diversified Pharmaceuticals)

2 people founded successful companies


Outreach - Workshops and Conferences

Provide a forum for the exchange of information among AHPCRC researchers, Army collaborators and other interested DoD researchers: ARL Intrusion Detection Systems Research Workshop

March 19-20, 2002 Higher Education and Applied Technology (HEAT) Center, Aberdeen, Maryland

Computational Electromagnetics (CEM) WorkshopJun 27-28, 2002 Sheraton College Park, Adelphi, Maryland

CFD / CSM for Projectile's Aerodynamics and Propulsion,Aug 13-14, 2002 Clark Atlanta University, Atlanta, GA

Contact Impact Modeling and SimulationSept. 2 (tentative) Minneapolis, Minnesota

Mesoscale Data Integration WorkshopSep 9-10, 2002 University of North Dakota, Grand Forks, ND


Outreach - Workshops and Conferences

10th Conference on Computational Chemistry (JSU, November 2001)Attended by two Nobel LaureatesBanquet speaker: Mr. W. Hollis

Second SIAM International Conference on Data Mining (April 11-13, 2002 – Hyatt Regency, Crystal City)

International Conference Series as a Follow-up of the Workshop on Mining Scientific and Engineering Datasets (held at the AHPCRC, 1999,200)


Synergistic Relationships Among Partners

Projectile Target Interaction UM, CAU, FAMU, HU, NCS

Signature Modeling HU, UND, UM, JSU, NCS

Chem-Bio Defense CAU, JSU, FAMU, UND, UM, NCS

ET & Computational Algorithms FAMU, HU, UM

VCE CAU, UM, NCS


Synergistic Relationships Example: Computational Chemistry

Goal is to predict chemical properties of interest. Sensitivity of energetic materials Properties of stealth coatings Conformational and reactive site variations in proteins

UM: Geometric modeling and data miningJSU: Computational chemistry and biologyFAMU: Energetic materialsNCS: Computational chemistry

Army High Performance Computing Research Center Vipin Kumar Director, AHPCRC Overview of Research...

Documents

Transcript of Army High Performance Computing Research Center Vipin Kumar Director, AHPCRC Overview of Research...