O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 1 Enabling Supernova Computations by...

1

OAK RIDGE NATIONAL LABORATORYU. S. DEPARTMENT OF ENERGY

Enabling Supernova Computations by Integrated Transport and Provisioning Methods

Optimized for Dedicated Channels

Nagi Rao, Bill Wing, Tony MezzacappaOak Ridge National Laboratory

Malathi VeeraraghavanUniversity of Virginia

DOE MICS PI Meeting: High-Performance Networking ProgramSeptember 14-16, 2004

Fermi National Laboratory


Outline

Background ORNL Tasks

Preliminary Results

UVA Tasks Preliminary Results


Terascale Supernova Initiative - TSI

Science Objective: Understand supernova evolutions DOE SciDAC Project: ORNL and 8 universities Teams of field experts across the country collaborate on

computations Experts in hydrodynamics, fusion energy, high energy

physics Massive computational code

Terabyte/day generated currently Archived at nearby HPSS Visualized locally on clusters – only archival data

Current Networking Challenges Limited transfer throughput

Hydro code – 8 hours to generate and 14 hours to transfer out

Runaway computations Find out after the fact that parameters needed adjustment


Data and File Transfers (terabyte – petabyte) Move data from computations on supercomputers Supply data to visualizations on clusters and supercomputers

Interactive Computations and Visualization Monitor, collaborate and steer computations Collaborative and comparative visualizations

Visualization channel

Visualization control channel

Steering channel

TSI Desired Capabilities

Computation orvisualization


Background on NSF CHEETAH Project Circuit-switched High-speed End-to-End Transport

arcHitecture (CHEETAH) Team: UVA, ORNL, NCSU, CUNY Concept:

Share bandwidth on a dynamic call-by-call basis End-to-end circuit:

Ethernet - Ethernet over SONET - Ethernet Network

Second NICs at hosts in a compute cluster/viz cluster Connected to MSPPs that perform Ethernet-SONET mapping GMPLS-enabled SONET crossconnects

Transport protocols and middleware To support file transfers on dedicated circuits To support remote visualization and computational steering

Applications to support TSI scientists SFTP Ensight + new visualization programs


Current DOE ORNL-UVA Project:Complementary Roles

•Project Components:•Provisioning for UltraScience Net - GMPLS•File transfers for dedicated channels•Peering – DOE UltraScience Net and NSF CHEETAH•Network optimized visualizations for TSI•TSI application support over UltraScience Net + CHEETAH

Peering

ORNL UVA

VisualizationTSI Application

ProvisioningFile Transfers

This project leverages two projects•DOE UltraScience Net•NSF CHEETAH


Peered UltraScienceNet-CHEETAH

CERN

Chicago

Sunnyvale

Atlanta

ANLFNAL

ORNL

CalTech

SLAC

LBL

NERSC

PNNL

10 Gbps

10 Gbps

DOE Science UltraNet + NSF CHEETAH

Seattle

BNL

JLab

University

DOE National Lab

Future Connections

UltraNetCHEETAH

UVa

NCSU

CUNY

Enables coast-to-coast dedicated channels

Phase I: TL1-GMPLS cross conversion

Phase II: GMPLS-based


ORNL: Year 1 Activities

• Peering CHEETAH - UltraScienceNet• Visualization

• Decomposable visualization pipeline• Analytical formulation• First implementation

• TSI support• Monitoring Visualizations


ORNL Personnel

Conference Papers• M. Zhu, Q. Wu, N. S. V. Rao, S. S.Iyengar, “Adaptive Visualization Pipeline Partition and

Mapping on Computer Network”, International Conference on Image Processing and Graphics, ICIG2004.

• M. Zhu, Q. Wu, N. S. V. Rao, S. S.Iyengar, “On Optimal Mapping of Visualization Pipeline On Optimal Mapping of Visualization Pipeline onto Linear Arrangement of Network Nodes”, International Conference on Visualization and onto Linear Arrangement of Network Nodes”, International Conference on Visualization and Data Analysis, 2005Data Analysis, 2005

Publications

Nagi Rao, Bill Wing, Tony Mezzacappa (PIs)

Qishi Wu (Post-Doctoral Fellow)

Mengxia Zhu (Phd Student – Louisiana State Uni.)


Modules of Visualization Pipeline

Visualization Modules Pipeline consists of several modules Some modules are better suited to certain network nodes

Visualization clusters Computation clusters Power walls

Data transfers between modules are of varied sizes and rates

Note:Commercial tools do not support efficient decomposition

filtering

transformation(topological surface

construction, volumetrictransfer function)

renderingframebufferfiltered data

transformed data(geometric model,volumetric values)raw data

Datasource

Display


Grouping Visualization Modules

Grouping Decompose the pipeline into modules Combine the modules into groups

Transfers on single node are generally faster Between node transfers take place over the network

Align bottleneck network links between modules with least data requirements


Optimal Mapping of Visualization Pipeline:Minimization of Total Delay

Dynamic Programming Solution Combine modules into groups Align bottleneck network links between modules with least

data requirements Polynomial-time solvable – not NP-complete

1

11 ,

( ) ,

( )( ) min

min ( ( ) )

m m m

vm

mm ton v V m m m m

u adj v v u v

c mT v pT v

c m c mT u p b

Note: 1. Commercial tools (Ensight) are not readily amenable to optimal

network deployment2. This method can be implemented into tools that provide appropriate

hooks

( )O n E

[ ], [ 1]

1 1

11 1 1 2 1[ ] [ ], [ 1]

( )1( )

i P i P i

i

q q q qi

total computing transport G L j ji i i j G and j iP i P i P i

m GT Path P of q nodes T T T T c m

p b


Optimal Mapping of Visualization Pipeline:Maximization of Frame Rate

Dynamics Programming Solution Align bottleneck network links between modules with least

data requirements Polynomial-time solvable – not NP-complete( )O n E

[ ], [ 1]

1,2, , 1

12[ ]

[ ], [ 1]1,2, , 1

12[ ]

( )

max ( ), ( ), ( )

1,

( )max ,

1

i

q

bottleneck

computing i transport P i P i computing qPath P of q nodesi q

j jj G and jP i

i

Path P of q nodesP i P ii q

j jj G and jP q

T Path P of q nodes

T G T L T G

c mp

m G

b

c mp

(2)

11

1 ,1

( ) ,

max ( ), ,

( ) min (6)

min max ( ), ,

i

i

i i i

mi m mm

iv

mi

m ton v Vm m m m

u adj v v u v

GS v c mF v p

F vc m mF u p b


First Implementation

Client/Server OpenGL implementation (leveraged from CHEETAH) Case 1: small cube geometry or frame-buffer Case 2: small geometry Case 3: small geometry CT scan: raw image or frame-buffer

Computer

Computer

LSU

NCSU

ORNL

Headnode

Slavenode

Slavenode

Slavenode

Slavenode

Dimension

Estimated bandwidth

Minimum delay

Raw data size/delay

Geometry size/delay

FB size/delay

Cube1

10x6x8 0.284Mbps 0.032sec 8 K / 0.257sec 1K / 0.032sec 1.8M/50.73sec

Cube2

50x20x39 0.300Mbps 0.034sec 610K / 16.3sec

16K / 0.46sec 1.8M/48.03sec

Cube3

150x210x139

0.277Mbps 0.033sec 71.6M / 34.4min

2.4M / 69.34sec

1.8M/52.01sec

Hand 256x256x80

0.239Mbps 0.033sec 81.9M / 45.69min

NA 1.8M/60.28sec


Requirements Light-weight server located at the computation site Remote client provides constant monitoring of variables

Our first implementation OpenGL server and client Client

Geometric operations Point, iso-surface, vector view

Commercial Visualization tools Not light weight – server on supercomputers Expensive – collaborative visualization by team Not optimized for network deployment

Monitoring Visualization


ORNL: Year 2 Activities

• MPLS Peering CHEETAH• Visualizations

• Computational Monitoring• Collaborative Visualization

• TSI support• Collaborative Steering• Integrated Data Transfers

O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 1 Enabling Supernova Computations by...

Documents

Transcript of O AK R IDGE N ATIONAL L ABORATORY U. S. D EPARTMENT OF E NERGY 1 Enabling Supernova Computations by...