The Drive Toward Dedicated IP Lightpipes for e-Science Applications

OSA’s 6th Annual Photonics & Telecommunications Executive Forum

Panel on "Back to the Future of Optical Communications:

Fiber Optics Opportunities Outside the Telco Bubble"

Los Angeles, CA

February 23, 2004

Dr. Larry Smarr

Director, California Institute for Telecommunications and Information Technologies

Harry E. Gruber Professor,

Dept. of Computer Science and Engineering

Jacobs School of Engineering, UCSD

Components of Cyberinfrastructure Enabled Science & Engineering

CollaborationServices

Knowledge managementinstitutions for collection buildingand curation of data, information,

literature, digital objects

High-performance computingfor modeling, simulation, data

processing/mining

Individual &Group Interfaces& Visualization

Physical World

Humans

Facilities for activation,manipulation and

construction

Instruments forobservation andcharacterization.

GlobalConnectivity

NSF Report on Revolutionizing Science and Engineering through Cyber-Infrastructure (Atkins Report)

www.communitytechnology.org/nsf_ci_report/

CERN Geneva Large Hadron Collider Cyberinfrastructure

Communications of the ACM, Volume 46, Issue 11 (November 2003)

High Energy and Nuclear Physics Major Links: Bandwidth Roadmap (Scenario) in Gbps

Year Production Experimental Remarks

2001 0.155 0.622-2.5 SONET/SDH

2002 0.622 2.5 SONET/SDH DWDM; GigE Integ.

2003 2.5 10 DWDM; 1 + 10 GigE Integration

2005 10 2-4 X 10 Switch; Provisioning

2007 2-4 X 10 ~10 X 10; 40 Gbps

1st Gen. Grids

2009 ~10 X 10 or 1-2 X 40

~5 X 40 or ~20-50 X 10

40 Gbps Switching

2011 ~5 X 40 or

~20 X 10

~25 X 40 or ~100 X 10

2nd Gen Grids Terabit Networks

2013 ~Terabit ~MultiTbps ~Fill One Fiber

Continuing the Trend: ~1000 Times Bandwidth Growth Per Decade;We are Rapidly Learning to Use Multi-Gbps Networks Dynamically

The OptIPuter Project – Removing Bandwidth as an Obstacle In Data Intensive Sciences

• NSF Large Information Technology Research Proposal– Cal-(IT)2 and UIC Lead Campuses—Larry Smarr PI– USC, SDSU, NW, Texas A&M, Univ. Amsterdam Partnering Campuses

• Industrial Partners– IBM, Sun, Telcordia, Chiaro, Calient, Glimmerglass, BigBangwidth

• $13.5 Million Over Five Years [www.optiputer.net]• Optical IP Streams From Lab Clusters to Large Data Objects NIH Biomedical Informatics NSF EarthScope

and ORION

http://ncmir.ucsd.edu/gallery.html

siovizcenter.ucsd.edu/library/gallery/shoot1/index.shtml

Research Network

Cyberinfrastructure in Design Phase--

Fiber OpticSatelliteWireless

NSF’s ORIONOcean Research Interactive Ocean Network

www.neptune.washington.edu

½ Mile

SIO

SDSC

CRCA

Phys. Sci -Keck

SOM

JSOE Preuss

6th College

SDSCAnnex

Node M

Earth Sciences

SDSC

Medicine

Engineering High School

To CENIC

Collocation

Source: Phil Papadopoulos, SDSC; Greg Hidley, Cal-(IT)2

The UCSD OptIPuter DeploymentUCSD is Prototyping

a Campus-Scale OptIPuter

SDSC Annex

JuniperT320

0.320 TbpsBackplaneBandwidth

20X

ChiaroEstara

6.4 TbpsBackplaneBandwidth

Dedicated Fibers Between Sites Link

Linux Clusters

Ultra-Resolution Displays Utilize Photonic Multicasting --Scaling to 100 Million Pixels

GlimmerglassSwitch Used to

Multicast and Direct TeraVision Stream

from One Tile to Another on the

Geowall-2

Glimmerglass Switch

Driven by

Linux Graphics Clusters

States are Acquiring Their Own Dark Fiber Networks -- Illinois’s I-WIRE and Indiana’s I-LIGHT

Source: Charlie Catlett, ANL

Edge and Core OptIPuter Nodes

Int’l GE, 10GE

Nat’l GE, 10GE

I-WIRE OC-192

16x1 GE 16x10 GE

16-dual Xeon Cluster

16x1GE

OMNInet 10GEs

128x128 Calient

64x64GG

All Processors also Connected by GigE to Routers

UIC/EVL

Future 64-bit Cluster

The OptIPuter Will Become aNational-Scale Collaboratory in 2004

Source: Tom West, CEO, NLR

Chicago OptIPuter

StarLightNU, UIC

SoCalOptIPuter

USC, UCI UCSD, SDSU

“National Lambda Rail” PartnershipServes Very High-End Experimental and Research Applications

4 x 10Gb Wavelengths Initially Capable of 40 x 10Gb wavelengths at Buildout

NASA Ames

NASA Goddard

NEPTUNE

In Discussion

LambdaGrids Link the WorldGlobal Lambda Integrated Facility: GLIF

DWDM SURFnet

10 Gbit/s

SURFnet10 Gbit/s

SURFnet10 Gbit/s

IEEAF10 Gbit/s

DwingelooASTRON/JIVE

DwingelooASTRON/JIVE

PragueCzechLight

PragueCzechLight

2.5 Gbit/s

NSF10 Gbit/s

StockholmNorthernLight

StockholmNorthernLight

CanadaCA*net4

2.5 Gbit/s

New YorkMANLANNew YorkMANLAN

TokyoWIDETokyoWIDE

10 Gbit/s

10 Gbit/s

10 Gbit/s

10 Gbit/sIEEAF

10 Gbit/s

10 Gbit/s

10 Gbit/s

2.5 Gbit/s

2.5 Gbit/s

TokyoAPANTokyoAPAN

AmsterdamNetherLightAmsterdamNetherLight

GenevaCERN

GenevaCERN

LondonUKLightLondonUKLight

ChicagoStarLightChicagoStarLight

Source: Kees Neggers, SURFnet

Invisible Nodes, Elements,

Hierarchical,Centrally Controlled,

Fairly Static

Traditional Provider Services:Invisible, Static Resources,

Centralized Management

OptIPuter: Distributed Device, Dynamic Services,

Visible & Accessible Resources, Integrated As Required By Apps

Limited Functionality,Flexibility

Unlimited Functionality,Flexibility

Source: Joe Mambretti, Oliver Yu, George Clapp

LambdaGrid Control Plane Radical Paradigm Shift

See Nov 2003 CACM For Articles on OptIPuter Technologies