Steve LloydThe Data Deluge and the GridSlide 1 The Data Deluge and the Grid The Data Deluge The...

Steve Lloyd The Data Deluge and the Grid Slide 1

The Data Deluge and the Grid

• The Data Deluge• The Large Hadron Collider• The LHC Data Challenge• The Grid• Grid Applications• GridPP• Conclusion

Steve LloydQueen Mary

University of [email protected]


The Data Deluge

Expect massive increases in amount of data being collected in several diverse fields over the next few years:– Astronomy - Massive sky surveys– Biology - Genome databases etc.– Earth Observing– Digitisation of paper, film, tape records etc to create

Digital Libraries, Museums . . .– Particle Physics - Large Hadron Collider– . . .1PByte ~1000 TBytes ~ 1M GBytes ~ 1.4M CDs [Petabyte Terabyte Gigabyte]


Digital Sky Project

Federating new astronomical surveys: ~ 40,000 square degrees ~ 1/2 trillion pixels (1 arc second) ~ 1 TB x multi-wavelengths > 1 billion sources

Integrated catalogue and image database:– Digital Palomer Observatory Sky Survey– 2 All Sky Survey– NRAO VLA Sky Survey– VLA FIRST Radio Survey

Later:– ROSAT– IRAS– Westerbork 327 MHz Survey


Sloan Digital Sky Survey

• ~ 1 million spectra• positions and images of 100 million objects• 5 wavelength bands• ~ 40 TB

Survey 10,000 square degrees of Northern Sky over 5 years


VISTA

Visible and Infrared Survey Telescope for Astronomy


Virtual Observatories

Crab Nebula

Optical

RadioInfra-red

X-ray

Jet in M87

HST optical

Gemini mid-IR

VLA radio

Chandra X-ray


NASA’s Earth Observing System

1 TB/day

Galapagos Oil Spill:


ESA EO Facilities

ESRIN

MATERA (I)

NEUSTREL.ITZ (D)

KIRUNA (S)- ESRANGE

MASPALOMAS (E)

TROMSO (N)

MATERA (I)

SEAWIFS

SPOT

IRS-P3

LANDSAT 7TERRA/MODIS

STANDARDPRODUCTION CHAINS

USERS

HISTORICALARCHIVES

USERS

PRODUCTS

AVHRR

GOME analysis detected ozone thinning over Europe 31 Jan 2002


Species 2000

To enumerate all ~1.7 million known species of plants, animals, fungi and microbes on Earth

A federation of initially 18 taxonomic databases - eventually ~ 200 databases


Genomics


The LHC

The Large Hadron Collider (LHC) will be a 14 TeV centre of mass proton proton collider operating in the existing 26.7Km LEP tunnel at CERN. Due to start operation > 2006

– 1,232 superconducting main dipoles of 8.3Tesla

– 788 quadrupoles– 2,835 bunches of 1011 protons

per bunch spaced by 25ns


Particle Physics Questions

• Need to discover (confirm) Higgs Particle– Study its properties– Prove that Higgs couplings depend on masses

• Other unanswered questions:– Does Supersymmetry exist?– How are quarks and leptons related?– Why are there 3 sets of quarks and leptons?– What about Gravity?– Anything unexpected?


The LHC


The LEP/LHC Tunnel


LHC Experiments

LHC will house 4 experiments:– ATLAS and CMS are large 'General Purpose'

detectors designed to detect everything and anything

– LHCb is a specialised experiment designed to study CP violation in the b quark system

– ALICE is a dedicated Heavy Ion Physics Detector


Schematic View of the LHC


The ATLAS Experiment

ATLAS Consists of – An inner tracker to measures the

momentum of each charged particle – A calorimeter to measure the energies

carried by the particles – A muon spectrometer to identify and

measure muons – A huge magnet system for bending charged

particles for momentum measurement

A total of > 108 electronic channels


The ATLAS Detector


Simulated ATLAS Higgs Event


LHC Event Rates

• The LHC proton bunches collide every 25ns and each collision yields ~20 proton proton interactions superimposed in the Detector i.e.– 40 MHz x 20 = 8x108 pp interactions/sec

• The (110 GeV) Higgs cross section is 24.2pb.• A good channel is H with a branching

ratio of 0.19% and a detector acceptance ~50%– At full (1034cm-2s-1) LHC luminosity this gives

1034 x 24.2x10-12 x 10-24 x 0.0019 x 0.5 = 2x10-4 H per second

A 2x10-4 needle in a 8x108 Haystack


'Online' Data Reduction

Collision Rate 40 MHz

Level 1 Special Hardware Trigger

Level 2 Embedded Processor Trigger

Level 3 Processor Farm

Raw Data Storage

104 - 105 Hz

102 - 103 Hz

10 - 100 Hz

Offline Data Reconstruction

Selecting interesting events based on progressively more detector information

10-100 GB/sec

40 TB/sec

1-10 GB/sec

100-200 MB/sec


Offline Analysis

Raw Data from Detector

Physics Analysis

1-2 MB/event @ 100-400 Hz

Data Reconstruction(Digits to

Energy/momentum etc)

Event Summary Data

0.5 MB/event

Analysis Event Selection

Analysis Object Data

10 kB/event

Total Data per year from one experiment

1 to 8 PBytes (1015 Bytes)


Computing Resources Required

CPU Power (Reconstruction, Simulation, User Analysis etc)– 20 Million SpecInt2000– (A 1 GHz PC is rated at ~400 SpecInt2000)– i.e. 50,000 of yesterday/today's PCs

'Tape' Storage– 20,000 TB

Disk Storage– 2,500 TB

Analysis carried out throughout the world by hundreds of Physicists


Worldwide Collaboration

CMS: 1800 physicists150 institutes32 countries


Solutions

• Distributed solution:– exploit established computing expertise &

infrastructure in national labs and universities– reduce dependence on links to CERN– tap additional funding sources (spin off)

Is the Grid the solution?

• Centralised Solution:– Put all resources at CERN

• Funding agencies certainly won't place all their investment at CERN

• Sociological problems


What is The Grid?

Analogy with the Electricity Power Grid:– Unlimited ubiquitous distributed computing– Transparent access to multipetabyte

distributed databases– Easy to plug in– Complexity of infrastructure hidden


The Grid

Ian Foster and Carl Kesselman, editors, “The Grid: Blueprint for a New Computing Infrastructure,” Morgan Kaufmann, 1999, http://www.mkp.com/grids

Five emerging models:

•Distributed Computing

- synchronous processing

• High-Throughput Computing

- asynchronous processing

• On-Demand Computing

- dynamic resources

• Data-Intensive Computing

- databases

• Collaborative Computing

- scientists


The Grid

Ian Foster / Carl Kesselman: "A computational Grid is a hardware

and software infrastructure that provides dependable, consistent, pervasive and inexpensive access to high-end computational capabilities."


The Grid

• Dependable - Need to rely on remote equipment as much as the machine on your desk

• Consistency - Machines need to communicate so need consistent environments and interfaces

• Pervasive - The more resources that participate in the same system the more useful they all are

• Inexpensive - Important for pervasiveness - i.e. built using commodity PCs and disks


The Grid

• You simply submit your job to the 'Grid'- you shouldn't have to know where the data you want is or where the job will run. The Grid software (Middleware) will take care of:– running the job where the data is or– moving the data to where there is CPU power

available


@#%&*!

The Grid for the Scientist

E = mc2

Grid Middleware

“Putting the bottleneck back in the Scientist’s mind”


Grid Tiers

• For the LHC we envisage a 'Hierarchical' structure based on several 'Tiers' since the data mostly originates at one place:– Tier-0 - CERN - the source of the data– Tier-1 - ~ 10 Major Regional Centres (inc

UK)– Tier-2 - smaller more specialised Regional

Centres (4 in UK?)– Tier-3 - University Groups– Tier-4 – My laptop? Mobile Phone?

• Doesn't need to be hierarchical e.g. for Biologists probably not desirable


Grid Services

Resource-specific implementations of basic servicese.g., Transport protocols, name servers, differentiated services, CPU schedulers, public keyinfrastructure, site accounting, directory service, OS bypass

Resource-independent and application-independent services authentication, authorization, resource location, resource allocation, events, accounting,

remote data access, information, policy, fault detection

DistributedComputing

Toolkit

Grid Fabric(Resources)

Grid Services(Middleware)

ApplicationToolkits

Data-Intensive

ApplicationsToolkit

CollaborativeApplications

Toolkit

RemoteVisualizationApplications

Toolkit

ProblemSolving

ApplicationsToolkit

RemoteInstrumentation

ApplicationsToolkit

Applications Chemistry

Biology

Cosmology

Particle Physics

Environment


Problems

• Scalability– Will it scale to thousands of processors,

thousands of disks, PetaBytes of data, Terabits/sec of IO?

• Wide-area distribution– How to distribute, replicate, cache,

synchronise, catalogue the data?– How to balance local ownership of resources

with the requirements of the whole?• Adaptability/Flexibility

– Need to adapt to rapidly changing hardware and costs, new analysis methods etc.


SETI@home

• A distributed computing project - not really a Grid project

• You pull the data from them rather than they submit the job to you– total of 4,591,332 users– 963,646,331 results received– 1,545,634 years of cpu time– 3.3x1021 floating point operations– 125 different cpu types – 143 different operating systems

Arecibo telescope in Puerto Rico


SETI@home


Entropia

Uses idle cycles on Home PCs for profit and non-profit projects:

Mersenne Prime Search• 42,519 machines active• 560 years of cpu per day

FightAIDS@Home• 60,000 Machines• 1,400 years of cpu time


NASA Information Power Grid

• Knit together widely distributed computing, data, instrumentation and human resources

• to address complex large scale computing and data analysis problems


Collaborative Engineering

Real-timecollection

Multi-sourceData Analysis

Unitary Plan Wind Tunnel

Archival storage


Other Grid Applications

• Distributed Supercomputing– Simultaneous execution across

multiple supercomputers• Smart Instruments

– Enhance the power of scientific instruments by providing access to data archives and online processing capabilities and visualisation e.g. coupling Argonne’s Photon

Source to a supercomputer


GridPP

http://www.gridpp.ac.uk


GridPP Overview

Provide architecture and middleware

Use the Grid with simulation data

Use the Grid with real data

Future LHC Experiments

Running US Experiments

Build prototype Tier-1 and Tier-2s in the UK and implement middleware in experiments


The Prototype UK Tier-1

March 2003 •560 CPUs (450Mhz-1.4GHz)•50 TB Disk•35 TB Tape in use (theoretical

tape capacity 366 TB)


Conclusions

• Enormous data challenges in next few years.

• The Grid is likely solution.• The Web gives ubiquitous access to

distributed information.• The Grid will give ubiquitous access to

computing resources and hence knowledge.• Many Grid projects and testbeds starting to

take off.• GridPP is building a UK Grid for Particle

Physicists to prepare for future LHC Data.

Steve LloydThe Data Deluge and the GridSlide 1 The Data Deluge and the Grid The Data Deluge The...

Documents

Transcript of Steve LloydThe Data Deluge and the GridSlide 1 The Data Deluge and the Grid The Data Deluge The...