Storage and Data Movement at FNAL

D. PetravickCHEP 2003

3/25/2003 DLP -- CHEP 2003 2

FNAL Overall Goals

• Provide competent one copy permanent store to all the Lab’s experiments.

• Provide scalable and performant data flows– Tape and Disk– Local area and Wide area.

• Provide standard interfaces allowing for interoperation with other sites.

• Collaborate! (DESY > 5 yrs) GLOBUS, LBL, JLAB, CERN

3/25/2003 DLP -- CHEP 2003 3

STK, ADIC libraries; STK, IBM/LTO tape drives

dCache: Buffering and Caching

Enstore: (staging files from tape)

PNFS Name Space

SRM (weak, kerberized,grid) FTP

AnywhereClents

dCap

“local” Clients

encp NFS (not data)

Local Clients

3/25/2003 DLP -- CHEP 2003 4

Overview

• Central Storage Systems capacious and scalable enough to be a hub of a data intensive system.– Linux as a hardware platform.– Permanent and temporary semantics.

• Competent local and GRID interfaces.ENCP, dCapFTP(s) and SRM (Storage Resource Manager)investigating GLUE schema and monitoriing

3/25/2003 DLP -- CHEP 2003 5

3/25/2003 DLP -- CHEP 2003 6

3/25/2003 DLP -- CHEP 2003 7

Simplified structure of Dcache

Read poolRead pool

Write poolWrite pool

User

Grid FTP SRM Kerberos FTPFTPDCCPDCAP APIDcachedoors

13March 19, 2003

3/25/2003 DLP -- CHEP 2003 8

6/11/2002 D. Petravick -- THIC -- NCAR 18

Bonus Disk -- dFarm

• Exploit the excess disk + competent networks we are currently blessed with.

• Provide redundant, temporary storage, robust against failure of any farm node

3/25/2003 DLP -- CHEP 2003 9

RunII Work Plan

• SAM– Grid FTP integration with SAM.– DCAP integration– SRM integration with SAM.

• SRM integration with legacy CDF AC++

• Restore rates for CDF experiment.• Provide hardware for D0 Wide area

transfers.

3/25/2003 DLP -- CHEP 2003 10

US CMS Work Plan

• Use dCache as a vehicle for Grid File access locally.

• Evaluate Storage elements for Tier II centers (dCache, DRM, NeST EDG wp 5 , Dfarm).

• Follow, help formulate LCG requirements.• Detailed work to meet CMS Data Challenge

on CMS deadlines.• Monitor and Improve Network

– Pinger IEPM (co-project)– FAST and other h/p TCP stacks. (co-project)

3/25/2003 DLP -- CHEP 2003 11

3/25/2003 DLP -- CHEP 2003 12

Grid Side

Storage Element

Storage Element

Worker Side

Job-Aware, Replica-Aware MdwPermanent Store Replica Stores

FTP Inter-face

SRMInter-face

DirectFile Access Interface

Monitoring Interface

Framework + job

3/25/2003 DLP -- CHEP 2003 13

Lattice Gauge Work Plan

• Support the Fermilab Lattice Computation Facility.

• Use GRID FTP, SRM to integrate with – JLAB (cluster) , BNL (QCDOC) facilities.

• Understand relationship of storage systems to QIO (community I/O package)

• Investigate utility of caching data at FNAL.• Investigate symmetries with earth science

type system.

3/25/2003 DLP -- CHEP 2003 14

Other Work Elements

• Work on Grid FTP V1.1 specification.– Include work on a few scaling issues for

“seas of linux box” type systems.

• Work on SRM protocol, futures.– V2.1 (co – pi @ FNAL)

• Look at “Object Based Storage” in conjunction with US CMS.– Proposed investigation is root integration.

• Understand the requirements of analysis systems.

3/25/2003 DLP -- CHEP 2003 15

Other work elements…

• Support Grid protocols for dFarm.– Make it a “storage element”– Another innovative package.

• Experimenting with Grid Authentication.• Prepare for special routing on the WAN

– E.g. lambda networking.

• Support the experimental community at FNAL.– Auger in GridFTP production.– Minos in Kerberized, Weak FTP.

3/25/2003 DLP -- CHEP 2003 16

•

3/25/2003 DLP -- CHEP 2003 17

Advanced Monitoring

• Animations were Crucial for debugging The RUN II LAN based data systems.

• Looking for analogous tools for grid based works

• Follow items such as GLUE.

3/25/2003 DLP -- CHEP 2003 18

Advanced Network Integration

• FNAL proposes to have two paths off site.

• Path to Starlight will allow R&D on advanced network concepts (e.g. lambda network)

• Central data movement systems will be an early user.

3/25/2003 DLP -- CHEP 2003 19

Summary

• Very substantial and successful support of FNAL Program.

• Very proactive work on fabric side.

• Data in Movement are interesting.

• Goal :Routine – performant flows– on LAN and WAN– with interoperation