Workshop Introduction - Indico€¦ · Workshop Introduction Ken Bloom US CMS Tier-2 Workshop March...

Workshop Introduction

Ken BloomUS CMS Tier-2 Workshop

March 7, 2011

Ken Bloom 2011 US CMS Tier-2 Workshop March 7, 2011

The first yearThe T2 sites have always been a key piece of in the CMS computing model -- this is where physicists do their physics.A lot of physics got done in 2010, thanks in no small measure to the T2’s.Our performance during this period has been really good.➨ As the biggest sites, the US sites really must be the best for CMS to

succeed

2


2010 in review: RSVThis is what is reported to WLCG for availability/reliability, and that in turn gets reported to the funding agencies.

3


2010 in review: SAM tests

4


2010 in review: SAM tests

5


2010 in review: job robot

6


2010 in review: site readinessThis was still in shake-down a year ago; now it’s what is used as the primary metric in CMS.

7


2010 in review: site readiness

8


2010 in review: production

9


2010 in review: inbound transfers

“Only” 11.9 of 17.7 TB is from T1’s -- taking advantage of T2-T2 mesh

10


2010 in review: outbound transfers

2.9 of 8.3 TB is to T1’s.

11


2009 in review: analysis job hostingz3PWmht.pngz3PWmht.png

12

42% of T2 analysis jobs run at OSG T2 sites


Looking aheadThe performance of the US T2 sites has been everything we needed it to be, in part because we took several years to prepare for this time, and now have systems that function well.

Thus, now we’re thinking about changing everything!

13


A resource-constrained world

Expecting T2 storage and CPU to be saturated in the next few months!➨ Will be shifting more MC production to T1➨ 2012 resource request will be about 1/3 bigger than 2011

Capacity assumes we deploy in April, when in fact it’s more like September.➨ What (besides money) keeps us from doing it sooner?

US CMS has submitted the NSF proposal that will cover the next 5 years of T2 funding, with a request to support additional hardware ($350K/year)➨ Can our infrastructure support this growth?

14

0

5000

10000

15000

20000

25000

30000

35000

Janu

ary

2010

A

pril

2010

Ju

ly 2

010

Oct

ober

20

10

Janu

ary

2011

A

pril

2011

Ju

ly 2

011

Oct

ober

20

11

Janu

ary

2012

A

pril

2012

Ju

ly 2

012

Oct

ober

20

12

Janu

ary

2013

A

pril

2013

Ju

ly 2

013

Oct

ober

Tier-2 Disk Storage Tier-2 Space

Production Space on Tier-2s User Space On Tier-2s

AOD MC on Disk

RECO MC on Disk

AOD Data on Disk

RECO Data on Disk

0

100

200

300

400

500

600

Janu

ary

2010

Apr

il 20

10

July

201

0

Oct

ober

201

0

Janu

ary

2011

Apr

il 20

11

July

201

1

Oct

ober

201

1

Janu

ary

2012

Apr

il 20

12

July

201

2

Oct

ober

201

2

Janu

ary

2013

Apr

il 20

13

July

201

3

Oct

ober

201

3

Tier-2 CPU Needs

Tier-2 Capacity

AOD Analysis Tier-2

RECO Analysis Tier-2

MC Processing at Tier-2


Technology changesCMSSW is now ready to start operating in multi-core mode➨ Probably MC production only rather than analysis➨ Need to configure systems to handle such jobs [Holzman]

The OSG continues to evolved [Roy/Quick/Fraser]Potential evolution of LHC data access models [Fisk]:➨ High-capacity networking more available/reliable than planned for➨ Speed of activities (data processing frequency, publication) faster➨ Want to use disk and CPU more dynamically and efficiently➨ Many ideas, implementations now afloat

The Xrootd redirector is a working prototype [Wuerthwein]:➨ Decouples data location and CPU location➨ Fallbacks for read errors, bypass overloaded sites, a diskless T3?➨ Want all US T2 sites to join in the redirector pool ASAP!

15


Our operationsWe are now generally more in operations rather than R&D mode➨ There are exceptions: Xrootd, Lustre, hadoop....

I claim that our biweekly meetings have thus gotten sort of boring!➨ Ian: “Operations meetings should be boring.”

Should we drop down to monthly meetings?➨ I’m not sure, can see pros and cons➨ Your feedback welcome

16


Let’s go! (I have a plane to catch....)Thanks in advance to:➨ Our hosts at Harvard Medical School for handling all of the logistics➨ The OSG for organizing the all-hands meeting, and for supporting the

platform that our CMS work rests on➨ All of our speakers for doing the work to make good presentations➨ All of you for coming to The Hub (even if it just was a T trip)➨ Everyone involved with the T2 project -- six successful years and

counting!

17

Workshop Introduction - Indico€¦ · Workshop Introduction Ken Bloom US CMS Tier-2 Workshop March...

Documents

Transcript of Workshop Introduction - Indico€¦ · Workshop Introduction Ken Bloom US CMS Tier-2 Workshop March...