Polish Contribution to the Worldwide LHC Computing Grid

WLCGM. Witek

On behalf of the team of Polish distributed Tier-2

Outline• Introduction• History and status of Polish Tier-2 Federation• Examples of physics results from LHC experiments

CERN - European Organisation for Nuclear Reseach

LHC – Large Hadron Collider

CMS

Alice

LHCb

Atlas

2

Polish WLCG Tier-2 team• Polish WLCG Tier-2 team

– A. Binczewski3, M. Bluj5, A. Cyz2, M. Dwużnik1, M. Filocha4, L. Flis1, R. Gokieli4,5, J.Iwaszkiewicz4, M. Kowalski2, P. Lasoń1, R. Lichwała3, M. Łopuszyński4, M. Magryś1, P. Malecki2, N. Meyer3, K. Nawrocki4,5, A. Olszewski2, A. Oziębło1, A. Padee4,5, H. Pałka2, M. Pospieszny3, M.Radecki1, R. Rowicki4, D. Stojda6,4, M. Stolarek4, T. Szepieniec1, T. Szymocha1,2, M.Turała1,2, K.Wawrzyniak4,5, W.Wiślicki4,5, M.Witek2, P.Wolniewicz3

• Institutions– 1 AGH University of Science and technology, ACC Cyfronet AGH, Krakow– 2 Institute of Nuclear Physics PAN, IFJ PAN, Krakow– 3 Poznan Supercomputing and Networking Center, PSNC, Poznan– 4 Interdisciplinary Centre for Mathematical and Computational Modelling, ICM, Warsaw– 5 Soltan Institute for Nuclear Studies, IPJ, Warsaw– 6Copernicus Science Centre, CSC, Warsaw, Poland

More details will be available in the article of PL-Grid Book

3

Eksperiment requirements

• Example of on-line conditions– ~108 electronic channels to read at 40

MHz– 1 PB/sec of raw info from the detector– ~few TB/sec of zero suppressed data

( channels with signal only)

• Multi-level trigger system– Reduction factor 106-107

– Output rate > 100 MB/sec

• 107 sec/year ~few PB of raw data per experiment

Level 1 Level 2

40 MHz (1000 TB/sec)

Level 3

75 KHz (75 GB/sec)5 KHz (5 GB/sec)100 Hz (100

MB/sec)Data Recording &Offline Analysis

WLCG

4

(Pre)History

• MONARC (Models of Networked Analysis at Regional Centres for LHC Experiments)– Simulate data processing (a kind of Monte Carlo model):

• a complex set of networked computing systems (CERN, Tier-1 and Tier-2 regional centres)

• the analysis process, composed of a dynamic workload of reconstruction and analysis• the behaviour of key elements of the system, such as distributed database servers and

the networks

Summary: CERN-LCB-2000-001

One of the problems @ 2000:data sharing difficult due to the prohibitively high cost of networking

Even more pronounced in Poland at that time !!!

5

From 2000 to 2011• 2000: DataGrid 2000 – HEP oriented EU project• 2000: CERN-Poland meeting (initiated by M. Turala & P. Malecki)• 2002: CrossGrid first Polish EU project with HEP task.• 2002: start of LCG (LHC Computing Grid)• 2003: First LCG testbed of 14 sites (Cyfronet participation)• 2004-2010: EGEE, EGEE II, EGEE III• 2003-2008: Participation in Data Challenges• 2005-2010: BalticGrid, BalticGrid II• 2007: Poland signs MoU for WLCG• 2009: start of PL-Grid project (WLCG infrastructure)• 2009-2011: LHC start. Smooth running of Polish WLCG sites.

6

Polish participation in networking and grid projects

Polish part of WLCG emerged out of many different projects. Some of them not directly related to HEP. Nevertheless they provided infrastructure and common tools.

PIONIER, CrossGrid, EGEE and PL-Grid were essential for the development of Polish WLCG infrastructure.

7

GTS 1,6 Gb/s

GDAŃSK

POZNAŃ

ZIELONA GÓRA

KATOWICE

KRAKÓW

LUBLIN

WARSZAWA

BYDGOSZCZ TORUŃ

CZĘSTOCHOWA

BIAŁYSTOK

OLSZTYN

RZESZÓW

Bielsko-Biała

GÉANT 10+10 Gb/s

KOSZALIN

SZCZECIN

WROCŁAW

ŁÓDŹ

KIELCEPUŁAWYOPOLE

RADOM

BASNET 34 Mb/s

CESNET, SANET

GÉANT/TELIA 2x2,5 Gb/s

DFN 10 Gb/s Gorzów

MAN

10 Gb/s(1 lambda)

2 x 10 Gb/s

1 Gb/s

CBDF 10 Gb/s

PIONIER’S FIBERS

Polish infrastructure in WLCG

Tier1 FZKKarlsruhe

Tier2 PCSSPoznań Tier2 ICM

Warszawa

Tier2 ACKCyfronet Kraków

HEP VLAN1 Gb/s

HEP VLAN1 Gb/s

8

Pionier – Polish Optical Internet• Nationwide broadband optical network represents a base for research and development

– Europe's first national academic network that uses its own dark fiber optics and DWDM 10GE transmission

– Connects 21 Academic Network Centers of Metropolitan Area Networks (MAN) and 5 of the HPC (High Performance Computing) centers.

• Cross-border links with: Germany (DFN), Czech Republic (CESNET), Slovakia (SANET), Ukraine (UARNET), Byelorus, Lithuania

Started in 2000, 5917,5 km of fibres cables @ 2010

Data sharing problem due to networking limits indicated by MONARC solved !

9

Towards physics results - that is what WLCG was built for

Every experiment (ALICE, ATLAS, CMS, LHCb) has its own computing model. All based on multi-tier structure

Successful LHC restart in 2009Excellent performance of the LHC and the experiments

3.5*1014 collisions delivered in 2011

Polish distributed Tier-2Kraków-Poznań-Warszawa

The RAW data reside in Tier-0 and Tier-1. The final physics result is an effect of many processing steps. Some steps have to be repeated when better detector calibration and alignment appear.

The role of Tier-2 sites:Monte Carlo productionAnalysis centre (larger sites)Limited reprocessing activity

10

LHCb computing model – DIRAC role

11

Polish WLCG resources

ACK Cyfronet AGH – Tier2

Polish sites provide about 2% of the total computing power in proportion to the engagement of Polish scientists in the LHC experiments.Most of the resources comes from PL-Grid infrastructure

Polish Tier-2 Federation 2010 2011 2012

CPU (HS06) 10540 13050 15800

Disk (TB) 599 810 1020 Tier-3 at IFJ PANWLCG pledges

Reliability

Availability

AtlasLHCb

Warsaw ICM – Tier2

CMSLHCb

Poznan PSNC – Tier2

AliceAtlas

July 2011

12

Poland in LCG accounting

A week with 9 % contribution of Cyfronet to the LHCb reprocessing of 2011 data.Congratulations for the site performance from the LHCb computing team.

LHCb3.9% delivered by PolandCMS

2% delivered by Poland

Month 2011

May June July Aug Sept

Usage as % of pledge

165% 266% 299% 173% 229%

Performance of Polish Tier-2 (From WLCG Accounting)

Increase over 100% mainly due to Cyfronet additional resources

Highlight from LHCb

Atlas1.6 %

Delivered by Poland

Tier-1/2

At present 69 Tier-2 federations in 33 countries. The size of Polish Tier-2 is in top 20.

13

The Standard Model – Particles and Forces

Bulding blocks of our Universe

14

ATLAS and CMS – direct discovery potential• Find the last undiscovered element of the Standard Model

– The Higgs bozon

• Direct discovery of New Physics – Supersymmetric particles or any object/phenomena beyond Standard Model we even did

not think about yet ATLAS

CMS

The Higgs boson search

15

LHCb – LHC beauty experiment• Hunting the signs of new physics by precision measurements

Bs0→μ+μ-

Bs0 (bs) meson contains two

quarks which are not present in the world around us: b beauty and s strange

Extremely rare decay3 out of 109 decays

Standard Model Supersymmery

New Physics

2 decays observed – 1.4 expected

No sign of New Physics yet2 events selected out of recorded 1010 WLCG 2 events selected out of 3.5*1014 initial collisions

New phenomena can significantly increase the decay rate

16

Summary• Polish physicists and computer scientists created distributed WLCG

Tier2, which availability and efficiency is high; the delivered resources are in the range 2-3 % of the total.

• The performance of the WLCG matches high requirements of the data processing of LHC experiments.

• Lots of highest quality physics results are being shown recently. Much more are on the way, hopefully showing New Physics.

AcknowledgmentsWe would like to thank all people for their contribution to the successful creation and operation of Polish Tier-2 Federation. In particular the management and staff of the three computing centres, ACC Cyfronet AGH in Krakow, ICM UW in Warsaw and PSNC in Poznan for their constructive cooperation. A supportive approach of the directorates of IFJ PAN Krakow and IPJ Warsaw is also appreciated. We are also grateful to many people from CERN, the FZK Karlsruhe and other WLCG collaborating institutions for their friendly advice and support.