Tier2 Centre in Prague

Tier2 Centre in Tier2 Centre in PraguePrague

JiJiří Chudobaří Chudoba

FZU AV ČR - Institute of Physics of tFZU AV ČR - Institute of Physics of the he Academy of SciencesAcademy of Sciences of the Czech Republic of the Czech Republic

19.10.2005, FZK19.10.2005, FZK [email protected]@fzu.cz22

OutlineOutline

Supported groups Supported groups HardwareHardware Middleware and softwareMiddleware and software Current statusCurrent status


Particle Physics in the Particle Physics in the Czech RepublicCzech RepublicGroups located atGroups located at Charles University in PragueCharles University in Prague Czech Technical University in PragueCzech Technical University in Prague Institute of Physics of the Academy of Sciences of the Czech Institute of Physics of the Academy of Sciences of the Czech

RepublicRepublic Nuclear Physics Institute of the Academy of Sciences of the Nuclear Physics Institute of the Academy of Sciences of the

Czech RepublicCzech Republic

Main ApplicationsMain Applications Projects ATLAS, ALICE, D0, STAR, TOTEMProjects ATLAS, ALICE, D0, STAR, TOTEM Groups of theoreticiansGroups of theoreticians Approximate size of the communityApproximate size of the community

6060 scientistsscientists, 2, 200 engineers engineers, 2, 200 technicianstechnicians andand 4 400 students students and PhD students and PhD students


HardwareHardware

2 independent computing farms2 independent computing farms– Golias located in FZUGolias located in FZU– Skurut from CESNET Skurut from CESNET (= “academic network provider”)

older hardware (29 dual nodes, PIII700MHz) older hardware (29 dual nodes, PIII700MHz) offered by CESNEToffered by CESNET

part used as a production farm, some nodes for part used as a production farm, some nodes for tests and support for different VOs (VOCE, tests and support for different VOs (VOCE, GILDA, CE testbed)GILDA, CE testbed)

contribution to Atlas DC2 at a level of 2% of all contribution to Atlas DC2 at a level of 2% of all jobs finished on LCGjobs finished on LCG


Available Resources - Available Resources - FZUFZU Computer hall Computer hall

– 2 x 9 racks2 x 9 racks– 2 air condition units2 air condition units– 180 kW electrical power 180 kW electrical power

available from UPS, backed available from UPS, backed up by Diesel generatorup by Diesel generator

– 1 Gbps optical connection to 1 Gbps optical connection to CESNET metropolitan networkCESNET metropolitan network

– direct 1 Gbps optical direct 1 Gbps optical connection to CzechLight connection to CzechLight

– shared with other FZU shared with other FZU activitiesactivities


Hardware @ FZUHardware @ FZU Worker NodesWorker Nodes

(September 2005)(September 2005)– 6767x x HP DL140, HP DL140, dual dual Intel Intel

Xeon 3.06Xeon 3.06 GHz withGHz with HT HT (enabled only on some (enabled only on some nodes), 2 or 4 GB RAM, 80 nodes), 2 or 4 GB RAM, 80 GB HDDGB HDD

– 11x dual AMD Opteron 1.6 x dual AMD Opteron 1.6 GHzGHz, 2 GB RAM, 40 GB HDD, 2 GB RAM, 40 GB HDD

– 24x HP LP1000r, 2xPIII1.13 24x HP LP1000r, 2xPIII1.13 GHz, 1 GB RAM, 18 GB SCSI GHz, 1 GB RAM, 18 GB SCSI HDDHDD

– WN connected via 1 Gbps WN connected via 1 Gbps (DL140) or 100 Mbps (DL140) or 100 Mbps (LP1000r)(LP1000r)

Network componentsNetwork components– 3x HP ProCurve Networking 3x HP ProCurve Networking

Switch 2848 (3x48 ports)Switch 2848 (3x48 ports)– HP 4108GL HP 4108GL

~ 30 KSI2K will be added this year


Golias Farm Hardware - Golias Farm Hardware - serversservers

PBS server: HP DL360 – 2xPBS server: HP DL360 – 2x Intel Xeon 2.8Intel Xeon 2.8, 2 GB RAM, 2 GB RAM CE, SE, UI: HP LP1000, 2xCE, SE, UI: HP LP1000, 2x1.13Ghz PIII, 1 GB RAM, 100 1.13Ghz PIII, 1 GB RAM, 100

Mbps (SE should be upgraded to 1 Gbps soon)Mbps (SE should be upgraded to 1 Gbps soon) NFS serversNFS servers

– 11x x HP DL145 – 2xHP DL145 – 2x AMD Opteron 1.6 GHz AMD Opteron 1.6 GHz, connected , connected disc array 30 TB (raw capacity), ATA discsdisc array 30 TB (raw capacity), ATA discs

– 1x HP LP4100TC, 1 TB disc array, SCSI discs1x HP LP4100TC, 1 TB disc array, SCSI discs– 1x embedded server in EasySTOR 1400RP (PIII), 10 TB, 1x embedded server in EasySTOR 1400RP (PIII), 10 TB,

ATA discsATA discs dCache serverdCache server

– HP DL140 upgraded by raid controller, 2x300GB discsHP DL140 upgraded by raid controller, 2x300GB discs– not used for productions, reserved for SC3 not used for productions, reserved for SC3

Some other servers (www, SAM)Some other servers (www, SAM)


Middleware, Batch Middleware, Batch SystemSystemGOLIAS: GOLIAS:

– LCG2 (2_6_0): LCG2 (2_6_0): CE, SE, UI – SLC3 CE, SE, UI – SLC3 WNs - RH7.3 (local D0 group not yet ready for WNs - RH7.3 (local D0 group not yet ready for

SLC3)SLC3)

– PBSPro server not on CEPBSPro server not on CE CE submits jobs to the node golias (PBSPro server)CE submits jobs to the node golias (PBSPro server) local users can submit to local queues on goliaslocal users can submit to local queues on golias

SKURUT:SKURUT:– LCG2 (2_6_0), OpenPBS server on CELCG2 (2_6_0), OpenPBS server on CE– all nodes - SLC3all nodes - SLC3


QueuesQueues

Separate queues for different Separate queues for different experiments and privileged users:experiments and privileged users:atlas, atlasprod, lcgatlas, lcgatlasprodatlas, atlasprod, lcgatlas, lcgatlasprodalice, aliceprod, d0, d0prod, auger, star, ...alice, aliceprod, d0, d0prod, auger, star, ...short, longshort, long

priorities are set by some PBS priorities are set by some PBS parameters:parameters:– max_running, max_user_running, priority, max_running, max_user_running, priority,

node properties node properties still not optimal in heterogeneous still not optimal in heterogeneous

environmentenvironment


Jobs statistics 2005/1-Jobs statistics 2005/1-66

Used CPU time (in days) per activity, for January – June 2005Used CPU time (in days) per activity, for January – June 2005

10553

4974

219455 1 886 23

d0 atlas starprod auger

alice long short other


Simulations for ATLASSimulations for ATLAS

All PILEUP jobs on LCG

TRIUMF; 153; 2%

TAIWAN; 3207; 33%

PRAGUE; 3843; 38%

ROMA; 1718; 18%

IFAE; 871; 9%


ALICE PDC – Phase2 ALICE PDC – Phase2 (2004)(2004)

2004: ALICE jobs submitted to Golias via AliEn2005: new version of AliEn is being installed


Tier1 – Tier2 relationsTier1 – Tier2 relations

Requirements defined by experiments Requirements defined by experiments ATLAS and ALICEATLAS and ALICE

Direct network connection between Direct network connection between FZU and GridKa will be provided by FZU and GridKa will be provided by GEANT2 next yearGEANT2 next year

““Know-how” exchange welcomedKnow-how” exchange welcomed


THE END

Tier2 Centre in Prague

Documents

Transcript of Tier2 Centre in Prague