ScotGRID:The Scottish LHC Computing Centre Summary of the ScotGRID Project Summary of the ScotGRID...
-
Upload
abner-tyler -
Category
Documents
-
view
215 -
download
0
Transcript of ScotGRID:The Scottish LHC Computing Centre Summary of the ScotGRID Project Summary of the ScotGRID...
Georg Raff elt, Max-Planck-I nstitut für Physik, München, Germany CMB & Cosmology: Where Are We?, Ringberg Castle, Tegernsee, Germany, 2-6 December 2002
ScotGRID:The Scottish LHC Computing ScotGRID:The Scottish LHC Computing CentreCentre
Summary of the ScotGRID ProjectSummary of the ScotGRID Project
Phase2 of the ScotGRID ProjectPhase2 of the ScotGRID Project
Phase1 of the ScotGRID ProjectPhase1 of the ScotGRID Project
Introduction: ScotGRID GoalsIntroduction: ScotGRID Goals
Georg Raff elt, Max-Planck-I nstitut für Physik, München, Germany CMB & Cosmology: Where Are We?, Ringberg Castle, Tegernsee, Germany, 2-6 December 2002
The LHC Computing
Challenge for Scotland
2000: JREI Bid The JREI funds will make possible to commission and fully exercise a prototype LHC computing centre in
Scotland
The Centre would provide:
1. Technical service based for the grid services
2. DataStore to handle samples of data Analysis
3. Significant simulation production capability
4. Excellent network connection RAL / Regional sites
5. Support grid middle devel. with CERN/RAL
6. Support core software devel. within LHCb/ATLAS
7. Support user applications in other scientific areas
This will enable us to answer: Grid viable solution for LHC computing challenge,TDR Can a two-site Tier-2 centre be setup and operate effectively How can network topology between Ed,GL, RAL & CERN
This will enable us to answer: Grid viable solution for LHC computing challenge,TDR Can a two-site Tier-2 centre be setup and operate effectively How can network topology between Ed,GL, RAL & CERN
ScotGRID: The Spirit of the ProjectScotGRID: The Spirit of the Project
http://www.scotgrid.ac.uk/
Georg Raff elt, Max-Planck-I nstitut für Physik, München, Germany CMB & Cosmology: Where Are We?, Ringberg Castle, Tegernsee, Germany, 2-6 December 2002
2002 200520042003
Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
Q1 Q2 Q3 Q4
Q1 Q2 Q3 Q4
2001
Q4
PrototypePrototypeTo To
ProductionProduction
PrototypePrototypeTo To
ProductionProductionDelivery of more Kit…
End of JREI fundingStart of ScotGRID-II
Start of GridPP-II
Links to other applications…
Production Tier-2 Site
Time-Line for a Production Tier-2 & beyondTime-Line for a Production Tier-2 & beyond
Completed: Phase1 Phase2
Future Upgrades ?
Georg Raff elt, Max-Planck-I nstitut für Physik, München, Germany CMB & Cosmology: Where Are We?, Ringberg Castle, Tegernsee, Germany, 2-6 December 2002Georg Raff elt, Max-Planck-I nsti tut f ür Physik, München, Ge rmany CMB & Co smology: Where Are We?, Ringberg Castle, Tegernsee , Ger many, 2-6 December 2002
ScotGRIDScotGRID-- PhasePhase-- 1:Completed Successfully1:Completed Successfully
HardwareHardware ApplicationsApplicationsMiddlewareMiddleware
ScotGRID Processing nodes at GlasgowScotGRID Processing nodes at Glasgow• 59 IBM X Series 330 dual 1 GHz Pentium III with 2GB memory • 2 IBM X Series 340 dual 1 GHz Pentium III with 2GB memory • 3 IBM X Series 340 dual 1 GHz Pentium III with 2GB memory • 1TB disk • LTO/Ultrium Tape Library • Cisco ethernet switches
ScotGRID Storage at EdinburghScotGRID Storage at Edinburgh• IBM X Series 370 PIII Xeon • 70 x 73.4 GB IBM FC Hot-Swap HDD
Georg Raff elt, Max-Planck-I nstitut für Physik, München, Germany CMB & Cosmology: Where Are We?, Ringberg Castle, Tegernsee, Germany, 2-6 December 2002
ScotGrid reached ~ 580,000th processing hour
Tier-2 Web-Based Monitoring
http://www.scotgrid.ac.uk/
Georg Raff elt, Max-Planck-I nstitut für Physik, München, Germany CMB & Cosmology: Where Are We?, Ringberg Castle, Tegernsee, Germany, 2-6 December 2002
ScotGrid accumulated statsScotGrid accumulated stats
Significant non-LHC application use
Q1Q2Q3
Georg Raff elt, Max-Planck-I nstitut für Physik, München, Germany CMB & Cosmology: Where Are We?, Ringberg Castle, Tegernsee, Germany, 2-6 December 2002
ScotGrid JREI project includes a mid-term hardware upgrade.
As part of GridPP planning, we need to upgrade from Prototype to Production Tier 2 status by 2004.
JREI funding left to be spent by June 2003:
Edinburgh £220k Glasgow £30k
£250k
ScotGRID Phase-2 : Just Started ScotGRID Phase-2 : Just Started
Georg Raff elt, Max-Planck-I nstitut für Physik, München, Germany CMB & Cosmology: Where Are We?, Ringberg Castle, Tegernsee, Germany, 2-6 December 2002
Upgrade Database Server in Edinburgh xSerier 440 (8 Way 1.9GHz Intel Xeon, 1MB L3 Cache, 32GB Memory, 36GB Internal Disk)
16 – 20 TB of Disk Storage in Edinburgh FAStT700 or FAStT900 (see next slide)
5 TB of Disk Storage in Glasgow Agreed to relocate existing FAStT500 from Edinburgh to Glasgow
Edge Servers for Edinburgh EDG SE:1 from Glasgow (x340) LCFG, CE: 2* x205, 1 x 1.8 GHz, 256MB, 40GB
Summary of ScotGRID UpgradeSummary of ScotGRID Upgrade
New Kit for Glasgow CDF & eDIRT Kit
Georg Raff elt, Max-Planck-I nstitut für Physik, München, Germany CMB & Cosmology: Where Are We?, Ringberg Castle, Tegernsee, Germany, 2-6 December 2002
Specifications IBM
FAStT700IBM
FAStT900
RAID Controllers Dual Dual
Max Cache Size 2 GB 2 GB
Host interface 1 or 2Gb FC 1 or 2 Gb FC
# Host Ports 8 8
Disk Drive Interface 1 or 2Gb FC 1 or 2 Gb FC
Max # of Drives 224 224
Max Raw Capacity (73 GB HDD) 33TB 33TB
Performance IOPS (cache reads) 110,000 148,000
Performance IOPS - read(random disk I/O) - write
40,0008,500
53,200*10,900*
Performance MBps - read(sequential disk I/O) - write
390240
772*530*
Max Partitions (with SW upgrades) 64 16/(64)
Max Logical Drives (with SW upgrades) 2048 2048
RAID Levels 0,1,3, 5, 10 0,1,3,5,10
20TB vs. 18TB
Storage SystemStorage System
Georg Raff elt, Max-Planck-I nstitut für Physik, München, Germany CMB & Cosmology: Where Are We?, Ringberg Castle, Tegernsee, Germany, 2-6 December 2002
Dual FastT900+ 24 TB
IBM@server: xSeries 440 8* Xeon
(1.9GHz) 32 GB, Scalable configuration
x340 dual PIII/1 GHz /2 GbyteRH7.3
x205 P4/1.8 GHz /256 MBRH7.3
x205 P4/1.8 GHz /256 MBRH7.3
Storage Nodes
CE
LCFG
ScotGRID-Edinburgh: Schematic ScotGRID-Edinburgh: Schematic
Fibre connectionGbps
Georg Raff elt, Max-Planck-I nstitut für Physik, München, Germany CMB & Cosmology: Where Are We?, Ringberg Castle, Tegernsee, Germany, 2-6 December 2002
The CDF kit has 1 rack with only UPS 1 rack with 10 Dell dual Xeon 2.4 GHz 4GB, 7.5 TB disk, UPS 1 rack with IBM x370 8 of 700 MHz Pentium III with 8GB, 1 TB via FAStT500, UPS 1 tape library
ScotGRID-Glasgow: CDF KitScotGRID-Glasgow: CDF Kit
IBM: http://www.xcat.org/CDF: http://www.scali.com/
Georg Raff elt, Max-Planck-I nstitut für Physik, München, Germany CMB & Cosmology: Where Are We?, Ringberg Castle, Tegernsee, Germany, 2-6 December 2002
The eDIKT kit has: http://www.edikt.org/ 4th rack on the right and has 6 IBM x335 (1U high) dual Xeon 2.4 GHz and 1.5 GB memory 2 IBM BladeCentres (7U high) each with
14 Blades x335 dual Xeon 2.4 GHz and 1.5 GB memory
ScotGRID-Glasgow: eDIKT KitScotGRID-Glasgow: eDIKT Kit
Georg Raff elt, Max-Planck-I nstitut für Physik, München, Germany CMB & Cosmology: Where Are We?, Ringberg Castle, Tegernsee, Germany, 2-6 December 2002
Possible ScotGRID ConfigPossible ScotGRID Config
Cluster
Worker
nodes
Front-end
Globus gatekeeper+ job manager
Globus GRIS
EDG Job GridManager
Information Providers: queues, jobs, users
PBS
Working Directories/session1/session2
...
GridFTP
StorageElement(GridFTP)
control connection
data connection
GlobusReplicaCatalog
GIIS
GIIS
GIIS
GIIS
Cluster
Cluster
Tier2 GateWay (Broker)
A
u
t
h
VirtualOrg.Catalog
Georg Raff elt, Max-Planck-I nstitut für Physik, München, Germany CMB & Cosmology: Where Are We?, Ringberg Castle, Tegernsee, Germany, 2-6 December 2002
Summary of Tier-2 ScotGRID ProjectSummary of Tier-2 ScotGRID Project
Have Successfully completed Phase-1
ScotGRID had been upgraded – Phase-2 eDIRT Kit, CDF Kit HPS + Larger DataStore
ScotGRID: will concentrate on commissioning the new hardware during the next few month. before going to EDG2.0
There will be an ad. for ScotGRID Technical Coordinator
We will have to start to think about restructing the ScotGRID team for production running – early next year
In the New Year plan to integrate the Durham CPU Farm
Georg Raff elt, Max-Planck-I nstitut für Physik, München, Germany CMB & Cosmology: Where Are We?, Ringberg Castle, Tegernsee, Germany, 2-6 December 2002
BackUp Slid
es
Georg Raff elt, Max-Planck-I nstitut für Physik, München, Germany CMB & Cosmology: Where Are We?, Ringberg Castle, Tegernsee, Germany, 2-6 December 2002
Internet VLAN
10.0.0.0 VLAN
100 Mbps
1000 Mbps
Masternode Storage Nodes Head Nodes
Compute Nodes
Campus Backbone
59 x330 dual PIII 1GHz/2 Gbyte
x340 dual PIII/1 GHz /2 Gbytex340 dual PIII/1 GHz /2 Gbytex340 dual PIII/1 GHz/0.5 Gbyte
ScotGRID-Glasgow: Schematic ScotGRID-Glasgow: Schematic
Georg Raff elt, Max-Planck-I nstitut für Physik, München, Germany CMB & Cosmology: Where Are We?, Ringberg Castle, Tegernsee, Germany, 2-6 December 2002
Disk Arrays(Total 4.6 Tb)Disk Arrays(Total 4.6 Tb)
FastT 500 Storage ControllerFastT 500 Storage Controller
Server (4*Pentium Xeon, 16Gb RAM)Server (4*Pentium Xeon, 16Gb RAM)
SRIF Network
ScotGRID-Edinburgh: Schematic ScotGRID-Edinburgh: Schematic