Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran...
-
Upload
conrad-palmer -
Category
Documents
-
view
215 -
download
2
Transcript of Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran...
![Page 1: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/1.jpg)
Flexibility, Manageability and Performance in a Grid Storage Appliance
John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley, Andrea Arpaci-Dusseau, Remzi
Arpaci-Dusseau, and Miron Livny
University of Wisconsin
![Page 2: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/2.jpg)
Two trends
Data setsPerformance
• Storage appliances address both trends.
![Page 3: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/3.jpg)
Storage Appliances: + and -
Storage Appliances: Great for basic file service Easy to manage: Plug in and it works Good performance: Specialized just for I/O Reliable and available too
Storage Appliances for the Grid: Mismatch? Inflexible: Few, specific protocols (e.g., NFS) Costly: 10x the cost of PC + a few disks Difficult to integrate: Just one piece of the puzzle
![Page 4: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/4.jpg)
A Solution: NeST
NeST: A Storage Appliance for the Grid
Flexible: Multiple simultaneous protocols Virtual protocol layer
Low-cost: Use commodity machines Dynamic adaptation
Grid-aware: Integrate w/ higher-level systems Design specifically for the Grid
![Page 5: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/5.jpg)
Outline
Introduction General architecture Design goals
Flexibility Low-cost Grid-aware features
NeST in the Grid example Conclusions
![Page 6: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/6.jpg)
Physical storage layer
Chirp HTTP Grid FTP NFSCommon protocol layer
Dispatcher
Transfer Mgr Concurrency ModelsStorage Mgr
Control flowDatal flow
NeST: Protocol layer
Physical network layer Virtualizes different protocols
Mediates access to network
![Page 7: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/7.jpg)
Physical storage layer
Chirp HTTP Grid FTP NFSCommon protocol layer
Dispatcher
Transfer Mgr Concurrency ModelsStorage Mgr
Control flowDatal flow
NeST: Dispatcher
Physical network layer Mediates interaction between other components
Gathers information, advertises
![Page 8: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/8.jpg)
Physical storage layer
Chirp HTTP Grid FTP NFSCommon protocol layer
Dispatcher
Transfer Mgr Concurrency ModelsStorage Mgr
Control flowDatal flow
NeST: Storage manager
Physical network layer Space management
Access control
Virtualizes physical storage
![Page 9: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/9.jpg)
Physical storage layer
Chirp HTTP Grid FTP NFSCommon protocol layer
Dispatcher
Transfer Mgr Concurrency ModelsStorage Mgr
Control flowDatal flow
NeST: Transfer manager
Physical network layer Implementsscheduling policies
Chooses concurrency model
![Page 10: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/10.jpg)
Outline
Introduction General architecture Design goals
Flexibility Low-cost Grid-aware features
NeST in the Grid example Conclusions
![Page 11: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/11.jpg)
Flexibility: Multiple protocols
Problem: How to support multiple protocols? One approach: Just a Bunch of Servers (JBOS)
Problems with JBOS Lack of control (scheduling) Painful administration No shared code Larger memory footprint nfsd
wu-ftpd
httpd
JBOS Server
![Page 12: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/12.jpg)
NeST: Flexibility By Design
NeST: Integrate protocols and gain advantage Implementation like VFS
Integration introduces new challenges Different protocols allow different auth models More expensive to add a new protocol Less fault isolation
![Page 13: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/13.jpg)
NeST vs JBOS
Chirp GridFTP HTTP NFS0
5
10
15
20
25
30
35
Ser
ver
band
wid
th (
MB
/s)
0
5
10
15
20
25
30
35
Total
• For each protocol, NeST is comparable to JBOS server.
wu-
ftpd
Apa
che
linu
x nf
sd
Linux cluster- Dual PIII- 1 GB Ram- linux 2.2.19Each protocol- 4 clients- 10 MB files
![Page 14: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/14.jpg)
Exerting scheduling control
Different scheduling policies FCFS Cache-aware [USENIX ‘02] Proportional share
Proportional share scheduling Allows administrators to set protocol proportions
e.g. favor NFS Very difficult in JBOS
![Page 15: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/15.jpg)
Proportional share
0
5
10
15
20
25
30
35
Ser
ver
band
wid
th (
MB
/s)
FCFS 1:1:1:1 1:2:1:1 1:1:1:4Scheduling configuration
• In most cases, achieves Jain’s metric of fairness > 0.98 (1 is “fair”).
Linux cluster- Dual PIII- 1 GB Ram- linux 2.2.19Each protocol- 4 clients- 10 MB files
![Page 16: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/16.jpg)
Outline
Introduction General architecture Design goals
Flexibility Low-cost Grid-aware features
NeST in the Grid example Conclusions
![Page 17: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/17.jpg)
Low-Cost: New challenges Desire: Run on arbitrary OS on arbitrary PC
Software-only, user-level storage appliance Currently on Linux (release 0.9) and Solaris (beta)
Problem: Portable performance Performance under load is platform / workload dependent
Threads or processes on some systems, events on others May also be workload dependent (e.g. whether in cache)
NeST approach: Dynamic adaptivity Simultaneously support multiple concurrency models Monitor performance using each model Bias towards better model over time
![Page 18: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/18.jpg)
Adaptive Concurrency
0
1
2
3
Solaris: 1K files
Ave
tim
e pe
r re
ques
t (se
c)
Eve
nts
Thr
eads
Linux: 10 MB files
Eve
nts
Thr
eads
0
1
0.5
1.5
Ave
tim
e pe
r re
ques
t (m
s)
Ada
ptiv
e
Ada
ptiv
e
• Dynamic adaptation approaches “ideal” without static information.
![Page 19: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/19.jpg)
Outline
Introduction General architecture Design goals
Flexibility Low-cost Grid-aware features
NeST in the Grid example Conclusions
![Page 20: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/20.jpg)
Grid-Aware Mechanisms
Basic functionality Users and groups: Dynamic creation/deletion
does not need administrative intervention Access control: Generic AFS-style ACLs
Advanced functionality QoS: Preferential scheduling Advertises into global scheduling systems Flexible protocol and authentication mechanisms Self-cleaning storage guarantees: Lots
![Page 21: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/21.jpg)
Storage guarantees: Lots
Characteristics of Lots: Capacity: Total amount of data lot can store Duration: Time for which data is guaranteed to exist Set of files: Multiple files may co-exist within lot
Self-cleaning Expired lots become “best-effort” lots
Lot management Either default set created by administrator, OR
use resource management protocol to create before usage Implementation: File system quotas
Advantage: Integrates cleanly with local access methods Disadvantage: Performance hit for large writes
![Page 22: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/22.jpg)
Outline
Introduction General architecture Design goals
Flexibility Low-cost Grid-aware features
NeST in the Grid example Conclusions
![Page 23: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/23.jpg)
NeST in the Grid
LinuxNeST
SolarisNeST
Home Remote
Global Execution Manager
computenode
computenode
computenode
computenode
Condor Tribune
Advertisement
![Page 24: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/24.jpg)
NeST in the Grid
N N
Home Remote
Global Execution Mgr
GridFTP
3)
3) Global coordinates xfer1)
1) Home submits jobs
6)
6) Global terminates space5) 5) Global coordinates xfer
NFS4)
4) Global starts jobs2)
2) Global reserves space
![Page 25: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/25.jpg)
Conclusions
NeST: A storage appliance for the Grid Gain manageability Without sacrificing performance
Design goals: Flexibility: Virtual protocol architecture Low-cost: Adaptation mechanisms Grid-aware: Space management
Current status: release 0.9 available Future work
Hot deployable NeSTs, lot management extensions
![Page 26: Flexibility, Manageability and Performance in a Grid Storage Appliance John Bent, Venkateshwaran Venkataramani, Nick Leroy, Alain Roy, Joseph Stanley,](https://reader035.fdocuments.net/reader035/viewer/2022070404/56649f355503460f94c5325f/html5/thumbnails/26.jpg)
Questions?
http://www.cs.wisc.edu/condor/nest