Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau...
-
Upload
dina-poole -
Category
Documents
-
view
220 -
download
0
Transcript of Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau...
![Page 1: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/1.jpg)
Evolving RPC for Active StorageEvolving RPC for Active Storage
Muthian Sivathanu
Andrea C. Arpaci-Dusseau
Remzi H. Arpaci-Dusseau
University of Wisconsin-Madison
![Page 2: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/2.jpg)
Architecture of the future Everything is “active”
Cheaper, faster processing power Example: “smart” disks
Everything is “distributed” Network between processors and devices Example: Network-attached storage
Need: Effective software paradigms Leverage power of active components But remains easy to use
![Page 3: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/3.jpg)
Software systems of the future Need: Tools to build “active”, “distributed” systems
Pragmatic: Easy for system developers to use Powerful: Exploit active nature of systems
Active systems permit extensibility Download code to device Tailor to needs of applications/system Simplicity, maintainability
• Provide primitives, allow clients to compose interface
Traditional “distributed” systems built w/ RPC Simple, easy-to-use communication paradigm But not designed for “active” world
Build better distributed systems w/ “active” components
![Page 4: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/4.jpg)
Scriptable RPC SRPC: Paradigm for extensible distributed systems
Pragmatic: RPC-like development process Powerful: Exploit active components easily
Case study: Active storage High Performance
• Efficient “composition” of primitives Rapid addition of new functionality
• Powerful: Advanced consistency semantics over NFS • Simple: Substantial functionality in < 20 lines of code
Simplicity in design• Obviate distributed locking, crash recovery
Compelling paradigm for future systems
![Page 5: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/5.jpg)
Outline
Motivation Scriptable RPC Case Study: Active Storage
Performance Functionality Simplicity
Summary
![Page 6: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/6.jpg)
Scriptable RPC (SRPC) Evolve Remote Procedure Call (RPC) Augment RPC interface of “server” with a scripting capability
ClientScript
Limited executioncontext
Active Disk
Script Interpreter
Result
Prototype uses Tcl as the scripting language
![Page 7: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/7.jpg)
SRPC : Key issues
Migration path Efficient execution of scripts Safety
![Page 8: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/8.jpg)
Migration path
Make transition to new paradigm less intrusive Code to embed scripting into server
automatically generated Existing unmodified clients co-exist with
scripting clients Development process exactly the same
![Page 9: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/9.jpg)
Development: RPC
Interface definition
Interface Implementatio
n
IDL Compile
r
Stubs
Native Compiler
Distributed Service
![Page 10: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/10.jpg)
Development: SRPC
Interface definition
Interface Implementatio
n
IDL Compile
r
Stubs
Native Compiler
Distributed Service
Tcl Wrappers
Interpreter
![Page 11: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/11.jpg)
Efficient execution of scripts
Hide script interpretation overhead Script caching
Exploit efficient Tcl bytecode representation Concurrency
Multiple interpreters run simultaneously “Fast” standard library of primitives
Implemented in C
![Page 12: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/12.jpg)
Safety
Guard against misbehaving client scripts Limited execution environment: SafeTcl
Even while loops can be turned off Runtime type-checking
Prevent illegal memory references Automatic tracking of locks
Safe concurrent execution
![Page 13: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/13.jpg)
Outline
Motivation Scriptable RPC Case Study: Active Storage
Performance Functionality Simplicity
Summary
![Page 14: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/14.jpg)
Case Study: Active storage
Utilize CPU power at disks for client-specific processing
Previous approaches Demonstrate performance benefits But, require radically new architectures
• No migration path for existing services Limited class of applications
• Parallel database primitives
![Page 15: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/15.jpg)
Evaluation environment
Platform P-III 550 MHz machines, 1GB mem, 100 Mb/s net Linux kernel v2.2.19
Case studies enhance ScFS using SRPC
Application
VFS
ScFS
ActiveDiskNFS - like protocolLinux
Kernel
Client
![Page 16: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/16.jpg)
ScFS: Performance enhancements
Combine dependent sequence of operations into single script Reduction in network round-trips needed for
a logical operation• Benefit sensitive to network delay• Significant savings over dialup, wide-area• Even across overloaded “fast” networks
Reduction in total network traffic Helps overcome limitations in interface
![Page 17: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/17.jpg)
Pathname lookup
Read (dir page “/”)
Client Disk
RPC
Lookup “/foo”
![Page 18: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/18.jpg)
Pathname lookup
Find inodenumber
Read (dir page “/”)
Page Data
Client Disk
RPC
abc 21def 39
bar 52foo 40
Lookup “/foo”
![Page 19: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/19.jpg)
Pathname lookup
Find inodenumber
Page Data
GetAttr (inode 40)
Client Disk
RPC
Lookup “/foo”
Read (dir page “/”)
![Page 20: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/20.jpg)
Pathname lookup
Find inodenumber
Page Data
GetAttr (inode)
Attributes
Client Disk
RPC
Lookup “/foo”
Read (dir page “/”)
![Page 21: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/21.jpg)
Pathname lookup
Find inodenumber
Page Data
GetAttr (inode)
Attributes
Client Disk Client Disk
ExecScript (Read-GetAttr)
Read page, findinode number, getattributes
RPC SRPCLooku
p “/foo”
Lookup “/foo”
Read (dir page “/”)
![Page 22: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/22.jpg)
Pathname lookup
Find inodenumber
Page Data
GetAttr (inode)
Attributes
Client Disk Client Disk
ExecScript (Read-GetAttr)
Read page, findinode number, getattributes
Attributes
RPC SRPCLooku
p “/foo”
Lookup “/foo”
Read (dir page “/”)
![Page 23: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/23.jpg)
Pathname lookup: Benefits
![Page 24: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/24.jpg)
Pathname lookup: Benefits
![Page 25: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/25.jpg)
Pathname lookup: Benefits
![Page 26: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/26.jpg)
Pathname lookup: Benefits
![Page 27: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/27.jpg)
Performance: Summary Examples only illustrative
Other “compositions” possible too!
Micro-benchmarks Benefit due to reduced network roundtrips
Macro-benchmarks Postmark: 54% less network traffic TPC-B: 96% less network traffic
Facilitates working around minimal interfaces
![Page 28: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/28.jpg)
Outline
Motivation Scriptable RPC Case Study: Active Storage
Performance Functionality Simplicity
Summary
![Page 29: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/29.jpg)
ScFS: Functionality enhancements
Implement enhanced virtual protocols over physical protocols State can be added to stateless protocols System provides primitives
• Clients compose them into desired functionality
Examples AFS consistency semantics over NFS Sprite consistency semantics over NFS
![Page 30: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/30.jpg)
Consistency semantics: NFS vs AFS
NFS Stateless server Client checks periodically for updates
AFS Write-on-close semantics Server tracks clients caching a file
• Notifies clients when modified file written Requires server-side state, participation
• Cannot implement using existing paradigms
![Page 31: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/31.jpg)
Scripted AFS consistency
Client A
Client B
![Page 32: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/32.jpg)
Scripted AFS consistency
Client A
Client B
ExecScript (AFS_OPEN)
Open
![Page 33: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/33.jpg)
Scripted AFS consistency
Client A
Client B
File Data A
ExecScript (AFS_OPEN)
Callback list
Cached file
![Page 34: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/34.jpg)
Scripted AFS consistency
Client A
Client B
ExecScript
(AFS_OPEN)
Open
A Callback list
Cached file
![Page 35: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/35.jpg)
Scripted AFS consistency
Client A
Client BFile Data
A
BExecScript
(AFS_OPEN)Callback list
Cached file
![Page 36: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/36.jpg)
Scripted AFS consistency
Client A
Client B ExecScript
(AFS_CLOSE_DIRTY)
A
B
File Close
Callback list
Cached file
![Page 37: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/37.jpg)
Scripted AFS consistency
Client A
Client B
B
INVALIDATE_CACHE
ExecScript
(AFS_CLOSE_DIRTY)Callback list
Cached file
![Page 38: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/38.jpg)
Functionality: Summary SRPC: Powerful
Possible to add complex functionality Even those requiring augmenting server state
SRPC: Simple AFS consistency
• 2 scripts, < 10 lines each
Sprite consistency• 3 scripts, < 20 lines each
Simple base system, compact scripts to extend it
![Page 39: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/39.jpg)
ScFS: Simplicity enhancements
Ability to group operations at server Simplifies implementation of atomic sets of
operations Often, obviates need for distributed locks,
distributed crash recovery Example - concurrent directory updates
![Page 40: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/40.jpg)
Concurrent directory updates
Client A Client B
abc 21def 39
![Page 41: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/41.jpg)
Concurrent directory updates
Client A Client B
REA
Dabc 21def 39
abc 21def 39
Create(“/foo”)
![Page 42: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/42.jpg)
Concurrent directory updates
Client A Client B
READ
abc 21def 39
abc 21def 39
Create(“/foo”) Create(“/bar”)
abc 21def 39
![Page 43: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/43.jpg)
Concurrent directory updates
Client A Client B
abc 21def 39
abc 21def 39foo 41
abc 21def 39bar 52
![Page 44: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/44.jpg)
Concurrent directory updates
Client A Client B
WRI
TE
abc 21def 39foo 41
abc 21def 39foo 41
abc 21def 39bar 52
![Page 45: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/45.jpg)
Concurrent directory updates
Client A Client B
WRITE
abc 21def 39bar 52
abc 21def 39foo 41
abc 21def 39bar 52
![Page 46: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/46.jpg)
Concurrent directory updates
Non-scripting Distributed locking, distributed crash recovery
• Clients acquire locks before read-modify-write• Recover from client failures while holding locks
SRPC Script acquires in-memory lock at server Just enforce mutual exclusion within single
address space
![Page 47: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/47.jpg)
Summary
SRPC High Performance, rapid extensibility, simplicity Makes effective use of “active” architecture
All scripts less than 20 lines of code Some implement non-trivial functionality
Fewer lines of code => Fewer bugs, more robust systems Ease of building systems with active components
Don’t have a complex system catering to all client requirements Provide primitives, enhance with compact scripts
![Page 48: Evolving RPC for Active Storage Muthian Sivathanu Andrea C. Arpaci-Dusseau Remzi H. Arpaci-Dusseau University of Wisconsin-Madison.](https://reader036.fdocuments.net/reader036/viewer/2022062408/56649f0b5503460f94c1e804/html5/thumbnails/48.jpg)
Wisconsin Network Disks Group
http://www.cs.wisc.edu/wind
Questions ?