Region-Based Software Distributed Shared Memory
description
Transcript of Region-Based Software Distributed Shared Memory
Region-Based Software Distributed Shared Memory
Song Li, Yu Lin, and Michael Walker
CS 656 -- Operating Systems
May 1, 2000
Outline
• Motivation
• Problems
• DSM Models
• Proposed Solution
• Conclusion
Motivation
• Share distributed resources to increase computing power– First Solution: Tightly-coupled system– Problem: Central memory bus– Alternative: Loosely-coupled system– New Problem: Need IPC in distributed systems
Motivation: RPC
• Solution: Remote Procedure Calls (RPC)– Snd/Rcv protocol– Pass-by-value– OS Assignment 1
• Problems with RPC– Explicit awareness of communication– Marshalling complex data structures is hard
Motivation: DSM
• Solution: Distributed Shared Memory (DSM) [Li86--Yale]– Idea: Nonresident pages are fetched from network
– Illusion of tightly-coupled system in a loosely-coupled one
– Abstraction on top of message-passing model– Programmers use memory-accessing paradigm
DSM: Potential Benefits
• Shared access to memory
• Avoids Von-Neumann bottleneck
• Familiar abstraction
• Pass-by-reference
• Spreads communication load over time
• Provides more memory than is locally available
Problems
• Who handles remote access?
• What should be shared?
• Cache, cache, cache…but how?
• Page Replacement & Thrashing
• More than one way to solve the problems...
DSM Models: Decisions
• Hardware: MMU controls message passing, data migration, and caching
• Software: Control managed by OS or library
• Page-based shared memory versus shared variable versus Object
• Cache consistency models
Proposed Solution: Overview
• Software-based model using libraries
• Variable-size regions -- similar to segments– Every granularity has its advantages/drawbacks– Make the region size flexible, provide
reasonable defaults (sound familiar?)
• Multiple reader, single writer (MRSW) caching support
Solution: User Interface
• r_handle sm_malloc(size);• int sm_regionat(r_handle, attrib);• int sm_read(r_handle, offset, buf, size);
• int sm_write(r_handle, offset, buf, size);
• int sm_regiondt(r_handle);• int sm_free(r_handle);
Solution: Mode of Operation
...
...Provider 1
Shared Memory
ManagerMaster Table
Client 1
Cache
Client Table
Client 2
Cache
Client Table
Client n
Cache
Client Table
Provider 2
Shared Memory
Provider n
Shared Memory
Solution: Mode of Operation
...
...Provider 1
Shared Memory
ManagerMaster Table
Client 1
Cache
Client Table
Client 2
Cache
Client Table
Client n
Cache
Client Table
Provider 2
Shared Memory
Provider n
Shared Memory
(1) Allocate memory
Solution: Mode of Operation
...
...Provider 1
Shared Memory
ManagerMaster Table
Client 1
Cache
Client Table
Client 2
Cache
Client Table
Client n
Cache
Client Table
Provider 2
Shared Memory
Provider n
Shared Memory
(2) Query table & attach region
Solution: Mode of Operation
...
...Provider 1
Shared Memory
ManagerMaster Table
Client 1
Cache
Client Table
Client 2
Cache
Client Table
Client n
Cache
Client Table
Provider 2
Shared Memory
Provider n
Shared Memory
(3) Send region info to client
Solution: Mode of Operation
...
...Provider 1
Shared Memory
ManagerMaster Table
Client 1
Cache
Client Table
Client 2
Cache
Client Table
Client n
Cache
Client Table
Provider 2
Shared Memory
Provider n
Shared Memory
(4) Client communicates with provider
Solution: Mode of Operation
...
...Provider 1
Shared Memory
ManagerMaster Table
Client 1
Cache
Client Table
Client 2
Cache
Client Table
Client n
Cache
Client Table
Provider 2
Shared Memory
Provider n
Shared Memory
(5) Client uses cached copy
Solution: Mode of Operation
...
...Provider 1
Shared Memory
ManagerMaster Table
Client 1
Cache
Client Table
Client 2
Cache
Client Table
Client n
Cache
Client Table
Provider 2
Shared Memory
Provider n
Shared Memory
(6) Client detaches and frees memory
Solution: Mode of Operation
...
...Provider 1
Shared Memory
ManagerMaster Table
Client 1
Cache
Client Table
Client 2
Cache
Client Table
Client n
Cache
Client Table
Provider 2
Shared Memory
Provider n
Shared Memory
(7) Memory is freed
Solution: MRSW Caching Model
• Replication is good for reads, so allow MR– No communication overhead
• Writer requires cache invalidation, so restrict to SW– MW is too complicated
Solution: MRSW Caching Model
...
...Provider 1
Shared Memory
ManagerMaster Table
Client 1
Cache
Client Table
Client 2
Cache
Client Table
Client n
Cache
Client Table
Provider 2
Shared Memory
Provider n
Shared Memory
Readers
(1) Multiple readers
Solution: MRSW Caching Model
...
...Provider 1
Shared Memory
ManagerMaster Table
Client 1
Cache
Client Table
Client 2
Cache
Client Table
Client n
Cache
Client Table
Provider 2
Shared Memory
Provider n
Shared Memory
Readers Writer
?
(2) Write request
Solution: MRSW Caching Model
...
...Provider 1
Shared Memory
ManagerMaster Table
Client 1
Cache
Client Table
Client 2
Cache
Client Table
Client n
Cache
Client Table
Provider 2
Shared Memory
Provider n
Shared Memory
Readers Writer
(3) Cache invalidation to selected clients only
Solution: MRSW Caching Model
...
...Provider 1
Shared Memory
ManagerMaster Table
Client 1
Cache
Client Table
Client 2
Cache
Client Table
Client n
Cache
Client Table
Provider 2
Shared Memory
Provider n
Shared Memory
Readers Writer
(4) Single writer, multiple readers
Conclusion: Current Status
• Basic client/manager model is implemented
Conclusion: Remaining Work
• Page replacement strategy: LRU
• Region protection mechanism
• Replicated managers for better performance and fault tolerance
• Evaluation...
Conclusion: Evaluation
• Find distributed apps to use DSM model
• Performance evaluation:– Test apps with DSM model and message-
passing model– Desired results: DSM performance is better