Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7...
Transcript of Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7...
![Page 1: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/1.jpg)
Systems Design and ImplementationI.7 –Memory Management
h
Jan Stoess
University of Karlsruhe
System Architecture Group, SS 2009
University of Karlsruhe
June 9, 2009
![Page 2: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/2.jpg)
Overview
Loading a program into memory Case Studies
4.3 BSD Virtual Memory on VAX-11SVR4 VM Architecture
2© 2009 University of Karlsruhe, System Architecture Group
SVR4 VM Architecture Sawmill Dataspaces
![Page 3: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/3.jpg)
The Road to a Running Program
SourceCode
Compiler/Assembler
ObjectFile
hDynamic
3© 2009 University of Karlsruhe, System Architecture Group
OtherObjectFiles
Linker
LoaderExecutable
File
SystemLibraries
RunnableIn-memory
Image
![Page 4: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/4.jpg)
How do we get there?
What do we load? Where do we load it?
4© 2009 University of Karlsruhe, System Architecture Group
RunnableIn-memory
Image
![Page 5: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/5.jpg)
What do we load ?
What we load is partially defined by ELF. Executable and Linkable Format Four major parts in ELF file
ELF header – roadmap Program headers describe segments directly
ELF header
Prog. Hdrs
.text
.rodata
.data
5© 2009 University of Karlsruhe, System Architecture Group
g g yrelated to program loading
Section headers describe contents of the file The data itself
.eh_frame
.sbss
.bss
.comment
.note
Section Hdrs
![Page 6: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/6.jpg)
An Example
Root Task (pager) Runnable Image
stack
Location defined by convention in
VM system
6© 2009 University of Karlsruhe, System Architecture Group
data
code
Location defined by linker
Location defined by linker
Dynamicdata
Location defined by VM system
![Page 7: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/7.jpg)
Load directly in physical memory?
Root task
Root Task
Test client
Test Client
Test client
Pros Simple one-to-one virtual-to-physical
mapping Easy
7© 2009 University of Karlsruhe, System Architecture Group
Testclientmodule
y
Cons Only one image per executable Must link at the correct address
error prone and cumbersome Fragmentation is a problem Limited to physical memory size
![Page 8: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/8.jpg)
Dynamic relocation
Root task
Root Task
Test client
Test Client 2
Test clientTest client Pros
Simple one-to-one virtual-to-physical mapping
8© 2009 University of Karlsruhe, System Architecture Group
Cons PIC code has performance penalty,
or relocation has a startup penalty Fragmentation is still a problem Still limited to physical memory size
Testclientmodule
![Page 9: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/9.jpg)
Address Translation / Segmentation
Root task
Root Task
Test client
Test Client 2
Test client
Test client Pros
Multiple instances of same executable No a priori constructed load map No relocation or PIC
9© 2009 University of Karlsruhe, System Architecture Group
No relocation or PIC Can use more than available physical
memory
Cons Need a translation table (base,limit) Fragmentation is still a problem (why?) Swapping is coarse grained
Testclientmodule
![Page 10: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/10.jpg)
Paged Virtual Memory
Pros Multiple instances of same executable No a priori constructed load map No relocation or PIC Simple physical memory management Fragmentation dramatically reduced
Root Task Client 2
P1
P2
P1
P1
Client 1
10© 2009 University of Karlsruhe, System Architecture Group
g y(internal fragmentation only)
Can use more than available physical memory
Cons Need a page-based translation table
and hardware This is not free
P2
P3
P3
P2
P3
P1
P1
P2
P3
P1
P2
P3
![Page 11: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/11.jpg)
Case Study4.3 BSD Virtual Memory on VAX-11
Three major components1. Core map
Global frame table
2 Page tables
11© 2009 University of Karlsruhe, System Architecture Group
2. Page tables Per Process translation table
3. Swap maps Per-process mapping table from virtual pages
space to disk blocks in swap space
![Page 12: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/12.jpg)
Core Map
P l d ib d b f t i
Nonpaged
pool
paged
pool
Error
buffer
Physical Memory
12© 2009 University of Karlsruhe, System Architecture Group
Page pool described by an array of cmap entries Cmap entries contain
Name <type, owner, virtual page number> Free list pointers <next, prev> For text pages <device, block number> Synchronization flags
![Page 13: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/13.jpg)
Virtual Address Space
text data stack kernel
P0 ReservedS0P10Gb 1Gb 2Gb 3G 4Gb
13© 2009 University of Karlsruhe, System Architecture Group
Note: Data and stack areas are free to grow. The page tables describe the layout of the address
space.
![Page 14: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/14.jpg)
Page States
Each individual page may be in one of the following states. Resident Fill-on-demand
Fill f t t
text data stack kernel
14© 2009 University of Karlsruhe, System Architecture Group
Fill-from-text Zero-fill
Swapped out
State encoded inpage table entries
![Page 15: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/15.jpg)
Process creation
Allocate page tables representing the address space. Initialize all page table entries as either
text data stack kernel
15© 2009 University of Karlsruhe, System Architecture Group
p g Fill-from-text Zero-fill
![Page 16: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/16.jpg)
Page-fault handling
Bounds error – access to inaccessible area. Easy to detect bounds error if
text data stack kernel
16© 2009 University of Karlsruhe, System Architecture Group
Easy to detect, bounds error if access to invalid page table entry, or access to non-existent page table entry.
Except, automatic stack/heap growth.
![Page 17: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/17.jpg)
Page fault handling
17© 2009 University of Karlsruhe, System Architecture Group
![Page 18: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/18.jpg)
Disadvantages
No support for shared memory. No support for memory mapped files. No support for shared libraries. No copy-on-write support.
18© 2009 University of Karlsruhe, System Architecture Group
VAX-specific architecture VAX specific optimisations and structure Not portable
Code not modular – difficult to add features.
![Page 19: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/19.jpg)
Memory mapped files
19© 2009 University of Karlsruhe, System Architecture Group
![Page 20: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/20.jpg)
Memory mapped files
20© 2009 University of Karlsruhe, System Architecture Group
Can be used as a mechanism to implement: Shared memory Shared libraries
![Page 21: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/21.jpg)
Case StudySVR4 VM Architecture
The basic concepts are Page – a frame of physical memory Address space
Segment a region in an address space
21© 2009 University of Karlsruhe, System Architecture Group
Segment – a region in an address space Hardware address translation – page tables Anonymous page – page with no
permanent storage
J. Moran, "SunOS Virtual Memory Implementation", European UNIX Users Group (EUUG) Conference, Spring 1988
![Page 22: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/22.jpg)
Case StudySVR4 VM Architecture
22© 2009 University of Karlsruhe, System Architecture Group
J. Moran, "SunOS Virtual Memory Implementation", European UNIX Users Group (EUUG) Conference, Spring 1988
![Page 23: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/23.jpg)
Physical memory
P l d ib d b f t i
Nonpaged
pool
paged
pool
Error
buffer
Physical Memory
23© 2009 University of Karlsruhe, System Architecture Group
Page pool described by an array of page entries Page entries contain
Name <vnode, offset> List pointers <next, prev> etc.
Free, I/O, hash chains, vnode
HAT info to locate all mappings Synchronization flags
J. Moran, "SunOS Virtual Memory Implementation", European UNIX Users Group (EUUG) Conference, Spring 1988
![Page 24: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/24.jpg)
Virtual Address Space
Address spaces are composed of memory objects called segments
text data stack kernellibrarySharedmem file
24© 2009 University of Karlsruhe, System Architecture Group
called segments. Segments are a mapping between address space
regions and backing-store objects (files, swap space, etc.)
Operations on an address space Alloc, free, dup, map, unmap, setprot, checkprot.
J. Moran, "SunOS Virtual Memory Implementation", European UNIX Users Group (EUUG) Conference, Spring 1988
![Page 25: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/25.jpg)
Anonymous memory
Memory that is private to a process. Not externally referable to, thus is “anonymous”
Memory that has no permanent storage. Contents are lost when process exits
25© 2009 University of Karlsruhe, System Architecture Group
Paged by the kernel to swap space. Zero-filled.
J. Moran, "SunOS Virtual Memory Implementation", European UNIX Users Group (EUUG) Conference, Spring 1988
![Page 26: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/26.jpg)
Hardware Address Translation Layer
Machine dependent – page tables. Operations
alloc, free. memload, devload, unload.
26© 2009 University of Karlsruhe, System Architecture Group
pageunload, pagesync. Data in the HAT layer is redundant – it can be rebuilt
from the machine independent layer. Interface is machine-independent.
J. Moran, "SunOS Virtual Memory Implementation", European UNIX Users Group (EUUG) Conference, Spring 1988
![Page 27: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/27.jpg)
Segments
Each region has a segment driver associated with it seg_vn – Mappings to regular files and anonymous object
(vnodes). seg_dev – Mappings to devices (frame buffers)
Segment driver data
27© 2009 University of Karlsruhe, System Architecture Group
Segment driver data Current and max protection Mapping type (shared or private) Pointer to vnode Offset to beginning of file
Segment drivers support the following operations create, dup, fault, setprot, checkprot, unmap, swap out,
sync.
J. Moran, "SunOS Virtual Memory Implementation", European UNIX Users Group (EUUG) Conference, Spring 1988
![Page 28: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/28.jpg)
Creating a Mapping
User calls mmap(“file”) Mmap checks permissions, existence, etc. Mmap calls as_map() to associate the file with a
region in the address space.
28© 2009 University of Karlsruhe, System Architecture Group
as_map() allocates segment data for the segment, and calls create() in the appropriate segment driver.
J. Moran, "SunOS Virtual Memory Implementation", European UNIX Users Group (EUUG) Conference, Spring 1988
![Page 29: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/29.jpg)
Creating a process
Simplistically, involves creating mappings (segments): Text -> appropriate region of executable file. Data -> anonymous memory
29© 2009 University of Karlsruhe, System Architecture Group
Stack -> anonymous memory Shared libraries are mmaped by the client itself as
needed.
J. Moran, "SunOS Virtual Memory Implementation", European UNIX Users Group (EUUG) Conference, Spring 1988
![Page 30: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/30.jpg)
Page fault handling outline
Kernel calls as_fault(). Is the fault within a segment?
No -> bounds error! Yes -> call the fault() routine of the associated segment
driver
30© 2009 University of Karlsruhe, System Architecture Group
driver. The segment driver locates/allocates the data associated
with the fault, and returns.
All the complexity is in the segment specific drivers.
J. Moran, "SunOS Virtual Memory Implementation", European UNIX Users Group (EUUG) Conference, Spring 1988
![Page 31: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/31.jpg)
Advantages
Design is modular Easier to extend, modify
Highly portable Machine-dependent translation functionality in HAT layer
Various types of memory sharingR d h i l
31© 2009 University of Karlsruhe, System Architecture Group
Reduces physical memory usage Powerful Mmap
Supports file access, shared memory, and shared libraries Flexibility through segment drivers
Allows use of any object representable by vnodee.g. NFS files.
J. Moran, "SunOS Virtual Memory Implementation", European UNIX Users Group (EUUG) Conference, Spring 1988
![Page 32: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/32.jpg)
Disadvantages
Kernel consumes more memory State associated with all the abstractions
More complex and slower algorithms Modularity restricts flexibility
Framework may prevent machine-specific optimizations
32© 2009 University of Karlsruhe, System Architecture Group
y p p p Copy-on-write not always faster than anticipatory copying
However, in general the benefits of new functionality outweighs the performance penalties.
![Page 33: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/33.jpg)
Implementing a Multi-Server OSwith Dataspaces
Concept & Implementation
![Page 34: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/34.jpg)
Traditional OSes: Kernel-based VM Application-control impossible Extensibility limited
Motivation
34© 2009 University of Karlsruhe, System Architecture Group
Extensibility limited
Multi-Server OSes: Kernel-based VM management primitives only VM management defined and implemented by user-
level pagers
![Page 35: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/35.jpg)
Observation: Address Space
Consists of regions Different semantics Different types
Diff t
35© 2009 University of Karlsruhe, System Architecture Group
Stack BinaryHeap
Different resources
Task A
File
![Page 36: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/36.jpg)
Involved Parties
Tasks that need „data“ Pager(s), Providers of „data“
36© 2009 University of Karlsruhe, System Architecture Group
![Page 37: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/37.jpg)
We want...
Diversity Customizable Control over policies
Dynamic extensibility
37© 2009 University of Karlsruhe, System Architecture Group
y a s b y Code reuse Easy implementation of policies
Performance Abstractions should not limit optimizations
![Page 38: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/38.jpg)
The SawMill framework for VM diversity
Framework to build and customize VMM Policy-free abstractions for VM management Decomposing of VMM into components
Dynamic configurability
38© 2009 University of Karlsruhe, System Architecture Group
Dynamic configurability
![Page 39: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/39.jpg)
Microkernel Provides:
Threads Address spaces IPC
Hierarchical VM system
39© 2009 University of Karlsruhe, System Architecture Group
Hierarchical VM system Mapping IPC Grant IPC
User-level-pager (per thread) map
grant
![Page 40: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/40.jpg)
Microkernel Provides:
Pagefault IPC Source tid Address
R/W
40© 2009 University of Karlsruhe, System Architecture Group
map
grant
R/W IP Mapping
![Page 41: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/41.jpg)
The Dataspace Concept
A dataspace is is memory concept denotes an abstract data container
represents unstructured data
41© 2009 University of Karlsruhe, System Architecture Group
represents unstructured data can be associated with files, memory, frame
buffers, …
![Page 42: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/42.jpg)
Dataspace vs. Region
Dataspace Entity of a dataspace
manager No size associated
Region In client‘s address
space fpage
42© 2009 University of Karlsruhe, System Architecture Group
Logical object Has a size Makes part of a
dataspace accessible
![Page 43: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/43.jpg)
accessing dataspaces
Dataspaces Server and Client
Dataspace Manager A Dataspace Manager B
43© 2009 University of Karlsruhe, System Architecture Group
Stack BinaryHeap
Region
File
Dataspace
Client
Region
Dataspace
![Page 44: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/44.jpg)
Multiple Dataspace Managers
DSM 0(i.e. files)
DSM 1(i.e. mem...)
44© 2009 University of Karlsruhe, System Architecture Group
Task D
Task A
Task C
Task B
![Page 45: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/45.jpg)
Region Mapper (RM)
One user level pager per task Customizable Efficient
45© 2009 University of Karlsruhe, System Architecture Group
Task ARM Pager
Worker Threads
![Page 46: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/46.jpg)
Attach Scenario
Stack File
Dataspace Manager
1. open2. return dataspace3. attach dataspace
46© 2009 University of Karlsruhe, System Architecture Group
Stack BinaryHeapFile
Client
Worker
RM
2. return
3. attach
1. open
![Page 47: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/47.jpg)
Involved Parties
Client address space Pagefault handler (region mapper) Dataspace manager
47© 2009 University of Karlsruhe, System Architecture Group
![Page 48: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/48.jpg)
Pagefault
Stack File
Dataspace Manager
1. access memory2. kernel pagefault IPC3. identify region4. request page mapping
5. memory „prepared“6 i
48© 2009 University of Karlsruhe, System Architecture Group
Stack BinaryHeapFile
Client
Worker
RM
2. kernel pagefault IPC
4. map_pagey p p
6. mapping to rm7. empty ipc to worker8. resuming
1. memory access
6. mapping
![Page 49: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/49.jpg)
Paging of the Region Mapper
External Region Mapper
RM External
49© 2009 University of Karlsruhe, System Architecture Group
Stack BinaryHeapFile
Task A
RM
![Page 50: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/50.jpg)
Dataspace Operations
Dataspace manager open
(signature depends on type of dataspace)
close
Region mapper attach detach pagefault
50© 2009 University of Karlsruhe, System Architecture Group
close map page share / transfer
![Page 51: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/51.jpg)
Hierachical Dataspaces
Stacking - build dataspaces with different semantics out of simple dataspaces
Specializing data container, allowing code reuse
51© 2009 University of Karlsruhe, System Architecture Group
![Page 52: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/52.jpg)
C-O-W Dataspace
C-O-W DS
read:1. read2. mapping from source
read olnywrite:1. write2. pf3. attach anonym backing4. copy from source5. return mapping
C-O-W DS
52© 2009 University of Karlsruhe, System Architecture Group
Anonym. Mem DS Source DS
C-O-W DSMAddress Space Source
copy
![Page 53: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/53.jpg)
Summary Dataspaces
Pros:+ decomposing+ distribution+ simple user pager
possible
Cons:- overhead (on pf.
crossing multiple protection domains)
- large virtual address
53© 2009 University of Karlsruhe, System Architecture Group
possible+ easy sharing+ flexible+ customizable
gspace needed
- leads to confusion on first contact
![Page 54: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/54.jpg)
Further reading
The Design and Implementation of the 4.3BSD UNIX Operating SystemS.J. Leffler, M.K. McKusick, M.J. Karels, J.S. QuartermanAddison Wesley, May 1989
SunOS Virtual Memory ImplementationJ.P. MoranSun Microsystems, Inc.
54© 2009 University of Karlsruhe, System Architecture Group
y ,
The Sawmill Framework for Virtual Memory DiversityM. Aron, L. Deller, K. Elphinstone, T. Jaeger, J. Liedtke, and Y. ParkSixth Australasian Computer Systems Architecture Conference (ACSAC2001), Bond University, Gold Coast, Queensland, January 29 - February 2, 2001
![Page 55: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/55.jpg)
Tasks and Virtual Memory in SDI OS
TestClient
?
TestClient 2
Wanted: Paged virtual memory for tasks
Multiple instances of the same binary
F t? ?
TestClient 3
55© 2009 University of Karlsruhe, System Architecture Group
L4 Micro kernel
NameServer
SimplePager
LogServer
Sigma0
GRUBFileserver
Features Create new task from executable file Command line support? Destroy tasks More?
What else do we need?
PrivSCServer
![Page 56: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/56.jpg)
SDI OS
We will implement page-based virtual memory
What are the implementation issues???
56© 2009 University of Karlsruhe, System Architecture Group
What are the implementation issues???
![Page 57: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/57.jpg)
Bits and Pieces
Frame Table – maintains usable memory What info needs to be in there? Initialize based on available memory
Where does the memory come from? Anonymous memory server
Page Table maintains virtual memory layout
57© 2009 University of Karlsruhe, System Architecture Group
Page Table – maintains virtual memory layout Suggest 2-level x86-like
Advantage of having 4K node sizes, easy (de)allocation Alternative: Section list
Process Table Bookkeeping about processes
Pointer to page table Thread id, state, etc
![Page 58: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/58.jpg)
Bits and Pieces
Loader Takes an ELF file
Builds an image using physical frames Builds a page table to map virtual page in the new address space to
frames in the root task
Don’t forget: You need a convention for stack handling.
58© 2009 University of Karlsruhe, System Architecture Group
g g
Pager Receives page faults. Sends mappings based on information about task
Stored in a page table Stored in section list
Process Manager Creating processes Destroying processes Listing processes?
![Page 59: Systems Design and Implementation I.7 –Memory Management · Systems Design and Implementation I.7 –Memory Management h Jan Stoess University of Karlsruhe System Arc itecture Group,](https://reader034.fdocuments.net/reader034/viewer/2022050504/5f96444eb75604521620285b/html5/thumbnails/59.jpg)
Upcoming talks (thursday in a week)
Memory and device server groups Think up how to achieve paged virtual
memory to support multiple testclients What components?
59© 2009 University of Karlsruhe, System Architecture Group
What components? How do they interact?
Thursday: Holiday Tuesday Lecture:
Device drivers