CS510 Concurrent Systems
28
CS510 Concurrent Systems “What is RCU, Fundamentally?” By Paul McKenney and Jonathan Walpole Daniel Mansour (Slides adapted from Professor Walpole’s)
description
CS510 Concurrent Systems. “What is RCU, Fundamentally?” By Paul McKenney and Jonathan Walpole Daniel Mansour (Slides adapted from Professor Walpole’s). Review of Concurrency. The Story So Far Critical Sections Shared global objects Blocking/Non-Blocking Reader/Writers - PowerPoint PPT Presentation
Transcript of CS510 Concurrent Systems
CS510 Concurrent Systems
“What is RCU, Fundamentally?”By Paul McKenney and Jonathan Walpole
Daniel Mansour(Slides adapted from Professor Walpole’s)
2
Review of Concurrency
The Story So Faro Critical Sections
• Shared global objects• Blocking/Non-Blocking
o Reader/Writers• Reads are “safer”• Writes are Fewer
o Hardware/Compiler Optimizations• Operations are not always in order
o Memory Reclamation• Deleted Objects can still be read
3
Read-Copy Update (RCU)
Read/Writeso Do not allow reads while writes are occurringo RCU solves this by allowing multiple versions
Insertiono Publish-Subscribe Mechanism
Deletiono Wait for Readers to completeo Non-Blocking
4
Publish-Subscribe Mechanism
1 struct foo { 2 int a; 3 int b; 4 int c; 5 }; 6 struct foo *gp = NULL; 7 8 /* . . . */ 9 10 p = kmalloc(sizeof(*p), GFP_KERNEL); 11 p->a = 1; 12 p->b = 2; 13 p->c = 3; 14 gp = p;
5
Publish-Subscribe Mechanism
1 struct foo { 2 int a; 3 int b; 4 int c; 5 }; 6 struct foo *gp = NULL; 7 8 /* . . . */ 9 10 p = kmalloc(sizeof(*p), GFP_KERNEL); 11 gp = p; 12 p->a = 1; 13 p->b = 2; 14 p->c = 3;
Reordered, concurrent readerswould get uninitialized values
6
Publish-Subscribe Mechanism
Publish the Objecto rcu_assign_pointer
1 p->a = 1; 2 p->b = 2; 3 p->c = 3; 4 rcu_assign_pointer(gp, p);
7
Publish-Subscribe Mechanism
Readerso Ordering can still be an issue
o p->a can be read before p in some compiler optimizations
1 p = gp; 2 if (p != NULL) { 3 do_something_with(p->a, p->b, p-c); 4 }
8
Publish-Subscribe Mechanism
Subscribe to the objecto rcu_dereference
o Implements any necessary memory barriers
1 rcu_read_lock() 2 p = rcu_dereference(gp) 3 if (p != NULL { 4 do_something_with(p->a, p->b, p->c); 5 } 6 rcu_read_unlock();
9
Implementation
Doubly Linked Listso list_head
o Head links to tail with prev and vice versa
10
Implementation - list_head
Publishing 1 struct foo { 2 struct list_head list; 3 int a; 4 int b; 5 int c; 6 }; 7 LIST_HEAD(head); 8 9 /* . . . */ 10 11 p = kmalloc(sizeof(*p), GFP_KERNEL); 12 p->a = 1; 13 p->b = 2; 14 p->c = 3; 15 list_add_rcu(&p->list, &head);
Still needs to be protected
11
Implementation - list_head
Subscribing 1 rcu_read_lock(); 2 list_for_each_entry_rcu(p, head, list) { 3 do_something_with(p->a, p->b, p->c); 4 } 5 rcu_read_unlock();
12
Implementation - hlist_head/hlist_node
Doubly Linked Listso hlist_head/hlist_node
o Linear list
13
Implementation - hlist_head/hlist_node
Publishing 1 struct foo { 2 struct hlist_node *list; 3 int a; 4 int b; 5 int c; 6 }; 7 HLIST_HEAD(head); 8 9 /* . . . */ 10 11 p = kmalloc(sizeof(*p), GFP_KERNEL); 12 p->a = 1; 13 p->b = 2; 14 p->c = 3; 15 hlist_add_head_rcu(&p->list, &head);
14
Implementation - list_head
Subscribing 1 rcu_read_lock(); 2 hlist_for_each_entry_rcu(p, q, head, list) { 3 do_something_with(p->a, p->b, p->c); 4 } 5 rcu_read_unlock();
Iteration requires checking for nullInstead of p = head
15
Reclaiming Memory
Waitingo rcu_read_lock and rcu_read_unlock
• Must not block or sleep
Reclaiming Memory
Basic Process
1. Alter the data (replace/delete an element)
2. Wait for all pre-existing RCU reads to completely finish• synchronize_rcu• New RCU reads have no way to access removed element
3. Reclaim/Recycle memory
16
Reclaiming Memory
Example
17
1 struct foo { 2 struct list_head list; 3 int a; 4 int b; 5 int c; 6 }; 7 LIST_HEAD(head); 8 9 /* . . . */ 10 11 p = search(head, key); 12 if (p == NULL) { 13 /* Take appropriate action, unlock, and return. */ 14 } 15 q = kmalloc(sizeof(*p), GFP_KERNEL); 16 *q = *p; 17 q->b = 2; 18 q->c = 3; 19 list_replace_rcu(&p->list, &q->list); 20 synchronize_rcu(); 21 kfree(p);
Reclaiming Memory
synchronize_rcuo rcu_read_lock and rcu_read_unlock do nothing on non-
CONFIG-PREEMPT kernelso Recall that code in rcu_read blocks can not sleep or block,
preventing context switcheso When a context switch occurs, we can deduce that the read
has completedo synchronize_rcu waits for each cpu to context switch
conceptually:
18
1 for_each_online_cpu(cpu) 2 run_on(cpu);
Maintaining Multiple Versions - Deletion
During deletion
Initial State
19
1 p = search(head, key); 2 if (p != NULL) { 3 list_del_rcu(&p->list); 4 synchronize_rcu(); 5 kfree(p); 6 }
Maintaining Multiple Versions - Deletion
list_del_rcu(&p->list)
p has been deleted, but other readers can still be accessing it
20
Maintaining Multiple Versions - Deletion
synchronize_rcu()
kfree(p)
21
Maintaining Multiple Versions - Replacement
During replacement
Initial State
22
1 q = kmalloc(sizeof(*p), GFP_KERNEL); 2 *q = *p; 3 q->b = 2; 4 q->c = 3; 5 list_replace_rcu(&p->list, &q->list); 6 synchronize_rcu(); 7 kfree(p);
Maintaining Multiple Versions - Replacement
kmalloc()
23
Maintaining Multiple Versions - Replacement
*q = *p
24
Maintaining Multiple Versions - Replacement
q->b = 2; q->c = 3
25
Maintaining Multiple Versions - Replacement
list_replace_rcu
There are now two versions of the list, but both are well formed.
26
Maintaining Multiple Versions - Replacement
synchronize_rcu
kfree()
27
Issues
Very much a read/write implementation, not ideally suited for rapidly updated objects
Responsibility on the programmer to make sure that no sleeping/blocking occurs in rcu_read_locks
RCU updaters can still slow readers via cache invalidation
28
