Department of Physics, Stanford University, Stanford, CA ...
Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson...
-
date post
21-Dec-2015 -
Category
Documents
-
view
226 -
download
0
Transcript of Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson...
![Page 1: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/1.jpg)
Testing Atomicity of Composed Concurrent Operations
Ohad Shacham Tel Aviv University
Nathan Bronson Stanford University
Alex Aiken Stanford University
Mooly Sagiv Tel Aviv University
Martin Vechev ETH & IBM Research
Eran Yahav Technion
![Page 2: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/2.jpg)
Concurrent Data Structures
• Writing highly concurrent data structures is complicated
• Modern programming languages provide efficient concurrent collections with atomic semantics
.
.
…
……
…
…
………
![Page 3: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/3.jpg)
Challenge
Testing the atomicity of composed operations
…
……
…
…
………
![Page 4: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/4.jpg)
TOMCAT Motivating Example
attr = new HashMap();…
Attribute removeAttribute(String name){ Attribute val = null; synchronized(attr) { found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } } return val;}
TOMCAT 5*.TOMCAT 6*.
Invariant: removeAttribute(name) returns the value it removes from attr or null
attr = new ConcurrentHashMap();…
Attribute removeAttribute(String name){ Attribute val = null; /* synchronized(attr) { */ found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } /* } */ return val;}
![Page 5: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/5.jpg)
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
attr.put(“A”, o);
attr.remove(“A”);
o
![Page 6: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/6.jpg)
Linearizability removeAttribute(“A”) {
Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
attr.put(“A”, o);
attr.remove(“A”);
attr.put(“A”, o);
attr.remove(“A”);
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) { return val;
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) { return val; attr.put(“A”, o);
attr.remove(“A”);
attr.put(“A”, o);
attr.remove(“A”);
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”) ; if (found) {
val = attr.get(“A”); attr.remove(“A”);
} return val;
o
o
null
null
o
null
null
o
null
null
o
null
![Page 7: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/7.jpg)
Challenge
Testing the linearizabiliy of composed operations
…
……
…
…
………
![Page 8: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/8.jpg)
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
attr.put(“A”, o);
attr.remove(“A”);
• Reconstruction in TOMCAT extremely challenging– Large traces– Large number of traces– Bugs occur in rare cases with specific key values
![Page 9: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/9.jpg)
Our Solution
ModularityGenerates simple traces
Base linearizability Restrict generated traces
Influence Restrict generated traces
Enables Env control
![Page 10: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/10.jpg)
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
attr.put(“A”, o);
attr.remove(“A”);
Modular Checking
![Page 11: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/11.jpg)
Challenge
Testing Linearizability of composed operation in a modular fashion
…
……
…
…
………
![Page 12: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/12.jpg)
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”) ;if (found) {return val;
removeAttribute(“B”) {Attribute val = null;
found = attr.containsKey(“B”) ;if (found) {return val;
removeAttribute(“GGG”) {Attribute val = null;
found = attr.containsKey(“GGG”) if (found) {return val;
attr.put(“A”, o);
removeAttribute(“GGG”) {Attribute val = null;
found = attr.containsKey(“GGG”) if (found) {
val = attr.get(“GGG”) ; attr.remove(“GGG”) ;
}return val;
attr.put(“GGG”, o);
removeAttribute(“K”) {Attribute val = null;
found = attr.containsKey(“K”) if (found) {
val = attr.get(“K”) ; attr.remove(“K”) ;
}return val;
attr.put(“K”, o’);
removeAttribute(“K”) {Attribute val = null;
found = attr.containsKey(“K”)
if (found) {return val;
attr.put(“K”, o’);
removeAttribute(“P”) {Attribute val = null;
found = attr.containsKey(“P”) if (found) {
val = attr.get(“P”) ; attr.remove(“P”) ;
}return val;
attr.put(“P”, o’);
removeAttribute(“P”) {Attribute val = null;
found = attr.containsKey(“P”) if (found) {
val = attr.get(“P”) ; attr.remove(“P”) ;
}return val;
attr.put(“P”, o’);
removeAttribute(“O”) {Attribute val = null;
found = attr.containsKey(“O”) if (found) {
val = attr.get(“O”) ; attr.remove(“O”) ;
}return val;
attr.put(“O”, o’);
removeAttribute(“LL”) {Attribute val = null;
found = attr.containsKey(“LL”) if (found) {
val = attr.get(“LL”) ; attr.remove(“LL”) ;
}return val;
attr.put(“LL”, o’);
removeAttribute(“L”) {Attribute val = null;
found = attr.containsKey(“L”) if (found) {
val = attr.get(“L”) ; attr.remove(“L”) ;
}return val;
attr.put(“L”, o’);
removeAttribute(“G”) {Attribute val = null;
found = attr.containsKey(“G”) if (found) {
val = attr.get(“G”) ; attr.remove(“G”) ;
}return val;
attr.put(“G”, o’);
removeAttribute(“R”) {Attribute val = null;
found = attr.containsKey(“R”) if (found) {
val = attr.get(“R”);
attr.remove(“R”) ;}
return val;
attr.put(“R”, o’);
attr.remove(“R”);
![Page 13: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/13.jpg)
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
attr.put(“A”, o);
attr.remove(“A”);
Influence Base Environment
![Page 14: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/14.jpg)
Running Example
Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}
![Page 15: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/15.jpg)
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
attr.put(“A”, o);
attr.remove(“A”);
Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}
![Page 16: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/16.jpg)
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) {
val = attr.get(“A”);
attr.remove(“A”); } return val;
attr.put(“A”, o);
attr.remove(“A”);
attr.put(“A”, o);
attr.remove(“A”);
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) { return val;
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”); if (found) { return val; attr.put(“A”, o);
attr.remove(“A”);
attr.put(“A”, o);
attr.remove(“A”);
removeAttribute(“A”) {Attribute val = null;
found = attr.containsKey(“A”) ; if (found) {
val = attr.get(“A”); attr.remove(“A”);
} return val;
o
o
null
null
o
null
null
o
null
null
o
null
![Page 17: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/17.jpg)
COLT
program
CO extractor
candidateCOs
Timeout
instrument linearizability
checking
CO
key/value driver
Non-Lin
Execution
library spec
influence driver
![Page 18: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/18.jpg)
Benchmark
• Used simple static analysis to extract composed operations– 19% needed manual modification
• Extracted 112 composed operations from 55 applications– Apache Tomcat, Cassandra, MyFaces – Trinidad, etc…
• Extracted all composed operations per application
• We did not find additional composed operations– Using Google Code and Koders
![Page 19: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/19.jpg)
112Unknown
![Page 20: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/20.jpg)
59Non
Linearizable
53Unknown
![Page 21: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/21.jpg)
53Unknown
42Non
Linearizable
17Open Non
Linearizable
![Page 22: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/22.jpg)
17Open Non
Linearizable
42Non
Linearizable
27Linearizable
26Globals
![Page 23: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/23.jpg)
Globals
…
…
……
…
……
…
…
……
…
…
…
…
……
…
m.put(k,v)
…
![Page 24: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/24.jpg)
27Linearizable
85Non-linearizable
![Page 25: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/25.jpg)
Easy Detection
Attribute removeAttribute(String name){ Attribute val = null; found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } return val;}
![Page 26: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/26.jpg)
Current Work
• Define a class of data-independent composed operations
• Use small model reduction to prove linearizability of data-independent composed operations
• Empirical study for the ratio of real life data-independent composed operations
![Page 27: Testing Atomicity of Composed Concurrent Operations Ohad Shacham Tel Aviv University Nathan Bronson Stanford University Alex Aiken Stanford University.](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649d6a5503460f94a487d6/html5/thumbnails/27.jpg)
Summary
• Writing concurrent data structures is hard• Employing atomic library operations is error prone• Modular linearizability checking• Leverage influence
• Sweet spot– Identify many important bugs together with a traces showing and
explaining the violations– Hard to find– Simple and efficient technique