From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic...
Transcript of From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic...
![Page 1: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/1.jpg)
From Symbolic Execution to
Concolic Testing
Daniel Paqué
![Page 2: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/2.jpg)
Structure
Symbolic Execution
Concolic Testing
Execution Generated Testing
Concurrency in Concolic Testing
From Symbolic Execution to Concolic Testing 2 28.11.2014
![Page 3: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/3.jpg)
Motivation
28.11.2014 From Symbolic Execution to Concolic Testing 3
Software Testing “usually accounts for 50% of software
development cost” [Source: “The economic impacts of inadequate infrastructure for software testing”, NIST]
complex and large Software Systems complicate finding
small test suites with high coverage
Symbolic Execution
automic test case generation
high code coverage
![Page 4: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/4.jpg)
Symbolic Execution - Idea
28.11.2014 From Symbolic Execution to Concolic Testing 4
execute the program in symbolic domain
explore all possible execution paths
for each path the constraints of the branching points are
collected
generate test input based on the constraints
![Page 5: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/5.jpg)
Symbolic Execution - Example
28.11.2014 From Symbolic Execution to Concolic Testing 5
![Page 6: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/6.jpg)
Symbolic Execution - Example
28.11.2014 From Symbolic Execution to Concolic Testing 6
symbolic domain:
symbolic state path condition
![Page 7: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/7.jpg)
Symbolic Execution - Example
28.11.2014 From Symbolic Execution to Concolic Testing 7
symbolic domain:
![Page 8: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/8.jpg)
Symbolic Execution - Example
28.11.2014 From Symbolic Execution to Concolic Testing 8
symbolic domain:
![Page 9: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/9.jpg)
Symbolic Execution - Example
28.11.2014 From Symbolic Execution to Concolic Testing 9
symbolic domain:
1
2
![Page 10: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/10.jpg)
Symbolic Execution - Example
28.11.2014 From Symbolic Execution to Concolic Testing 10
1
2
3
symbolic domain:
![Page 11: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/11.jpg)
Symbolic Execution - Example
28.11.2014 From Symbolic Execution to Concolic Testing 11
![Page 12: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/12.jpg)
Limits of Symbolic Execution
28.11.2014 From Symbolic Execution to Concolic Testing 12
1
2
symbolic domain:
![Page 13: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/13.jpg)
Solution
28.11.2014 From Symbolic Execution to Concolic Testing 13
Mix Symbolic Execution with Concrete Execution
Concolic Testing
Execution Generated Testing
![Page 14: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/14.jpg)
28.11.2014 From Symbolic Execution to Concolic Testing 14
2005,
Godefroid, Sen
Symbolic Execution
Concolic Testing Execution Generated
Testing (EGT)
mix concrete with
symbolic execution
2006
Cadar et. al
1979
J.C. King
+ improvements in
constraint solving
![Page 15: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/15.jpg)
Concolic Testing
28.11.2014 From Symbolic Execution to Concolic Testing 15
execute program with concrete values and collect
symbolic constraints during execution
explore paths sequentially instead of forking
infer input for next execution
use concrete values to solve problematic constraints
![Page 16: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/16.jpg)
Concolic Testing - Example
28.11.2014 From Symbolic Execution to Concolic Testing 16
symbolic state:
![Page 17: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/17.jpg)
Concolic Testing - Example
28.11.2014 From Symbolic Execution to Concolic Testing 17
symbolic state:
false
![Page 18: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/18.jpg)
symbolic domain:
Concolic Testing - Example
28.11.2014 From Symbolic Execution to Concolic Testing 18
new input:
![Page 19: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/19.jpg)
Concolic Testing - Example
28.11.2014 From Symbolic Execution to Concolic Testing 19
![Page 20: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/20.jpg)
Concolic Testing - Example
28.11.2014 From Symbolic Execution to Concolic Testing 20
symbolic domain:
evaluate
condition in
concrete
true
![Page 21: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/21.jpg)
Concolic Testing - Example
28.11.2014 From Symbolic Execution to Concolic Testing 21
symbolic state:
evaluate bar()
in concrete
![Page 22: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/22.jpg)
28.11.2014 From Symbolic Execution to Concolic Testing 22
sequential path exploration
guided by concrete input
Symbolic Execution
Concolic Testing Execution Generated
Testing (EGT)
mix concrete with
symbolic execution
fork execution for each path
guided by symbolic execution
1979
J.C. King
![Page 23: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/23.jpg)
Execution Generated Testing
28.11.2014 From Symbolic Execution to Concolic Testing 23
further differences to Concolic Testing:
EGT dynamically checks if all operands are concrete
if so the operation can be executed in concrete
elsewise the operation is executed symbolical
concrete
symbolic
![Page 24: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/24.jpg)
How to deal with
concurrent programs?
28.11.2014 From Symbolic Execution to Concolic Testing 24
![Page 25: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/25.jpg)
Main Challenge
28.11.2014 From Symbolic Execution to Concolic Testing 25
Thread 0 Thread 1
…
Problem
![Page 26: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/26.jpg)
Main Challenge
28.11.2014 From Symbolic Execution to Concolic Testing 26
Thread 0 Thread 1
…
Idea
summarize redundant
interleavings
![Page 27: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/27.jpg)
28.11.2014 From Symbolic Execution to Concolic Testing 27
Koushik Sen & Gul Agha:
(2006)
„race-detection and flipping algorithm“
minimize redundant executions in concurrent
programs
uses vector clocks to identify races
jCUTE
![Page 28: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/28.jpg)
Redundant Executions
28.11.2014 From Symbolic Execution to Concolic Testing 28
Thread t0: Thread t1:
Execution 1:
{4, 0, 16}
Execution 2:
{4, 0, 16}
Execution 3:
{3, 0, 15}
Execution 4:
{3, 0, 16} result:
![Page 29: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/29.jpg)
Redundant Executions – Race Detection
28.11.2014 From Symbolic Execution to Concolic Testing 29
two events are in a race if… they stem from different threads
both access the same memory location (without locking)
the order both events can be permuted by changing the schedule
Execution 1:
{4, 0, 16}
Execution 2:
{4, 0, 16}
Execution 3:
{3, 0, 15}
Execution 4:
{3, 0, 16}
result:
races:
![Page 30: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/30.jpg)
The Race-Detection and Flipping Algorithm
28.11.2014 From Symbolic Execution to Concolic Testing 30
generate a random input and a schedule
execute code with the generated input and schedule
compute the race conditions and symbolic constraints
generate a new schedule or a new input
continue until all possible distinct execution paths have
been explored (depth-first search strategy)
init:
loop:
![Page 31: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/31.jpg)
Generating new inputs/schedules
28.11.2014 From Symbolic Execution to Concolic Testing 31
new input: concolic testing
new schedule:
pick two events which are in a race
delay the first event as much as possbile
schedule1: schedule 2:
![Page 32: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/32.jpg)
How to identify
races?
28.11.2014 From Symbolic Execution to Concolic Testing 32
![Page 33: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/33.jpg)
How to identify races?
28.11.2014 From Symbolic Execution to Concolic Testing 33
vector clocks
- each thread t gets it‘s own vector clock Vt
- each memory location gets another two
![Page 34: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/34.jpg)
Vector Clocks - Example
28.11.2014 From Symbolic Execution to Concolic Testing 34
![Page 35: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/35.jpg)
Vector Clocks - Algorithm
28.11.2014 From Symbolic Execution to Concolic Testing 35
![Page 36: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/36.jpg)
Vector Clocks – Example
28.11.2014 From Symbolic Execution to Concolic Testing 36
![Page 37: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/37.jpg)
Vector Clocks – Example
28.11.2014 From Symbolic Execution to Concolic Testing 37
![Page 38: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/38.jpg)
Vector Clocks – Example
28.11.2014 From Symbolic Execution to Concolic Testing 38
![Page 39: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/39.jpg)
Vector Clocks – Example
28.11.2014 From Symbolic Execution to Concolic Testing 39
![Page 40: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/40.jpg)
Vector Clocks – Example
28.11.2014 From Symbolic Execution to Concolic Testing 40
![Page 41: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/41.jpg)
Vector Clock Theorem
28.11.2014 From Symbolic Execution to Concolic Testing 41
![Page 42: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/42.jpg)
Questions?
28.11.2014 From Symbolic Execution to Concolic Testing 42
![Page 43: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/43.jpg)
Precise Definitions
(just in case)
28.11.2014 From Symbolic Execution to Concolic Testing 43
![Page 44: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/44.jpg)
Race Relation – Simple Defintion:
28.11.2014 From Symbolic Execution to Concolic Testing 44
![Page 45: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/45.jpg)
sequentially related
28.11.2014 From Symbolic Execution to Concolic Testing 45
![Page 46: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/46.jpg)
access precedence related
28.11.2014 From Symbolic Execution to Concolic Testing 46
![Page 47: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/47.jpg)
causally related
28.11.2014 From Symbolic Execution to Concolic Testing 47
![Page 48: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/48.jpg)
race related
28.11.2014 From Symbolic Execution to Concolic Testing 48
![Page 49: From Symbolic Execution to Concolic Testing - TU · PDF fileexecute the program in symbolic domain ... The Race-Detection and Flipping Algorithm 30 From Symbolic Execution to Concolic](https://reader034.fdocuments.net/reader034/viewer/2022051523/5a78fd2a7f8b9a217b8bba96/html5/thumbnails/49.jpg)
Race-Detection and Flipping Algorithm
Detailled Example
28.11.2014 From Symbolic Execution to Concolic Testing 49