Post on 17-Apr-2018
ITLTracer: Runtime Verification of Properties expressedin ITL
Dr. Helge Janicke (heljanic@dmu.ac.uk)
De Montfort University
Version: 906913f9b356(September 30, 2010)
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 1 / 29
Introduction
Runtime Verification
Definition (Runtime Verification)
Combines formal verification and Program Execution.
Detects faults by passively observing system behaviour during normaloperation.
Validates that the system under test (SuT) satisfies formallyexpressed requirements (e.g. in Interval Temporal Logic (ITL)).
Pros
Testing in the real world!
No (little) overhead duringdevelopment.
Reactive Systems.
Cons
Failure Recovery.
Performance overhead.
Not a substitute for rigoroustesting.
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 2 / 29
Introduction
Runtime Verification in Context
When is RuntimeVerification useful?
Information onlyavailable at run-time.
Dependent onenvironment (reactivesystem).
(security) criticalsystem.
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 3 / 29
Introduction
Runtime Verification in Context
When is RuntimeVerification useful?
Information onlyavailable at run-time.
Dependent onenvironment (reactivesystem).
(security) criticalsystem.
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 3 / 29
Introduction
Problem Statement
Currently we use AnaTempura for Runtime Verification.
AnaTempura uses information obtained from the system via assertionpoints as input to Tempura which computes a trace. Properties arethen evaluated as state invariants1.
Tempura does not allow properties to contain negation of formulae,non-deterministic choice or implication, as these are not executable.(David’s Functional Programming implementation may be analternative!)
Properties, such as SANTA Policies, use some of these, which madetheir encoding cumbersome and lead to restrictions in the language.
1This including references to next or previous statesheljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 4 / 29
Introduction
Contribution and Applications
Semantics based on partitions instead of states.This allows for a more efficient evaluation of formulae containingsequential compositions (Chop).
Efficient encoding of partitions.Traces are stored in a compacted form saving space and access time.
Prototype of a Runtime Monitor (pure Java).See www.tech.dmu.ac.uk/~heljanic
Applications include:
Runtime Verification, where temporal assertions are needed.
SANTA Policy Enforcement.
Formal Log-File/ Attack analysis.
...
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 5 / 29
Framework Overview
Motivation
Observation
Most programs are sequential, or contain large parts that are executedsequentially.
In a sequential program only few variables can change at a time(typically only one).
Depending on the interpretation of states this means that only veryfew variables are changed from one state to the next.
Storing all variable/value mappings for each state can be expensiveand it is more sensible to store only the changes.
This is what partitions do ...
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 6 / 29
Semantics
Definition (Partitioning)
Let a partitioning partx(σ) for a variable x ∈ Var be a nonempty, finitesequence of tuples (l0, r0), (l1, r1), ..., (ln, rn) where:
l0 = 0 start at 0
rn = |σ| end with length of sigma
∀0≤i≤n · li ≤ ri order
∀0≤i<n · ri + 1 = li+1 progression
∀0≤i≤n · ∀li≤t≤ri · σl(x) = σt(x) constant mapping
Example
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 7 / 29
Semantics
Definition (Partitioning)
Let a partitioning partx(σ) for a variable x ∈ Var be a nonempty, finitesequence of tuples (l0, r0), (l1, r1), ..., (ln, rn) where:
l0 = 0 start at 0
rn = |σ| end with length of sigma
∀0≤i≤n · li ≤ ri order
∀0≤i<n · ri + 1 = li+1 progression
∀0≤i≤n · ∀li≤t≤ri · σl(x) = σt(x) constant mapping
Example
grouping of states
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 7 / 29
Semantics
Definition (Partitioning)
Let a partitioning partx(σ) for a variable x ∈ Var be a nonempty, finitesequence of tuples (l0, r0), (l1, r1), ..., (ln, rn) where:
l0 = 0 start at 0
rn = |σ| end with length of sigma
∀0≤i≤n · li ≤ ri order
∀0≤i<n · ri + 1 = li+1 progression
∀0≤i≤n · ∀li≤t≤ri · σl(x) = σt(x) constant mapping
Example
partitioning
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 7 / 29
Semantics
Definition (Partitioning)
Let a partitioning partx(σ) for a variable x ∈ Var be a nonempty, finitesequence of tuples (l0, r0), (l1, r1), ..., (ln, rn) where:
l0 = 0 start at 0
rn = |σ| end with length of sigma
∀0≤i≤n · li ≤ ri order
∀0≤i<n · ri + 1 = li+1 progression
∀0≤i≤n · ∀li≤t≤ri · σl(x) = σt(x) constant mapping
Example
alternative partitioning
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 7 / 29
Semantics
Definition (Partition)
Let δx map from any given partition in the partition interval of partx(σ)and variable x ∈ Var to the set of values (Val).
δx : partx(σ) 7→ Val
such that: ∀l≤t≤r · δx(l , r) = σt(x)
Definition (Unrolling)
We define unrol(δx) as a function:
unrol(δx) = λi ·
if (l0 ≤ i ≤ r0 → δx(l0, r0)
[] (l1 ≤ i ≤ r1 → δx(l1, r1)
...
[] (lk ≤ i ≤ rk → δx(lk , rk)
fi
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 8 / 29
Semantics
Definition (Equivalence)
Two partitions are equivalent δx ≈ δ′x iff unrol(δx) = unrol(δ′x)
δx(0, 1) = 0
δx(2, 4) = 1
δx(5, 5) = 0
δ′x(0, 1) = 0
δ′x(2, 3) = 1
δ′x(4, 4) = 1
δ′x(5, 5) = 0
The partitions contain sufficient information to reconstruct the underlyingσ via unrolling. Equivalence with respect to a variable x is established bychecking if both underlying intervals σ and σ′ are identical w.r.t. thevariable x .
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 9 / 29
Semantics
Definition (Aligned)
We say any two partitions partx(σ) and party (σ) are aligned iff:
|partx(σ)| = |party (σ)|∀0≤i≤|partx (σ)| · (li , ri ) ∈ partx ≡ (li , ri ) ∈ party (σ)
Example
Not aligned!
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 10 / 29
Semantics
Definition (Aligned)
We say any two partitions partx(σ) and party (σ) are aligned iff:
|partx(σ)| = |party (σ)|∀0≤i≤|partx (σ)| · (li , ri ) ∈ partx ≡ (li , ri ) ∈ party (σ)
Example
Aligned.
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 10 / 29
Semantics
Definition
Let the partition interval δ be defined as a tuple of all partitions.
δ = 〈δx0, δx1, ..., δxn〉
where xi ∈ Var . With the restriction that all partitions are of equal length:
∀xi∈Var · len(δx0) = len(δxi )
where len(δx) = r xn is the right index of the last tuple in the partitioning.We also use len(δ) to denote the length of the partition interval.
We refer to ∆x(σ) as the set of all possible partitionings for the variable x .Correspondingly to ∆ for the set of all possible partition intervals.
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 11 / 29
Semantics
The syntax below:
e ::= g(e1, ..., en) | fin v | ©v | v | µf ::= p(e1, ..., en) | f1 ; f2 | f1
∗ | skip | f1 ∧ f2 | ¬ f1
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 12 / 29
Semantics
Semantics of Expressions
The semantics of expressions is defined by induction over the syntax:
E [[v ]](δ) = δv (0, r v0 )
E [[g(e1, ..., en)]](δ) = g(E [[e1]](δ), ..., E [[en]](δ))
E [[ fin v ]](δ) = δv (lvn , rvn )
E [[©v ]](δ) =
χ(Val) if r vn = 0
δv (0, r v0 ) if r v0 > 0
δv (1, r v1 ) otherwise
E [[µ]](δ) = µ
Where lvi and r vi are the left element and the right element of the i ’s tuplein the partition partv (σ).
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 13 / 29
Semantics
Semantics of Formulae
The semantics of formulae is defined by induction over the syntax:
δ � f1 ; f2 iff ∃δ′ ∈ ∆ · δ′ ≈ δ and δ′ is aligned. And
δ′[0, k] � f1 and δ′[k , |δ′|] � f2 for some k , 0 ≤ k ≤ |δ′|δ � f1 ∧ f2 iff δ � f1 and δ � f2
δ � ¬ f iff σ 2 f
δ � skip iff len(δ) equal to 1
δ � empty iff len(δ) equal to 0
δ � p(e1, ..., en) iff p(E [[e1]](δ), ..., E [[en]](δ))
δ � f ∗ iff δ � empty or δ � f ∗ ; f
The complexity of the chop is now reduced from being proportional to theinterval length len(δ) to the number of partitions |δ|.
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 14 / 29
Tool-Support
Tool-Support
Pure Java implementation
javacc for parsing and rewriting.
Java interface as well as stdin option.
Evaluation of properties on demand.
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 15 / 29
Tool-Support
Keeping a TraceVariables
States
...
...
0 1 2 3 4 5
x
y
z
0 0 1 1 1 0
1 1 0 0 0 0
0 0 0 0 0 1
Figure: Trace
Variables
...
x
y
z
0@0 1@2 0@5
1@0 0@2
0@0 1@5
Figure: Partitioned Trace
Instead of recording the values of variables in each state only the changesare recorded. Implicitly a variable keeps its value if it is not changed. Theimplementation keeps record of rn the interval length an thus only needsto store the li values of the tuples together with the value as the all otherri ’s can be computed.
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 16 / 29
Tool-Support
UML Classdiagram: Trace Recording
DataStore<T>- state:long = 0L- variables:HashMap<String,VariableStore<T»+ add(id:String, value:T)+ advanceState(delta:int)+ get(state:long, id:String) : T+ getInv(state:long, id:String, inv:long[]):long[]+ getState():long
VariableStore<T>- data:List<DataEntry<T»- lastpivot:long+ (final) varid:String+ VariableStore(varid: String)+ add(state:long, value:T)+ get(state:Long):T+ getDataEntries():List<DataEntry<T»+ getEntry(state:Long):DataEntry<T>+ getEntryIndex(state:long):int+ getInv(state:long,inv:long[]):long[]
DataEntry+ (final) state : long+ (final) value : Object+ DataEntry(state: long, value: Object)
uk.ac.dmu.heljanic.satchk
0..n
containsID -> Variablemappings
containsstate -> value
mappings
Checker
Figure: Class Diagram
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 17 / 29
Tool-Support
Unrolling and Partitions
The unrol function is implemented by the methodget(state:long):T which returns the value for a given state byidentifying the corresponding partition using an adapted binary search.
The function uses a cursor, so that repeated calls for close statesresult in constant algorithmic complexity.
The function getInv(state:long, inv:long[]):long[]
implements the partitioning. For any state it returns thecorresponding partition element.
Data is recorded using the add(state:long, value:T) functionthat records the new variable value for the state.
The class DataStore collates all Variable stores and handles theidentifier mappings. It also is responsible for keeping the overallinterval length.
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 18 / 29
Tool-Support
Alignment
When evaluating a “chop”, the partitions must be aligned.The alignments are computed “on the fly”, ie. no restructuring of therecorded trace takes place. This is achieved through the efficientimplementation of the unrol function.Only those partitions of variables that are evaluated as part of the“chop” are aligned. The partitioning of other variables cannot affectthe outcome.
Example
true ; (x = 1 ∧ y = 3)heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 19 / 29
Tool-Support
Tool Interface
ITLTracer can be used interactively, via the stdin/stdout interface. Thisallows ITLTracer to trace almost any program and is also useful for testingand verification of recorded traces (e.g. log analysis)
Starting the tool
$ java uk.ac.dmu.heljanic.itltracer.ITLTracer
This is ITLTracer $Revision: 906913f9b356 $ (c) 2010 H. Janicke
itlt:ESCAPE <escapecode> ... set the escape code for assertions.
default = itlt:
escape [x=v] [@s] ... records value v for variable x in state s.
Omitting x=v advance only the state to s.
Omitting @s records for the last set state.
itlt:EXIT ... quit.
itlt:PRINTSIGMA ... prints the recorded trace (sigma).
itlt:PRINTDELTA ... prints the recorded trace (delta).
itlt:CHECK <property> ... checks the <property> on the trace.
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 20 / 29
Examples
Recording a Trace
To record a trace, we set the escape sequence thatis recognised by ITLTracer (default “itlt:”). Anystring received on stdin that is prefixed by theescape sequence will be interpreted as an assertionpoint.
The example on the right is a sequence thatconstructs the interval used in the previousexamples.
Illegal inputs
Attempting to record a variable with a differentvalue in the same state or attempts to reduce thecurrent state will result in an exception.
Exampleitlt:ESCAPE
x=0
y=1
@2
x=1
y=2
y=3 @4
x=0 @5
y=7
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 21 / 29
Examples
Displaying Trace
ITLTrace can also output the trace tostdout in two formats:
itlt:PRINTDELTA This command
will output the partition interval δ.
itlt:PRINTSIGMA This commandwill reconstruct the underlying intervalσ using the unrol function.
Example (itlt:PRINTSIGMA)
SIGMA=
0 y= 1 x= 0
1 y= 1 x= 0
2 y= 2 x= 1
3 y= 2 x= 1
4 y= 3 x= 1
5 y= 7 x= 0
Example(itlt:PRINTDELTA)
DELTA=
DELTA(y)
(0, 1) -> 1
(2, 3) -> 2
(4, 4) -> 3
(5, 5) -> 7
DELTA(x)
(0, 1) -> 0
(2, 4) -> 1
(5, 5) -> 0
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 22 / 29
Examples
Checking a Property
ITLTracer can check a property using the check command:
itlt:CHECK some suffix {x=1 and y=2}ITLTracer only processes the primitive syntax and derived operators suchas “some suffix” (viz. ♦) are expanded during the parsing:
Example (Rewriting ♦f with true ; f (prefix notation))
rewriting DIAMOND node
IN = Diamond(And(Eq(Id[x], ILit[1]), Eq(Id[y], ILit[2])))
OUT = Chop(BLit[true], And(Eq(Id[x], ILit[1]), Eq(Id[y], ILit[2])))
parsed successfully
Once reduced to primitive operators the property can be evaluated againstthe trace.
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 23 / 29
Examples
Checking a Property (step by step)DELTA=
DELTA(y)
(0, 1) -> 1
(2, 3) -> 2
(4, 4) -> 3
(5, 5) -> 7
DELTA(x)
(0, 1) -> 0
(2, 4) -> 1
(5, 5) -> 0
Processing: l=0 r=5 f=Chop(BLit[true], And(Eq(Id[x], ILit[1]), Eq(Id[y], ILit[2])))
chop: k=5Processing: l=5 r=5 f=And(Eq(Id[x], ILit[1]), Eq(Id[y], ILit[2]))
Processing: l=5 r=5 f=Eq(Id[x], ILit[1])
Processing: l=5 r=5 f=Id[x]
return= 0 : [5, 5, 5, 5]
Processing: l=5 r=5 f=ILit[1]
return= 1 : [0, 5, 5, 5]
return= false : [5, 5, 5, 5]
return= false : [5, 5, 5, 5]
chop: k=4Processing: l=4 r=5 f=And(Eq(Id[x], ILit[1]), Eq(Id[y], ILit[2]))
Processing: l=4 r=5 f=Eq(Id[x], ILit[1])
Processing: l=4 r=5 f=Id[x]
return= 1 : [2, 4, 4, 5]
Processing: l=4 r=5 f=ILit[1]
return= 1 : [0, 5, 4, 5]
return= true : [2, 4, 4, 5]
Processing: l=4 r=5 f=Eq(Id[y], ILit[2])...
Evaluation is backwards and uses left and right indicies instead of partitionindicies, to avoid repartitioning. In this case a k in the partition (2,3)evaluates to true.
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 24 / 29
Conclusion and Future Work
Conclusions
ITLTracer checks ITL properties against a (recorded or streamed)system trace.
Overcomes (executable) restriction of AnaTempura, viz. a wider rangeof properties can be used.
Partitions can reduce algorithmic complexity, as evaluation of chop isO(|δ|) rather than O(|σ|).
Checks can be initiated by the target program. The checking ofproperties can be reduced to specific assertion points within thetarget program.
Prototype implementation is available atwww.tech.dmu.ac.uk/~heljanic
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 25 / 29
Conclusion and Future Work
Applications
Runtime Verification, where temporal properties are important.
Policy Based Management: History Based Access Control Policies canbe enforced. (Without refinement to executable form!)
Compliance Checking:
Lean Engineering, Standard Operations.Management Applications: Business Process Monitoring.Service Oriented Computing: Contract Monitoring.
Incident Response and Forensic Analysis: Checking for knownmalicious behaviours (properties) against log files (system trace)
Intrusion Detection Systems: Checking for known behaviour patterns(properties) in network traffic or access patterns (trace).
Trust in Collaborative Systems: Establishing trust based onbenevolent and malicious behaviour patterns.
Archaeology (Properties of Data Analysis Processes).
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 26 / 29
Conclusion and Future Work
Future Work (I)
Reuse results from previous checks:
σ = σ0...σn σ′ = σ0...σnσn+1
Reuse known prefix results.
σ � f ; true =⇒ σ′ � f ; true
σ 2 f ; true and σ′ � f =⇒ σ′ � f ; true
σ 2 f ; true and σ′ 2 f =⇒ σ′ 2 f ; true
Reuse suffix results (state formulae only)
σ � true ; w =⇒ σ′ � true ; w
σ 2 true ; w and σn+1 � w =⇒ σ′ � true ; w
σ 2 true ; w and σn+1 2 w =⇒ σ′ 2 true ; w
By annotating the formulae with the last evaluation result andreference to δ (including the computed partitioning).
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 27 / 29
Conclusion and Future Work
Future Work (II)
Extend supported Data Types (All immutable Java Objects can besupported).
Allow query calls to stored objects in the trace (query means thatcalling the function is free of side-effects)
Locking mechanism: Properties require only certain variables andbased on previous result evaluations part of the trace becomesirrelevant (ie. σ � f ; true =⇒ σ′ � f ; true) therefore any futurequery will return true, thus the record of σ can be released (if noother property requires this part of the trace).
Temporal Formulae compute partitions based on invariants (currentlyexpressed on σ). This will have to be converted to the new partitionsemantics (δ).
testing, publications...
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 28 / 29
Conclusion and Future Work
Questions?
Any Comments or Questions?
Dr. Helge Janicke Software Technology Research Laboratory
Faculty of Technology
E: heljanic@dmu.ac.uk De Montfort University
P: ++44 (0) 116 257 7617 Bede Island 1.16
W: www.tech.dmu.ac.uk/~heljanic LE1 9BH Leicester, UK
heljanic@dmu.ac.uk (DMU) ITLTracer 906913f9b356 29 / 29