Tutorial / lab 2: Code instrumentation
-
Upload
octavius-booker -
Category
Documents
-
view
31 -
download
1
description
Transcript of Tutorial / lab 2: Code instrumentation
© Dr. A. Williams, Fall 2008 - Present
Software Quality Assurance – Clover Lab 1
Tutorial / lab 2: Code instrumentation
• Goals of this session:
1. Create some tests for a Java class.
2. Measure the code coverage.
© Dr. A. Williams, Fall 2008 - Present
Software Quality Assurance – Clover Lab 2
Code Instrumentation Tools
• Tools that modify production code for various purposes during code execution:
– Debuggers– Add information to allow for break points
or step-by-step execution.
– Coverage measurement– Report on the number of times various
code elements have been executed.
– Profilers– Measure amount of time spent in various
parts of the code.
© Dr. A. Williams, Fall 2008 - Present
Software Quality Assurance – Clover Lab 3
Coverage tools
• Program is typically compiled with special options, to add extra source or object code.
– Additional data structures, such as a flow graph, may also be created.
• Program is run, possibly via test cases
– During execution, information is accumulated and written to an output file.
• Post-processing phase:
– User report is generated from output file.
© Dr. A. Williams, Fall 2008 - Present
Software Quality Assurance – Clover Lab 4
How to measure coverage?
• Instrument the source code before compilation
• Instrument the virtual machine or object code.
– The Apache Byte Code Engineering Library (BCEL) is used by several Java coverage tools
– http://jakarta.apache.org/bcel
• Use a customized virtual machine.
© Dr. A. Williams, Fall 2008 - Present
Software Quality Assurance – Clover Lab 5
Source code instrumentation
• Create a copy of the source code, instrument the copy, and compile the instrumented copy to .class files.
– This is the approach used by the Clover code coverage tool.
• Typical method:
– Set up an array of counters.
– Insert source code at points of interest that increment specific counters.
– Coverage tool calls main() method
– After main method returns, dump values of counters.
© Dr. A. Williams, Fall 2008 - Present
Software Quality Assurance – Clover Lab 6
Source code before instrumentation
public final void add( final MatrixRow<T> newValues ){ if ( getNumColumns( ) == 0 ) { contents = newValues.copy( ).getContents( ); } else { contents.addAll( newValues.copy( ).getContents( ) ); }}
• A counter is inserted for each method, statement, and branch.
• Statement counters are before each statement, in case the statement throws an exception.
• Branch counters need to identify true and false branches taken.
© Dr. A. Williams, Fall 2008 - Present
Software Quality Assurance – Clover Lab 7
Same code after instrumentation
public final void add( final MatrixRow<T> newValues ){ CLOVER.M[348]++; CLOVER.S[1814]++; if ( ( getNumColumns( )==0 && ++CLOVER.CT[338]!=0 | true )
|| ( ++CLOVER.CF[338] == 0 & false ) ) { CLOVER.S[1815]++; contents = newValues.copy( ).getContents( ); } else { CLOVER.S[1816]++; contents.addAll( newValues.copy( ).getContents( ) ); }}
methodcounter
statementcounters
branchcounters
© Dr. A. Williams, Fall 2008 - Present
Software Quality Assurance – Clover Lab 8
Counting branches
• Count number of times that a branch condition evaluates to true and false.
if ( ( aBoolean && ++CT[338]!=0 | true )
|| ( ++CF[338] == 0 & false ) )
• If aBoolean is true, then the && is also evaluated, which will increment the CT counter. We want the expression’s overall evaluation to be true.
• If aBoolean is false, then execution switches to the right side of the || clause. Increment the CF counter, but then ensure that the entire expression still evaluates to false to take the else branch.
• Short-circuit evaluation means that as soon as the expression result can be determined, evaluation stops. This avoids:– Incrementing the true counter if aBoolean is false.– Incrementing the false counter if aBoolean is true.
© Dr. A. Williams, Fall 2008 - Present
Software Quality Assurance – Clover Lab 9
Analysis
• Advantages:– Results are directly mapped to source code.– If a statement throws an exception, it will be counted
as having been executed.
• Disadvantages:– Requires keeping a second version of source code,
and ensuring that original source is not over-written.– Instrumented code must not affect original code
execution.– Management of .class files from compiling alternate
source code. – Memory usage: for statement coverage alone, every
statement requires one integer as a counter.
© Dr. A. Williams, Fall 2008 - Present
Software Quality Assurance – Clover Lab 10
Byte code instrumentation
• Can be done in two ways:
– Instrument object files in advance to produce a alternate version.
– Example: create instrumented .class files for Java.
– Instrument object files as they are used.
– Example: create customized .class file loader that inserts the instrumentation at run time.
• This is the approach used by the Emma code coverage tool
– This is an open-source tool; see http://emma.sourceforge.net
– Eclipse plug-in version: http://www.eclemma.org
© Dr. A. Williams, Fall 2008 - Present
Software Quality Assurance – Clover Lab 11
Analysis
• Pre-instrumentation:
– Advantages:
– source code is not modified.
– can be done as a batch process before execution
– Disadvantages:
– management of two .class file versions.
– mapping of source code statements to/from object code
– counting source code statements can be difficult
– object code may need extra information such as special compiler options