B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by:...
-
Upload
tyrone-newman -
Category
Documents
-
view
219 -
download
1
Transcript of B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by:...
![Page 1: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/1.jpg)
B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda
Presented by:
Priyanka Puri
1000676053
SOFTTEST: A FRAMEWORK FOR SOFTWARE TESTING OF JAVA PROGRAMS
![Page 2: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/2.jpg)
STRUCTURAL SOFTWARE TESTING
• Assure quality, robust software
• Collect coverage information about the program
• E.g., branch coverage records which edges are hit
• Uses program instrumentation
![Page 3: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/3.jpg)
..Continued..
• Automatically apply multiple test strategies
• Multiple languages and platforms
• Handle large programs
![Page 4: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/4.jpg)
Static instrumentation approaches
• Not scalable: Instrumentation left in place• Time, memory
• Inflexible: Only certain tests, languages, platforms
• Generally not detailed
![Page 5: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/5.jpg)
Approach
• Specification driven: User written test
• Test plans: Recipe of how & where to test
• Path specific: Instrument only what is needed
• Dynamic: Insert & remove instrumentation
![Page 6: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/6.jpg)
TEST FRAMEWORK INSTANCE FOR JAVAWizard for creating the test specification
From test specification determine how to testTest plan uses instrumentation primitives
Source level coverage results
![Page 7: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/7.jpg)
• Apply different tests to various code regions.
• may be applied with different coverage criteria
• A GUI for
• What tests to apply, where to apply, under what conditions
• Select code regions
• Conditions can be specified
TEST SPECIFICATION
![Page 8: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/8.jpg)
public class Example
{
public int foo(int x)
{
if(x==200)
x = x+100;
else x = x-100;
return x;
}
}
DEFINITIONS
{
NAME: X, REGION_D,
LOCATION: FILE Example.java {
CLASS example, METHOD foo
}
}
BODY
{
DO BRANCH_TEST
ON REGION X UNTIL:90%
}
Example test specification
![Page 9: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/9.jpg)
TEST PLANNER• Test plan: Where and how to test program
• Test table
• Where to instrument in the target machine code
• Test information (results)
• Branch coverage: when to insert & remove instrumentation, records which edges are hit/not hit
![Page 10: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/10.jpg)
• Test payload
– What to do at an instrumentation point
– Target machine code
– Branch coverage: Update edge coverage
![Page 11: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/11.jpg)
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 N,N
2 4 N
3 4 N
4 1 N
Instrumentation
Hit
![Page 12: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/12.jpg)
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 N,N
2 4 N
3 4 N
4 1 N
Instrumentation
Hit
![Page 13: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/13.jpg)
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 N,N
2 4 N
3 4 N
4 1 N
Instrumentation
Hit
![Page 14: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/14.jpg)
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 N,N
2 4 N
3 4 N
4 1 N
Instrumentation
Hit
![Page 15: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/15.jpg)
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 Y,N
2 4 N
3 4 N
4 1 N
Instrumentation
Hit
![Page 16: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/16.jpg)
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 Y,N
2 4 N
3 4 N
4 1 N
Instrumentation
Hit
![Page 17: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/17.jpg)
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 Y,N
2 4 Y
3 4 N
4 1 N
Instrumentation
Hit
![Page 18: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/18.jpg)
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 Y,N
2 4 Y
3 4 N
4 1 N
Instrumentation
Hit
![Page 19: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/19.jpg)
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 Y,N
2 4 Y
3 4 N
4 1 Y
Instrumentation
Hit
![Page 20: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/20.jpg)
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 Y,Y
2 4 Y
3 4 N
4 1 Y
Instrumentation
Hit
![Page 21: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/21.jpg)
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 Y,Y
2 4 Y
3 4 N
4 1 Y
Instrumentation
Hit
![Page 22: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/22.jpg)
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 Y,Y
2 4 Y
3 4 Y
4 1 Y
Instrumentation
Hit
![Page 23: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/23.jpg)
BRANCH COVERAGE EXAMPLE
Test Table
Test Payload
Mark edge hit
Insert at next point
Remove instrumentation
1
2 3
4
Block Next Hit
1 2,3 Y,Y
2 4 Y
3 4 Y
4 1 Y
Instrumentation
Hit
![Page 24: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/24.jpg)
Example CFG with block as its own successor
![Page 25: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/25.jpg)
TEST VIRTUAL MACHINE• Provides functionality to insert and remove
instrumentation at run-time.
• Implements an interface for inserting and removing instrumentation with fast breakpoints*.
• API provides primitives, such as
• the placement of successor breakpoints, • storing test- specific data• removal of breakpoints, for constructing fast
breakpoints with varying payloads
![Page 26: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/26.jpg)
TEST ANALYZER
• Displays the results of tests
• For branch coverage, the analyzer displays the CFG for a method
• CFG for target machine code
![Page 27: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/27.jpg)
Observations• SoftTest significantly reduces the number of tests resulting in savings in
scripting effort.
• Significant impact on the acceptability of code coverage.
• Provides cost effective approach to testing.
• Low performance and memory overhead.
• For the benchmarks, the memory overhead was 178 to 822 bytes for the test table.
• The performance overhead for coverage varied from 1% to 5.5% of the total execution time.
![Page 28: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/28.jpg)
Preliminary Results: Branch Coverage Testing
Overhead includes:
1. Test Planner
2. Instrumentation
Run-time performance overhead
[3]
![Page 29: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/29.jpg)
REFERENCES
[1] http://veryoldwww.cs.pitt.edu/copa/eclipse.html
[2] SoftTest: A Framework for Software Testing of Java Programs
[3] http://www.cs.pitt.edu/coco/papers/SoftTest-Etx-2003
[4] http://www.softwaretestingmentor.com/types-of-testing/static-testing.php
![Page 30: B. Childers, M. L. Soffa, J. Beaver, L. Ber, K. Cammarata, J. Litman, J. Misurda Presented by: Priyanka Puri 1000676053 SOFTTEST: A FRAMEWORK FOR SOFTWARE.](https://reader035.fdocuments.net/reader035/viewer/2022062515/56649d055503460f949d8f4d/html5/thumbnails/30.jpg)
QUESTIONS?