(1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text...
-
Upload
stephen-holt -
Category
Documents
-
view
222 -
download
1
Transcript of (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text...
![Page 1: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/1.jpg)
(1)
Verification MethodsCS2110: SW Development Methods
•Reading: Study Section 1.2.5 in the MSD text •These slides supplement the textbook
– These touch on other important methods, but focus on execution-based testing
![Page 2: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/2.jpg)
(2)
Some Definitions
• Verification: showing that something meets specifications– We verify “against” something– Important: if you don’t have requirements
specifications documented, then you can’t verify!– If behavior is not defined, then what counts as
incorrect behavior?• Verification can happen on any kind of software work-
product– designs, code, etc.
![Page 3: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/3.jpg)
(3)
Verification Techniques
At least four approaches to verifying code quality:1. Execution-based Testing (running tests)2. Inspections
– People examine the work product by eye3. Static Analysis by a tool
– For C programs, an old tool called “lint”– For Java programs, some nice tools integrate
with Eclipse: CheckStyle and PMD4. Formal Correctness Proofs
– Use mathematics to demonstrate the desired property
![Page 4: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/4.jpg)
(4)
Execution-based Testing
Some Basic Definitions:• Fault: What we think of as a bug
– Something in the code that’s wrong• Failure: Observed incorrect behavior of a system
resulting from a fault• Error: Mistake made by a developer
A possible definition of execution-based testing:– Process of inferring certain behavioral properties
of a product based, in part, on results of executing it in a known environment with selected inputs.
![Page 5: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/5.jpg)
(5)
Famous and Important Quote
• Program testing can effectively show the presence of bugs but is hopeless for showing their absence.– Edsger Dijkstra
– Famous computer scientist– Turing Award winner– Wrote "Go To Statement
Considered Harmful“ (1968)– Shortest-path algorithm
![Page 6: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/6.jpg)
(6)
Types of Testing
• Unit testing– An individual class or function is tested thoroughly– Done as part of coding process– Done by the programmer him/herself– Usually by a test harness (or driver) or using a
test-class• Integration testing
– Multiple parts of a system are combined and tested to see if they work together
• Acceptance testing– The customer is involved. (Alpha or beta testing)
![Page 7: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/7.jpg)
(7)
Approaches to Unit Testing
• Two approaches– Test only based on inputs, outputs (i.e. the
specification) without looking at the code– Test based on the logic of the code (must study it)
• The 1st of these is called black-box testing• The 2nd of these is glass-box or white-box testing• Some principles of glass-box testing
– Make sure every line (or “case”) gets executed– Make sure loops get done zero times, >zero times– Etc.
![Page 8: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/8.jpg)
(8)
Limitations of Glass-Box Testing
• Why not fully exercise our code? • Logic of a piece of code defines a set of possible
execution paths, “flow-paths,” through the function– Think of a flow-chart– The inputs control which path is taken
• A good set of test data might make sure every possible path is followed– This tests every possible behavior
• Problem: for even small programs with loops and conditionals, the total number of paths becomes too large too quickly– See MSD text, p. 24-25
![Page 9: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/9.jpg)
(9)
Black-Box Testing
• This means testing based on inputs to the code-module without basing tests on code’s internals– You can’t see inside a black-box– Note: MSD’s definition on p. 26 isn’t the usual one
• Choose test cases for common problems– Boundary values, boundary conditions– Off-by-one errors– Incorrect inputs
• Important: You have to know in advance what the expected result of any test should be!
![Page 10: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/10.jpg)
(10)
Lab on Monday/Tuesday
• The JUnit (v. 3) framework is a tool to help people generate and format unit tests in a systematic way.
• Stand-alone and alsobuilt into Eclipse
• Example JUnit test– Class contains all
tests– setUp() performs
preconditions– “test…” is an
individual test case– “assert” checks a
program condition
public class CoffeeMakerTest extends TestCase { private CoffeeMaker cm; private Recipe recipe1;
public void setUp() { cm = new CoffeeMaker();
recipe1 = new Recipe(); recipe1.setName("Coffee"); recipe1.setPrice(50); recipe1.setAmtCoffee(6); recipe1.setAmtMilk(1); }
public void testAddRecipe() { assertTrue(cm.addRecipe(recipe1)); }}
![Page 11: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/11.jpg)
(11)
![Page 12: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/12.jpg)
(12)
When we test a class’ method to see if its loop goes to size-1 and not size, is this
clearly an example of:
1. A black box test
2. A glass-box test
3. A unit test
4. Both 1 and 3
5. Both 2 and 3
![Page 13: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/13.jpg)
(13)
Planning Your Tests
• How do we plan a “test case”?Define three parts first:– A purpose (why are we doing this particular test?)– An input– An expected result
• We execute the test to see if observed output matched the expected result– It’s really just like a scientific experiment
• When testing, have a plan (at least in your head)– Don’t test blindly or randomly
![Page 14: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/14.jpg)
(14)
Planning Your Tests
• Poorly specified test cases…
Test ID Description Expected Results
Actual Results
1 Player 1 rolls dice and moves.
Player 1 moves on the board.
2 Player 2 rolls dice and moves.
Player 2 moves on the board.
![Page 15: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/15.jpg)
(15)
Planning Your Tests
• Why are these test cases better than the previous ones?
Test ID Description Expected Results
Actual Results
1 Precondition: Game is in test mode, GameBoard is loaded, and game begins.
Number of Players: 2
Player 1 dice roll: 3
Player 1 moves to square Blue 3.
2 Precondition: TestID 1 completed successfully.
Player 2 dice roll: 3
Player 2 moves to square Blue 3.
![Page 16: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/16.jpg)
(16)
Unit Testing: Boundary values
• Often programs misbehave because we make mistakes dealing with the “extremes”
• Examples:– The last item or first item in a collection– The start or end of a range of values
• Equivalent classes of test cases– Don’t have test cases or data that test the same
situation (e.g. the middle of a range)• Boundary or extreme conditions
– An empty list, an empty file– One item in a list or file
![Page 17: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/17.jpg)
(17)
Unit Testing: Off-by-one errors
• Related to boundary conditions• Using the nth value instead of n-1• Using 0 instead of 1• Executing a loop body once when it the body should
not be executed at all– Wait – is this one a black-box testing method?
Why or why not?• Etc.
![Page 18: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/18.jpg)
(18)
Unit Testing: Invalid Inputs
• Rely on pre-conditions defined for your functions– They specify what the agreement is about what
you are responsible for when its time to handle mis-use
• If a function is mis-used in this way– Return an error condition if the interface provides
a way to do this– Otherwise, halt program! Better than producing
unexpected results• Use assertions (more later!)
![Page 19: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/19.jpg)
(19)
Example: Program with Student and Course classes
• Homework 1 this term– Define class Student
• methods include:
– Define class Course• methods include:
• Focus on defining:– Black-box test-cases for the method
boolean drop(Course c) in class Student• Let’s make a list of possible test-cases for this
method!
![Page 20: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/20.jpg)
(20)
Example of Full Test-Case Description
• Test Case #1: Drop last course in list with >1 item– Purpose: Drop Course c for Student s where that
course is the last course stored in the Student object's courses list. This tests the boundary condition of last item in list for Student.drop(c).
– Input: Student s is enrolled in three courses, and the course to be dropped is the last of these stored in the list.
– Expected output: The Student s is unchanged except that only two courses are stored for that student. Also, the list of students stored for that course is altered by having that student removed.
![Page 21: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/21.jpg)
(21)
Other Test Cases for this Program
• There are a lot!• Boundary conditions?
– for Student.drop(): end of list, start of list– same for Course.drop()– Question: affected by whether you used ArrayList
instead of Java arrays?– Note: also test “typical” case, item in middle of list– Empty lists: Course.cancel(), Student.dropAll()
• Off-by-one errors?– Really included in boundary condition testing here
![Page 22: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/22.jpg)
(22)
Other Test Cases for this Program
• Invalid inputs?– See the specifications!– Test return values
• For boolean, test both success and failure• Remember: make it fail!
• Test of equals() methods?– If you get this wrong, trouble!– Easy to test. (Easy to forget to.)
![Page 23: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/23.jpg)
(23)
Good or Bad Test?
1. Good Test?
2. Poor Test?
Student s adds a course and the course appears in the schedule
![Page 24: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/24.jpg)
(24)
Good or Bad Test?
1. Good Test?
2. Poor Test?
Student s currently is inthe system and currentlyhas no courses.Student s adds a course number 394 and it appearsat the end of the schedule.
![Page 25: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/25.jpg)
(25)
Good or Bad Test?
1. Good Test?
2. Poor Test?
Student s has 4 courses,drops 3, adds 1. Check if thestudent is then deletedfrom the system.
![Page 26: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/26.jpg)
(26)
Extra Slides
![Page 27: (1) Verification Methods CS2110: SW Development Methods Reading: Study Section 1.2.5 in the MSD text These slides supplement the textbook – These touch.](https://reader035.fdocuments.net/reader035/viewer/2022062421/56649cf85503460f949c922c/html5/thumbnails/27.jpg)
(27)
#2: Inspections: We Might See These Later
• Many different “styles” or approaches– Walk-throughs, Inspections
• Advantages:– You can inspect all of a product but usually it is
impossible to test all of a product.– Applicable to all work products (e.g. code,
designs, requirements documents, etc.)– Fairly easy: training, application, follow-up
• Disadvantages:– Staff time is expensive– Works best if follows a rigorous process