3 Test-Driven Development

8/6/2019 3 Test-Driven Development

1/22

Software Testing & Test-Driven

Development

JAMS Workshop

Makerere UniversitySeptember 2010


2/22

Agenda

Intro to Software Testing

Test-Driven Development

Writing a Test Plan Test Frameworks

JUnit

Visual Studio


3/22

Software Testing

Without testing, theres no proof that the software doeswhat it is intended to do Testing == Quality

Testing should be incorporated into the developmentprocess from the beginning The need to test your software will impact how you develop it

The earlier you find a problem, the easier it is to fix

Time

Introduced

Time Detected

Requirements Design Implementation QA Post-Release

Requirements 1x 3x 5-10x 10x 10-100x

Design - 1x 10x 15x 25-100x

Implementation - - 1x 10x 10-25x

Software testing is the process of verifying that a software program

works as expected.

Cost of fixing a defect depending on the stage it was found


4/22

Who is Involved with Testing?

Business Stakeholder Participates in internal

alpha testing before thesoftware is released

Program Manager

Writes functional specthat enables tester to

write a test plan Reviews the test plan

Developer Writes implementation

spec that enables testerto complete a test plan

Reviews the test plan

Implements unit testsTest Engineer

Writes test plan

Writes and executes test

cases, files bugs Signs off on final product

Customer The person who is

ultimately affected if thecode was not well-tested

May participate in betatesting or perform user-acceptance testing


5/22

Testing Methods

The box analogy describes the point of view theengineer takes when testing

Tester has access to internal data structures and algorithms

API testing

Code coverage

Fault injection

Tester treats the code as a black box no knowledge ofimplementation

Boundary-value analysis

Alpha/beta testing, dogfooding

Tester has knowledge of internal data structures andalgorithms, but tests at the user/black-box level

Integration testing (between two modules of code at the interface level)

Reverse engineering

WhiteBox

Black

Box

Grey

Box


6/22

Many Levels of Testing

Tests that verify a specific section of code, e.g.classes or methods

Usually white box tests written by developers

Unit Testing

Verifying interfaces between components

Iteratively add elements until the whole systemis verified

Integration Testing

Testing a completely integrated system to verifythat it meets requirements

System Testing

Testing that a system is integrated to an externalor 3rd party system

Each system has already passed system testing

System Integration Testing

Tests that serve to prevent old bugs fromcoming back

Usually written as part of the bug fix process

Regression Testing

Also known as User Acceptance Testing

Testing performed by the customer, in their ownlab environment

Acceptance Testing

Testing by bleeding edge internal users orexternal customers

Could also refer to internal dogfood efforts

Alpha Testing

A broader test deployment, usually happensafter alpha testing

Usually involves an external audience

Beta Testing


7/22

Test-Driven Development

A development technique that relies on the concept ofwriting the test cases before the product code Validates that spec and requirements are well-understood

Test cases will initially fail

Developer writes code to make them pass

The test is the proof that the code works Developer must clearly understand user requirements in

order to write tests

There should be no functionality in product code that isnttested

Encourages simple designs and inspires confidence Clean code that works


8/22

TDD Workflow

1. (Re)write a

test

2. Run all

tests & see if

the new one

fails

3. Write some

code

4. Run the

automatedtests & see

them pass

5. Refactor code

Repeat

Test succeeds

Test fails

Test(s) fail

All tests

succeed

Always start by writing a test

Test must fail because the

feature isnt implemented (if it

succeeds, the need for the

feature is obviated)

To write a test, you must fully

understand the features

specification and requirements

Write minimal product code tomake the test compile and run

Validates that test harness works

Tests the test itself: make sure it

doesnt pass without new code

Test should fail for the expected

reason ensure that it is testing

the right thing

Write some code to make the

test pass

May be inelegant or suboptimal;

we will improve it later

Make the test pass only, no

extra untested functionality

If all test cases pass, all tested

requirements are met

If they fail, keep iterating

Assuming the tests are

comprehensive, move on to the

final stage

Clean up the code as necessaryto achieve production quality

Focus on removing duplication,

including duplication between

test & product code

Re-run the test cases to ensure

that refactoring isnt breaking

any functionality


9/22

Writing a Test Plan

Like a functional specification for your test code

Test Plan Template

Test Plan Objectives

Scope

Features to be tested

Features NOT to be tested

Test Strategy Test Cases

Open Issues


10/22

Test Plan Objectives & Scope

Similar to functional spec What are we trying to

accomplish with our testcases?

What is in and out of scopefor testing?

Test Plan Objectives

[P1] Test functional correctness of the games

underlying methods using white-box unit testing

[P1] Test that only valid moves are allowed

[P1] Validate that the game can correctly switch

between players and declare the end of the game

[P2] Test advanced features, like the statistics

table, play timer, and computer player, if

implemented

[P3] Performance testing

Scope

Features to be tested

Basic game play

o The game alternates between

players

o Players can place moves

oOnly legal moves are allowed Ending the game

o The game can declare a

winner

o The game can declare a tie

Player statistics table (if

implemented) Timer (if implemented)

Machine player (if implemented)

Features NOT to be tested

User interface rendering

Networked play


11/22

Test Strategy

Describe the testing methodologyyou plan to use E.g. white box, black box, grey box,

or a combination

Explain which testing frameworksyoull use, if any E.g. JUnit, Visual Studio

Will your tests require anysophisticated infrastructure,setup, or tools? E.g. mock objects, load simulation,

test bridges

Test Strategy

We plan to employ mostly

white-box unit tests in order

to test the application,using Visual Studios built-in

unit-testing framework.

Each of the following major

classes will have at least

one unit test:

Game GameBoard

Player

GameStatistics


12/22

Test Case Detail

Test Case ID Test Case Name Steps Expected Result

2.1 Test basic moves 1. Place an X at TopLeft

2. Assert that the board

records an X at TopLeft

3. Place an O at Center

4. Assert that the board

records an O at Center

Markers are

present at

expected positions

(assertions pass)

2.2 Disallow player from

placing his marker on a

square occupied by his

opponent

1. Place an X at TopLeft

2. Place an O at TopLeft

Move 2 fails

2.3 Disallow player from

playing on his own

occupied square

1. Place an X at TopRight

2. Place an O at Center

3. Place an X at TopRight

Move 3 fails

Feature 2: Validate Basic Moves/Allow Legal Moves Only


13/22

Testing Frameworks

JUnit is a unit-testing framework for Java

Developed by the same people who pioneered TDD

Uses source code annotations to decorate special

methods to be run by the test harness Integrated with Java IDEs like Eclipse and JCreator

Visual Studio has a test framework for any .NETlanguage

Based on the same ideas as JUnit

Supports unit tests, database unit tests, generic tests,manual tests, load tests, web tests


14/22

Test Methods

JUnit Annotate test case

methods with @Test

Use methods fromorg.junit.Assert to check

your test conditions orfail the test case

Visual Studio Annotate test case

methods with

Use methods from theAssert class to checkyour test conditions orfail the test case

import org.junit.Test;

public class AdditionTest {

private int x = 1;

private int y = 1;

@Test public void testAdd() {

int z = this.x + this.y;assertEquals(2, z);

}

}

Imports Microsoft.VisualStudio.TestTools.UnitTesting

Public Class AdditionTest

Private x As Integer = 1Private y As Integer = 1

Public Sub TestAdd ()

Dim z As Integer = Me.x + Me.y

Assert.AreEqual(2, z)

End Sub

End Class


15/22

Initialization & Cleanup Methods

Common code that runsbefore/after each testcase

also known as Fixturemethods Add a field for each part of

the fixture

Annotate a method withBefore/TestInitialize andinitialize the variables in

that method Annotate a method with

After/TestCleanup to cleanup before the next test

@Before protected void setUp() {

this.x = 1;

this.y = 1;

}

@After protected void tearDown() {

this.x = 0;this.y = 0;

}

Public Sub SetUp()

Me.x = 1

Me.y = 1End Sub

Public Sub TearDown()

Me.x = 0

Me.y = 0

End Sub


16/22

Running Tests in the IDE

Visual Studio & Eclipse bothenable running tests inside theIDE Select specific tests to run or run

the whole suite IDE reports which cases

passed/failed

Run selected tests again

Easily switch between test codeand product code Change each as needed and rerun

tests

Easy to debug a test case, setbreakpoints in test or productcode


17/22

Further Reading

TDD

Test-Driven Development: By Example (Google

Books)

testdriven.com

JUnit

JUnit Cookbook

JUnit Javadoc

An early look at JUnit 4


18/22

APPENDIX


19/22

Suite Initialization Methods

Similar to setUp and tearDown, but they run before & after theentire test suite

Useful for expensive config operations that dont need to be run foreach unit test

E.g. setting up a DB or network connection, redirecting System.errwhen testing 3rd-party libraries

Be careful that your unit tests dont make changes to static state thatwill impact other unit tests later in the suite

// This class tests a lot of error conditions, which Xalan annoyingly logs

// to System.err. This hides System.err before the run, restores it after.

private PrintStream systemErr;

@BeforeClass protected void redirectStderr() {

systemErr = System.err; // Hold on to the original valueSystem.setErr(new PrintStream(new ByteArrayOutputStream()));

}

@AfterClass protected void tearDown() {

// restore the original value

System.setErr(systemErr);

}


20/22

Testing Exceptions

It is easy to test for expected exceptions Annotate your test with the expected exception

If the exception isnt thrown (or a different one is), the test will fail

Limitation: if you need to test the exceptions message or

other properties, use a different structure:

@Test(expected=ArithmeticException.class)

public void divideByZero() {

int n = 2/

0;}

@Test public void divideByZero() {

try { int n = 2 / 0;

fail("Divided by zero");

}

catch (ArithmeticException(success) {

assertNotNull(success.GetMessage());

}

}


21/22

Timed Tests

Simple performance bench-marking:

Network testing:

@Test(timeout=500)

public void retrieveAllElementsInDocument() {

doc.query("//*");

}

@Test(timeout=2000)

public void remoteBaseRelativeResolutionWithDirectory()

throws IOException, ParsingException {builder.build("http://www.ibiblio.org/xml");

}


22/22

Creating JUnit Tests with Eclipse

Using JUnit with Eclipseis easy

Create new JUnit testsusing File -> New ->JUnit

Specify JUnit 4

Select location, classyou want to test,

method stubs to create Then select which

methods you want totest, Eclipse willgenerate test stubs

3 Test-Driven Development

Documents

Transcript of 3 Test-Driven Development