Functional System Testing

7/29/2019 Functional System Testing

1/25

Functional System Testing

Written by

Aishwarya Bisht


2/25

Black Box Testing2

Outline

Goal of testing

Test cases, test suites and test data

What is functional system testing?

Coverage

Functional testing techniques:

Functional analysis

Equivalence partitioning

Boundary value analysis


3/25

Black Box Testing3

The goal of software testing

The process of uncovering evidence of defects insoftware systems

Does not include efforts associated with tracking downbugs and fixing them

No amount of testing will improve the quality of acomputer program

The more testing we do of a system, the moreconvinced we might be of its correctness

Testing cannot in general prove a system works 100%correctly


4/25

Black Box Testing4

Test cases

The basic component of testing is a Test Case

In its most general form: (inputs, expected-result)

inputs include system state, user commands and datavalues to be processed

expected resultincludes visible/audible interface

changes or changes in the system state

Test cases are organized into Test Suites functionality, security, performance,


5/25

Black Box Testing5

Test case execution

A running of the software (under test) that

provides the inputs specified in the test case

and observing the results and comparingthem to those specified by the test case

If the actual result varies from the expected

result, then a failure has been detected


6/25

Black Box Testing6

Test data

An effective test strategy requires careful acquisition andpreparation of test data prior to testing

Testing can suffer if test data is poor

Test data concerns: Depth: quantity and size of data

Breadth: variance of data values and data types

Scope: completeness, relevance and accuracy of data

Result of a query should be valid for the specific purpose of the

query, and not due to a missing or inappropriate value Conditions: data should reflect specific conditions in the domain

Data that would otherwise arrive after performing specific operationsover time

Test data and test results are expensive to construct


7/25

Black Box Testing7

Example: Test data for TVRS

Name:test1.db

Description:Violation records designed for validating violation lookup

Violation IDOffenders first name

Offenders last name

Issuing policeman ID

243567 Rachel Josef 8700342

237812 Dan Levi 6386541

264683 Dan Porat 1346329

255245 Dina Josef 8245731

000345 longFirstN longLastN 8700342


8/25

Black Box Testing8

Specification Vs. Implementation

The basic approaches to testing software are based on its

specification and implementation

White box testingtest cases and data are constructed

based on the code that implements the software

Quality and correctness of computations is validated

Will not be further discussed in this tutorial

Black box testingtest cases and data are constructed

based solely on the softwares specification


9/25

Black Box Testing9

Functional System Testing

Testing of a completed application to determine

that it provides all of the behaviors required of it

Testing of completed increments that provide somedegree of end-user functionality

Search for defects that are variances between the

actual operation of the system and the

requirements for the system System is treated as a black box


10/25

Black Box Testing10

How much testing is adequate?

Completely validating IEEE 754 floating-point division requires 264 test-cases!

float divide(float x, float y)

From practical and economic perspectives,exhaustive testing is usually not possible

Which software pieces should we test?

Which test cases should we choose?


11/25

Black Box Testing11

Coverage

Coverage is a measure of how completely a test

suite exercises the capabilities of a piece of

software Each line of code should be executed at least once

One test case should be constructed from each

specified requirement

It is necessary to use testing techniques thatnarrow down the number of test cases allowing the

broadest testing coverage with the least effort


12/25

Black Box Testing12

Technique: Functional Analysis

Analyze the expected behavior of the system

according to its functional specification

Generate a test procedure for each of the possibleusage scenarios

Corresponds to use case scenarios

Analyze how a change in one part of the system affects

other parts

Grand tour test cases: the result of one test case

produces the data that is the input to the next test case


13/25

Black Box Testing13

Example: Functional Analysis I

Use Case: Remove Traffic Violation

1. Supervisor calls for deletion of the chosen Traffic Violation

2. TVRS prompts Supervisor for confirmation

3. Supervisor confirms

4. TVRS requests OffendersDB to delete the Traffic Violation

from the offenders record

5. OffendersDB approves that the Traffic Violation has beendeleted

6. TVRS allows Supervisor to look up a new Traffic Violation as

described in the Lookup Traffic Violation UC


14/25

Black Box Testing14

Example: Functional Analysis IITest case ID: 134543

Pre-conditions: 1. TVRS initialized with test1.db database

2. Violation 243567 displayed in the Lookup Violation dialog

Related use cases: Lookup Traffic Violation, Remove Traffic Violation

Expected resultActionConfirmation dialog is displayedPress Delete button

Lookup Violation dialog is displayedPress the Yes button

A message dialog stating that violation

243567 is not stored in TVRS

Enter 243567 at Violation ID text field

and press the Search button

Test results

Actual results:Passed

Failed

Defect diagnosis:


15/25

Black Box Testing15

Example: Functional Analysis III

Verify effects

of change

Filled when

the test case

is executed

How do we know that

violation 243567 is

stored in the system?

Can a tester

diagnose the cause

of a defect?

In addition, a querycould be run on the

Offenders database


16/25

Black Box Testing16

Technique: Equivalence

Partitioning Identifies ranges of input and initial conditions

that are expected to produce the same result

A group of test cases form an equivalence class if: They test the same feature/scenario

If one test reveals a fault, the other ones (probably) willtoo

If a test does not reveal a fault, the other ones(probably) will not either

It is adequate to use only a single representative ofthe equivalence class


17/25

Black Box Testing17

Example: Equivalence Partitioning I

Input value specification for Lookup Violation form:

Valid valuesField name

[0-9]{0, 9}Violation ID

[a-zA-Z]{0, 10}Offenders first name

[a-zA-Z]{0, 10}Offenders last name


18/25

Black Box Testing18

Example: Equivalence Partitioning II

Field Valid

equivalent

classes

Valid

representative

values

Invalid

equivalent

classes

Invalid

representative

values

Violation ID Known violation 00243567 ID < 0 or ID >

999999999

-1, 1234567890

Unknown violation 32456720 Non numeric ID 23ab@

Empty

Offenders

first name

Unknown violation David Character# > 10 Hasalongname

Single known

violation

Rachel Invalid character ad0@am

Many known

violations

Dan

Empty


19/25

Black Box Testing19

Example: Equivalence Partitioning III

The number of test cases to choose from is reduced to(3 + 2) (4 + 2) (4 + 2) = 180

The actual number can be further limited

Single invalid field per test case (3 4 4 + 6 = 54) Importance of use case

Resources available

Most frequent input

Life-critical software

Infeasible test cases

Randomly

...


20/25

Black Box Testing20

Technique: Boundary Value

Analysis Based on experience / heuristics

Testing boundary conditions of equivalence classes is

more effective Choose input boundary values as equivalence

classes representatives

Choose inputs that invoke outputboundary values

Examples: (0, 10] validate using 0, 1, 2, 9, 10, 11

Read up to 5 elements validate reading 0, 1, 4, 5, 6 elements


21/25

Black Box Testing21

BVA as an equivalence

partitioning extension Choose one (or more) arbitrary value(s) in

each equivalence class

Choose valid values exactly on lower andupper boundaries of equivalence class

Choose invalid values immediately below

and above each boundary (if applicable)


22/25

Black Box Testing22

General purpose test-suite

construction technique May be used to obtain reasonable coverage with little

effort Use cautiously!

Unsuitable when values of different fields are related

1. While test cases can be added

Each new test case should include as many un-included valid non-boundary

equivalence class representatives as possible

2. While test cases can be added

Each new test case should include as many un-included valid boundary

equivalence class representatives as possible3. While test cases can be added

Each new test case should include a single invalid equivalence class

representative that has not been included before

4. Manually replace/remove redundant or infeasible test-cases


23/25

Black Box Testing23

Example: Country Club I

Day Sunday - Thursday Friday - Saturday

Guest

status

Visitor Member Student Visitor Member Student

Age

(years)

Admission fee

[0, 16) 25 10 20 35 10 30

[16, 60) 50 25 45 70 25 65

[60, 120] 35 15 30 50 15 45

Specification


24/25

Black Box Testing24

Example: Country Club II

Field Valid

equivalent

classes

Valid

representative

values

Invalid

equivalent

classes

Invalid

representative

values

Day Sun - Thu Mon, Sun, Thu

Fri - Sat Fri, Sat

Guest

status

Visitor Visitor

Member Member

Student Student

Age [0, 16) 2, 0, 15 Non-numeric

value

14@a

[16, 60) 34, 16, 59 Age < 0 or

Age > 120

-1, 121

[60, 120] 100, 60, 120

A combo box is used

for choosing the day

and guest status


25/25

Black Box Testing25

Example: Country Club IIITest case ID Day Guest status Age Result

1 Mon Visitor 2 25

2 Fri Member 34 25

3 Mon Student 100 30

4 Sun Visitor 0 255 Sat Member 16 10

6 Thu Student 60 30

7 Sun Member 15 10

8 Sat Student 120 45

9 Thu Visitor 59 50

10 Mon Member 14@a Invalid age

11 Fri Student -1 Invalid age

12 Fri Visitor 121 Invalid age

valid

valid

(boundary)

invalid

Functional System Testing

Documents

Transcript of Functional System Testing