PTesting vs Ctesting Touseef

8/3/2019 PTesting vs Ctesting Touseef

1/38

Understandablity of Concrete versus Parametric Test Cases

By

Touseef Tahir, Ali Jafar, Mohan Maharajan

Lahore


2/38

hIntroduction to Unit Testing

Concrete Testing AND Parametric Testing

h

The research QuestionshHypothesis

hPlanning and design of Experiment

hExperiment

hAnalysis and discussion

hConclusion


3/38

hA test case is a description of a specific interactionthat a tester will have in order to test a singlebehavior of the software. Test cases are very

similar to use cases, in that they are step-by-stepnarratives which define a specific interactionbetween the user and the software.

hTest case is a triplet [I, S, O] where

I is input dataS is state of system at which data will be input

O is the expected output

hTest suite is set of all test cases


4/38

Testing of functional and non functional

behavior of the software programs.

Verifies correctness of the units of program inisolation.

Detect and remove faults earlier from the

specified units.Provides 100% code coverage, checks all the

corner cases and detects error in components

logic


5/38

executes for a single test input at an instance.

execute a method of a class under test withfixed arguments and verify that it returns the

expected result or not.

The concrete input values are provided to the

unit to determine its failure or success.


6/38

Numerous test inputs that has to be examined

and revised.

Tedious, labor intensive, error-prone, time

consuming, unlikely to cover all possible

behavior and often difficult task.


7/38

Does not ensure the high code coverage since

only single examplary input value is provided

as an input.

Increase in effort, cost, time, numerous test

cases, and numerous iterations.


8/38

Parameters as an input to the unit of code.

Exercised different code paths for the set of

test inputsDetects more faults than concrete test cases.

High code coverage with small test suite.

Can generate more test instances and executesautomatically.

Less time consuming, and likely to cover all

the possible behavior.


9/38

Assist in reusability, evolution, and

maintainability

Hard to measure since depends on cognitive

behavior of human


10/38


11/38

Documentation of source code

Understanding of Structure and component

Code Complexity

Psychological Complexity


12/38

Reading Scalability for general unit test

questions

Closed ended questionnaires


13/38


14/38

Types Definition

Subjects MSc. Software Engineering

MSc. Computer Sciences

MSc Security engineering

Environment Class Room

Apparatus Document containing Questioners

(Close ended question)

Experimental Design Method Factorial Design

Programming Languages Java, C#

Testing Tools JUnit, NUnit, Pex

Groups JUnit Concrete Testing (6 Subjects)

JUnit Parameterized Testing(6

Subjects)

NUnitConcreter(6 Subjects)

Pex Parameterized Testing (6Subjects)


15/38

Variable type Variables

Independent Subjects

Programming languages (Java,C#)

JUnit, NUnit, Pex

Treatment Concrete Testing

Parameterized Testing

Dependent Understandability

Time

Accuracy (Correctness)

Extraneous Participants Gender/Sex, Age

Experiment situation(noise, temperature)

Equipment (fault equipment, calibration)

The time of the experiment

Conduct of experiment

Carry over effect


16/38

Understandablity ofConcrete versus

Parametric TestCases


17/38

hPurposed Hypothesis H0: Concrete test

cases are difficult to understand than

Parameterized test cases and thus makeoverall software testing difficult.

hAlternative Hypothesis H1: Parameterized

test cases are difficult to understand than

concrete test cases and thus make overall

software testing difficult.


18/38


19/38

General questions

Questions regarding concrete test cases and

parameterized test cases related to the codeand Test Code.


20/38


21/38

0

0.5

1

1.5

2

2.5

3

3.5

4

SCALE


22/38

Note:

CC=C# Concrete, JP=Java Concrete

Time is measured in Minute, Correct Answer out of 37


23/38

Note:

CP=C# Parameterized JP=Java Parameterized

Time is measured in Minute, Correct Answer out of 37


24/38

Descriptive Statistics

Hypothesis Testing

Wilcoxon Signed Test


25/38

hSummarize the available data


26/38


27/38


28/38

hAccept OR Reject hypothesis


29/38


30/38

Note: As there is one tie in the data so the number of participants

for this group will be 11 and the table value is 10


31/38


32/38

hTime

No significant difference

Not practical to conclude on the basis of time

hCorrectness ratio

Significant difference as well as participants

hWe reject null hypothesis

Correctness ratio and general testing questionswe concluded that

hMore general test cases are harder to understand than concrete ones

and thus would lead to tests that are harder to understand.


33/38

Understandability of concrete test cases

versus general test cases using mixed

approach.

Using practitioners besides using students.

Using more than two programminglanguages, more subjects and more examples.


34/38


35/38


36/38

h After the early analysis of data it was seen that subjects

have taken different amount of time to solve a specific

example. In some cases the time difference was huge so it

was not fitted for parametric hypothesis testing methods.The data was not in intervals as well so it was decided non

parametric statistics was used on the collected data. The

only factor involved in experiment was unit testing with

two different treatments which were Parameterized test

cases and Concrete test cases. The best suitable method forhypothesis testing was Wilcoxon signed test


37/38


38/38

PTesting vs Ctesting Touseef

Documents

Transcript of PTesting vs Ctesting Touseef