Software Testing•Software testing is a critical element of software quality assurance and represents the ultimate review of specification, design and code generation.•What is it?Once source code has been generated, software must be tested to uncover (and corrected) as many errors as possible before delivery. The goal is to design a series of test cases that have a high likelihood of finding errors.These techniques provide guidance for designing tests that verify the internal logic of software components to uncover errors in program function, behavior and performance reviews and other activities can and do uncover errors, but are not sufficient. 

Testing..• Who does it?• During initial stages, a software engineer

performs all tests. And later testing specialists are involved

• What are the steps?• Software is tested from two different

perspectives: internal logic is tested using the “white

box” testing requirements are tested using the “black

box” testing• In both the cases, the intent is to find maximum

number of errors with minimum amount of effort and time

Software Testing Fundamentals• Testing Objectives

• The various testing objectives are: It is a process of executing a program with the intent of

finding an error A good test case is one that has a high probability of

finding an as-yet-undiscovered error A successful test is one that uncovers an as-yet-

undiscovered error.• Benefits of a successful testing• It uncovers errors in the software• It demonstrates that software functions appear to be

working according to specification, that behavioral and performance requirements are met

• Data collected during testing provides a good indicator for the program quality

• Testing does not show the absence of error and defects, it can show that errors and defects are present

Attributes of a “good” test• The followings are the attributes of a good test:   The good test has a high probability of

finding an error.   A good test is not redundant. A good should be “best of breed”. The test

that has the highest likelihood of uncovering a whole class of errors should be used.A good test should be neither too simple nor

too complex.

Bugs• Defect Failure• Anomaly• Variance• nconsistency• Product Anomaly• Fault• Problem• Error• Incident

Defective software and sources of problems

• We develop programs that contain defects – How many? What kind?• Sources of Problems :-

• Requirements Definition: Erroneous, incomplete, inconsistent requirements.

• Design: Fundamental design flaws in the software.

• Implementation: Mistakes in chip fabrication, wiring, programming faults, malicious code.

• Support Systems: Poor programming languages, faulty compilers and debuggers, misleading development tools.

Sources of Problems• Inadequate Testing of Software: Incomplete

testing, poor verification, mistakes in debugging.

• Evolution: Sloppy redevelopment or maintenance, introduction of new flaws in attempts to fix old flaws, incremental escalation to inordinate complexity.

Adverse Effects of Faulty Software

• Communications: Loss or corruption of communication media, non delivery of data.

• Space Applications: Lost lives, launch delays.• Defense and Warfare: Misidentification of

friend or foe.• Transportation: Deaths, delays, sudden

acceleration, inability to brake.• Safety-critical Applications: Death, injuries.• Electric Power: Death, injuries, power outages,

long-term health hazards (radiation).• Money Management: Fraud, violation of

privacy, shutdown of stock exchanges and banks, negative interest rates.

Problems Occurred• Military Aviation Problems:- An F-18 crashed because of a missing

exception condition: if ... then ... without the else clause that was thought could not possibly arise.

• Bank Generosity:- A Norwegian bank ATM consistently dispersed 10 times the amount required.

• Many people joyously joined the queues as the word spread.• AT & T Bug :- In mid-December 1989, AT&T installed new software in 114 electronic

switching systems. • On January 15, 1990, 5 million calls were blocked during a 9 hour period nationwide.

The bug was traced to a C program that contained a break statement within an switch clause nested within a loop.

The switch clause was part of a loop. Initially, the loop contained only if clauses with break statements to exit the loop.

When the control logic became complicated, a switch clause was added to improve the readability of the code ...

• Year Ambiguities:- In 1992, Mary Bandar received an invitation to attend a kindergarten in

Winona, Minnesota, along with others born in '88. Mary was 104 years old at the time Mr. Blodgett’s auto insurance rate tripled when he turned 101. He was the computer program’s first driver over 100, and his age was

interpreted as 1.

Goal of a software tester• … to find bugs • … as early in the software development

processes as possible• … and make sure they get fixed.

• Software is :- requirements specification documents• design documents• source code• test suites and test plans• interfaces to hardware and software operating environment• internal and external documentation• executable programs and their persistent data• Software Effort :- Specification• Product reviews• Design• Scheduling• Feedback• Competitive information acquisition• Test planning• Customer surveying• Usability data gathering• Look and feel specification• Software architecture• Programming

Test documents• Test plan

– Quality objectives, resource needs, schedules, assignments, methods, etc.

• Test cases– Inputs and expected outputs.

• Bug reports– E.g., the Bugzilla web-based bug tracker.

• Test tools and automation• Metrics, statistics, and summaries

– Number of unresolved bugs, mean time to repair a bug, etc.

Software Project Staff• Project managers

– Write product specification, manage the schedule, critical decision tradeoffs

• Software architects, system engineers– Design the software, work closely with programmers

• Programmers, developers, coders– Write code, fix bugs

• Testers, quality assurance staff– Find bugs, document bugs, track progress of open bugs

• Technical writers– Write manuals, on line documentation

• Configuration managers, builders– Packaging and code, documents, and specifications

Quality Assurance• Quality Assurance consists of a set of auditing and reporting

functions that assess the effectiveness and completeness of quality control actions

• Its goal is to provide mgmt and technical staff with the necessary data i.e. product quality, theory gaining insight and confidence that actions to achieve qualitative product

• Elements of s/w quality assurance:- – Standards :- IEEE,ISO and other std.organizations have

produced a broad array of s/w engg std and related docs– Reviews and Audits:- technical review are quality control

activity performed by s/w engg for s/w engg. This is to uncover errors. Audits are a type of review performed by SQA personnel with the intent of insuring the quality guidelines

– Audit is the review process might be conducted to ensure review conducted to lead to the highest likelihood of uncovering errors

Quality Assurance•Testing:- it is done to find the errors. job of SQA is to ensure that testing is properly planned and efficiently conducted •Error/defect collection and analysis:- SQA collects and analyzes errors and defects to understand how errors are introduced and s/w engg activities are suited to eliminate them•Change management: changes should be saved in logs from time to time•Security management : -to protect the data at all levels,establish firewall protection, ensuring that s/w has not been tampered internally•risk management : - risk related contingency plans have to be established

Software quality assurance (SQA) include

• Formal technical reviews, quality.• Configuration audits • Performance monitoring.• Simulation.• Feasibility study. • Documentation review.• Database review• Algorithm analysis.• Development testing. • Qualification testing and installation testing

Six sigma for s/w Engg•Six Sigma is a business management strategy originally developed by Motorola, USA in 1981•Six Sigma seeks to improve the quality of process outputs by identifying and removing the causes of defects (errors) and minimizing variability in manufacturing and business processes.•Is a set of quality management methods

Six Sigma…•Software Six Sigma is an overall strategy to accelerate and to sustain continuous improvements in the software development process and in software product quality. • It is based on the application of statistical tools to the process and product metrics that characterize a stable software development process.  •Software Six Sigma enables quantitative management of product quality.  This in turn:

•speeds up integration and test•allows delivery of very high quality product (very few latent defects)•improves the repeatability and predictability of the entire software development process.

Six Sigma…•Software Six Sigma involves the application of Statistical Process Control (SPC) and related techniques to the production of software requirements, architecture, modules, and test cases. •These statistical techniques are known collectively as the six sigma tool kit.•Software Six Sigma's attributes are:

•trustworthy metric data•accurate planning based on historical data•use of statistical tools for real time analysis and decision support•quantifiable Software Process Improvement cost and benefits•predictably high product quality

Six sigma…•It is a rigorous and disciplined methodology that uses data and statistical analysis to measure and improve a company’s operational performance by identifying and eliminating defects•It has 3 core steps:-

•Define customer requirements and deliverables and project goals via well defined methods of customer communication•Measure the existing process and its o/p to determine current quality performance(collect defect metrics0•Analyze defect metrics and determine vital few causesif an existing process is in place,but improvement required, six sigma suggests 2 additional steps:- improve the process by eliminating the root causes of defectsControl the process to ensure that future work does not reintroduce the causes of defects

Six sigma…•If an organization is developing a s/w process(rather than improving existing process) the core steps could be:-

design the process to avoid root causes of defects and to meet customer requirementsVerify that the process model will, avoid defects and meet customer requirements

•This variation is called as DMADV(define,measure,analyze,design and verify )method

•http://www.softwaresixsigma.com/SixSigma_Main_SixSigma.htm

Static Testing•Static testing is a form of software testing where the software isn't actually used.• It is generally not detailed testing, but checks mainly for the sanity(judging) of the code, algorithm, or document. •It is primarily syntax checking of the code and/or manually reviewing the code or document to find errors. •This type of testing can be used by the developer who wrote the code, in isolation. Code reviews, inspections and walkthroughs are also used.•Code review is systematic examination (often as peer review) of computer source code. •It is intended to find and fix mistakes overlooked in the initial development phase•Walkthrough is often used in the software industry to describe the process of inspecting algorithms and source code by following paths through the algorithms or code as determined by input conditions and choices made along the way.

Static Testing..•Static Analysis: Analysis of a program carried out without executing the program.

•Static Analyzer: A tool that carries out static analysis. •Static Testing: Analysis of a program carried out without executing the program. •Static analysis, also called static code analysis, is a method of computer program debugging that is done by examining the code without executing the program. •The process provides an understanding of the code structure, and can help to ensure that the code adheres to industry standards.• Automated tools can assist programmers and developers in carrying out static analysis. •The process of scrutinizing code by visual inspection alone (by looking at a printout, for example), without the assistance of automated tools, is sometimes called program understanding or program comprehension.

Static testing..• Static testing is less cost effective for languages which has

strong type checking like Java hence can detect many errors during compilation

• Static testing highlights semantic problem areas in the source code regarding misuse of types and variables

• Semantic deals with the meaning of language and include the distinctions made between the meaning of words and symbols. 

• A complete static testing may be performed for all the modules in the product before unit testing starts, and after each modification to the unit later in development

• Static testing may reveal the faulty programming constructs at various levels. Some of the examples are:-

– It enables the checking of variables used before initialization,declared but never used,assigned twice but never used

– It looks for possible array bound violation and undeclared variables– It analyzes the unreachable codes and unconditional branches into loops– It checks for interface fault and will assure for parameter type and

number mismatch as well as uncalled functions and procedures– It checks unassigned pointers and pointer arithmetic

Principles of static testing•Review all coding std violations:- all violations of coding std should be removed•Review complexity of code: the complexity of the code must be reviewed. The static analysis reports contain many measures of complexity including lines of code, complexity metric, and depth of loop testing•Inspect and remove unreachable code, unreferenced proc and unused variables•Report all types of data flow anomalies:- following are the several types of anomalies

•A variable is used before it is initialized•A variable is assigned a value and runs out of scope without being used:- all such anomalies should be examined by developer and documented in the code•A variable is assigned a value and then reassigned without having been used in between :- a variable is assigned a value before a loop is entered and in the loop this variable is reassigned. So again the majority of anomalies reported of this type do not represent real bugs•A parameter mismatch:- difference in type of the parameter declaration and the actual variable being passed as a parameter to a procedure. many compilers do automatic conversion(casting) , that may be dangerous •Suspicious casts: using explicit casting is a solution for the same instead of allowing automatic casting

Taxonomy of static testing•It does not necessitate the execution of the program•It checks whether representations and descriptions of s/w are consistent ,non contradictory and unambiguious•Takes care of correct description, specifications and representations of s/w system•Covers lexical analysis of the program syntax and investigates and checks the structure and usage of individual stmt•Scope of s/w testing are:-

•Checking completeness and consistency of the program•Considering predefined rules•Comparing the pgm with its specification or documentation

Types of testing•The software developer is always responsible for testing the individual units (components) of the program the function for which it was designed. •In many cases, the developer also conducts integration testing – a testing step that leads to the construction (and test) of the complete program structure. •Only after the software architecture is complete does an independent test group become involved•Unit Testing:- Unit Testing focuses on verification effort on the smallest unit of software design - the software component or module. •Using the component-level design description as a guide, important control paths are tested to uncover errors within the boundary of the module. •The relative complexity of tests and uncovered errors is limited by the constraint scope established for unit testing.

Unit test considerations•The module interface is tested to ensure that information properly flows into and out of the program unit under test.•The local data structure is examined to ensure that data stored temporarily maintains its integrity during all steps in an algorithm’s execution.•Boundary conditions are tested to ensure that all modules operate properly at boundaries established to limit or restrict processing.•All independent paths (basis paths) through the control structure are exercised to ensure that all statements in a module have been executed at least once.

•And finally, all errors handling paths are tested.

Unit testing diagram

Errors found during unit testing

•Misunderstood or incorrect arithmetic precedence.•Mixed Mode operations.•Incorrect initialisation.•Precision accuracy.•Incorrect symbolic representation of an expression.•Comparison of different data types.•Incorrect logical operators or precedence.•Expectations of equality when precision error makes equality unlikely.•Incorrect comparisons of variables.•Improper or non-existent loop termination.•Failure to exit when divergent iteration is encountered.•Improperly modified loop variables.

Integration testing•Integration testing is a logical extension of unit testing. In its simplest form, two units that have already been tested are combined into a component and the interface between them is tested. •A component, in this sense, refers to an integrated aggregate of more than one unit.• In a realistic scenario, many units are combined into components, which are in turn aggregated into even larger parts of the program. The idea is to test combinations of pieces and eventually expand the process to test your modules with those of other groups.• Eventually all the modules making up a process are tested together•

Integration testing…•integration testing has following ways :- •The top-down approach to integration testing requires the highest-level modules be test and integrated first. This allows high-level logic and data flow to be tested early in the process and it tends to minimize the need for drivers. However, the need for stubs complicates test management and low-level utilities are tested relatively late in the development cycle. Another disadvantage of top-down integration testing is its poor support for early release of limited functionality.•The bottom-up approach requires the lowest-level units be tested and integrated first. These units are frequently referred to as utility modules. By using this approach, utility modules are tested early in the development process and the need for stubs is minimized. The downside, however, is that the need for drivers complicates test management and high-level logic and data flow are tested late. Like the top-down approach, the bottom-up approach also provides poor support for early release of limited functionality.

System testing•Goal is to evaluate the system as a whole, not its parts •Techniques can be structural or functional •Techniques can be used in any stage that tests the system as a whole (acceptance, installation, etc.) •Techniques not mutually exclusive •During system testing, the system is incorporated with other system elements (like the hardware, people, information ) and a series of Integration and validation tests are conducted.•The Software engineer should Anticipate (accept) potential integrating problems and thus Design error-handling path test•Conduct a series of tests to detect bad data.•Record the results of tests to use as evidence if finger pointing does occur.•Participate in Planning and Design of system to ensure that software is adequately tested.

4 types of system testingThere are 4 types of system testing:

•Recovery testing•Security testing•Stress testing•Performance testing

Recovery Testing

•It is desired that a computer system must recover from faults and resume processing with a pre-specified time •While at times it might be desired that the system must be fault tolerant i.e. processing faults must not cause overall system function to cease.

• It may also be desired that the system failure must be corrected within a specified period of time else server

economic damage will occur.•If the recovery is automatic, re-initialisation, check pointing mechanisms, data recovery and restart are evaluated for correctness

Security Testing

• Any computer-based system that causes actions that can improperly harm the system is a target for illegal penetration.•Example:

•Hackers who attempt to penetrate system for sport.•disgruntled employees who attempt revenge•Dishonest individuals who attempt to penetrate for personal gain.

• Security testing verifies protection mechanism of a system to protect it from Illegal penetration.• During security testing the tester plays the role of the individual who desires to penetrate a system.

•He may attempt to acquire password.•He may attack the system to breakdown any defence mechanism. •He may purposely cause system errors and penetrate during recovery.•He may browse through data

• The role of the system designed to make penetration cost more than the value of the information that will be obtained

Stress Testing• Stress tests are designed to confront programs with abnormal

situations.• Stress testing executes a system in a manner that demands

resources in abnormal quantity, frequency or volume•  Example:

– Special tests may be designed that generates ten interrupts per sec. when one or two is the average rate.

– Input data rate may be increased– Test cases that require maximum memory or other resources are executed.

• A variation of stress testing is a technique called sensitivity testing. It attempts to uncover data combinations within valid input

classes that may cause instability.

Performance Testing

•For real-time and embedded systems, software that provides required functions but doesn’t confirm to performance requirements are unacceptable.•Performance testing is thus designed to test the run-time performance of the software.• Performance are often coupled with stress testing and usually require both hardware and software instrumentation.

It monitors --1. Execution intervals.2. Log events (E.g.: interrupts).3. Machine states at regular intervals.

Software Verification and Validation

– ... is the discipline of verifying that products of each software life cycle phase comply with previous life-cycle phase requirements and establish the proper basis for initiating the next life cycle phase, and of validating that the completed end product complies with established software and system requirements.

– objective:• verification: “are we building the product right” • validation: “are we building the right product”

Validation and Acceptance• Validation testing• At the Highest level of Integration testing, software is

completely assembled as a package, errors are uncovered and corrected and a final series of Software tests i.e. the Validation Testing begins.

•  The expectations of a system are defined in Software Requirements Specification. The specification contains a

section called Validation Criteria.• Information contained in that section is the basis for a

Validation testing approach.•  The three components of validation testing are

• Validation test criteria• Configuration review• Alpha & Beta testing

• Configuration Review• An important elements of validation process is

Configuration Review

Alpha and Beta Testing• It Checks all element of Software have been properly developed, are catalogued and have the necessary details to support the Support Phase of the Software Life Cycle.• It is also called as Audit. • Alpha and Beta Testing• When software is built, a series of acceptance test are conducted to validate all requirements. An Acceptance test can range from ‘Test Drive’ to a planned and systematically executed series of tests.• Most Software product builders use a process called Alpha and Beta Testing to uncover the errors.• Alpha Test•Conducted by a customer at the developer’s site.•They are generally conducted in a Controlled Environment.• Beta Test•They conducted at one or more customer sites by the End-User. •Unlike alpha test, the developer is generally not present. •It is a live application of software in an Environment that cannot be controlled by developer.•The Customer records all problems that are encountered during Beta test and reports to developer. •Software Engineer makes modifications and then prepares for release.

Acceptance Testing• it addresses the testing of the system documentation by the user representative •It corresponds to the requirements phase of the SDLC•acceptance testing is black-box testing performed on a system (for example: a piece of software, lots of manufactured mechanical parts, or batches of chemical products) prior to its delivery.•It is also known as functional testing•A smoke test is used as an acceptance test prior to introducing a build to the main testing process.•Acceptance Testing: Testing conducted to enable a user/customer to determine whether to accept a software product. Normally performed to validate the software meets a set of agreed acceptance criteria.

Black Box Testing•Black-Box testing, also called behavioural testing, focuses on the functional requirements of the software.•Black box testing attempts to find errors in the following categories:

•Incorrect or missing functions•Interface errors•Errors in data structures or external data base access•Behaviour or performance errors•Initialization and termination errors.

• Tests during black box testing are designed to answer the following Questions:

•How is functional validity tested?•How is a system behaviour and performance case?•What class of input will make good test case?•Is the system particularly sensitive to certain input values?•How are the boundaries of a data class isolated?•What data rates and data volume can the system tolerate?•What effect will specific combinations of data have on system operation?

Functional Testing• Testing the features and operational behavior of a product to

ensure they correspond to its specifications. • Testing that ignores the internal mechanism of a system or

component and focuses solely on the outputs generated in response to selected inputs and execution conditions.

• White Box Testing• White box testing strategy deals with the internal logic and structure of the

code. White box testing is also called as glass, structural, open box or clear box testing. The tests written based on the white box testing strategy incorporate coverage of the code written, branches, paths, statements and internal logic of the code etc.

• In order to implement white box testing, the tester has to deal with the code and hence is needed to possess knowledge of coding and logic i.e. internal working

• Advantages: • i) As the knowledge of internal coding structure is prerequisite, it becomes

very easy to find out which type of input/data can help in testing the application effectively.

• ii) It helps in optimising the code • iii) It helps in removing the extra lines of code, which can bring in hidden

defects

White box testing contd..•Disadvantages: •i) As knowledge of code and internal structure is a prerequisite, a skilled tester is needed to carry out this type of testing, which increases the cost. •ii) And it is nearly impossible to look into every bit of code to find out hidden errors, which may create problems, resulting in failure of the application.•Using White-box testing methods, the software engineer can derive test cases that:•Guarantee that all independent paths within a module have been exercised at least once.•Exercise all logical decisions on their true and false sides.•Execute all loops at their boundaries and within their operational bounds.•Exercise internal data structures to ensure their validity.

Use of White box testing•Why are we using White-box testing?•Logical errors and incorrect assumptions: Errors tend to creep when we design and implement function, conditions, or control that are out of the mainstream•We often believe that a logical path is not likely to be executed when , in fact, it may be executed on a regular basis•Typographical errors are random.• White Box Testing Techniques•The white box testing techniques we are going to study are:

•Basis Path Testing•Graph Matrix (for self study)Pg134- 136—mustafa book•Flow graph•Control Structure Testing

Flow Graph

1

2,3

6

7

9

8

4,5

10

R2

R1

R4

Node

RegionR3

Edge

11(B)Flow Graph

Control Structure Testing•Condition Testing

•Data Flow Testing•Loop Testing•Condition Testing•This method tests for logical conditions contained in a program module. A simple condition is a Boolean variable or a relational expression. A relational expression takes the form• E1 <relational-operator> E2•Where, E1 and E2 are arithmetic expressions• Relational-operator is one of the following: <, <=, >, >=, <> or =•A compound condition is consist of two or more simple conditions, boolean operators and parenthesis.•Boolean operators are OR, AND and NOT.•The possible types of errors in a condition includes any of the followings:•Boolean operator error (incorrect/ missing/ extra operator)•Boolean variable error•Parenthesis error•Relational operator error•Arithmetic expression error•The condition testing method focuses on testing each condition in the program. This technique enables also to test for other errors in a program.

Condition testing•Possible relational operators are >, =, <•Example 1:• C1: B1 & B2 where B1 and B2 are boolean variables.•The condition constraint for C1 is (D1, D2) where each D is either t or f.•The constraint set for C1 is { (t, t), (t, f), (f, t)}•Example 2:• C2: B1 & (E3 = E4) where B is a boolean variable and each E an arithmetic expression•The condition constraint for C2 is (D1, D2) where each D1 is either t or f and D2 is = for t and < , > for f •Thus constraint set for C2 is { (t, =), (t,<), (t,>), (f,=) }•Example 3:• C3: (E1 > E2) & (E3 = E4)•The condition constraint for C3 is (D1, D2) where each D1 is either t for > or f for < , = and D2 is = for t and < , > for f •Thus constraint set for C3 is { (>, =), (>, >), (> ,<), (< , =), (= ,=) }

Data Flow Testing•This method selects test paths of a program according to the locations of definitions and uses of variables in the program.•Here, each statement is assigned a unique number. We define• DEF(S) = {X| statement S contains a definition of X}•USE(S) = { X| statement S contains a definition of X}•Consider the following example-•proc example• B1;• do while C1• if C2• then• if C4• then B4;• else B5;• end if;• else• if C3• then B2;• else B3;• end if;• end if;• end do;• B6;•end proc example;

Loop testing•Simple•Nested•Concatenated•Unstructured•Pg.145-148 Mustafa book diagrams and explanation

Black Box testingGraph-Based testing method (control Flow graph )•The software testing begins by creating a graph of important objects and their relationships. And then devising a series of tests that will cover the graph so that each object and relationship is exercised and errors are uncovered.•This method involves creating a graph i.e. a collection of nodes that represent objects, links that represent the relationships between objects, node weights that describe the properties of a node and link weights that describe some characteristic of a link.•Equivalence Partitioning•Equivalence partitioning strives to define a test that uncovers classes of errors.•An equivalence class represents a set of valid or invalid states for input conditions.•Typically, an input condition is either a specific numeric values, Range of values,a set of related values or A Boolean condition•Ex. For a software that computes square root of an input integer which can assume values between 0 and 5000. there are 3 equivalence classes:- •the set of negative integers, the set of integers between 0 and 5000 and integers larger than 5000. hence test cases must include the values from each of the equivalence classes and a not a possible test set can therefore be: •{-5,500,5500}

Boundary Value Analysis•BVA use to find the errors at the boundaries of the input domain.•Boundary Value Analysis leads to a selection of test cases that exercise boundary values.•BVA also derives test cases from the input as well as output domain.

•Guidelines for BVA Partitioning:•If ‘a’ & ‘b’ are boundary value then test should be just above & below a & b respectively.•If input condition specifies a number of values, test cases should be developed that use minimum & maximum number. Values just above & below of maximum and minimum are also tested.•The above two guidelines are applied to output conditions also

•If internal data structures have prescribed boundaries (e.g. an array of size 100), a test case to exercise the data structure at its boundary.•Ex. For a function that computes the square root of integer values in the range between 0 and 5000,the test cases must include the following values:- {0,-1,5000,5001}

Mutation testing (Diag.pg150)•It’s a fault based technique used to test quality of the test suite•It is techq.for unit testing s/w that is powerful, is computationally expensive•The principal cost of mutation is that many variants of the test program called mutants must be executed repeatedly•Mutating(altering) some stmt of source code and checking if test code is able to find the errors •In this, the set of mutation operators are given, if a test set kills the mutants generated by these operators then it is finding the real faults•During this, many versions of a program are created, created versions are mutated to seed a single fault•The mutant pgm generated is run against the test cases.•The goal is to get each faulty version fail,each time a test case causes the faulty version to fail, the mutant is called killed mutant•Pg 149-150

Test Plan•It is a general document for the entire project that defines the scope ,approach to be taken and schedule of testing as well as identifies the test items for the entire testing process and the personnel responsible for diff.activities of testing•The test plan can be done before the actual testing starts, can be done along with coding and design activities•The i/p for test plan are:-

•Project plan•Requirements doc•System design docA test plan should contain the following:-

otest unit specification

ofeatures to be testedoapproach for testing

otest deliverablesoschedule and task allocation

Test unit ,features to be tested,approach for testing,test deliverables,schedule

• Test unit is a set of one or more modules, together with associated data, that are from a single comp.prgm and that are the objects of testing. It may be a module,a few modules and a complete system.

• Testability of a unit is v.imp factor. It should be such that it can be easily tested.

• To form test cases and execute the unit without much effort with these test cases

• Features to be tested include all s/w features and combinations of features that should be tested

• A s/w feature is s/w characteristic specified/implied by the requirements or design docs

• Approach for testing specifies the overall approach to be followed in the current project

• This is used to judge the testing efforts. This is called as testing criterion for evaluating set of test cases used in testing

• Testing deliverables should be specified in the test plan before actual testing begins. Deliverables could be a list of test cases used, detailed results of testing including the list of defects found, test summary report and the data about the code coverage

• Test summary report summarizes the results of the testing activities and evaluates the results; defines the items tested, the environment in which testing is done and summary of defects found

Test procedure specification•The steps to be performed to execute test cases are specified in separate document called test procedure specification•Various o/p are produced as a result of test case execution for the unit under test.•The most common is test summary report and error report. •Test summary report is meant for project mgmt, where summary of entire test execution is provided. The summary gives tot.no.of test cases executed, the number and nature of errors found, and a summary of metrics data collected•Testing effort is the total effort actually spent by the team in testing activities,and is an indicator of whether sufficient testing performed or not•Testing requires careful monitoring, as it consumes max.effort and has a great impact on final quality•The estimated effort is used for monitoring•Computer time consumed during testing is another measure which gives valuable info. to project mgmt, how much time given for testing other than coding•The error report gives the list of all defects found. To facilitate reporting and tracking of defects found during testing , defects found must be properly recorded(by using tools)

All the best……..