SOFTWARE QUALITY ASSURANCE

Lecture 3Lecture 3

Instructor: Mr. Natash Ali Mian

Department of CS and ITDepartment of CS and ITThe University of Lahore

`

Switch off mobile phones during lectures or put them into silent modelectures, or put them into silent mode

CONTENTS

Term PaperpCharacteristics of good and bad qualityProject Management and Software QualitySQA PlanEconomics of Software QualityQuality Measurements

Please Obey Traffic Signals

KEEP YOUR SURROUNDING CLEAN

TERM PAPER

Finalize Group Members 26-Feb-2013

Finalize Topic 12-Mar-2013S h P d S t S l t d (At l t 15) 19 M 2013Search Papers and Sort Selected (At least 15) 19-Mar-2013Go Through the Abstract and Introduction of Selected Papers 26-Mar-2013Submit a Summary and Comments on related papers TBDSubmit Initial Draft TBDFinal Paper Submission TBDFinal Presentation TBD

Please note that Every Phase has Marks

How to find Topics???What are the latest

h d ?research trends?

CHALLENGES

ChallengesgDealing with increasing complexity of software processes and productsDefining improved quality modelsIntegrating better with development processesIncreasing the evidence that standards that promote higher quality software bring indeed significant benefitsFinding out ways to compose standardsImproving QA models and processes Increasing software quality-centric adaptabilityg q y p yIncreasing SQA independence from software developmentDealing with the perception that QA is less glamorousDealing with the perception that QA is less glamorous than development

TRENDS IN SOFTWARE QUALITY

Metrics and measurementsMethods and toolsTesting effectiveness and code coverageInspections, reviews, and walkthroughsS fSoftware architecturesManagement and certification

RESEARCH DIRECTIONS

QA for COTS-based softwareQQA for agile development processesQA for open-source software Metrics for software reliability prediction100% automatic testing

fPersonal and Team software processes

Lets now move to the theory part

DEFECT PREVENTION

DEFECT PREVENTION

We do not want defects or faults to enter our work products, requirements, design, code, or other documents

We try to eliminate the error sources in defect preventionprevention

Defect prevention is very difficult to understand, study, and quantify

PHILOSOPHY OF DEFECTP 1PREVENTION - 1

If human misconceptions are the error sources, p ,education and training can help us remove these error sources

If imprecise designs and implementations that If imprecise designs and implementations that deviate from product specifications or design intentions are the causes for faults, formal

h d h l h d i imethods can help us prevent such deviations

PHILOSOPHY OF DEFECTP 2PREVENTION - 2

If non-conformance to selected processes or pstandards is the problem that leads to fault injections, then process conformance or standard enforcement can help use prevent the injection of enforcement can help use prevent the injection of related faults

If certain tools or technologies can reduce fault injections under similar environments, they should be adoptedshould be adopted

EDUCATION AND TRAINING - 1Education and training provide people-based g p p psolutions for error source elimination

The people factor is the most important factor that determines the quality and, ultimately, the success or failure of most software projectssuccess or failure of most software projects

EDUCATION AND TRAINING - 2Education and training of software professionals g pcan help them control, manage, and improve the way they work

FOCUS OF EDUCATION & TRAINING

Product and domain specific knowledgep gSoftware development knowledge and expertiseKnowledge about Development methodology, technology, and toolsDevelopment process knowledge

PRODUCT AND DOMAIN SPECIFICKKNOWLEDGE

If the people involved are not familiar with the p pproduct type or application domain, there is a good chance that wrong solutions will be implementedimplemented

FORMAL METHODS

Formal methods provide a way to eliminate p ycertain error sources and to verify the absence of related faultsF l th d i l dFormal methods include

Formal specificationFormal verification

FORMAL SPECIFICATION

Formal specification is concerned with producing p p gan unambiguous set of product specifications so that

t i t customer requirements, environmental constraints design intentions,

are correctly reflected, thus reducing the chances f id t l f lt i j tiof accidental fault injections

FORMAL VERIFICATIONS

Formal verification checks the conformance of software design or code against these formal specifications,

thus ensuring that the software is fault free with respect to its formal specificationsp p

ROOT CAUSES OF POOR SOFTWAREQUALITY

ROOT CAUSES OF POOR SOFTWAREQUALITY - 1

Inadequate training of managers and staffq g g

Inadequate defect and cost measurement

Excessive schedule pressure

ROOT CAUSES OF POOR SOFTWAREQ 2QUALITY - 2

Insufficient defect removal

High complexity levels

Ambiguous and creeping requirements and d i (f t & i i k )design (feature race & gimmicks)

INDUSTRY STATISTICS

Lets look at the status of the software industry’s yseriousness of the software industry with respect to the software quality assurance

QUALITY ASSURANCE ORGANIZATIONS

No quality assurance 60%q yToken quality assurance 20%Passive quality assurance 15%Active quality assurance 5%

POINT TO REMEMBER

There is another point that must be remembered pthat software varies from industry to industryThe focus on software quality naturally is d d t th i d t ll th dependent on the industry, as well as the importance of the software application. More critical applications, naturally, need to More critical applications, naturally, need to have higher software quality than others

SOFTWARE QUALITY IN SIXSUB- INDUSTRIES

Systems software that controls physical devicesy p yInformation systems that companies build for their own useOutsource or contract software built for clientsOutsource or contract software built for clientsCommercial software built by vendors for lease or saleMilitary software built following various military standardsEnd-user software built for private use by End user software built for private use by computer literate workers or managers

CHARACTERISTICS OF QUALITYLLAGGARDS

We’ll now discuss the characteristics of companies, which produce poor quality software

QUALITY LAGGARDS - 1No software quality measurement program of q y p gany kindNo usage of formal design and code inspectionsNo knowledge of the concepts of defect potentials and defect removal efficiency

QUALITY LAGGARDS - 2Either no quality assurance group or a group that q y g p g pis severely understaffed

No trained testing specialists availableNo trained testing specialists available

Few or no automated quality assurance toolsq y

No quality and reliability estimation capability

QUALITY LAGGARDS - 3Minimal or no software project management tools p j gavailable

No automated risk assessment or avoidance capability

From a low of one to a high of perhaps four distinct testing stages

QUALITY LAGGARDS - 4No test library or test-case management tools y gavailable

No complexity analysis tools utilized

D f t t ti l i th 6 d f t Defect potentials averaging more than 6 defects per function point

QUALITY LAGGARDS - 5Defect removal efficiency averaging less than y g g80%

Executive and managerial indifference (and ignorance) of quality matters

OTHER RELATED ISSUES

Staff morale and voluntary attritiony

Market shares and competitive positioning

Litigation and product recalls

QUALITY LAGGARDS

Quality laggards are the companies with highest Q y gg p gprobability of cancelled projects, several schedule overruns, and severe overrunsIt i i id th t ft It is no coincidence that software groups among the quality laggards also tend to be candidates for immediate replacement by outsource organizations

CATEGORIES OF QUALITY ATTRIBUTES

Product-specific quality attributesp q y

Organization-specific quality attributes

PRODUCT-SPECIFIC ATTRIBUTES - 1Ease of use

Documentation

Defect tolerance

Defect frequency

PRODUCT-SPECIFIC ATTRIBUTES - 2Defect impactp

Packaging

Price versus reliability

Performance

ORGANIZATION-SPECIFIC ATTRIBUTES

Service and supportppInternal processes

ACHIEVING HIGH LEVELS OF SOFTWAREQUALITY - 1

Enterprise-wide quality programsp q y p gQuality awareness and training methodsQuality standards and guidelinesQuality analysis methodsQuality measurement methods

ACHIEVING HIGH LEVELS OF SOFTWAREQUALITY - 2

Defect prevention methodspNon-test defect removal methodsTesting methodsUser-satisfaction methodsPost-release quality control

BEST IN CLASS QUALITY RESULTS - 1Quality measurementsQ yDefect preventionDefect and quality estimation automationDefect tracking automationComplexity analysis tools

BEST IN CLASS QUALITY RESULTS - 2 Test coverage analysis toolsg yFormal inspectionsFormal testing by test specialistsFormal quality assurance groupExecutive and managerial understanding of

litquality

FINAL WORD

Reductions in total defect potentials using methods of defect prevention

Improvements in cumulative removal efficiency levelslevels

PROJECT MANAGEMENT APPROACHES ANDHIGH SOFTWARE QUALITY - 1

Use of automated project estimation methodsp jUse of automated project planning methodsUse of early and automated estimates of software defect potentialsdefect potentialsUse of early and automated estimates of software defect removal efficiencyFormal risk-analysis

48

PROJECT MANAGEMENT APPROACHES ANDHIGH SOFTWARE QUALITY - 2

Provision of adequate time for pre-test q pinspectionsHistorical quality data from similar projects availableavailableMilestone tracking automated and thoroughDefect tracking automated and thoroughManagement focus concentrated on achieving excellent results

49

PROJECT MANAGEMENT APPROACHES ANDPOOR SOFTWARE QUALITY

Exact opposite of the project management pp p j gapproaches correlating with high software quality

50

SQA GROUP - 1

Every company, which wants to establish Every company, which wants to establish a reputation for producing high quality software, must establish a Software Quality Assurance (SQA) Group within the companyThis groups must be funded properly and management must pay attention to the reports and presentations made b this reports and presentations made by this group

51

SQA GROUP - 2The SQA group report directly to the higher Q g p p y gmanagement and not to the project managementThe personnel of the SQA group must work with th j t t t d i t the project management team, and vice versa to produce high quality software for the company –which is the ultimate goal

52

The SQA group is needed to monitor the quality Q g p q yassurance-related activities in a company

53

SQA GROUP’S ACTIVITIES - 1Preparation of an SQA plan for a projectp Q p p jParticipation in the development of the project’s software process descriptionReview of software engineering activities to verify compliance with the defined software processprocess

54

SQA GROUP’S ACTIVITIES - 2Audit of designed software work products to g pverify compliance with those defined as part of the software process

55

SQA GROUP’S ACTIVITIES - 3Ensure that deviations in software work and work products are documented and handled according to a documented procedureR d li d t t i Record any noncompliance and reports to senior management

56

SQA PLAN - 1Evaluations to be performedp

Audits and reviews to be performed

Standards that are applicable to the project

Procedures for error reporting and tracking

57

SQA PLAN - 2Documents to be produced by the SQA groupp y Q g p

Amount of feedback provided to the software project teamproject team

58

SOFTWARE QUALITY PERSONNEL

Unfortunately are under-paidy pUsually are let go first in times of crisis“Top-gun” SQA personnel and managers with proven track record are in high demand from companies that have active QA programs

59

COSTS OF SOFTWARE QUALITY - 1 Defects prevention costspUser satisfaction optimization costsData quality defect prevention costsData quality defect removal costsQuality awareness/training costsNon-test defect removal costsTesting defect removal costs

60

COSTS OF SOFTWARE QUALITY - 2Post-release customer support costsppLitigation and damage award costsQuality savings from reduced scrap/reworkQuality savings from reduced user downtimeQuality value from reduced time-to-market i t lintervals

61

COSTS OF SOFTWARE QUALITY - 3Quality value from enhanced competitivenessQ y pQuality value from enhanced employee moraleQuality return on investment

62

ECONOMICS OF SOFTWARE QUALITY - 1High quality software applications have shorter g q y ppdevelopment schedules than low quality applications because they do not get hung up in integration and testing due to excessive defect integration and testing due to excessive defect levels

63

ECONOMICS OF SOFTWARE QUALITY - 2High quality software applications have lower g q y ppdevelopment and maintenance costs than low quality applications. This is because the cumulative costs of finding and fixing bugs is cumulative costs of finding and fixing bugs is often the major cost driver for software projects

64

ECONOMICS OF SOFTWARE QUALITY - 3High quality software applications have better g q y ppreliability levels and longer mean times to failure than low quality applicationsHi h lit i l ft k h High quality commercial software packages have larger market shares than low quality commercial software packages

65

ECONOMICS OF SOFTWARE QUALITY - 4High quality software achieves better user-g q ysatisfaction ratings than low quality softwareHigh quality software projects score better on

l l th d l lit employee morale surveys than do low quality software projects

66

ECONOMICS OF SOFTWARE QUALITY - 5

High quality software produced under High quality software produced under contract or an outsource agreement has a much lower probability of ending up in court for breach of contract or malpractice litigation than low quality softwareHigh quality software benefits or augments the performance levels of users, while poor quality tends to degrade while poor quality tends to degrade worker performance

67

ECONOMICS OF SOFTWARE QUALITY - 6Poor quality software can trigger truly massive q y gg yunplanned expense levels. Denver airport example

68