02 principles-of-testing
Transcript of 02 principles-of-testing
![Page 2: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/2.jpg)
Understanding the role of testing vis-à-vis other life cycle activities w.r.t quality
Getting an appreciation of the basic principles of testing
Basis Material for this session: Software Testing – Principles and Practices, Srinivasan Desikan and Gopalaswamy Ramesh, Pearson Education, 2006-2007
2
Today’s Agenda
![Page 3: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/3.jpg)
Requirements gathering Design Development Testing Deployment and maintenance
3
Phases of software development (not necessarily
sequential!)
![Page 4: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/4.jpg)
4
Quality during various phases…
Eff
ecti
ven
ess
of
Tes
tin
g p
has
e
LOW
HIGH
Quality of other phases
LOW HIGH
Not sustainable!
High Rework Costs Ideal State!
Risky!
![Page 5: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/5.jpg)
5
The car is complete…You just have to paint it!
![Page 6: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/6.jpg)
6
This car can accelerateFrom 0 to 60 in 5 secs Yeah, but it accelerates
when I press the brake!
![Page 7: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/7.jpg)
A product should meet the customers’ needs Customers need not be “external” customers
Testers are customers for developers A software should be “testable”
7
Principle #1
Testing should find out defects before customer finds them out!
![Page 8: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/8.jpg)
Practice of incorporating customer supplied test cases into battery of tests
Including customers and / or “customer facing” into the testing functions
Concept of Alpha / Beta testing
8
Principle #1:Implications
![Page 9: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/9.jpg)
Remember the “triangle” problem? Remember the last class “6 character
problem”?
9
Principle #2
Dijkstra’s Doctrine:Program Testing can never prove the absence of bugs,
Only their presence
![Page 10: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/10.jpg)
Choose your tests intelligently Choose tests that have a higher probability
of uncovering defects Boundary Values, Choice of regression tests
10
Principle #2:Implications
![Page 11: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/11.jpg)
11
Principle 3:
Correct Reqmts.
Defects in reqmts.
Correct design
Defects in reqmts
Defects in design
Correct code
Defects in reqmts
Defects in design
Defects in code
Defects found
Defects in reqmts
Defects in design
Defects in code
Defects not found
Testing Phase
Coding Phase
Design Phase
RequirementPhase
Testing Phase
Coding Phase
Design Phase
RequirementPhase
![Page 12: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/12.jpg)
Reqmts Design Coding Testing Post release
10x
100x
1000x
12
Principle #3
Defects should be detected and corrected at the earliest –Postponing detection of defects compounds cost!
![Page 13: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/13.jpg)
Different types of testing interspersed during product cycle
Different methods of integration Concept of Regression Testing
13
Principle #3: Implications
![Page 14: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/14.jpg)
14
Principle #4:The Cat and the Saint
Knowing why you are testing something is as important asKnowing what you are testing
![Page 15: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/15.jpg)
White Box Testing Starting from code
Black Box Testing Starting from external functionality
Integration Testing Stuff works together!
System / Performance Testing Works in real life conditions
Regression Testing Makes sure changes don’t affect existing stuff
Domain / adhoc testing Bringing domain expertise into testing
15
Principle #4:Implications
![Page 16: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/16.jpg)
16
Principle #5
Test the tests first!A defective test is even more dangerous than a defective product!
![Page 17: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/17.jpg)
A test should document the Input data expected results the test process
The behavior has to be externally corroborated No Turing Machine can verify itself!
17
Principle #5:Implications
![Page 18: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/18.jpg)
18
Principle #6:The Pesticide Paradox
Bugs are like bacteria – they develop resistance to antibiotics;New antibiotics have to be developed!
![Page 19: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/19.jpg)
Need to redefine and refine tests as we move forward
As we “pass” old tests, new defects will surface
Regression Tests have to be “retired” and new regression tests have to be designed
19
Principle #6:Implications
![Page 20: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/20.jpg)
20
Principle #7:The Rags and The Convoy
![Page 21: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/21.jpg)
Number of defects found already
Def
ects
yet
to
be
fou
nd
21
Principle #7:
Defects come in convoys ; Fixing a defect in the convoy is likely to add more defects
![Page 22: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/22.jpg)
Look for side effects Look for “spaghetti code” or “rags” Tie maintenance and testing functions
closely
22
Principle #7:Implications
![Page 23: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/23.jpg)
23
Principle #8:Cop on the Bridge
Prevention is better than cure –Prevention can open substantially new vistas
![Page 24: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/24.jpg)
Quality Assurance Process oriented Defect Prevention oriented Proactive Staff function
Quality Control Product oriented Defect detection / correction oriented Reactive Line function
24
Prevention and Cure:Quality Assurance Vs Quality Control
![Page 25: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/25.jpg)
Double edged sword!
Excessive Process orientation
Lack of Testing makes defects reach the customers
Not a healthy state!
Lack of standards foster “defect breeding”
Lack of Testing makes defects reach the customers
May be resource intensive but gives better payback
Institutionalizes Quality
Makes Quality visible to customers
Last minute rushes
Higher People dependency
Testers as “heroes” and “adversaries”
Double edged sword!
Excessive Process orientation
Lack of Testing makes defects reach the customers
Not a healthy state!
Lack of standards foster “defect breeding”
Lack of Testing makes defects reach the customers
May be resource intensive but gives better payback
Institutionalizes Quality
Makes Quality visible to customers
Last minute rushes
Higher People dependency
Testers as “heroes” and “adversaries”
Low High
Low
Hig
h
Defect prevention focus
Def
ect d
etec
tion
foc
us
25
Principle #8 leads to Principle #9:
![Page 26: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/26.jpg)
Defect Detection
Defect Prevention
26
Principle #9:End of the Pendulum
![Page 27: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/27.jpg)
Achieve a balance between (Defect Prevention) Quality Assurance and (Defect Detection) Quality Control
View QA and QC as supplementary and not adversary job functions
Distribute QA / QC functions throughout for early detection (Principle #1)
27
Principle #8 and #9:Implications
![Page 28: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/28.jpg)
The IBM experiment (Peopleware, DeMarco & Lister)
Issues of “pride” we discussed last class
28
Principle #10:The Men in Black
![Page 29: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/29.jpg)
Instilling pride in testers Establishing an identity for testers Showing them a career path
29
Principle #10:Implications
![Page 30: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/30.jpg)
30
Principle #11:Automation Syndrome
Focus on WHAT you want to automate;Relate it to the goals;
Automation is not a panacea
![Page 31: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/31.jpg)
#1: Goal of testing is to find defects BEFORE customers find them
#2: Program testing can only show presence of bugs, never their absence
#3: Test early and often #4: Understand the “why” and not just the “what” / “how” #5: Test the tests first #6: Tests develop immunity – discover new tests! #7: Focus on convoys of defects #8: Testing encompasses defect prevention #9: Strike a fine balance between defect prevention and
defect detection #10: Plan the automation initiatives #11: Instill pride in testers
31
Putting it all together…
![Page 32: 02 principles-of-testing](https://reader033.fdocuments.net/reader033/viewer/2022061401/5583cd5ad8b42a7c758b5222/html5/thumbnails/32.jpg)
Understanding the role of testing vis-à-vis other life cycle activities w.r.t quality
Getting an appreciation of the basic principles of testing
32
Today’s Agenda Recap