Post on 29-Dec-2015
slide 2COMP319 S Coope 2011
ObjectivesObjectives
• At the end of this lecture you should be able to;– list and describe factors which contribute to software quality– name and explain the processes used to ensure software quality
slide 3COMP319 S Coope 2011
QualityQuality“Characteristics of the software and/or its supporting documentation which make it suitable for its
purpose".
slide 4COMP319 S Coope 2011
Product design and conformance
Product design and conformance
• Design– characteristics of the materials and processes which have been specified for the system– includes all aspects of specification and software design
• Conformance– degree to which design specifications are followed during manufacturing– code implementation
slide 5COMP319 S Coope 2011
DeMarco’s QualityDeMarco’s Quality“A products quality is a function of how much it changes the world for the better”
slide 6COMP319 S Coope 2011
Quality controlQuality control
Productiondesign
process 1
Productiondesign
process 2
ReviewSample
TestMeasure
slide 7COMP319 S Coope 2011
Software production quality cost
Software production quality cost
• Each individual product needs producing only once• Benefits
– economy of scale (sometimes)
• Drawbacks– all products from one release will contain same defects– implementation details may change radically from product to product– statistical sampling techniques not applicable
slide 8COMP319 S Coope 2011
Software verses other manufacturing
Software verses other manufacturing
• Manufacturing industry– design once, make many– production process refined– design faults fed back on-going basis
• Software industry– design once, make once
slide 9COMP319 S Coope 2011
Component quality conceptComponent quality concept• In manufacturing
– electronics, cars, aviation– components made to specification and determined to be of high quality– build product out of blackbox components
• In software– components conform to interface specification– MS Component Object Model– Sun Java Beans
slide 10COMP319 S Coope 2011
Quality and reuseQuality and reuse
High reuse
should
improve quality
as code needs
to only be
tested or
designed once
only name : stringaddress : stringdob : date
Person
setAddress(String)print()
account_number :Long
Customer
pin : Integer
Bank accountholder
checkPin()
points : Integer
Clubcardholder
printStatement()
allergies: String
Patient
prescription()
Bank Supermarket Clinic
slide 11COMP319 S Coope 2011
Need for quality control Cost of correcting errorsNeed for quality control
Cost of correcting errors• IBM research
– 7053 hours inspecting 200,000 lines of code, 3112 potential defects discovered– total cost $282,000– cost per defect (at $40/hour) $91/defect– IBM estimate for correcting defects in the field is $25,000– total cost of 3112 defects $5 million
slide 12COMP319 S Coope 2011
Reliability EstimationReliability Estimation• Calculating probability of fault over time• Why do we need it?
– schedule maintainance– commercial (costing for warrantees, support contracts)– scheduling software release dates– determining safety of critical systems
slide 14COMP319 S Coope 2011
Reliability ProblemsReliability Problems• Therac 25• Airbus 320
– altimeter reading issues
• Patriot defence missile failure– Scud missile killed 28
– 0.000000095/0.1 second over 100 hours
slide 15COMP319 S Coope 2011
Reliability modellingReliability modelling
• Defect density models– analyses structure/size of software– input/outputs/complexity– can be determined before execution
• Reliability growth models– test performance– execution time– correlates defect density over time
slide 16COMP319 S Coope 2011
Reliability growth modelReliability growth model• Parameters
– Total bugs in software– Total bugs discovered– Time– Residual bugs
slide 19COMP319 S Coope 2011
Growth model examplesGrowth model examplesA = a(1-e-bt)
– Goel-Okumoto or Musa model– Concave a=total defects
A = a( 1-(1+bt)e-bt) – Yamada,83 Modification of G-O model
• A= (l/c)ln(ct+l)– Infinite poisson Musa
slide 20COMP319 S Coope 2011
Reliability growth model assumptions
Reliability growth model assumptions
• Defects are repaired immediately– Defects are not repaired immediately
• Defect repair doesn't introduce new defects– Generally not true
• Test cases represent operation profile of software– hard to duplicate in practise, need for alpha release phase
• Failures are independent
slide 21COMP319 S Coope 2011
Using a modelUsing a model• Parameter estimation
– maximum likelihood– least squares
• Maximum likelihood– Given a set of data– Which distribution gives us the highest probability of achieving this data
slide 22COMP319 S Coope 2011
Maximum liklihood exampleMaximum liklihood example• A coin is tossed 10 times• Chance of getting N heads is
– (10!/(N!(10-N)!) x (phead)^N x (1-phead)^(10-N)
• So given we get 3 heads– P=(10!/(3!(7)!) x (phead)^3 x (1-phead)^7
• What is the chance of– phead being 0.1, 0.2, 0.3 etc.
• We can define these as – P(3 | 0.1) P(3 | O.2) P(3 | 0.3) or generally– P(3 | p)
slide 23COMP319 S Coope 2011
Maximum liklihood exampleMaximum liklihood example• Now imagine we have the following data (number of heads)
– 2,2,2,3,3,5
• We end up with the following data– P(2 | p) P(2 | p) P(2 | p) P(3 | p) P(3 | p) P(5 | p)
• We can multiply all those probabilities together (since they are independent)– P(2 | p) x P(2 | p) x P(2 | p) x P(3 | p) x P(3 | p) x P(5 | p)
• We then solve for max prob. Against p
slide 24COMP319 S Coope 2011
Least Squares (regression)Least Squares (regression)• In this approach the parameters are chosen which minimise the error squares• (F(a,b,t)-f(t))^2
slide 25COMP319 S Coope 2011
Useful estimationUseful estimation• Stable
– not vary widely in terms of total bugs estimated week per week
• Give a good indication of residual errors after release– release conditions can be very different than test conditions
slide 26COMP319 S Coope 2011
Case study (Tandem)Case study (Tandem)• Tandem
– fault tolerant computer systems (now part of HP)
• Results– Release Predicated faults Faults year 1– 1 33 34– 1/2 33 28– 3 10 9
slide 27COMP319 S Coope 2011
Tandem conclusionsTandem conclusions• Simplest models are the best• Execution time is the only time measure worth using• Reliability growth is a useful measure• Problem reports are a good substitute for actual defects counts (problem reports can include duplicates)• Data can be analysed on a weekly basis
slide 28COMP319 S Coope 2011
McCall's software quality factors
McCall's software quality factors
ProductRevision
Producttransition
Product operations
slide 29COMP319 S Coope 2011
Product operationsProduct operations• Correctness
– conformance to specification (verification)– conformance to user requirements (validation)– conformance to module specification
• Reliability– results produced to expected precision in all circumstances
• Efficiency– efficient use of hardware resource
slide 30COMP319 S Coope 2011
Product operationsProduct operations
• Integrity– keeps data secure– commonly conflicts with functionality requirements
• Useability– easy to use– intuitive interface
slide 31COMP319 S Coope 2011
Product revisionProduct revision• Maintainability
– can the software be fixed at a later date
• Flexibility– can the software be modified to perform new or different functions
• Testability– can the software be tested
slide 32COMP319 S Coope 2011
Product transitionProduct transition• Portability
– can the software be used on another hardware or software platform
• Reusability– can the software be used to produce new software
• Interoperability– will we be able to interface to another system
slide 33COMP319 S Coope 2011
Evaluating and assuring quality
Evaluating and assuring quality
• Product operations– testing (user, system, security, module, stress, usability)
• Possible metrics– faults per line of code/module– mean time between software failure– time to perform operations (usability)– mean time between break ins (security)– mean response time under stress
slide 34COMP319 S Coope 2011
Product revisionquality control
Product revisionquality control
• Maintainability– review (documentation, code complexity)
• Flexibility– where to put parameters– review internationalisation issues
• Testability– reduce complexity– encapsulate data (OO paradigm)
slide 35COMP319 S Coope 2011
Flexibilityconfiguration parameters
Flexibilityconfiguration parameters
• Hardcoding– very bad practise
• Using #defines or constants– better, good for system constants
– more control than configuration file
– requires recompilation
• Configuration file– highest flexibility
– allows users to break software !!
slide 36COMP319 S Coope 2011
ExamplesExamplesExample 1 Hard coding
Account accounts[]=new Account[20];
for (j=0;j<20;j++) {
accounts[j]=file.readAccount();
}
Example 2 pre-processor definition
#define MAX_ACCOUNTS 20
Account accounts[]=new Account[MAX_ACCOUNTS];
for (j=0;j<MAX_ACCOUNTS;j++) {
accounts[j]=file.readAccount();
}
slide 37COMP319 S Coope 2011
Example (configuration file)Example (configuration file)Example 3 configuration file
int max_accounts;
readConfig(&max_accounts,"config.dat","maxaccounts");
Account accounts[]=new Account[max_accounts]
for (j=0;j<max_accounts;j++) {
accounts[j]=file.readAccount();
}
slide 38COMP319 S Coope 2011
TransitionTransition• Portability
– effected by language, software library support– note C does not have a standard size for int
• Reusability– code should be generalised as much as possible (including variable naming)– e.g. replace pin with authorisation_code– important to de-couple I/O from processing
slide 39COMP319 S Coope 2011
PortabilityPortability• Approaches
– use wrapper classes– define UI using XML– use plain old Java classes– use model/view/controller seperation
slide 40COMP319 S Coope 2011
TransitionTransition• Interoperability
– use standards e.g. Internet standards, JDBC, HTTP, XML– test against standard products e.g. browsers– next use in-house protocols if standard technique available
slide 41COMP319 S Coope 2011
Software reviewsSoftware reviews• Review functionality of system
– by development team– specification team– with customer (validation)– beta release to customer
slide 42COMP319 S Coope 2011
FTRsFormal technical reviews
FTRsFormal technical reviews
• Purpose– uncover defects (code, spec, design)
• Before– agenda, attendance list, documentation
• While– walkthrough, code inspection
• After– hit list, next review meeting scheduled
slide 43COMP319 S Coope 2011
Specification reviewSpecification review• Is it
– clear and unambiguous– understandable– free of errors– complete– what the customer wants
slide 44COMP319 S Coope 2011
Design reviewDesign review• Check
– is it broken up enough– in agreement with specification– uses correct notation– makes sense
slide 45COMP319 S Coope 2011
Code reviewCode review• Is it
– bug free– well document and formatted– uses meaningful names– flexible (easy to modify)– easy to understand/test
slide 46COMP319 S Coope 2011
Defect amplification(importance of review)
Defect amplification(importance of review)
• Error in specification– errors in design– errors in code– errors in product
• Error in design– errors in code– errors in product