Lecture11 - Software Quality
-
Upload
sarah-huang -
Category
Documents
-
view
219 -
download
0
Transcript of Lecture11 - Software Quality
-
7/23/2019 Lecture11 - Software Quality
1/26
28 May 2015 CSS 360 Spring 2015 Lecture 11
Conways Law:The structure of a computer
program reflects the structure ofthe organization that built it.
-
7/23/2019 Lecture11 - Software Quality
2/26
Outline Software quality what it is and who wants it? [discussed previously] Good enough quality
How can we measure quality?
Quality / Testing myths How, when, and what to test?
Bugs: stages and tracking over their lifetime
Test-Driven Development (TDD) with Unit tests
How can we improve software quality?
How to avert quality disasters?
28 May 2015 CSS 360 Spring 2015 Lecture 11
-
7/23/2019 Lecture11 - Software Quality
3/26
References Professional Software Development, by Steve
McConnell
lots of discussions with professionals in the fieldover the years
28 May 2015 CSS 360 Spring 2015 Lecture 11
-
7/23/2019 Lecture11 - Software Quality
4/26
Food for Thought:Quality in Different Contexts
The software Gold Rush fever periods Goal: being first-to-market in an unclaimed segment Customer base: tech savvy, willing to forgive bugs Project type: high-risk, potentially high pay-off Typical environment: two friends in a garage Processes: often code-and-fix, very informal
The in-between (post-Gold Rush) periods Goal: sustained, productive competition with others
Customer base: demands reliability Project type: lower-risk; lower, more predictable pay-off Typical environment: larger teams, formal processes Processes: careful, quality-driven development, emphasis
on quality (reliability, interoperability, usability, etc.)
28 May 2015 CSS 360 Spring 2015 Lecture 11
-
7/23/2019 Lecture11 - Software Quality
5/26
Components of Quality
Quality comprises (but is not limited to): Requirements quality Design quality
Code quality Test quality Documentation quality
These are all aspects that QA engineers work on,
during the various software lifecycle stages.
Q:Given limited resources, which of these do youconsider more important to pay attention to? Why?
28 May 2015 CSS 360 Spring 2015 Lecture 11
-
7/23/2019 Lecture11 - Software Quality
6/26
-
7/23/2019 Lecture11 - Software Quality
7/26
How Would You Test This?
28 May 2015 CSS 360 Spring 2015 Lecture 11
Activity
-
7/23/2019 Lecture11 - Software Quality
8/26
The Quality / Testing Mindset Value proposition of quality assurance and testing: providing
information to the project team on the quality of the productand associated processes Notto break the code, but to discover where the code is already
broken, and to appropriately communicate these findings The conceptual model and assumptions about quality often
differ between testers and developers (non-testers) Broken/suspicious until proven otherwise vs.
Works/clean until proven otherwise Different mindsets of testers and developers
The two groups have somewhat different incentives. looking at a project through a different set of backgrounds/skills and thus could discover complementary useful aspects
28 May 2015 CSS 360 Spring 2015 Lecture 11
-
7/23/2019 Lecture11 - Software Quality
9/26
Quality Assurance (QA) vs.Quality Control (QC) Software engineers and managers often treat QA
and QC as synonyms incorrectly! QC is reactive, after-the-fact, pure product testing. QA isproactive, provides informationto stakeholders
about the (objective) state of the project, the risks, andhow they may be mitigated/improved
QA looks holisticallyat the developmentprocess, and Proposes changes to improve it, so problems don't happen or recur
When problems happen, helps to discover them as close to thesource as possible (i.e., quickly and with lower project costs)
Overall goal of a QA organization is to: Prevent bugs as much as possible (through good processes) For bugs that are there, to discover them as early as possible (in
order to reduce costs and reduce waste)28 May 2015 CSS 360 Spring 2015 Lecture 11
-
7/23/2019 Lecture11 - Software Quality
10/26
Quality Assurance / Testing Myths
Myth 1: Developers dont own the code once itsreleased to testing.
Testers can help to ascertain the actual product quality,but developers must continue to be engaged for bug
fixes, critical decisions, code/design extensions, etc.
Myth 2: The QA organization is the bottleneckand is responsible for projects being late.
after all, arent they the last to touch the project, andto take precious project time while working on it.
Suggestion: Reviewing the value-stream mapcan helpto identify where problems originate.
28 May 2015 CSS 360 Spring 2015 Lecture 11
-
7/23/2019 Lecture11 - Software Quality
11/26
Myth 3: Automation reduces the need for othertesting and QA.
Many things cant be automated (easily).
New features come up all the time, so someone needs towrite new tests.
Automated tests are only as good as the conceptualmodel on which they were based. The same challenge as developing a proxy: how much it can
understand vs. how simple it is to develop
Automation can yield both false positives and falsenegatives.
28 May 2015 CSS 360 Spring 2015 Lecture 11
Quality Assurance / Testing Myths
(cont.)
-
7/23/2019 Lecture11 - Software Quality
12/26
Testing Approachesthroughout the Lifecycle Requirements / design / documentation inspections Acceptance / End-to-end testing System testing Integration testing Component testing Unit testing
28 May 2015 CSS 360 Spring 2015 Lecture 11
V-modelillustrates mappingto stages in the lifecycle Question:Which correspond
to verification testing andwhich to validation testing?
http://thep
cweb.com/wp-content/up
loads/2008/08/v-model.jpg
-
7/23/2019 Lecture11 - Software Quality
13/26
How Do You Prioritize
What to Test? Factors to consider:
organizational mission statement
external requirements (e.g., safety, standards, etc.) feature priority
typical areas of high risk for bugs
recently developed or major features
new technologies external dependencies
complex algorithms
28 May 2015 CSS 360 Spring 2015 Lecture 11
-
7/23/2019 Lecture11 - Software Quality
14/26
How Do You Know
When to Stop Testing? Consider the notion of good enough quality
Look at recent bug trends, including: If all high-priority test cases have passed
If there are no critical bugs left unresolved
If there have been no (very) recent code changes
(which may lead to problems if not testedthoroughly)
Hint: The above look like exit criteria.
28 May 2015 CSS 360 Spring 2015 Lecture 11
-
7/23/2019 Lecture11 - Software Quality
15/26
Main Types of Testing
Black-box testingbased on specification (inputs)
White-box testingbased on dataflow (code paths)
Automated testing / Scripted testinguseful forrepeating tests
Manual testingexpensive, infrequent, error-prone,but sometimes necessary
Ad-hoc testing / Exploratory testinguseful for quickfeedback, usually when time is very limited
Regression testingbugs that were fixed must remainfixed (or be discovered quickly, ideally automatically)
28 May 2015 CSS 360 Spring 2015 Lecture 11
-
7/23/2019 Lecture11 - Software Quality
16/26
Common Test Case Format Test case description/titlevery brief, intended for
quick identification among many other test caseswithout reading details
Specific testing instructionsdescribing how toperform the test, starting from a commonly knownplace/state
Expected outcome/resultsif the test succeeds, what
should be the results; this tells you how to identify atest case failure
Note: Including more aspects is possible, but optional.
28 May 2015 CSS 360 Spring 2015 Lecture 11
-
7/23/2019 Lecture11 - Software Quality
17/26
A Bugs Life
Throughout its lifetime, a bug can transitionbetween several states:
Open (or re-opened)Work-in-progress
External Resolved
Cant reproduce By design
Wont fix Postpone Fixed
Closed
28 May 2015 CSS 360 Spring 2015 Lecture 11
-
7/23/2019 Lecture11 - Software Quality
18/26
Tracking Bugs and Trends
Graph the number of bugs overtime, often grouped by variousproperties (e.g., open, howrecent, assigned to whom,priority, severity, etc.)
Track bug trends over the lifetimeof a project. Even if a bug isresolved, keep its record forproject tracking (and learning),and also in case it resurfaces.
28 May 2015 CSS 360 Spring 2015 Lecture 11
-
7/23/2019 Lecture11 - Software Quality
19/26
Test-Driven Developmentwith Unit Tests
1. Write a test first (!) for a given (yet unwritten) classmethodthis way, tests serve as a specification (i.e.,expressing the intent) for the class code (methods)
2. Write stubs for the relevant method(s) of a class just enough to make the test compile3. Fail the testthis crucial step ensures falsifiability.4. Make changes to the class methodsjust enough to
pass the test
5. Pass the test ensuring that step #4 was successful6. Refactor (i.e., make changes), as necessary7. Pass the test ensuring that refactoring didnt break code8. Return to step #1 to cover additional intents of the class28 May 2015 CSS 360 Spring 2015 Lecture 11
-
7/23/2019 Lecture11 - Software Quality
20/26
Unit TestingAn Example
Example using JUnit: import org.junit.*;(where the JUnit methods are)import org.junit.Test;
@Beforefor setting up a test suite (i.e., set of tests) @Testfor describing an individual test in a suite Assert.assertEquals(commentString,
intendedValue, actualValue)
[Discussing sample JUnit code shown separately]
28 May 2015 CSS 360 Spring 2015 Lecture 11
-
7/23/2019 Lecture11 - Software Quality
21/26
Best Practices for Testing
Tests must fail the first timeyou run them. to ensure that they arefalsifiable
The ideal tests are independent of each other: e.g.,
with no side effects and no special order of execution. Tests that are simple to write and quick to execute
create an incentive to be written and executed often.
Seek equivalence classesfor the domain youre testing.
Use an established testing methodology(e.g., SFDPO)rather than making up tests at random.
28 May 2015 CSS 360 Spring 2015 Lecture 11
-
7/23/2019 Lecture11 - Software Quality
22/26
Important Practices for
Good Quality Processes Perform testing in an environment separate
from the production (live, online) environment.
The QA organization must have a seat at thetable of strategic decision makers.
The hierarchy of the QA organization mustideally be independent and not answerable tothe Development organization.
28 May 2015 CSS 360 Spring 2015 Lecture 11
Activity
-
7/23/2019 Lecture11 - Software Quality
23/26
Mechanisms for Raising
the Quality of SoftwareAssume you are brought in on an ongoing software
project plagued by poor quality. What one or twoapproaches (mechanisms) would you propose to helpraise the quality of the software in production?
Make assumptions as needed, to concretize the question.
28 May 2015 CSS 360 Spring 2015 Lecture 11
Activity
-
-
h f h
-
7/23/2019 Lecture11 - Software Quality
24/26
Mechanisms for Raising the
Quality of Software: Some IdeasWhich mechanisms may help your project(s)?a. Involvement / frequent iterations with customers and other
stakeholdersb. Pair programmingc. Code reviews (not limited to code; reqs/design review, etc.)
d. External auditinge. Using automated tools (e.g., static analysis, code coverage, IDEs,
etc.) to help discover non-trivial properties that affect qualityf. Refactoringg. Code integration (if not already in place)
h. Testing: integration testing, regression testing, acceptance testing;automated testing; test-driven development (with unit testing)i. Component reuse
j. Team building activitiesk. Establish (or ensure) clear responsibilities within the teaml. Realistic up-to-date scheduling
28 May 2015 CSS 360 Spring 2015 Lecture 11
-
7/23/2019 Lecture11 - Software Quality
25/26
Perennial Recipes for Disaster(a.k.a. Poor Quality Products)
Ignore what the customers say they want developers surely must know better.
Put in all the features that could potentially ever be
useful. Do not worry about quality (and ignore related
prudent practices) until a deadline looms or someonecomplains loudly.
Do not waste time on design or documentation afterall, code is the most important thing, and time isalready short to do all that needs to be done anyway.
28 May 2015 CSS 360 Spring 2015 Lecture 11
-
7/23/2019 Lecture11 - Software Quality
26/26
Questions?
28 May 2015 CSS 360 Spring 2015 Lecture 11