Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem...
Transcript of Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem...
![Page 1: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/1.jpg)
1
Quality Tactics Session 9, Part 1
Steve Chenoweth
Phone: Office (812) 877-8974
Cell (937) 657-3885
Email: [email protected]
Comic version of soccer tactics – from a disgruntled fan.
http://nowthatisfunny.blogspot.com/2007/04/football-tactics.html
![Page 2: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/2.jpg)
2
Acknowledgements
Some of the material in these slides is
taken from Software Architecture in
Practice, 3rd edition by Bass, Clements
and Kazman. Ch 5 - 12.
![Page 3: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/3.jpg)
3
Outline
How to achieve all the attributes Bass
introduces in –
Availability
Interoperability
Performance
Testability
We expand here on the ones in red!
The other 3 – see the following slide
presentations for tonight.
Modifiability
Security
Usability
![Page 4: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/4.jpg)
4
Availability
Try one of these 3 Strategies:
Fault detection
Fault recovery
Fault prevention
And, we have a separate slide set on
this tonight, too…
![Page 5: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/5.jpg)
Interoperability
Locate
Discover Service
Manage Interfaces
Orchestrate
Tailor Interface
5
![Page 6: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/6.jpg)
Interoperability Challenges
Every implementation is standard
But every one wants to be open to special
features!
And standards
evolve
And some
standards are
bad
6
Right – CNN’s latest news reduction tactic – “Is
this a good thing or a bad thing?” Picture from
http://www.businessinsider.com/cnn-asks-the-
wire-is-wikileaks-good-or-bad-2010-11.
Interoperability, cntd
![Page 7: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/7.jpg)
The Telecom Strategy
Calling grandma - How do you make
one of these talk to one of those?
7
?
Interoperability, cntd
![Page 8: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/8.jpg)
Solution – “Half call model”
Basic Idea:
Niceties:
8
Caller
Protocol (100 varieties)
Callee
Protocol (100 varieties)
Common
Protocol
From The Basics of Telecommunications, by Stephan S. Jones.
Interoperability, cntd
![Page 9: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/9.jpg)
States Are Important?
Much of the Internet used to be “stateless” – IP
and HTTP
You issue an http request to a server
It responds
Next time, the server has no idea you are the same
requestor
Some real-time systems work this way.
There’s no time to set-up sessions.
A single message contains everything.
But…
9
Interoperability, cntd
![Page 10: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/10.jpg)
States Are Important!
But most things can’t work this way.
TCP, Telnet, FTP, …
Require initiation and termination of “sessions”
• The “handshake” to get started is complex.
• Resources get reserved.
Both sides have to know “what’s next”
• And share error handling
In OO software design, usually this amounts to filling-
in the right stuff for methods in an interface.
With REST, for example, you have to use HTTP
methods GET, POST, PUT and DELETE to execute
different operations.
10
Interoperability, cntd
![Page 11: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/11.jpg)
11
Modifiability
Try one of these 3 Strategies:
Localize changes
Prevent ripple effects
Defer binding time
See next slides for details on each
We have a whole separate course
addressing this – CSSE 575.
![Page 12: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/12.jpg)
12
Localize Changes
Maintain semantic coherence
abstract common services
Anticipate expected changes
Generalize module
Limit possible options
Modifiability, cntd
![Page 13: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/13.jpg)
13
Prevent Ripple Effects
Hide information
Maintain existing interfaces
Restrict communication paths
Use an intermediary
Ripple in action. From
http://www.crystalinks.com/rippleffect.html
Modifiability, cntd
![Page 14: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/14.jpg)
14
Defer Binding Time
Runtime registration
Configuration files
Polymorphism
Component replacement
Adherence to defined protocols
Modifiability, cntd
![Page 15: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/15.jpg)
15
Performance
Try one of these 3 Strategies:
Resource demand
Resource management
Resource arbitration
Tonight’s next set of slides has more
details on each of these
![Page 16: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/16.jpg)
16
Security
Try one of these 3 Strategies:
Resisting attacks
Detecting attacks
Recovering from an attack
See next slides for details on each
And, we have a separate elective
course addressing this – CSSE 542.
![Page 17: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/17.jpg)
17
Resisting Attacks
Authenticate users
Authorize users
Maintain data confidentiality
Maintain integrity
Limit exposure
Limit access
Security, cntd
![Page 18: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/18.jpg)
18
Detecting and Recovering
Intrusion detection system -
anomalous patterns
Restoration - see availability tactics
Identification - maintain audit trail
Security, cntd
![Page 19: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/19.jpg)
19
Testability
Strategies:
Manage Input/Output Record/playback
Separate interface from implementation
Specialize access
Internal monitoring Built-in monitors
And, our required CSSE 576 expands on how to build this into a system.
Right – Typical tool for
reading out error codes
logged by your car’s
computer, to help analyze its
problems. From
http://niceparts.blogspot.co
m/2007_12_01_archive.html
.
![Page 20: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/20.jpg)
What’s Bass say about this QA?
Problem – Testing needs to be more efficient & effective
It consumes a high percentage of software development time & effort
The heuristic is “half”
Goal – Allow easier testing when an increment of software development is completed.
Motivation – The overall arch of the system, and of its testing methods, can help make testing easier
Scenarios – What you need to develop, to see what’s important in your system.
What is Testability “about”?
What are some good tactics?
Testability, cntd
20
![Page 21: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/21.jpg)
Bass’s testability scenarios
Source: Unit developer Increment integrator
System verifier
Client acceptance tester
System user
Stimulus: Analysis, architecture, design, class, subsystem integration completed; system delivered
Artifact: Piece of design, piece of code, complete application
Environment: At design time, at development time, at compile time, at deployment time
Response: Provides access to state values; provides computed values; prepares test environment
Response Measures: (examples) Percent executable statements executed
Probability of failure if fault exists
Time to perform tests
Length of longest dependency chain in a test
Length of time to prepare test environment
Testability, cntd
21
![Page 22: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/22.jpg)
Example scenario
Source: Unit tester
Stimulus: Performs unit test
Artifact: Component of the system
Environment: At the completion of the component
Response: Component has interface for controlling behavior, and output of the component is observable
Response Measure: Path coverage of 85% is achieved within 3 hours
Testability, cntd
22
![Page 23: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/23.jpg)
Testability situations
It’s a development-time attribute:
Probability of fault discovery
Need to control
components
Need to observe
component failure
Right: Ren & Stimpy tell why people
ignore system testability. From web
site sctest.cse.ucsc.edu/ .
Testability, cntd
23
![Page 24: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/24.jpg)
Tactics to achieve testability
Manage Input/Output
Record/playback
Separate interface from implementation
Specialize access
Internal monitoring
Built-in monitors
Generally, the goal is to automate as
much testing as you can – why?
Testability, cntd
24
![Page 25: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/25.jpg)
Examples - 1
Test harness – like the stimulator you might have done for performance Automates some aspect of testing
But, for finding errors, the test cases need to be more complete (unless performance / reliability are what you’re testing!)
Can be for internal classes, etc., as well as external – See next slide
Give this itself an “architecture” with options, different modes, etc.
Testability, cntd
25
![Page 26: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/26.jpg)
Examples – 1, cntd
Typical test harness:
In MVC, replace the “View” with a stimulator
that fires test cases at the rest of the system:
Replace with
stimulator, for
testing
Testability, cntd
26
![Page 27: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/27.jpg)
Examples – 1, cntd
How about stimulating the GUI?
A slightly harder trick, replacing the human with
the mouse and keyboard.
How do we test
the GUI itself?
(Or the whole
System!?)
See list of open source tools at http://en.wikipedia.org/wiki/List_of_GUI_testing_tools
Testability, cntd
27
![Page 28: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/28.jpg)
Examples - 2
Built-in monitors – the system itself
is “instrumented” to provide test
results.
Usually, this can be turned on/off so it
doesn’t interfere with normal
operations.
May have multiple special modes.
The “heart” of white-box testing!
Testability, cntd
28
![Page 29: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/29.jpg)
Industrial strength tools
Good example – IBM’s Rational
PurifyPlus*
Shows code coverage
Analyzes memory problems (e.g., buffer
overflow, leaks)
Identifies performance issues like
bottlenecks
Runs on Linux, Unix, Windows
What do you use?
*See http://www-01.ibm.com/software/awdtools/purify/?cn=agus_rtnlrp-
20100514&cm=k&csr=google&cr=rational_purify&ct=AGRAK605&ck=rational
_purify&mkwid=s28L9ro6h_5281300654_432dt32930&cmp=109HG
Testability, cntd
29
![Page 30: Quality Tactics - Rose-Hulman Institute of Technology · What’s Bass say about this QA? Problem – Testing needs to be more efficient & effective It consumes a high percentage](https://reader033.fdocuments.net/reader033/viewer/2022041702/5e426455211e8b76af61542d/html5/thumbnails/30.jpg)
30
Usability
Strategies:
Design time Separate UI from rest of system
Runtime Support user initiative
• Cancel, Undo
Support system initiative
• User/System/Task models
Tonight there’s a separate set of slides has more details on each of these
See also your Interaction Design book from CSSE 571! (Rogers, Sharp and Preece, now in its 3rd edition)