(Exploratory) TestingWith Scripted Support
Simon Morley October 2014
Experience & Considerations
Saturday 25 October 14
My Situation Context• Complex Environment
• Meaning overhead & set-up costs
• Meaning preparation, tooling or ordering
• Many signaling protocols
• Meaning script support for signal injection and check
• Agile & Incremental SW Development
Saturday 25 October 14
Testing in Agile
• A common “agile” development model:• Backlog item / Opportunity Analysis
• Plan
• Design
• Test
• Build
• Iterate
Saturday 25 October 14
Testing in Agile
• A common “agile” development model:• Backlog item / Opportunity Analysis
• Plan
• Design
• Test
• Build
• Iterate
Where testing prep & execution happens
Saturday 25 October 14
Testing in Agile
• A common “agile” development model:• Backlog item / Opportunity Analysis
• Plan
• Design
• Test
• Build
• Iterate
Where testing prep & execution happens
Saturday 25 October 14
Testing in Agile
• A common “agile” development model:• Backlog item / Opportunity Analysis
• Plan
• Design
• Test
• Build
• Iterate
Where testing prep & execution happens
Saturday 25 October 14
Testing in Agile
• A common “agile” development model:• Backlog item / Opportunity Analysis
• Plan
• Design
• Test
• Build
• Iterate
Where testing prep & execution happens
Saturday 25 October 14
Testing in Agile
• A common “agile” development model:• Backlog item / Opportunity Analysis
• Plan
• Design
• Test
• Build
• Iterate
Where testing prep & execution happens
Saturday 25 October 14
Testing in Agile
• A common “agile” development model:• Backlog item / Opportunity Analysis
• Plan
• Design
• Test
• Build
• Iterate
Where testing prep & execution happens
Saturday 25 October 14
Testing in Agile
• A common “agile” development model:• Backlog item / Opportunity Analysis
• Plan
• Design
• Test
• Build
• Iterate
Where testing prep & execution happens
Saturday 25 October 14
Testing before “testing”• Testing in the “Opportunity Analysis”,
“Planning” & “Design” “phases” might be:
• Feature Investigation - from a testing perspective
• Testable?
• Creates issues with existing framework, tools and maintenance?
• Feature Walkthrough
Saturday 25 October 14
Possible Testing Approach• Investigate the new feature
development
• Develop framework and tools, develop scripts, comms, test scripts & framework & feature, comms, debug scripts, comms, debug environment, comms, test feature, comms, report completion, archiving.
Saturday 25 October 14
Feature Investigation
• Feature Walkthrough
• Test environment impacts & walkthrough
• Test Analysis summary
Saturday 25 October 14
Test Analysis Summary• Describe* or reference the current
feature, test environment and framework situation
• Describe* impacts to test environment and framework
• Describe* approach
• Include coverage goals for product, feature and testing
Saturday 25 October 14
Comms?• Communication
• Can be: Verbal & Written
• Can be: Formal & Informal
• Always: Two-way!
• Reporting - stand-ups, progress boards, meetings
• Reflect & decide about next steps
Saturday 25 October 14
Reflect & Decide?• What has the testing so far revealed?
• Product coverage goals met?
• Testing coverage goals met?
• Perceived risk acceptable?
• Stakeholder view on these?
Saturday 25 October 14
Reflect & Decide #2
• Next steps?
• Additional test steps?
• Enough already?
• Change direction or approach?
Saturday 25 October 14
Reflect & Decide #3• Pause & Reflect
• Control Q: “What don’t we know?”
• Silent Evidence
• Note: This pause & reflection is important - see thinking about heuristic discovery.
Saturday 25 October 14
Necessary support?• Can the tools & framework support
changes?
• Is the tool & framework architecture modular?
• Remember to test the tool architecture!!!
• Do the tools & framework allow quick prototyping?
• If not -> fix it!
Saturday 25 October 14
Describe*• Describe: Meaning the level sufficient in
the team or project.
• If coordination is needed between teams this should be a format that is commonly understood (picture/symbols, dashboard or text document).
• You agree what’s appropriate!
Saturday 25 October 14
Exploratory Testing Traps• Sometimes people think of ET means
less:-
• Documentation
• Structure
• Traceability
• And more
• Random banging on a keyboard
Saturday 25 October 14
Exploratory Testing Traps• Sometimes people think of ET means
less:-
• Documentation
• Structure
• Traceability
• And more
• Random banging on a keyboard
Wrong !!!
Saturday 25 October 14
ET Traps #2• Actually ET is about more
responsibility in skilled testing
• The skilled tester is responsible for communicating his/her approach, thinking, results and any mandated archiving
Saturday 25 October 14
ET Traps #2• Actually ET is about more
responsibility in skilled testing
• The skilled tester is responsible for communicating his/her approach, thinking, results and any mandated archiving I.E. Plenty of structure,
traceability & documentation
where appropriate!
Saturday 25 October 14
ET Traps #3• No support for skilled testers and
testing.
• Support for skilled testing is a pre-requisite (hand-in-hand) to realizing emphasized tester responsibility.
Saturday 25 October 14
ET Traps #3• No support for skilled testers and
testing.
• Support for skilled testing is a pre-requisite (hand-in-hand) to realizing emphasized tester responsibility.
Fix these if you want good & valuable testing!
Saturday 25 October 14
Discussion• Exploratory testing can be performed
with scripting support
• Good testing (in complex environments) requires elements of:
• Domain knowledge - built through early investigation
• Test environment knowledge & understanding
Saturday 25 October 14
Discussion #2
• Plus
• Rapid learning
• System and problem analysis
• Decision making
• Framing, Consensus & Commumnication
Saturday 25 October 14
References• Experience Report: Introducing Exploratory
Testing
• Deliberated Exploration - A Walk in the Woods
• Experiences with Semi-Scripted Exploratory Testing
• Framing: Some Decision Analysis and Frames in Testing
• Silent Evidence in Testing
• On Thinking about Heuristic Discovery
Saturday 25 October 14
Thanks!• Simon Morley
• Twitter: @YorkyAbroad
• Blog: testers-headache.blogspot.com
• Questions?
• Get in touch!
Saturday 25 October 14
Feature Walkthrough
Saturday 25 October 14
Top Related