Test Automation - Moving Beyond the Test Tool

Test Automation:

Moving Beyond the Tool

Aligning your automation to you

26 February 2014

2 ©Alliance Global Services 2014

Today’s Presenter

As Director of Solution Engineering, Joshua Berry has 20 years

of industry experience, with more than a decade in leadership

and innovation. He is a leader within Alliance’s Solution

Engineering organization in developing cutting edge engineered

solutions. His focus includes test automation and tooling,

problem solving, product testing, and agile techniques in

software development.

Today’s Host

As Director of Marketing, Sharon Lee heads the marketing

strategy and brand messaging for Alliance. With over

12 years of experience in both digital and traditional marketing,

she is focused on the effective use of media for compelling brand

messaging and creating successful marketing programs with

measurable results that impact revenue. Sharon holds a B.A.

from the University of Pennsylvania.


Overview

• Introduction

• Webinar Origins

• Topic deep dives

• Agenda

• The Tooling Trap

• Problems with current Automation Training

• Engineering a Solution

• Selecting a Tool(s)

• Customizing the Tool(s)


The “Textbook” Definition

What is Software Test Automation?

In other words,

Test Automation is using software to test software. One part test engineering + one part software engineering.

Software Test Automation is the application of computer software and electro-mechanical equipment to:

• Dynamically Control the execution of tests, • Compare actual outcomes to predicted outcomes, • Set up test preconditions and cleanup test modifications, • Perform other test control and test reporting functions.


FALLING INTO THE TEST TOOL TRAP:

FORGETTING ABOUT THE ENGINEER


A Common “New to Automation” Tooling Cycle

“Phil”


How did that happen?

• Test Automation is a software engineering project Omitted basic engineering practices

– An experienced architect

– Seasoned engineers

– Requirements / Design

– A framework

– Coding standards, version control, etc…

• Phil’s Automation Engineer Transformation

– “The Solution” was the tool without investing in Phil

– A three day course on a tool is not enough

– Automation Engineer career progression

• Test Engineering + Software Engineering + Time

– Phil had no frame of reference to judge tools

• Would you reassign your best Business Analysts by giving them Visual Studio and expecting them to code?

– A tool is only as good as the craftsman using it


A Tooling Allegory – Limiting to a single tool

• Craftsman Bolt-On Kit

• Good tool but with flaws:

– Pulling a nail?

– Cutting a thick piece of wood? Or metal?

• Screws and wood could be your web app,

nails your mobile app,

metal your web service


The Dangers of Current Tool Training

• Tool specific training is designed to

teach you how to use the tool not

generic cross tool automation.

• What if the Craftsman Bolt-On was

only tool and training to build a

house?

• Tradesmen learn the trade first, then how to use specific tools

• Why has the IT industry gotten this backwards?

– No need to train until tools existed

– To lock you into their process and thus their tool

• Expecting a Rookie to build good automation because of a tool is like expecting your accountant to build you a good house because you buy him a good hammer and saw.


AN ENGINEER’S APPROACH:

THE TOOL BECOMES A COMMODITY


Step 1: Requirements

• With your architect in place…

• Understand the application(s) to be tested

– GUI / Web service / API

– PC / Web / Mobile

– Technology stack / Flash / Applets

– Custom Controls

– Code Quality / Coding Consistency

– What interfaces will you be automating?

• Understand your environment

– Engineering skillset

– Upfront and ongoing budget for tooling and customizations

– Enterprise wide or project automation?

– How many people to create framework and script

– Regulatory issues for logging and reporting

– Corporate politics


Step 2: Design • Framework Design Points

– Scripting paradigm

– Function Modularity and Reusability

– Reporting and logging

– Error and Exception Handling

– Object versus Image-based testing

– Data Management

– Development vs Execution stations

• Integrate into the SDLC Design Points and Process

– Waterfall vs Agile

– Metrics

– Continuous Integration

– Source code control

– Test Case Management integration

– Departmental structure

• Design the framework based on your needs!

– Switch tools and keep the framework design


Step 3: Selecting the Tool (Finally!!)

• Toolkit Mentality

– Most tools will not do everything out of the box

– Can a secondary tool do the few things the primary tool can’t?

• Research all tools available + internal build option

– Clearly communicate requirements and design to tool vendors

– Consider “consultant” tool re-seller

• Eliminate non-Feasible tools

– Application(s)

– Framework

• Final Tools selection

– Do you need all of the tools features?

– Scorecard comparison

– Vendor POC / Bake-off


Customize Tool Training to you

• Seek generic automation training

• Skip unneeded components

– e.g. recovery scenarios

• Address your specific test requirements

– Teacher should be generic expert on the tool

• Labs can be done using your application


Customize your tools

• Search internet for solutions

• Think of the tool as a development platform

– Use the tools native scripting language

– VB, Java, C/C++, C#, Perl / Python, etc…

• Don’t need to use tools native widgets

• Integrate with COM DLL’s

• Create your own framework

– Scriptless / Codeless Scripting engines

• Build what doesn’t exist

• Iterative framework implementation


Applied Automation

• Using Automation for other purposes other than

Regression Testing

• Application-level production monitor

– Be your own canary!

• Configure applications for manual tester

• Validate environments

• Environment troubleshooting

• Automate manual production process


Questions?

Thank You

Joshua Berry

Director of Solution Engineering

[email protected]

www.allianceglobalservices.com

mailto:[email protected]

Test Automation - Moving Beyond the Test Tool

Technology

Transcript of Test Automation - Moving Beyond the Test Tool