© 2013 LogiGear Corporation. All rights reserved.
Augmenting Coded UIwith Action-BasedTest AutomationMicrosoft | LogiGear
© 2013 LogiGear Corporation. All rights reserved.
Test Requirements for Quality Enablement
10x
Speed
MaintainabilityReusability Scalability
© 2013 LogiGear Corporation. All rights reserved.
- Quickly and easily create a large number of tests
Test Requirements for Quality Enablement
Speed
10x
© 2013 LogiGear Corporation. All rights reserved.
- Quickly and easily create a large number of tests
- Maintain tests when the application or GUI changes
Test Requirements for Quality Enablement
Speed
10x
Maintainability
© 2013 LogiGear Corporation. All rights reserved.
- Quickly and easily create a large number of tests
- Maintain tests when the application or GUI changes
- Focus on reusable test design and flow, not test programming
Test Requirements for Quality Enablement
Speed
10x
MaintainabilityReusability
© 2013 LogiGear Corporation. All rights reserved.
- Quickly and easily create a large number of tests
- Maintain tests when the application or GUI changes
- Focus on reusable test design and flow, not test programming
- Execute data-driven testing for coverage and scalability
Test Requirements for Quality Enablement
Speed
10x
MaintainabilityReusability Scalability
© 2013 LogiGear Corporation. All rights reserved.
- Quickly and easily create a large number of tests
- Maintain tests when the application or GUI changes
- Focus on reusable test design and flow, not test programming
- Execute data-driven testing for coverage and scalability
- Enable test understanding and team collaboration
Test Requirements for Quality Enablement
Speed
10x
MaintainabilityReusability Scalability
© 2013 LogiGear Corporation. All rights reserved.
Action-Based Testing Methodology
Keyword-driven methodology
Modular test design
Solution architecture and
functionality
Automation Engine
ActionLibrary
Interface
Test ModuleTest ModuleTest Module
Test Development Plan
© 2013 LogiGear Corporation. All rights reserved.
• The test developer creates tests using "actions“
• Each action consists of a “keyword” and “arguments”
def action_NewProduct . . . .
def action_AddQuantity . . . .
def action_CheckQuantity . . . .
Keywords and Actions
number name quantity
new product P-9009 Sledge Hammer5
number quantity
add quantity P-9009 20add quantity P-9009 3add quantity P-9009 6
number quantity
check quantity P-9009 34
© 2013 LogiGear Corporation. All rights reserved.
© 2013 LogiGear Corporation. All rights reserved.
Low-level action: mouse Click
Group of low-level actions: Select a product Click “Add to Cart” Click “Buy”
High-level action: Checkout.
Higher-level action: Order Entry
Low-level actions …
… combined to make higher-level
actions
… combined to make higher-level actions
© 2013 LogiGear Corporation. All rights reserved.
Action-Based Testing Methodology
Keyword-driven
© 2013 LogiGear Corporation. All rights reserved.
Original functionality was OKNew functionality is SUBMIT
Every instance of the changedaction is automatically updated
Tests are easily maintained when application requirements change
Action-Based Testing Methodology
Reusability
© 2013 LogiGear Corporation. All rights reserved.
© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.
CHECKPAYMENT
CLOSEAPP
RENTCAR
Action Library
Custom Actions
STARTAPP
TEST MODULE Car Rental
user key
TEST CASE TC 01 Rent Cars
first name last name car
last name amount
FINAL Exit App
window
start app sam
rent car Sam Doe Ford Escape
check payment Doe 89.95
close app reservation
PROCESSORDER
CONFIGACCNT
ADDINS
CHANGEORDER
LISTCARS
Create tests with reusable Actions
PRINTRECEIPT
Easy-to-use test creation for readable, understandable test flows
CHECKPAYMENT
© 2013 LogiGear Corporation. All rights reserved.
… results in only a small increase in additional actions
2000tests
200actions
20 functions
A large increase in test cases …
4000tests
250actions
22 functions
Reusability enables scalability, for high-quality tests
© 2013 LogiGear Corporation. All rights reserved.
Cash FlowOptimization
ReconcileStatements
Management
Image Verification
Payment SystemAuthorization
Authentication
Access Mgmnt
Access
Secu
rity
Risk/FraudMitigation
CrimeDetection
TransactionMonitoring
Data Comm
Tran
sactio
ns
Real-time Processing
Action keywords need a methodology …
Test Development Plan
Business
Objective
Business Objective
Business
Objective
Business ObjectiveBusiness Objective
Business
ObjectiveBusiness Objective
Business Objective Business
ObjectiveBusiness
ObjectiveBusiness Objective
Business Objective
Cash Flow
Optimization
Reconcile
StatementsManagement
Image Verification
Authorization
Authentication
Access MgmntRisk/FraudMitigation
CrimeDetection
TransactionMonitoring
Data Comm
Real-time Processing
© 2013 LogiGear Corporation. All rights reserved.
Action-Based Testing Methodology
Keyword-driven methodology
Modular test design
Solution architecture and
functionality
Automation Engine
ActionLibrary
Interface
Test ModuleTest ModuleTest Module
Test Development Plan
© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.
Modular Test Design
Objective-driven process at the Business-object level
Test Development Plan• Top-down Test Development Plan
© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.
Objective
Test Module
Objective
Test Module
Objective
Test Module
Modular Test Design
Test Modules, not Test Cases, are the main testing component
Test Development Plan• Top-down Test Development Plan
• Business-level Test Module objectives
© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.
Test ModuleTest ModuleTest Module
TEST
CASE
Modular Test Design
The best method to scale automation to 100’s or 1000’s of test cases
Test Development Plan• Top-down Test Development Plan
• Business-level Test Module objectives
• Objective-driven Test Cases with similar “scope”
TEST
CASE
TEST
CASE
© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.
Test Module
TC
Modular Test Design
Test Development Plan• Top-down Test Development Plan
• Business-level Test Module objectives
• Objective-driven Test Cases with similar “scope”
• Action Library and Interfaces drive test case operation
Test Module
TC
Test Module
TC
ACTIONLIBRARY
INTERFACE
© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.
Modular Test Design
• Top-down Test Development Plan
• Business-level Test Module objectives
• Objective-driven Test Cases with similar “scope”
• Action Library and Interfaces drive test case operation
• Automation engine to execute Test CasesSeparate test development and test automation
environments
Test Module
TC
Automation Engine
Test Development Plan
Test Module
TC
Test Module
TC
ACTIONLIBRARY
INTERFACE
© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.
• Suited for complex, enterprise-level test projects
• Test modules, with readable, understandable test flows
• High team involvement, collaboration and communication
• Emphasis on test design, test flow and test project organization− Less focus on tools, technology, and
programming details
Maintainable, Reusable and Scalable Test Automation
Focus on test design and test flow, not on test programming
Test Module
TC
Automation Engine
ACTIONLIBRARY
INTERFACE
Test Development Plan
Test Module
TC
Test Module
TC
© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.
Cash FlowOptimization
ReconcileStatements
Management
Image Verification
Payment SystemAuthorization
Authentication
Access Mgmnt
Access
Secu
rity
Risk/FraudMitigation
CrimeDetection
TransactionMonitoring
Data Comm
Tran
sactio
ns
Real-time Processing
Action keywords need a methodology …
Test Development Plan
Business
Objective
Business Objective
Business
Objective
Business ObjectiveBusiness Objective
Business
ObjectiveBusiness Objective
Business Objective Business
ObjectiveBusiness
ObjectiveBusiness Objective
Business Objective
Cash Flow
Optimization
Reconcile
StatementsManagement
Image Verification
Authorization
Authentication
Access MgmntRisk/FraudMitigation
CrimeDetection
TransactionMonitoring
Data Comm
Real-time Processing
© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.
Cash FlowOptimization
ReconcileStatements
Management
Image Verification
Payment SystemAuthorization
Authentication
Access Mgmnt
Access
Secu
rity
Risk/FraudMitigation
CrimeDetection
TransactionMonitoring
Data Comm
Tran
sactio
ns
Real-time Processing
Automation Engine
ACTIONLIBRARY
INTERFACE
Security Business-level
Objectives
Transaction Business-level
Objectives
AccessBusiness-level
Objectives
Test Module Test Module Test Module
Test Development Plan
Action-Based Testing: Organized, Modular Test Design
© 2013 LogiGear Corporation. All rights reserved.
Action-Based Testing Methodology
Keyword-driven methodology
Modular test design
Solution architecture and
functionality
Automation Engine
ActionLibrary
Interface
Test ModuleTest ModuleTest Module
Test Development Plan
© 2013 LogiGear Corporation. All rights reserved.
Action-Based Testing Solution
Test Design
TestObjectives
Test Lines(with Actions)
TestModules
ActionDefinitions
InterfaceDefinitions
CustomScripts
Test Development Test Automation
Test and Action
Repository
Architecture and functionality designed for Action-Based Testing
Integrated in the Quality Enablement process
Extension in Microsoft Visual Studio 2012, and TFS/MTM Continuous Integration environment
© 2013 LogiGear Corporation. All rights reserved.
Action-Based Testing Solution
Architecture and functionality designed for Action-Based Testing
Integrated to the Quality Enablement process
Extension in Microsoft Visual Studio 2012, and TFS/MTM Continuous Integration environment
source: Microsoft, with permission
© 2013 LogiGear Corporation. All rights reserved.
Action-Based Testing Solution
Architecture and functionality designed for Action-Based Testing
Integrated to the Quality Enablement process
Extension in Microsoft Visual Studio 2012 and TFS/MTM Continuous Integration environment
Team Foundation Server
Microsoft Test
Manager
Visual Studio
© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.
Fragile Test Scripts
TC n
src ctrl
click add OK
TC n
src ctrl
click add OK
TC n
src ctrl
click add OK
TC n
src ctrl
click add OK
TC n
src ctrl
click add OK
Test Scripts
ApplicationUnder Test
50Quantity
use nailsDescr
123Number
hammerName
Add
OK
Test Scripts are fragile when linked directly to the application
© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.
50Quantity
use nailsDescr
123Number
hammerName
Add
TC n
src ctrl
click add OK
TC n
src ctrl
click add OK
TC n
src ctrl
click add OK
TC n
src ctrl
click add OK
TC n
src ctrl
click add OK
Test ScriptsFragile Test Scripts
OK
ApplicationUnder Test
When application code changes, test scripts require maintenance
© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.
INTERFACE Add Product
Name ID TA Name
ORDER_Button 34 OK
50Quantity
use nailsDescr
123Number
hammerName
Add
TC n
src ctrl
click add OK
TC n
src ctrl
click add OK
TC n
src ctrl
click add OK
TC n
src ctrl
click add OK
TC n
src ctrl
click add OK
Test CasesStable and Maintainable Test Cases
INTERFACEApplicationUnder Test
Test Cases are linked to a user-defined logical name in the interface
© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.
INTERFACE Add Product
Name ID TA Name
ORDER_Button 34 OK
50Quantity
use nailsDescr
123Number
hammerName
Add
TC n
src ctrl
click add OK
TC n
src ctrl
click add OK
TC n
src ctrl
click add OK
TC n
src ctrl
click add OK
TC n
src ctrl
click add OK
Test CasesStable and Maintainable Test Cases
INTERFACEApplicationUnder Test
Interface is linked to the control properties of the application
© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.
number name description quantityrow 57463292 Tires Radial tires 8row 45892135 Axles Between the wheels 10row 32198327 Wheels 17 inch wheels 24row 32014827 Taillights Red plastic 326row 03512305 Headlights For nighttime driving 22
name criterion
filter high quantity quantity > 8filter low quantity quantity < 100
number name description quantityrow 57463292 Tires Radial tires 8row 45892135 Axles Between the wheels 10row 32198327 Wheels 17 inch wheels 24row 32014827 Taillights Red plastic 326row 03512305 Headlights For nighttime driving 22
name criterion
filter high quantity quantity > 8filter low quantity quantity < 100
name
use data set /Inventory Data Set
number name description quantity
add product #number #name #description #quantity
name number description quantity
check product #name #number #description #quantity
repeat for data set
number name description quantityrow 57463292 Tires Radial tires 8row 45892135 Axles Between the wheels 10row 32198327 Wheels 17 inch wheels 24row 32014827 Taillights Red plastic 326row 03512305 Headlights For nighttime driving 22
name criterion
filter high quantity quantity > 8filter low quantity quantity < 100
Data-driven Test Cases
Multiple data sets applied to a single Test Module for code coverage
Test Module
TESTCASE
TESTCASE
TESTCASE
TESTCASE
© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.
Visual Studio 2012 Test Environment
Team Foundation Server
Visual Studio IDEMicrosoft Test Manager
Unit testing
Performance testing
Load testing
Code quality analysis
Test planning and
management
Test case manageme
nt
Lab Manager
Manual testing
Exploratory testing
Work item tracking Source control Lab management
servicesTest execution
services Build services Reporting and BI
UI automatio
ntesting
source: Microsoft, with permission
© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.
Test Case Association
Test Case
Visual Studio 2012 UI Test Automation
Coded UI Test Authoring
Manual Coding
Import Existing
Test Case
Record Actions
Execution Methods
Direct Executio
n
Build Integration
Test Manager
Visual Studio IDE
Unit testing
UI automation
testing
Performance testing
Load testing
Code quality analysis
Defect Reporting Quality Analytics
Code Coverag
e
Build Reports
Bug Creation
Keyword
Based
source: Microsoft, with permission
Test Case Association
Test Case
© 2013 LogiGear Corporation. All rights reserved.
• Connect to TFS• Define MTM tests• Build and check-in
tests into TFS• Associate the tests
to MTM tests• Run the tests in MTM
and examine the results
Microsoft TFS / MTMContinuous Integration
© 2013 LogiGear Corporation. All rights reserved.
© 2013 LogiGear Corporation. All rights reserved.
Automatically interact with the app like a user would
Follow steps your team has recorded—faster, unattended, continuously
Coded UI Example
© 2013 LogiGear Corporation. All rights reserved.
© 2013 LogiGear Corporation. All rights reserved.
Convenient Record & Playback easily replicates user actions to automatically generate test code
Coded UI tests are run as part of the Continuous Integration automated build
C# code defines methods, params, and property details
Coded UI tests run as part of the unit testing
suite
Coded UI tests in the familiar Solution
Explorer, are a part of source control
Coded UI Example
© 2013 LogiGear Corporation. All rights reserved.
© 2013 LogiGear Corporation. All rights reserved.
Keyword-driven example
© 2013 LogiGear Corporation. All rights reserved.
© 2013 LogiGear Corporation. All rights reserved.
Keyword-driven example
© 2013 LogiGear Corporation. All rights reserved.
© 2013 LogiGear Corporation. All rights reserved.
Keyword-driven example
© 2013 LogiGear Corporation. All rights reserved.
© 2013 LogiGear Corporation. All rights reserved.
Keyword-driven example
Add one plane to cart
© 2013 LogiGear Corporation. All rights reserved.
© 2013 LogiGear Corporation. All rights reserved.
Create a high-level action:add plane to cart
Keyword-driven example
© 2013 LogiGear Corporation. All rights reserved.
© 2013 LogiGear Corporation. All rights reserved.
High-level action:add plane to cart
Keyword-driven example
© 2013 LogiGear Corporation. All rights reserved.
© 2013 LogiGear Corporation. All rights reserved. © 2013 LogiGear Corporation. All rights reserved.
Identical keyword actions and Coded
UI functionality
Keyword-driven example
Lower-level programming details are hidden from the
test flow
© 2013 LogiGear Corporation. All rights reserved.
Scalable, reusable Action-Based Testing methodology
Readable, understandable
test flow
add plane to cart
Re-use the same high-level action multiple times
© 2013 LogiGear Corporation. All rights reserved.
© 2013 LogiGear Corporation. All rights reserved.
Augmenting Coded UIwith Action-BasedTest Automation
A single, integrated solution for:
• Test Development – fast, easy-to-use
• Test Management – reusable, maintainable
• Test Automation – scalable, customizable
www.TestArchitect.com
© 2013 LogiGear Corporation. All rights reserved.
© 2013 LogiGear Corporation. All rights reserved.
Thank you !
Top Related