Automated Acceptance testing by Developers & Automated ... Automated Acceptance testing by...

download Automated Acceptance testing by Developers & Automated ... Automated Acceptance testing by Developers

of 19

  • date post

    19-Jul-2018
  • Category

    Documents

  • view

    227
  • download

    0

Embed Size (px)

Transcript of Automated Acceptance testing by Developers & Automated ... Automated Acceptance testing by...

  • Gowrishankar Sundararajan

    QA Manager

    Tata Consultancy Services, Canada

    Automated Acceptance testing by Developers &

    Automated Functional Testing by Testers

  • Overview on Traditional Agile Testing Approach

    Comparison of Traditional Vs New Proposed Approach

    Recommended New Agile Approach with Benefits

    Executive Summary

    2Gowrishankar Sundararajan Nordic Testing Days 2016

    Case Study An implementation overview with KPIs

  • Introduction to Agile

    Introduction

    Frequent delivery of deployable business value

    Frequent changes in requirements

    Close collaboration between entities

    Prioritization of work is done by the

    stakeholders on the basis of requirements and

    focus is then paid on the highest risk issues as

    the work progresses

    Customers/business owners make business

    decisions, developers make technical decisions

    Agile Flow

    3Gowrishankar Sundararajan Nordic Testing Days 2016

  • Traditional Agile An Overview

    Test Design

    Day 1 Day 2 - 8 Day 9- 13 Day 14-15 Day 1

    Coding Defect Fix ClosurePlanning

    Test Execution ClosurePlanning

    QA Plan day wise

    Day 1 Previous Sprint Handover to UAT/Business and

    Scope finalization for new Sprint

    Day 2-8 Test Design phase in parallel with Coding

    Day 9-13 Test Execution Functional Testing

    Day 14-15 Test Execution Regression Testing for the

    current sprint

    In a normal 3 weeks Sprint window, below is the QA plan/duration

    for each testing Phase80%

    Design

    Execution

    Sprint Duration

    Pla

    nn

    ing

    Week1 Week2 Week3

    Test

    Co

    mp

    leti

    on

    %

    100%

    Closure

    4Gowrishankar Sundararajan Nordic Testing Days 2016

  • Drawbacks/Key Issues faced in Traditional Approach

    Comparison

    Issues

    Satisfaction

    On day 9, when testing team starts

    with Smoke testing there is always

    greater chance to identify

    Blocker defects.

    Due to blocker defects, a day or

    two will be lost which makes

    shorter duration for testing team

    to complete test execution in 5

    days.

    Sometimes, even regression for

    the current sprint release

    cannot be completed.

    In Many Cases, regression of

    prior releases are ignored which

    will have higher impact in UAT.

    Finally, test suite preparation of

    regression & End-End testing

    involves considerable effort,

    cost.

    Due to time constraint, few items needs to be removed

    from the current sprint.

    Ultimately, Product team is unhappy with the

    items not getting delivered which was promised.

    5Gowrishankar Sundararajan Nordic Testing Days 2016

  • The Solution New Best in Class Approach

    QA Plan day wise

    Day 2 4 : Testers develop the core automated script and test data for Critical path to satisfy Smoke/Sanity Criteria and

    hands it to Dev team to run the script before sending the new build to QA

    Day 5 8 : Testers add scripts to cover all possible test scenarios

    Day 9 11 : Functional Test Execution using Automation Scripts

    Day 12 13: In-sprint Defect Resting & Regression Test Execution

    Day 14 15: Regression for Prior sprints

    6

    Day 1 Day 2 - 4Day

    9 - 11 Day 14-15 Day1

    Coding Defect Fix ClosurePlanning

    Test Design Test Execution ClosurePlanning

    Day 5 - 8

    Core ScriptAll possible

    Combination

    Day

    12 - 13

    Automated Script Exec

    In-sprint Regression

    Prior sprint Regression

    Sanity Test

    Gowrishankar Sundararajan Nordic Testing Days 2016

  • New Best in Class Approach vs. Traditional Approach Comparison Chart

    Week1 Week2 Week3

    Test

    Co

    mp

    leti

    on

    %

    80%

    Pla

    nn

    ing Sprint

    DesignSprint

    ExecutionClosure

    Sprint Duration

    Week1 Week2 Week3

    Test

    Co

    mp

    leti

    on

    %

    Pla

    nn

    ing Sprint

    DesignSprint

    Execution

    Closure

    Core Regression

    Traditional Approach New Approach

    Acceptance test

    7Gowrishankar Sundararajan Nordic Testing Days 2016

    Sprint Duration

  • 8

    The Solution New Best in Class Approach

    Early Automation Start automation early in SDLC from smoke/Sanity testing rather

    than waiting for functional testing

    Testers will no longer perform Smoke/Sanity testing

    Testing team develops automated smoke/sanity test Scripts

    Developers executes automated smoke test on new build before handover to QA

    Automation

    Smoke/

    Sanity Test

    Functional testing is no longer manual

    Maximized Automation

    Scripts reuse for Regression/End-End testing

    Functional

    Required Change in Approach

    Gowrishankar Sundararajan Nordic Testing Days 2016

  • Benefits with New Approach

    9

    Defect Leakage

    Cost Saving

    Customer Satisfaction

    Coverage

    Cost saving due to

    automated smoke,

    functional testing

    Automation ROI for

    regression and end to

    end testing is much

    faster

    Zero Showstopper

    Leakage from Dev

    to SIT to UAT

    Reduced Major

    defects leakage

    from SIT to UAT

    Increased coverage

    by ensuing room for

    in-sprint regression

    Ensure coverage for

    core regression

    Deliver what is

    promised for each

    print

    More confidence on

    delivered product

    Improved overall

    customer satisfaction

    Key

    Benefits

    Gowrishankar Sundararajan Nordic Testing Days 2016

  • Case Study Testing Support for a Leading Canadian Money Transfer Payment Network Provider

    Client Profile and Background

    Canada's largest money transfer Payment network and Gateway provider.

    The client does the services similar to VISA, PAYPAL etc. and carries out 98% of money

    transfer across people belonging to same/different FIs with in Canada.

    Key Features

    Sending Money across FIs/Credit Unions

    Requesting Money across FIs/Credit Unions

    Project Information

    No of resources

    7 (1 On : 6 Off)

    Product Usage

    Money Transfer across FIs

    Technologies

    Java, Xml, Batches, Web

    Automation Tool

    SOAP SONAR

    Duration

    Jan 2013 Dec 2015

    Tracking / Listing the transactions

    Consolidated Reports

    10Gowrishankar Sundararajan Nordic Testing Days 2016

  • Case Study Testing Support for a Leading Canadian Money Transfer Payment Network Provider

    11Gowrishankar Sundararajan Nordic Testing Days 2016

    Scope of Work

    Current feature is built on Java Technology (Struts/JDBC) -> System heavy and affecting its performance.

    Rewrite of existing application + new features on Spring/Hibernate feature.

    No change in the XML Structure that is being used across FIs for Sending Money.

    Regression Testing of Old Features Functional Testing of New Features Integration Testing of Both Features

    Project Intro

    Web banking customers of participating FI -> electronic payments to anyone else -> web banking

    customer of a participating FI.

    Current functionality one-time, immediate money transfer.

    Future Release includes Requesting Money, Auto-deposit etc.

    The system uses an application service provider model (XMLs) with an interface to the FIs banking site

  • Case Study Testing Support for a Leading Canadian Money Transfer Payment Network Provider

    Types of Testing

    Smoke/Acceptance Testing

    System/Functional Testing

    System Integration Testing

    (End to End)

    Regression Testing

    Test Automation

    User Acceptance Testing

    Beta Testing

    Performance Testing

    12Gowrishankar Sundararajan Nordic Testing Days 2016

    Customer

    Banks /

    Credit

    Union

    Gateway

    Banks /

    Credit

    Union

    Customer

    Send Money Receive Money

    Send

    Request

    Receive Request

    Receive

    Money

    Fulfill Request

    Customer

    Info update

    Flow Diagram

  • Work Load Split-Up

    13

    MODULES USAGE SPLIT % FUNCTIONAL TESTING EFFORT SPLIT %

    Though our application uses web services only by 50%, actual functional testing effort to test other modules

    in turn requires to execute web services again. Overall Functional testing effor