Adopting Agile Testing

Transforming a Traditional QA team to Agile: A 4 Step Approach

Agile Testing

White Paper

Executive SummaryA growing need for quicker and adaptive solutions to tech problems is pushing firms to adopt the agile methodology. Today more and more companies are addressing different technology issues by adopting this iterative approach to software development and releasing high quality software, faster and more efficiently. Organizations see agile software development as a faster way to create products, thereby reducing the Go To Market time. The benefits of adopting the agile software methodology include improved quality, flexibility to midcourse correction, improved business satisfaction, better alignment between the teams and improved time to market.

However, adopting the agile development methods means that traditional software development organizations must change, in particular the functioning of QA teams. This white paper discusses the approach to transition a traditional QA team into an agile QA team, when an organization decides to adopt the agile methodology

2 P a g e

ADOPTING AGILE TESTING

The objective of any testing is to uncover defects (devia-tions from the defined business requirements) in software and to ensure that it works before it is shipped to customers. This objective stays the same for tradi-tional as well as agile QA; and the traditional testing techniques are very relevant to testing in agile as well. In fact, testers on agile teams exploit the benefits of tradi-tional QA within an iterative development process, but the structure, communication, test execution frequency and escalation methods are different in agile testing.

In the traditional QA model, testing is performed by a separate group with its own line of reporting and organi-zation. In this model, while the developers design the code, the QA team develops the test plan. For both teams, the requirements specifications from the requirements engineering group form the baseline. The QA team receives the product, often after months of design and coding for an agreed set of requirements by stakeholders. Then the QA team tests this code and reports defects for the development team to fix. When developers fix the defects filed by QA, they release the newer version of the software to QA; this cycle repeats until the product is ready to be shipped.

Although, this practice has been followed for many years across different organizations, it has its own limitations.

3

Developers test their code less and rely more on testers, leaking defects to testers who find defects late in development lifecycle, impacting time to market

Significant delays before developers receive feedback from testers, again impacting time to market

Frequent changes to test plan and test cases, as business requirements change.

If the changed requirements are not communicated to all stake holders, then there is a mismatch in expec-

tations in the final product across all stake holders

QA is usually the last activity before the published release date, and any delay in either requirements gathering or development, leads to a shorter time frame for testing. Extreme pressure from management to complete testing within the available time will compromise quality of the final product.

Business owners are involved in UAT after testing is formally completed. Observations from business users come very late in the project lifecycle before deploy-ment, and development may have moved to the next version of the release.

Agile testers are part of a cross functional team and talk directly to developers and have their say in all phases of the software development lifecycle.

The test team is colocated with the dev team and their participation in daily scrum meetings ensures informa-tion from all stakeholders is uninterrupted.

Continuous feedback loop to development team

Frequent iterative releases mean, more regression, hence the agile test team frequently makes use of test automation as a way to cut down testing time.

•

•

•

•

•

•

•

•

•

On the contrary, in the agile methodology, QA is normally entrenched in the agile project team, working alongside the developers and business users at every stage of the process. This early involvement and transparency ensures that the QA team is engaged much earlier in the software development life cycle, and gets the requirements directly from business users. This leads to early identification of dependencies, technical or testing challenges, project roadblocks and these are discussed with stake holders and built into estimation and project planning.

Some of the characteristics of agile testing include:

•

Agile testing drives development by refining accep-tance criteria and questioning stories during iteration planning

Test cases always exist, are expanded / modified and executed continuously

Usually the unit, integration and acceptance tests are automated.

What is Agile QA?

P a g e


Agile testers are part of a cross functional team and talk directly to developers and have their say in all phases of the software development lifecycle.

The test team is colocated with the dev team and their participation in daily scrum meetings ensures informa-tion from all stakeholders is uninterrupted.

Continuous feedback loop to development team

Frequent iterative releases mean, more regression, hence the agile test team frequently makes use of test automation as a way to cut down testing time.

4

Agile testing drives development by refining accep-tance criteria and questioning stories during iteration planning

Test cases always exist, are expanded / modified and executed continuously

Usually the unit, integration and acceptance tests are automated.

•

•

•

•

•

•

These characteristics give agile testing certain advantages over traditional testing, such as

Ongoing feedback to developers, allowing testers to ask the right questions and fine tune testing

As the final acceptance criteria are established prior to the work beginning, testing is always embedded in agile process and cutting corners such as skipping testing cannot happen. In fact testing measures the project progress in this methodology

The agile process emphasizes testing at lower levels such as unit and integration tests. Regression risks are managed in lower levels of testing, improving time to market.

When an organization decides to move from traditional development to agile development, transitioning the traditional QA team to an agile QA team is a critical task for the management. This task is not difficult if planned properly. This transformation not only requires changing

the way the testing team works, but also requires chang-es throughout the entire organization, from manage-ment to business organizations to programmers and support teams. This section describes the different aspects to be considered while transitioning a traditional QA team to an Agile QA team. Management must emphasize the following critical aspects though training, counseling and coaching the traditional QA team before starting the transition.

Team Work: In the traditional model, testers start testing after the coding is complete. Testers and developers often communicate through the defect tracking system and they belong to different groups within the organization. Howev-er, an agile tester is tightly integrated with the develop-ment process, and participates in planning, estimating, scheduling and story definition. An agile tester does not merely check the quality of the software; instead he/she moves from a mindset of “how can I break the system” to a mindset of “how can I deliver quality software success-fully”.

Collaborate and communicate: Agile teams do not produce hundreds of pages of requirements documents that form the basis for test cases in traditional QA. Agile testers must proactively work with customers and devel-opers to get the requirements. In fact, agile testers need to serve as a bridge between customers and developers. When requirements are unclear; customers, program-mers, and agile testers discuss together to clarify require-ments. The agile methodology values constant feedback and communication in order to avoid surprises in the later stages of development. This means that unlike the tradi-tional QA team which usually keeps test design, test docu-ment, test progress and QA specific metrics within QA boundaries, and publish them once in a while, agile testers provide visibility into testing activities during daily standup meetings, so issues can be addressed early in the sprint cycles.

Unlearn: When an organization moves from the tradition-al approach to agile, many testers from the traditional QA team move to agile QA with the traditional QA mindset. Learning the agile way of testing is pretty straight forward, as agile is a light weight process. However unlearning past behaviors and practices is the difficult part. For example, in traditional QA, an organization may be comfortable with manual testing, and may not invest in test automation. However, with the time boxed nature of agile, investing in test automation will pay dividends, since manual testing will increase cost and time. In the beginning, when test automation is in progress, it makes sense to keep the tradi-tional manual testing. Once test automation is completed, manual system testing is both expensive and inefficient. Practices and beliefs such as manual testing, that have made positive contribution in the traditional QA, now need to be unlearned if the team is to reach full potential.

P a g e

Agile Testing Transformation in Organizations

1. Change in mindset


Team Work: In the traditional model, testers start testing after the coding is complete. Testers and developers often communicate through the defect tracking system and they belong to different groups within the organization. Howev-er, an agile tester is tightly integrated with the develop-ment process, and participates in planning, estimating, scheduling and story definition. An agile tester does not merely check the quality of the software; instead he/she moves from a mindset of “how can I break the system” to a mindset of “how can I deliver quality software success-fully”.

Collaborate and communicate: Agile teams do not produce hundreds of pages of requirements documents that form the basis for test cases in traditional QA. Agile testers must proactively work with customers and devel-opers to get the requirements. In fact, agile testers need to serve as a bridge between customers and developers. When requirements are unclear; customers, program-mers, and agile testers discuss together to clarify require-ments. The agile methodology values constant feedback and communication in order to avoid surprises in the later stages of development. This means that unlike the tradi-tional QA team which usually keeps test design, test docu-ment, test progress and QA specific metrics within QA boundaries, and publish them once in a while, agile testers provide visibility into testing activities during daily standup meetings, so issues can be addressed early in the sprint cycles.

5

Unlearn: When an organization moves from the tradition-al approach to agile, many testers from the traditional QA team move to agile QA with the traditional QA mindset. Learning the agile way of testing is pretty straight forward, as agile is a light weight process. However unlearning past behaviors and practices is the difficult part. For example, in traditional QA, an organization may be comfortable with manual testing, and may not invest in test automation. However, with the time boxed nature of agile, investing in test automation will pay dividends, since manual testing will increase cost and time. In the beginning, when test automation is in progress, it makes sense to keep the tradi-tional manual testing. Once test automation is completed, manual system testing is both expensive and inefficient. Practices and beliefs such as manual testing, that have made positive contribution in the traditional QA, now need to be unlearned if the team is to reach full potential.

Automation: The time boxed nature of the agile practice presents some challenges to the QA team. Traditional QA considers testing as a mass effort at the middle or near the end of the project. However in agile, the dev team adds new features in each sprint and testing has to ensure that the features previously built, continue to work by incorpo-rating testing in each sprint. Also agile encourages change and must keep changes under control, preventing bugs from leaking into production. To achieve this, an agile QA team must shorten the feedback cycle to the development team, on features implemented. This is where the impor-tance of test automation comes into picture. Teams that are transiting from traditional QA to agile QA, must invest more in test automation. Without automation, the test team will be bogged down with running an ever increasing set of test cases manually, under a tight schedule. This is not scalable and sustainable. With automation at all levels of development i.e. unit, integration and system levels and

continuous execution of automated test cases, the agile test team achieves speed, quality and efficiency

Change the Way Traditional QA Measures Quality: Many organizations transitioning from traditional QA to agile QA, continue using the same metrics they used for traditional QA. Because the two processes are fundamentally differ-ent, organizations and project teams adopting the agile practice must come up with a different set of metrics for quality. For example, the metric “test pass and fail ratio”, in traditional QA, does not have any meaning in the agile methodology, since the moment tests start failing, the dev team stops and fixes the regression right away. Teams transitioning to agile QA, can come up with metrics related to parameters such as velocity, cycle time, code coverage, burn down and defects raised by customers; depending on business needs of stakeholders, customers and projects.

Programming Skills: Any tester who can write a test script in a simple word processor should be able to work on an agile team. However, a tester with very good programming and automation skills will be able to make the testing activity more productive and efficient by continuously automating features in each sprint.

Exploratory Testing: Test automation is an integral part of agile testing to address the challenges of the time boxed nature of the agile practice. However, test automation poses its own challenges to development work, as auto-mation cannot always address issues related to usability, reliability, performance, compatibility and other quality criteria. Using exploratory testing, some of the gaps left by test automation may be covered. The other reason why agile testers need exploratory testing is that in an agile project, the requirements may not be concrete, and changes are part of any agile project. As it is difficult to add test cases for unexpected changes and review them, exploratory testing is an essential skill to uncover the additional considerations.

2. Develop new Skills

P a g e


6

Exploratory testing helps agile test teams to cope with the ever changing context of the project. Management must invest in training the QA team on exploratory testing skills. Once the testers adopt exploratory testing, they will understand which risks are important to the team, where they should focus, how to improve test coverage and make appropriate daily decisions to deal with unexpected changes in the project. Working with Lean Process and Documentation: A typical agile practice moves QA activities upstream in the project lifecycle, enabling QA to prevent defects rather than finding them at the end of project lifecycle as in tradition-al QA. Agile believes in less documentation and agile testers may not get the required documents at the begin-ning of the project lifecycle or for that matter, throughout project lifecycle. For this reason, agile testers must devel-op skills to document test case/test script or test require-ments. Testers can share these test requirements with customers, so the gaps in the original requirements can be filled immediately. Test Driven Development (TDD) or Behavior Driven Development (BDD) can be adopted for this purpose. A traditional QA team, transitioning to agile

A traditional QA team transitioning to an agile QA, must be deeply involved with advance practices such as Test Driven Development, Behavior Driven Development, Test Auto-mation and Continuous Build and Integration, All these have significant impact on the day to day activities of the QA team. These new practices also influence the selection of tools for testing activities. In addition, as the Agile QA team is tightly integrated with development and project activities, they cannot distinguish between QA, Project Management and Development tools, instead should have knowledge on all the tools used by different personnel on the agile team as a whole.

Some of the ALM tools that are used in the agile methodol-ogy that testers must be familiar with, are shown in the table below.

QA, must learn how techniques such as TDD and BDD work and how to write these test cases.

ALM Phase

Project Planning

Project Status

Story Capture

Documentation management

Software Development

Test Management

Defect Management

Tools

Project Management tool

Sprint and Release status, Resource allocation and status, Burn Down charts

Theme, epic and story capture, acceptance criteriaStory Capture

Simple tools like wiki or any other tools

Continuous integration, unit testing tools, build tools, source control, checklist, code coverageSoftware Development

Manual test case creation, TDD & BDD creation, automated test case management, Defect Management

Defect creation and tracking, Defect integration with sprints, stories and requirements.

Exepctations from QA

Testers must be familiar

Testers must be hands on




Integral part of Agile QA team

P a g e

3. Adapt new tools


Integral part of Agile QA team


7

As can be seen from the table, an Agile QA team uses most of the tools in different stages of the ALM. Organi-zations must provide, tools, support and training to agile QA teams, to help them adapt to this new landscape of tools.

In traditional QA, System Testing is positioned towards the end of the development lifecycle, starting after development is completed, but before the product is released to customers. This is well controlled by entry criteria to start system testing. In contrast, the agile process demands that the build from every sprint cycle, be ready for deployment. When organizations transition to agile QA, they must come up a proper strategy, addressing the following issues:

Typically, traditional QA segregates system testing specialists to a single group, performing system testing for different development teams. Breaking down the virtual walls between different teams and tightly coupling the system testers to agile teams may be a challenge. Organizations must design a strategy where system and performance testing are an integral part of the agile development process.

Testers may not get fully completed features in each sprint, making system testing a challenge. Organiza-tions must work with different stakeholders and set expectations that system testing cycles will be execut-ed every iteration (say, every third or fourth sprint) rather than at the end of every sprint.

An agile process is adaptive, designed to accommo-date and encourage inevitable changes. This may lead to frequent changes in the system, and the testers

•

•

•

may get frustrated to test multiple builds with changed requirements in the same iteration. It is the responsibility of management to make these QA teams understand the importance of iterative system testing and motivate them to adopt the agile testing practice.

In many organizations, system testing slots are booked for different projects to optimize the hard-ware resources shared by many projects. If agile test teams decide to execute system testing whenever a feature is changed within any given iteration, it is very difficult to block the hardware in advance. Organiza-tions adopting agile practices must come up with plans to address such situations, including any exter-nal dependencies.

•

Organizations adopt the agile software development methodology as a faster way to create and release prod-ucts. The benefits of adopting the agile software meth-odology include improved quality, flexibility to midcourse correction, improved business satisfaction, better alignment between the teams and improved time to market. However, adopting agile development meth-ods means that traditional software development orga-nizations must change the way QA teams work. Tradi-tional QA teams must change their mindset, break down the virtual wall with development teams, learn newer testing techniques such as exploratory testing, testing based on TDD and BDD. Organizations must see auto-mated testing as an integral part of agile testing and not just rely on manual testing. The testers in the agile meth-odology must learn newer tools and skills. They must ask the right questions, while working collaboratively with other groups, in overcoming some of the challenges of testing in the agile methodology.

P a g e

4. Position your System Testing Phase in Agile QA

Conclusion


About the AuthorHarsha B N works as a Test Architect in the Mobility division of Idexcel. He has twelve years of experience in develop-ment and testing mobile applications. Prior to joining Idexcel Harsha worked with Nokia for eight years in various capacities as Program Manager, Chief Test Engineer, Project Manager working on OTA infrastructure development, Mobile Payments services, S60 SDK.

About IdexcelIdexcel is an innovative provider of IT Products & Services focused on emerging technologies. We help world leading companies build efficiencies and stronger businesses. With more than 15 years into existence Idexcel’s main focus is client satisfaction and technology innovation. Our industry expertise and a global, collaborative workforce forms the backbone of our services. We offer high degree of skills in Enterprise Applications, Cloud Services, Data-warehousing, Big Data, Analytic, QA & Testing Services, IT consulting and Staffing. Idexcel product line includes: NDS, ERP, and Cync - A revolutionary credit monitoring application for the manufacturing and �nancial management.For more information log on to www.idexcel.com.

Global Head quarters459 Herndon Parkway Suite 11Herndon, VA 20170Tel: 703-230-2600Fax: 703-467-0218Email: [email protected]

India Operations“Crystal Plaza” 9, 10 ,11Bhuvanappa Layout, Hosur RoadBengaluru – 560 029KarnatakaTel: +91-80-2550 8830Email: [email protected]

© Copyright, Idexcel. All rights reserved. No part of this document may be reproduced, stored in a retrieval system, transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the express written permission from Idexcel. The information contained herein is subject to change without notice. All other trademarks mentioned herein are the property of their respective owners.


Adopting Agile Testing

Technology

Transcript of Adopting Agile Testing