Testing Tools Landscape 2010-1

14
Making Leaders Successful Every Day April 27, 2010 The Testing Tools Landscape: 2010 by Margo Visitacion and Mike Gualtieri for Application Development & Delivery Professionals

Transcript of Testing Tools Landscape 2010-1

Page 1: Testing Tools Landscape 2010-1

Making Leaders Successful Every Day

April 27, 2010

The Testing Tools Landscape: 2010by Margo Visitacion and Mike Gualtierifor Application Development & Delivery Professionals

Page 2: Testing Tools Landscape 2010-1

© 2010, Forrester Research, Inc. All rights reserved. Unauthorized reproduction is strictly prohibited. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change. Forrester®, Technographics®, Forrester Wave, RoleView, TechRadar, and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. To purchase reprints of this document, please email [email protected]. For additional information, go to www.forrester.com.

For Application Development & Delivery Professionals

EXECUTIVE SUMMARYGone are the days when application development and delivery teams could cavalierly ask the business to pick two: cost, time, or quality. !e business wants and needs all three. Delivery teams have responded by experimenting with or adopting Agile development practices, choosing "t-to-purpose development platforms, and designing application architectures for faster change. Now it is time for quality management and testing to respond to this faster-moving environment. Functional testing tools are not enough. Quality must move beyond the purview of just the testing organization and must become an integrated part of the entire so#ware development life cycle (SDLC) to reduce schedule-killing rework, improve user satisfaction, and reduce the risks of untested nonfunctional requirements such as security and performance. To achieve this balance of cost, time, and quality, application development organizations must spread a focus on good testing practices across every role. !ese new requirements have motivated vendors to provide tools that support every role, considerably broadening the testing tools landscape.

TABLE OF CONTENTSDeveloping Quality Applications Is Not Getting Any Easier

The Testing Tools Landscape: Neither Suites Nor Specialized Tools Dominate

Choose Tools That Make Quality Pervasive Across Your Entire SDLC

WHAT IT MEANS

App Dev Teams Will Continue To Need Multiple Testing Tools

Supplemental Material

NOTES & RESOURCESForrester interviewed nine vendor and user companies, including Borland Software, CAST Software, CUNA Mutual Group, HP, IBM Rational, Interactive TKO (iTKO), Micro Focus, Original Software, and SmarteSoft.

Related Research Documents“The Top Five Changes For Application Development In 2010” January 4, 2010

“A Few Good (Agile) Testers” December 18, 2009

“The Dawn Of Dynamic Software Quality Assurance” February 2, 2009

April 27, 2010

The Testing Tools Landscape: 2010Functional Testing Tools Are Not Enoughby Margo Visitacion and Mike Gualtieriwith Mike Gilpin and Adam Knoll

2

5

8

11

12

Page 3: Testing Tools Landscape 2010-1

© 2010, Forrester Research, Inc. Reproduction ProhibitedApril 27, 2010

The Testing Tools Landscape: 2010 For Application Development & Delivery Professionals

2

DEVELOPING QUALITY APPLICATIONS IS NOT GETTING ANY EASIER

What business doesn’t demand high-quality so#ware? What application team does not want to deliver it? Delivery on time and on budget is challenge enough, but quality done right can help teams reduce the pernicious e$ects of rework on their budget and delivery schedule. However, delivering quality applications has become more challenging because:

· Budgets remain tight. !e economy is improving, but "rms remain cautious. Lean and mean, more for less — whatever you call it, it is a continued reality, and testing professionals are not immune to this attitude.1

· Application delivery cycles are shortening. Both internal and external customers expect to get features faster. Internal customers need new features to improve productivity, o$er more products and/or services to their customers, and respond to competitors. Customers expect you to keep up with competitors, and the business wants to maximize switching costs by o$ering more reasons for customers to stay put. All this means that application development teams are under more pressure to deliver more, faster, and that quality had better not su$er.

· Applications are increasingly composed of heterogeneous components. Although your app dev teams are using improved tools and platforms, the improvement is o#en o$set by the fact that applications are increasingly composed of multiple services or components that your team may or may not have developed. !e resulting dependencies make testing and development more di%cult. As they become part of the landscape, cloud platform-as-a-service (PaaS), so#ware-as-a-service (SaaS), and infrastructure-as-a-service (IaaS) will further complicate dependencies for many app dev teams.

· Application development teams are adopting Agile methods. Many application development teams are adopting or adapting Agile methods to do more iterative development.2 !is a$ects every part of the so#ware development life cycle (SDLC), including testing and quality. In many cases, as requirements evolve, quality goals become a moving target. To manage this, team members must work closely together to evolve test plans, testing artifacts and scripts along with requirements.

· Extra coding time is o!en at the expense of testing time in the project plan. Project managers and developers notoriously underestimate the time it takes to code a new application or make changes to an existing application due to a combination of optimism and pressure from the business to deliver faster. More coding time doesn’t always mean the project timeline gets extended; o#en it means the testing part of the plan gets squeezed.

Page 4: Testing Tools Landscape 2010-1

© 2010, Forrester Research, Inc. Reproduction Prohibited April 27, 2010

The Testing Tools Landscape: 2010 For Application Development & Delivery Professionals

3

Quality And Testing Must Be A Whole-Team E!ort

Improving quality in applications is not just about "nding more defects a#er most of the development is completed just prior to release; it’s about baking quality and testing practices into your entire SDLC process. !is realization is not new; it’s a goal that development teams have struggled with for many, many years (see Figure 1). However, today’s development teams are learning from past successes and mistakes and taking ownership of quality throughout the life cycle. It is not enough to talk about starting the quality process with requirements; teams must own quality from initial requirements through "nal acceptance. !at means responsibility for requirements, unit, functional, nonfunctional, system, integration, and user acceptance testing (UAT). To achieve this, application development teams must:

· Make quality everyone’s job. Distributing ownership for quality works best when the whole team owns it — with everyone on the team held accountable to the same quality metrics. Teams sharing information face to face (whenever possible) while developing user stories and de"ning requirements supports the ability to plan for less rework, more quality, and streamlined approaches to testing.3

“We make our developers accountable for their own unit testing and functional testing for their module . . . they have to look forward to determine both the quality levels and the functional levels for the modules. !eir test results are included with the build for the [testing] team. !ey own it.” (Leader of an engineering team, global independent so#ware vendor)

· Empower all team members with tools and practices. Today’s teams must collaborate on several levels; however, while informal collaboration is critical, teams need more-structured ways of communicating about potential problems in the project. Application life-cycle management (ALM) tools must span the entire life cycle of the project, not just the development life cycle. Test management (TM) tools must focus on broader aspects of quality; creating and organizing automated test cases is only a partial solution. Today’s teams must focus on roles and responsibilities as well as expand testing to include system, automated, manual, and performance testing. !e impact of and relationships among these tests are as important as the tests themselves.

· Own and drive the quality process. Requirements may seem like a good place to start, but for many teams, requirements happen too late in the process. Quality ownership stems from initial project proposal through "nal acceptance; therefore, quality assurance includes responsibility for requirements or user story validation; unit, system, and integration testing; as well as functional and UAT testing. When teams collaborate, they should focus on the essentials for guaranteeing a good release and trim away what won’t add value.

Page 5: Testing Tools Landscape 2010-1

© 2010, Forrester Research, Inc. Reproduction ProhibitedApril 27, 2010

The Testing Tools Landscape: 2010 For Application Development & Delivery Professionals

4

Figure 1 Not A New Problem, But Today App Dev And Testing Teams Take A Di!erent Approach

Source: Forrester Research, Inc. 56795

OvTT ,

w.

ocus.ts to code,

.

ocusTPrT

process.T eprocess.

Late

tcus

TPrT

It e

Late

Testing Tools Must Adapt To Serve Multiple Roles

Testing tools are a critical element of what app dev teams need to be e$ective, especially when multiple roles are involved in the testing e$orts. Unfortunately, the way most organizations are using tools today supports a siloed approach over an integrated one. !e result is that teams catch the need for rework too late and may not test all conditions. Some of the issues contributing to ine$ective testing include:

· Quality assurance (QA) teams buy functional testing tools but do not use them e"ectively. Functional testing tools are not new. However, organizations o#en buy them and neglect to implement them to the fullest extent. In a 2009 Methods and Tools survey, fewer than half of the respondents reported that they have implemented tools; in fact, 60% reported either that they have not bought tools or that they are not using the tools purchased.4

· Results aren’t shared across the board. Information about quality is o#en a one-way street: Quality and testing professionals send defect information to developers from their testing activities but don’t receive information back about the outcome of developers’ unit tests and defect analysis. !erefore, testers cannot paint a complete picture of application quality. Defect tracking tools o#en vary by team, or only part of the project team uses them. QA managers may not have insight into ALM tools, developers don’t have insight into TM tools, and neither side is pushing for greater access. Some tools, such as HP Quality Center and IBM Rational Quality Manager, integrate with ALM suites, but the broader team o#en doesn’t use the tools with regularity.

Page 6: Testing Tools Landscape 2010-1

© 2010, Forrester Research, Inc. Reproduction Prohibited April 27, 2010

The Testing Tools Landscape: 2010 For Application Development & Delivery Professionals

5

· Developers use testing professionals as human debuggers. Many developers don’t use functional or test management tools. Too o#en, leaders measure developers on their ability to code a feature and not on the quality of that feature. Developers code as quickly as possible and throw features over the wall to the testing team, saying they have "nished coding these features when in reality the features might not work at all — or at least still have defects. Developers who do test use unit-testing frameworks, such as JUnit, or manually test the application, focusing on the feature they are working on.

· Nonfunctional tests are out of QA professionals’ hands. Security, performance, availability, scalability, and usability testing is critical to quality, but organizations o#en assign responsibility for these kinds of testing to speci"c roles outside the application development and quality teams. Waiting until the very end of the application development life cycle to run stress tests or security tests can increase the need for rework and thus increase development costs and time-to-delivery.

THE TESTING TOOLS LANDSCAPE: NEITHER SUITES NOR SPECIALIZED TOOLS DOMINATE

Today’s testing tools still embrace speci"c functions based on the type of testing they support; however, collaboration, metrics, and cross-functional requirements are coming to the forefront (see Figure 2). As developers expand their usage of the tools that were once part of the tester’s domain and as testers adopt a more comprehensive approach, the borders between the roles are beginning to blur.

Test Suites Are Taking A Comprehensive Approach To Quality Management

As they have evolved, test suites have added more-comprehensive functionality to address organizations’ testing needs:

· Test management tools provide strategic quality planning, scheduling, and collaboration. !e solutions in this category have expanded beyond the organization and tracking of test plans and automated test scripts.5 Vendors such as HP, IBM, Micro Focus, MKS, Original So#ware, and Zephyr emphasize a more comprehensive path that includes resource planning, risk, and/or cost planning — either through the tool itself or via integration with project portfolio management (PPM) tools.

Vendors such as Microso#, SmarteSo#, and !oughtWorks Studios are a bit less comprehensive in scope but are strongly focused on the planning and delivery of e$ective test management. All of the vendors in this space now acknowledge that comprehensive test coverage is becoming a must-have element for QA teams.

· Functional testing tools are growing to embrace real-world requirements. At their core, functional testing tools still address quality from the user’s perspective; however, testing tools are moving away from focusing on “capture and play” or scripting elaborate text cases, instead embracing scenario or scriptless testing.

Page 7: Testing Tools Landscape 2010-1

© 2010, Forrester Research, Inc. Reproduction ProhibitedApril 27, 2010

The Testing Tools Landscape: 2010 For Application Development & Delivery Professionals

6

Figure 2 Suites And Specialized Tools Are Available For Every Stage Of The SDLC

Source: Forrester Research, Inc. 56795

*Tools in red are quality or test focused

AcceptSoftware

SoftwareSystems

Rationalicro Focus

viewally Software

DevelopmentRavenflow

Software VersionOne

ALM

ollabNetRational

icro Focusicrosoft

ally SoftwareDevelopment

SerenaSoftwareVersionOne

IDE

icrosoftVisual Studio

System-leveltesting/code qualitymodeling/virtualization

ftwareonformiq

rityitNesse

Fortify Software tive O

work

rasoft

AutomatedQAZephyr byD Software

Rationalicro Focus icrosoft

Softwarey

Software

teSoft orks

Studios

Deploydobe

Software

Rational

s

ytaporks

Studios

Load testingAutomatedQA

Rationaltive Oe

Systems

cro Focus

teSoftbmetrics

Requirements Analysis andn

/construction T Deployment/

support

Specialized Tools Fill In The Gaps To Ensure Comprehensive Test Coverage

Functional tools o$er broad support for the testing process, but they are not comprehensive. To ensure coverage, especially in service-oriented architecture (SOA) and Internet applications, teams need specialized tools to augment automated and manual testing. Some of these tools "ll gaps where certain types of testing won’t detect problems, and some assist in managing complex environments. Specialized testing tools include:

· Services testing tools, which ensure adequate coverage of the middle tier. As companies take advantage of cloud computing, Internet applications, and other service-oriented application strategies, testing at the graphical user interface (GUI) level only scratches the surface, yet manual testing isn’t always enough to detect defects that may exist in the middle tier or at the service level. Service testing used to reside purely in developers’ domain, but as tools and testing skills have improved, QA now shares in the process.

Page 8: Testing Tools Landscape 2010-1

© 2010, Forrester Research, Inc. Reproduction Prohibited April 27, 2010

The Testing Tools Landscape: 2010 For Application Development & Delivery Professionals

7

· Test modeling tools, which require upfront e"ort but can save time during regression testing. Test modeling tools leverage UML modeling to create paths of application behavior when automating a business process, detailing the work&ow of that process and then creating the test framework for that process — including all the permutations that should be tested. While it takes time upfront to create these models, the value comes in when it’s time to rework scripts. Developers or test analysts can make changes to the model and then re-run the model to execute changes to the related tests. !is can save countless hours for team members who previously would have had to review and make changes manually.

· Test lab virtualization, which increases productivity and enables faster test cycles. Creating and maintaining pristine and realistic test environments can be expensive and time-consuming. Too o#en organizations skimp on areas such as performance and security testing because they cannot safely access a production environment or lack the budget to build a test lab that is an adequate replica. Virtualization and testing in the cloud enable organizations to create secure test environments to perform load testing, and there are multiple investment options. Companies can invest in private clouds or secure environments in public clouds, or they can

“pay by the drink,” provisioning a replica of their production environment to test only when they need it.

· ALM suites, which integrate with or provide test management and test coverage tools. Many ALM suites provide integration of test management functions with HP Quality Center and other test management tools.6 ALM integration is an important feature for getting maximum value out of testing and other quality-related metadata. It ensures that the testing functionality is available as broadly as possible across the extended team and throughout the development and delivery life cycle and links that metadata to other life-cycle artifacts such as requirements or change requests.

· Unit-testing frameworks. Developers who need to do unit testing o#en use unit-testing frameworks such as JUnit for Java or NUnit for .NET development. !ese frameworks are code-based, so they typically execute within the code, and developers can easily use them within Eclipse or Microso# Visual Studio.

· Nonfunctional quality tools, which provide insight into coverage and application quality. Determining the e%cacy of all forms of testing and ensuring compliance with organizational and industry requirements are beyond the scope of functional testing tools. Doing this manually is time-consuming and error-prone. Leveraging code coverage, code quality, and compliance so#ware from vendors such as Black Duck So#ware, CAST So#ware, and Coverity supports the ability to review code quality to determine potential risks, determine test coverage, and make sure that code and/or asset usage is compliant with corporate standards. Other tools such as Adobe BrowserLab or Gomez Web Cross-Browser Testing tools can help ensure that a Web site will render and perform in many di$erent browsers and browser versions.

Page 9: Testing Tools Landscape 2010-1

© 2010, Forrester Research, Inc. Reproduction ProhibitedApril 27, 2010

The Testing Tools Landscape: 2010 For Application Development & Delivery Professionals

8

· Security tools. Developers and test professionals o#en think that security is about user IDs, passwords, and data encryption. But it is also about "nding and mitigating the vulnerabilities and underlying platforms that malicious hackers can use to steal valuable information such as credit card numbers, execute fraudulent transactions, or cause other mischief. Developers use static analysis tools such as those o$ered by Coverity, Fortify So#ware, and Klocwork to scan code for known vulnerabilities such as bu$er over&ows and SQL injection.7 Such tools can only uncover known vulnerabilities, so developers should also perform threat modeling on their application design to uncover design vulnerabilities using a tool such as the Microso# Security Development Lifecycle (SDL) !reat Modeling Tool.8

· Performance and load testing tools. A key nonfunctional requirement for applications is that they perform acceptably under both normal and heavy-load conditions.9 Knowing an application’s breaking point is important to predicting whether the application will be able to handle the expected volume and to determining how much headroom the application has for future growth. Load testing tools such as HP LoadRunner provide a test harness to put load on the application under several conditions to test performance and availability as well as determine capacity. !e challenge in load testing is o#en getting a test lab similar to production, which can be expensive. A popular use case for public clouds is to rapidly provision and tear down test labs. Cloud vendors such as Skytap focus on cloud for test labs. Equally important to traditional load testing within the "rewall is realistic testing from your user’s perspective. Gomez, Keynote Systems, and Webmetrics all provide load testing services outside the "rewall to test how your Web applications are a$ected by all the vagaries of the Internet, third-party components and services, and geographic latency.10

CHOOSE TOOLS THAT MAKE QUALITY PERVASIVE ACROSS YOUR ENTIRE SDLC

Establish the business case for your testing activities not only on the bene"ts of quality but also on how your e$orts can reduce app dev cost and speed time-to-market. In today’s application development organizations, testers are gaining stature as critical elements in the process and valuable members of the delivery team; they o$er a level of focused skills and business awareness that brings a contextual richness to the delivery process. To maintain this momentum, QA testers must expand their approach to quality beyond just testing, because functional or performance testing alone is not enough. Testing organizations must adopt a more holistic approach to testing — ranging from strategic quality management planning that estimates risks and e$ort for the testing process, to becoming involved earlier in the process. As testers work more collaboratively with developers, they must learn technical skills that they can apply in system and integration testing.

Developing new skills is crucial, but tools are also an important part of the equation. Today’s tools provide greater opportunities for QA organizations to build a complete feedback loop between them and the development team (see Figure 3).

Page 10: Testing Tools Landscape 2010-1

© 2010, Forrester Research, Inc. Reproduction Prohibited April 27, 2010

The Testing Tools Landscape: 2010 For Application Development & Delivery Professionals

9

Figure 3 Quality Belongs At Every Stage Of The Life Cycle

Source: Forrester Research, Inc. 56795

Codeconstruction

Requirements

Analysisdesign

Quality

Testing

Deployment

y tests

Code coverage

unctional testsend verification

and validation

Performance tests

T tests

Testing Professionals Must Lead The Charge

Making quality pervasive throughout the SDLC must be everyone’s responsibility, but testing professionals must lead the charge. By creating an environment in which teams seek to remove barriers and promote collaborative quality management techniques, testing professionals can elevate the importance of testing by:

· Holding everyone accountable for quality. Quality assurance may be a job title, but it is the responsibility of everyone on the project team. From conception through deployment, quality practices and the e%cient use of testing tools must be on par with the processes and tools developers use to create the application. QA professionals may lead in devising the quality strategy, but everyone has a role to play in making sure that quality remains job number one.

· Embracing nonfunctional testing. Nonfunctional quality is just as important as functional quality. QA professionals must expand their skill sets to include development-level expertise, or at least a strong understanding of an application’s logic, to be able to review and test an application’s underlying components. According to one vendor interviewed for this document,

Page 11: Testing Tools Landscape 2010-1

© 2010, Forrester Research, Inc. Reproduction ProhibitedApril 27, 2010

The Testing Tools Landscape: 2010 For Application Development & Delivery Professionals

10

“Testers don’t learn new technologies as quickly, but they cannot ignore them — they can’t sign o$ on quality without that knowledge.”

· Focusing quality and testing e"orts on the most valuable and riskiest areas. It’s not cost-e$ective to test everything, so you need a methodology to determine the most critical and risky areas for testing. !is is where strategic test management becomes a critical component of the quality process. Early involvement and continual collaboration enables the team to collectively determine priority and testing risks. Test management tools from vendors such as HP, IBM Rational, Micro Focus, and MKS support a testing team’s ability to group and build test scenarios, track progress, and collect the required metrics to measure e$ectiveness.

· Using automation wisely. Don’t put more e$ort into automating testing than it is worth. If your application changes are minor and quick to test manually, it may not make sense to automate testing in that instance. Or, if you have an application in the very early stages of development, you may "nd yourself repeatedly rerecording or modifying test scripts (although this is an area where the tool vendors have really improved functionality); in this case, it may be better to wait to automate modules until the test scripts begin to stabilize.

Make Testing More Attractive For Developers

Developers live and breathe in their integrated development environment (IDE) in an almost endless cycle of coding, testing, and debugging. !e speed of this cycle is paramount to their productivity, so the last thing they want to do is leave the warm and cozy interface of Eclipse or Visual Studio to use a test management system or testing tool. To bring developers onboard:

· Don’t make developers leave their beloved IDE. Choose testing tools that can integrate with the IDEs your developers use. Don’t just choose a tool that says it integrates with Eclipse or Visual Studio; walk through the process of how a developer will use the test management tool for tasks such as reading and closing defects. Not all tools you need will be available through the IDE, but the more testing features that are accessible within the IDE, the more likely developers are to use them.

· Give developers automated testing tools. Developers hunkered down in the code, test, debug cycle o#en repeat the same tests several if not hundreds of times per day. Provide them with a tool such as one of the o$erings from FitNesse, iTKO, Paraso#, or Selenium that can easily provide them with single-click automation of this testing, and you will improve productivity as well as quality.

· Tell developers that it is not coded until it is tested. Do not let developers say they are done coding a feature unless it is successfully tested. Developers o#en get credit for saying they "nished coding a feature even though later testing may "nd several defects. !e developer gets credit for completing the feature on time, but in reality it did not work. Still, don’t go overboard

Page 12: Testing Tools Landscape 2010-1

© 2010, Forrester Research, Inc. Reproduction Prohibited April 27, 2010

The Testing Tools Landscape: 2010 For Application Development & Delivery Professionals

11

with unit testing; not every line of code a developer writes should necessarily be unit tested, as this would only add unnecessary time to a project. Have developers determine what code should be unit tested based on an estimate of the cost to create a test harness now compared with the impact of waiting until later, when the code can be tested in combination with other parts of the application that exercise that code.

W H A T I T M E A N S

APP DEV TEAMS WILL CONTINUE TO NEED MULTIPLE TESTING TOOLS

There are suites that o!er test management, functional test automation, and perhaps even some nonfunctional testing tools such as stress testing tools. Despite these comprehensive o!erings, app dev teams that inject quality throughout their SDLC will continue to need multiple tools, especially for nonfunctional testing such as security, multibrowser, and mobile testing. Project teams should:

· Start with a suite, then augment as needed, looking for integration. Testing teams must look for the tool set that brings the most bang for the buck, and that means tool suites. However, for comprehensive coverage, teams should go with vendors that are open to integration with nonfunctional testing tools, coverage tools, and ALM (if it’s not already part of the environment).

· Ensure that testers expand their skill sets. Business acumen alone is not enough. To move quality and testing up in the life cycle and build the notion of quality as an integral part of the life cycle, testers have to share the nonfunctional testing load wherever possible. For example, testers may encounter usability or security issues when testing interactive portions of an application. With some basic training in user experience and security principles, testers can catch some of these issues, augmenting the capacity for nonfunctional testing across the organization.

· Link quality and testing e!orts to reduced development cost and time-to-market. Better and earlier testing leads to less rework, which reduces the cost of development and makes delivery schedules more predictable. The quality case for testing’s impact on improved application service levels is usually clear to everyone. What is less evident is how pervasive quality reduces cost and speeds time for development, too. Make the case to your entire team and management.

Page 13: Testing Tools Landscape 2010-1

© 2010, Forrester Research, Inc. Reproduction ProhibitedApril 27, 2010

The Testing Tools Landscape: 2010 For Application Development & Delivery Professionals

12

SUPPLEMENTAL MATERIAL

Companies Interviewed For This Document

Borland So#ware

CAST So#ware

CUNA Mutual Group

HP

IBM Rational

Interactive TKO (iTKO)

Micro Focus

Original So#ware

SmarteSo#

ENDNOTES1 Lean and mean is the attitude that will help you deliver better, faster, and more cheaply. See the January 4, 2010,

“!e Top Five Changes For Application Development In 2010” report.

2 Forrester published report that details how "rms are adopting and adapting Agile methods to improve application development and delivery. See the January 20, 2010, “Agile Development: Mainstream Adoption Has Changed Agility” report.

3 Consolidated and iterative project teams that work together to prioritize test strategies can leverage earlier testing that approaches both system and functional testing requirements. See the December 18, 2009, “A Few Good (Agile) Testers” report.

4 !e 2009 poll noted that while two-thirds of the respondents had tools, these tools weren’t widely used and their adoption had not grown signi"cantly since the "rst poll in 2005. Source: !e So#ware Development Jobs Board: Functional Test Tools: Adopted but not Used? (http://www.methodsandtools.com/dynpoll/oldpoll.php?FuncTest2).

5 Test management tools are strategic quality planning applications that are increasingly borrowing features from other planning tools such as project management and requirements management tools to create a management-level view of quality progress, cost, and risk. See the March 24, 2009, “Test Management: !e Secret Weapon In Your Arsenal To Build Dynamic SQA” report.

6 Forrester evaluated Agile development management (ADM) tool vendors against 152 criteria to determine which vendors had the strongest current o$erings and strategy. See the upcoming Forrester Wave™.

7 For more information on how static analysis tools can help uncover security vulnerabilities in code, see the November 20, 2009, “Know Your Code: How Static Analysis Tools Make Applications More Secure” report.

8 To learn more about threat modeling and the Microso# SDL !reat Modeling Tool, see the March 10, 2009, “Use !reat Modeling To Develop More-Secure Applications” report.

9 Forrester published a report detailing how to design applications to achieve the best possible performance. See the February 4, 2009, “Best Practices: Attaining And Maintaining Blazing Fast Web Site Performance” report.

10 For more information on performing realistic performance testing for customer-facing applications, see the June 16, 2009, “Perform Realistic Web Testing To Ensure Blazing Fast Web Site Performance” report.

Page 14: Testing Tools Landscape 2010-1

Forrester Research, Inc. (Nasdaq: FORR) is an independent research company that provides pragmatic and forward-thinking advice to global leaders in business and technology. Forrester works with professionals in 20 key roles at major companies providing proprietary research, customer insight, consulting, events, and peer-to-peer executive programs. For more than 26 years, Forrester has been making IT, marketing, and technology industry leaders successful every day. For more information, visit www.forrester.com.

Headquarters

Forrester Research, Inc.400 Technology SquareCambridge, MA 02139 USATel: +1 617.613.6000 Fax: +1 617.613.5000Email: [email protected] symbol: FORRwww.forrester.com

M a k i n g L e a d e r s S u c c e s s f u l E v e r y D a y

56795

For information on hard-copy or electronic reprints, please contact Client Support at +1 866.367.7378, +1 617.613.5730, or [email protected]. We o!er quantity discounts and special pricing for academic and nonpro"t institutions.

For a complete list of worldwide locationsvisit www.forrester.com/about.

Research and Sales O!cesForrester has research centers and sales o#ces in more than 27 cities internationally, including Amsterdam; Cambridge, Mass.; Dallas; Dubai; Foster City, Calif.; Frankfurt; London; Madrid; Sydney; Tel Aviv; and Toronto.