of Automated Testing

of Automated Testing
of Automated Testing
of Automated Testing
of Automated Testing
download of Automated Testing

of 4

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of of Automated Testing

  • 1. Avoiding the Pitfallsof Automated Testingby Allan McNaughton Q uality assurance engineers are being asked to take on more complex QA Wizards simplicity also signicantlyreduces long-term maintenance costs. This projects than ever before. Application complexity has increased tois a major benet as the only certainty the point that formerly simple test plans now encompass hundreds, if not in automated testing is that test scripts thousands, of test cases that must be executed under stringent deadlines.eventually break. Tests are created using These demands have resulted in quality assurance teams seeking higherassumptions about the application undertest. When the application changes in a efciencies than traditional manual testing provides.meaningful way it is quite likely scripts Much of the focus of this search has beenthe selected tool be a good t with yourwill fail. Users of language-based tools on automated testing tools. It is commonly software technologies (MFC, .NET, Web,will nd themselves laboriously poring thought that automated testing is theetc.) and your testing teams technical over lengthy scripts trying to rectify the panacea for all quality assurance woes. But is capabilities. problem. Users of QA Wizard can see at a that always the case? While a well-executedglance why tests failed and can update out-Automated testing tools basically fall into automated testing strategy can measurablyof-date scripts without touching a singletwo camps: those that rely on scripting improve productivity, in reality there are line of code.languages and those that use a graphical many pitfalls waiting to ensnare the unwary. approach. Code-based tools have complex Plan for Success Quality assurance teams need to carefullyscripting languages but can be customized Automated testing projects must be analyze the path before them. By choosingin enumerable ways. While this exibility carefully planned to ensure success. The the right tool for their needs and using itmay seem desirable, it can cause more rst step is to set realistic goals. Many wisely, they can reap the much heraldedproblems than it solves. These tools aretesters think the goal should be to automate benets of automated software testing. extremely complex, and they can be100% of the test plan. Experience has Select the Right Tooldaunting to all but the most technicallyshown that just reaching the 75% level is Nothing will sink an automated softwaresophisticated users. This complexity oftena noteworthy accomplishment. The fact is testing project faster than adopting the results in the tool getting shelved after the that manual testing is still required even in wrong tool for the job. Testers are facedinitial enthusiasm fades. the most heavily automated environments. with a bewildering array of choices that A better solution is to seek out a tool thatHowever, by using QA Wizard you can range from loosely supported open source does not require programming skills yet reach higher levels of automation faster solutions to commercial tools that claim offers functionality comparable to those that than what is possible with language-based to solve most every automated testingdo. Seapine Softwares QA Wizard fulllstools. problem. this requirement by combining the power Determining which parts of the application Which tool is right for the job? The of a language-based tool with an easy-to- are right for automated testing and which question is best answered by honestlyuse graphical interface. This dramaticallyparts should continue to be tested manually evaluating your organizations needs against reduces the time it takes to learn the tool is straightforward when the following the products that are available in todays and lets all team members participate inguidelines are heeded: competitive marketplace. To succeed at automated testing projects, regardless of automated testing, it is imperative that their technical skills.1

2. Identify portions of the application thatprojects. Projects exist within workspaces require data comparison or validation. and multiple workspaces can be created. Text comparison and data validationSome teams create one workspace while are areas that manual testing can easily others create separate workspaces for each overlook. QA Wizard can compare andapplication being tested. Likewise, some validate the same information with total teams create projects for each component in accuracy.an application while others create projects Identify which parts of the applicationbased on the type of testing taking place deal with complex business rules.(See Figure 1). Although QA Wizard can easily direct Think Small the ow of a script using context checks Develop short maintainable scripts. Longer and conditional statements, it is generallyscripts are harder to debug and less reusable. more effective to manually test complexWith QA Wizard, you can record an business rules.individual script for each test in the plan Avoid automating areas of the applicationor easily break down a longer script into that change constantly. Automatedmultiple smaller scripts after it has been testing is better suited for more stable recorded. components (such as core functionality)The process of script decomposition that can reuse test scripts. highlights a key difference between QA Adopt Best Practices Wizard and code-based tools. In a code- As with any tool there are many ways tobased tool you must locate the code that approach a problem. To get the most out of implements the test, cleanly extract it, and automated testing it is recommended that create a new script that seamlessly integrates a number of easy-to-follow best practicesthis logic. As any tester knows, it is all too be followed. These simple steps will helpeasy to make a mistake when doing this type you reap signicant rewards by streamliningof task. QA Wizard visually presents what the creation of automated test scripts and each step does, not how it is accomplished. reducing maintenance of those scripts down This makes moving a step from one script to the road.another a simple matter of copying the stepto its new location (See Figure 2). Organize Your Process Ad hoc scripting should not be permitted. Every test script should t into an overall architecture. Figure 1 Project Explorer The importance of properly Identify components of the applicationorganizing test scripts rises in that require large amounts of data entry. direct proportion to the number Data entry is time consuming and can be of tests that must be automated. easily automated. QA Wizard can retrieveWhen there are hundreds of these data values from a spreadsheet or automated tests it is easy to be database and enter them for you.overwhelmed without a well- dened system for managing Identify areas of the application that them. require repetitive testing with different data values for each iteration. QAQA Wizard scales to support the Wizard can retrieve these values from a largest test suites by allowing spreadsheet or database and run through users to organize test scripts the test multiple times using new datahierarchically into workspaces andFigure 2 Step copy and paste values with each run. 3. Plan for ReuseSOAP servers or to access an operating Create reusable scripts to test commonsystem service such as reading and writing user interface components. For instance,system registry keys. Assemblies can also be an application often contains multipleused for custom data manipulation such as occurrences of drop-down menus. While generating unique login names. Methods drop-downs always work the same, testerscan be called directly from QA Wizard so usually implement separate test logic for individual script steps can be customized as each one. A more reusable implementationneeded. is to develop a callable function thatCapture Knowledge determines the number of items in the Test scripts should be documented so they menu and iterates through them one by one Figure 3 Creating checkpoints can be easily understood by others. QA until all choices have been tested. Wizards graphical approach to automated After each test run QA Wizard generates a Validate Resultstesting relieves testers of much of this work. report that shows whether a checkpoint or While automated testing tools are Every step of a QA Wizard test script has any other step in a test script failed. Testers commonly used to control an application asan image showing exactly what eld the can quickly ascertain the nature of failures if it were being used by a manual tester, theyscript is testing (See Figure 4). This visual by examining the actual and expected results may not catch all the errors that a persondocumentation can be quickly understood of the script. Analysis is further simplied can unless additional measures are taken. by those who are new to a project. Testers as QA Wizard captures actual screenshots of For example, validating a spreadsheet may can further clarify scripts by inserting failing script steps as they occur. require the tester to enter specic values into script steps that only contain descriptive a worksheet and then verify the results of aUse Data Driven Scripts comments. calculation. If the script does not validateTest plans often require that the same test results, the test may pass even though thebe executed using different parameters application generated an incorrect answer.(user names, account codes, etc.). The Testers can add validation steps to their work required to accomplish this can test scripts by creating checkpoints usingbe dramatically reduced by having QA QA Wizards point-and-click interface Wizard retrieve the test inputs from a (See Figure 3). QA Wizard supports thedata le. When a script is run, the input following types of checkpoints: information is read from the external data le. Subsequent iterations of the script use Value checkpoints verify that an object Figure 4 Previewer a fresh set of records from the le. Data can cont