The Server Side Symposium 2009 Test Tools...

Test Tools Meet-up

Frank Cohen, CEO(408) 871-0122,[email protected] 18, 2009

Selenium, soapUI, HTMLUnit, TestMaker, TestGen4Web

The Server Side Symposium 2009

Open Source Test Automation

About PushToTest‣Mission: Make Internet Services Run Faster, More Reliably

‣Founded by Frank Cohen in 2001

‣Functional Test, Load and Performance Test, Service Monitors

‣Privately Held

‣http://www.pushtotest.com

2

Morgan Kaufmann 2007 Prentice Hall 2005 Sams 2003

Next let’s talk about


The Problem

‣RIA and SOA Is Everywhere

‣Complexity and Lack of Standards Driving Test Costs Up

‣Solutions for Test Development

3

How do we achieve rapid test development

in a haze of new protocols and techniques?


RIA, SOA Is Everywhere

5

AIR is only one RIA/SOA success story of Dozens MoreEveryone is building software this way.


6

6.7 Million People1.6 Billion Internet Users

How Do I Test?

0 RIA, SOA Standards


7

Test Biosphere‣Test Development/Authoring

‣Protocols, Data, Events

‣Record/Playback

‣Scripting

‣Deployment and Orchestration

‣Repurposing

‣Root Cause Analysis and Mitigation

‣Automation, Continuous Build


Applies Everywhere

8

Application

Testing Model

Environment

Methodology

Architecture

Model

Build Tests and Monitors of

Web 2.0, SOA, RIA, Ajax, Flex,

Flash

Record/Playback, Unit Test,

Object/Component Oriented

Test Development

Agile Test-First Methodology

Service-Oriented Architecture,

All Tests are Implicitly SOA-

based and Ready for Use

But Mostly...


9


10


11

➌

➊➋


Selenium Script RunnerHTMLUnit

public class CheckEmail { public void setUp() {} ... public void runTest { selenium.setBaseUrl( "http://myurl" ); selenium.open("/login.srf”); selenium.type("search", "ctest"); selenium.click("//Item[6]"); selenium.assertTextExists("mytext"); }}

TestMaker Transformer: Java Unit Test

12

➊ Enter Search Text

Test Script (Selenium)Use Case

➋ Click Item 6

➌ AssertTextExists


13

DeploymentYour cloud or mine? Or both!


Clouds Enable New Levels

14

‣Run At User Levels Never Before Possible

‣Run From Multiple Locations Around The World

‣Pay As You Go


Repurposing

15

Developers Unit Test Coverage, As Many As Possible, Jython=9000

1000

QA Functional Test Test Each Step In A Business Process 500

QA Load, Performance Test

Test Heavyweight Tasks 100

Ops Production Monitoring

Just Enough To Know The App is Up 10

Investment


Business Flow of a Test

16


Check-In Unit Test Using DPL

17

Unit Test

Data Production

Library (DPL)Protocol Handler

CSV RDBMS Object

Developer Data

QA Load Data

Preproduction Data

Production Data


Unit Test Using Protocol

18

Unit Test

Data Production

Library (DPL)Protocol Handler

HTTP SOAP Telnet

Extensible for other protocols


‣Data Production Library (DPL) and Protocol Handlers Enable Repurposing

Unit Tests To Test Flows

19

Check-in Buy Chips Buy Dinner


20

From Unit Tests To Functional Tests


21



22




Results Oriented Approach‣Surface Performance Issues Quickly

‣Automated Functional Tests

‣SLA Compliance Monitoring

23

Surface and Solve Performance Issues in RIA, Ajax, SOA, Web apps

‣Root Cause Analysis & Mitigation

‣Product Search Latency Becomes The Problem Over Time

‣Due To Network Saturation When Running More Than 500 Simulated Users

Surface Issues Quickly

24


Automated Functional Tests‣Avoid Manual Testing

‣Functional Test Operates a Test Flow

‣Multiple Steps In Flow

‣Build, Test, Deploy, Monitor in One Script

25


SLA Compliance Monitoring‣Reuse Tests To Monitor Compliance with SLAs

‣Provide Immediate Dashboard To Surface Production Performance Issues

26



Update On The Tools

‣Selenium, soapUI, HTMLUnit, TestMaker, TestGen4Web

27


Update On OST Tools

28

‣HTMLUnit 2.3 = Ajax oriented release

‣soapUI 2.5 = REST

‣Selenium RC 1.0 release candidate (finally)

‣Windmill, http://trac.getwindmill.com/wiki/AboutWindmill

‣PushToTest TestMaker

‣5.2 with Selenium

‣5.3 with Glassbox integration

‣Appcelerator TestMonkey = Ajax unit testing in JavaScript

Open Source Build and Test Workshop

Web Application Record/Playback‣Simple record/playback using TestGen4Web

‣TestGen4Web from SpikeSource

‣Firefox Plug-in

‣Outputs TestGen4Web XML file format

‣Reuse in PushToTest

‣PushToTest TestGen4Web Script Runner

‣Transform to Java or Jython and run using HTMLUnit

‣Simple, Easy, Not Very Extensible

29

Record/Playback Only Gets Us So Far

CoffeeMaker Example


Ajax Record/Playback‣Simple record/playback using Selenium IDE

‣Selenium from ThoughtWorks, OpenQA.org

‣Largest Enterprise User: Google

‣Selenium Core, Selenium RC, Selenium IDE

‣Outputs Selenese HTML file format


‣PushToTest Selenium Script Runner in TestMaker 5.2

‣Transform to Jython and run

‣Complex DSL, Requires JavaScript Skills, XPath, Very Extensible31


32

Repurposing In Ajax Testing


SOAP, REST, Web Service Testing‣Fast And Easy WSDL/SOAP Test Authoring

‣Most Popular Open Source Test project for SOA

‣soapUI and soapUI Pro from Eviware

‣Outputs tests to soapUI XML file format


‣PushToTest soapUI Script Runner in TestMaker 5.0

‣ Ideal for Agile “Test First” Development with Mock Services

33

Show Mock Services


Where To Go From Here‣Attend a PushToTest Training (soapUI, Selenium, TestMaker)

‣http://soapuitraining.pushtotest.com

‣Attend An Open Source Test Workshop

‣http://workshop.pushtotest.com

‣Ask Peter Schumacher, 512 468-6679 (USA, Texas), [email protected], For A Proposal on Licenses, Training, Support, Consulting

34

The Server Side Symposium 2009 Test Tools...

Documents

Transcript of The Server Side Symposium 2009 Test Tools...