Automated Browser Testing

AutomatedBrowser Testing

Darren Hickling September 2014

Overview

3 / 22Automated Browser Testing Darren Hickling

In a Nutshell

• Automate browsers to create repeatable integration and regression tests

• Those tests can be run in parallel on different browsers on different hardware to more efficiently ensure consistent behaviour


The Tech

• Selenium emerged as the obvious choice• Installed and managed via NuGet for C#• Supports all major browsers, including mobile

• Uses the same API for each• Makes direct calls to the browser where possible

• To provide e.g. file downloading• Easy to learn and very powerful

http://seleniumhq.org/

Examples


Browser Switching

Download a browser driver from NuGet and load it

using OpenQA.Selenium.Chrome;using OpenQA.Selenium.IE;

// Setup...RemoteWebDriver driver = new InternetExplorerDriver();

// Later...Driver = new ChromeDriver();


Element Selection

Select and filter elements by their tags and attributes

var tableContentRows = WebDriver .FindElements(By.TagName("table")) .FirstOrDefault(e => e.GetAttribute("media") == "screen") .FindElements(By.TagName("tr")) .Skip(1);


Actions

Queue and perform a set of actions fluently

new Actions(WebDriver) .ClickAndHold(firstListItem) .MoveToElement(secondListItem) .MoveByOffset(0, 5) .Release() .Perform();

Impressions


Patterns Rock

• Two useful patterns have already emerged• Use a Page to interact with the elements of a page

• Fill forms• Click buttons

• Build a Navigator to perform a user journey• Take the name a problem domain• Chain many Page interactions


Problems

• Complex element selection can be convoluted• Finding a parent element, then filtering its

descendants• Using XPath• jQuery selector plugins are also available!• However, you might be masking the fact that this is

unnecessarily difficult?• No other complaints so far


Act as Documentation

• Watching a full integration test suite in action is surprisingly magical and enlightening!

• Make a change here, watch it impact that thing there, understand the domain more

• Great for new engineers and those unfamiliar with a product/concept


Theme and Resize Testing

• Selenium can take a screenshot of the browser• Pick specific screens, screenshot them and compare to

the originals• Resize the browser and screenshot at the different

sizes for responsive testing• Store a hotspot overlay of the failed comparison to

quickly identify the problem• No drawing red ovals in Paint!


Project Workflow Changes

• QA write/evolve test plans from:• High-level project goals• Diagrams/wireframes• Experience!

• SE create automated tests from the plans• Code-reviewed with their BI logic and unit tests

• QA review:• By manually following their test plans

• Checking recordings via supported cloud VM service


Bug Workflow Changes

Triage an issue

QA test and release

Add or update unit tests

Fix the problem!

Ensure browser test now pass

Replicate with browser tests

Code review all changes


Cross-Browser Testing

• Browsers limited to those available on the box used for testing

• At a minimum, use installed browsers on the build server

• Will slow down other builds, though• Not parallelised, either• VMs seem a better and recommended option• Kick off the move to Azure now? Or use…


Sauce Labs

• Provide 359 device/OS/browser platforms via VMs• Used by EventBrite, Yelp, Mozilla and others• Free to evaluate at a very small scale• Automated tests capped per subscription• Unlimited manual tests• C# sample project on Github• Integrates with TeamCity and Bamboo

http://saucelabs.com/

https://github.com/saucelabs/sauce-dotnet-examples


Test Dashboard


Profile the Test Server

• Run a profiler on the box that the tests hit• Pick from one of four contenders, such as ANTS• Provide consistent, historical profile trends

• Have our changes affected performance?• Automated performance issue alerts

• Including advertising huge improvements!• Could mostly prevent (lucky?) devs with a profiler

license from needing to help others

http://www.red-gate.com/products/dotnet-development/ants-performance-profiler/

In Conclusion


Selling Points

• Confidently assert the product functions effectively on many platforms

• Resolved issues are less likely to reappear• Worth the investment for this alone?

• Identify and fix performance problems before they reach clients

• Advertise significant performance gains


More Selling Points

• Reduce the time taken for engineers to understand a issue/product/concept

• Reduce the likelihood that large changes will adversely affect existing functionality

• Clients do/will expect this level of testing

Automated Browser Testing

Software

Transcript of Automated Browser Testing