Beginner’s QA Testing of Websites

Who I Am...• Part of the team at Lucidus, an Internet

Strategy & Services Firm• interface designer; HTML/CSS

developer• QA tester• Analytics and usability consultant• Regional account manager• The “South Florida office”

The Commodity Is Not As Important As

The Way It Is Delivered.We deliver a proven process that combines

engineering precision with creative flexibility

What We Care About:

What isWhat isQuality Assurance?Quality Assurance?QA is part of all good production

processes.

Quality Assurance (QA) is part ofthe process that ensures:•quality in work•activities are being performed effectively•the product meets requirements

What’s unique What’s unique about Website QA?about Website QA?

=

It’s Not EqualIt’s Not EqualWebsite QA has some differences:•functions, benefits and production costs

are not as easily measured as a physical product

•when a website is deployed, it is not necessarily “complete”... it lives, it evolves, is updated. “Finished” is a term applied for the purpose of releasing it for use.

The ChallengeProcesses and methods to manage, monitor, and measure quality in websites and apps can be as fluid and elusive as the defects they are meant to keep in check.

For processes and methods to be most effective, you need to implement both web standards and company guidelines.

QA Testing is not limited to Usability

or Cross-Browser

Testing• Usability is an essential part of QA • Cross-browser testing is tests for

acceptable display in modern browsers• QA tests the entire process, including

functional accuracy

QAQA Function,Function, DesignDesign andand UsabilityUsability

The Benefits of QA• Improved client satisfaction:

profitable relationships, good testimonials, waves of referrals

• Reduced cost of development: fewer defects, streamlined and simplified stages of development, retesting runs smoothly, deployment on time, on budget

• Reduced cost of maintenance: good development is less troublesome to support, support is costly

Methodology of QAValidation testing: entering erroneous data to test for an expected result

• what is the expected result?• should this error have happened?• if this error is likely to happen, is

there any way we might prevent its occurrence?

Form Validation• character limit, add

more chars than the field

• do the unexpected, try to break the form

• form submits when numbers or characters are entered in the fields:

1,2,3,4,5 etc.“ ” & ‘ ( ) ’

Methodology of QAData comparison: compares the output of an application to previously entered data

• if the appropriate data types are entered, are we getting the correct result?

• does the way the page delivers or displays those results make sense?

• is there anything about this part of the application that should be more obvious?

Data Input vs. Output

• does it make sense?

• any inconsistencies with the data entry?

• test and change the data entry and see if output updates the same way

Methodology of QAUsability Testing:tests out how users actually use a website, to match it more closely to what user needs

• is the user doing what we expected in the manner we expected?

• are they finding and performing tasks?• where are the weaknesses in the design?• where and why are tasks not being

completed? • how can we assist the user?

Usability• does it make

sense?

• how can we improve the design?

• how can we help the user know what to do?

• how can we assist workflow?

Testing often and little is far more valuable and cost effective than doing one whopping big usability test of an entire site when it is almost finished. - Steve Krug, “Don’t Make Me Think”

Testing Guidelines• Test by developer repeatedly as critical

sections are finished, to validate functionality

• Test when all sections are ready and “code complete”

• not involved with the site in any way• new to the website (don’t ask them

twice)• familiar with the web in general

• Use an outside user or colleague who is:• colleagues may provide very useful

feedback

Learn the requirements

Examples of requirements:• A streamlined, purposeful interface that

is easy to understand and navigate, with obvious links, and no clutter

• Forms use inline validation to assist user

• Actions which cannot be undone should ask for confirmation

• An online forum where users can add topics, edit their posts, and comment

Gather use cases from the developer and account management team

Use cases specify how users carry out a task in the website. Examples include:

• Complete application start to finish• Start, Save, then Continue later• Change status; Verify update to

status• Filter and reorder list• Add or Edit an item• Search for an item

Draft a Checklist for the tester for what to test

• Content• spelling, grammar,

mechanics• all critical information is

present• titles, headers, and

navigation labeled correctly

• Graphics and layout• image quality• download time• text layout• alignment of elements• color accuracy• User preference• font size and link colors

• Browser compatibility• check on different platforms• check on different browsers• Functionality• form validation• erroneous data• input vs. output• meets expectations• makes sense

Draft a questions for the tester to ask during testingDoes the user:• gets the point of the page(s)• understands the navigation system• can guess where to find things.In a general test you want to know:• how do users interact with the web site?• what is difficult to do?• where do they get lost?• what makes sense to them?• what makes them feel distrustful or insecure?• what do they like and hate?In a specific test you might want to know, for example:• can the user accomplish a key task?• can the user find something specific

Create a testing matrix

Then... Test!Test in all browsers needed to meet requirements:•Firefox 2•Safari•Internet Explorer 6•Internet Explorer 7

Document findings!!!

Get the Benefits• No surprises late in the game• You KNOW it works and can stand

by it confidently• You can sleep at night• Happy clients• Long-term clients• Deliver real value

Thank You!Resources and Q&A

• “Designing the Obvious: a common sense approach to web application design” by Robert Hoekman, Jr.

• Hoekman’s Design Description Documents http://rhjr.net/ddd/

• Web Commandments http://rhjr.net/eReader/webCommandments.html

• “About Face 2.0: The Essentials of Interaction Design” by Alan Cooper & Robert Reimann

• “The Elements of User Experience: User-centered Design for the Web” by Jesse James Garrett

• “Communicating Design: Developing Web Site Documentation for Design and Planning” by Dan M. Brown

• Website Checklist http://www.chromaticsites.com/web-design-blog/2008-05-26/the-official-successful-website-checklist-challenge/