Accessibility Support Baseline: Balancing User Needs Against Test Effort

29
THE ACCESSIBILITY SUPPORT BASELINE Aidan Tierney @AidanA11y CSUN March 25, 2016

Transcript of Accessibility Support Baseline: Balancing User Needs Against Test Effort

Page 1: Accessibility Support Baseline: Balancing User Needs Against Test Effort

THE ACCESSIBILITY SUPPORT BASELINE

Aidan Tierney

@AidanA11y

CSUN March 25, 2016

Page 2: Accessibility Support Baseline: Balancing User Needs Against Test Effort

"It can be difficult to know where to start,

and more difficult to know where to stop."

- Chetan Bakhru @cbakhru

Page 3: Accessibility Support Baseline: Balancing User Needs Against Test Effort

Accessibility Support Baseline

"the minimum set of combinations of

operating systems, web browsers,

assistive technologies, and other user

agents that the website is expected to

work with"

Website Accessibility Conformance Evaluation

Methodology (WCAG-EM) 1.0

http://www.w3.org/TR/WCAG-EM/

Page 4: Accessibility Support Baseline: Balancing User Needs Against Test Effort

Examples of 'combinations'

Windows 10, IE 11,

Jaws 16,

Android 5.1.1, TalkBack 4.2,

Nexus 6

iOS 9.2.x, Safari,

VoiceOver, iPad Air 2

Page 5: Accessibility Support Baseline: Balancing User Needs Against Test Effort

Balancing…

User needs &

EXPECTATIONS

Effort to

develop, test

& SUPPORT

Page 6: Accessibility Support Baseline: Balancing User Needs Against Test Effort

Lip service

Insincere support or

respect expressed but

not put into practice.

Page 7: Accessibility Support Baseline: Balancing User Needs Against Test Effort

Support

Anticipating and addressing user needs & expectations

Page 8: Accessibility Support Baseline: Balancing User Needs Against Test Effort

Evidence of support for AT

• We could speak to the person reporting the

issue and not say something embarrassing like

"what's JAWS?"

• We have knowledge of the AT and ability to use

it on a device to replicate an issue within a

day or two

• We have already tested the app with the AT

• We can investigate or fix the issue

• We have licensing, firewall clearance, and

basic training in place for this AT

Page 9: Accessibility Support Baseline: Balancing User Needs Against Test Effort

Assistive Technology

(AT)

Version #

E.g. JAWS 17, NVDA 2016.1

Operating System (OS)

Version #

E.g. Windows 10, OSX 10.11,

iOS 8.4

Browser

Version #

E.g. IE 11, Chrome 49

Device

Mostly for mobile

E.g. iPhone 6 Plus, Samsung Galaxy

S6, iPad Air 2

Things to account for in baseline

And users of course!

Page 10: Accessibility Support Baseline: Balancing User Needs Against Test Effort

Tens or even hundreds of

possible combinations

Page 11: Accessibility Support Baseline: Balancing User Needs Against Test Effort

Support for additional combinations will

likely impact effort, cost & timelines

Development QA

Customer/user support teams

Project delivery timelines

Tools & training

Page 12: Accessibility Support Baseline: Balancing User Needs Against Test Effort

Levels of support

• Full

• Reduced

• Targeted

• On Demand

• None (at this time)

Page 13: Accessibility Support Baseline: Balancing User Needs Against Test Effort

Support level before & after launch -

May not need to be the same

• QA before launch

• Customer support

• E.g. Projects tests

with JAWS 17 but will

support customers on

JAWS 15, 16 also

Page 14: Accessibility Support Baseline: Balancing User Needs Against Test Effort

Level Before launch:

QA & Dev

After launch:

User –reported issues

Full QA tests all screens and user flows QA validates & Dev addresses all

issues

Reduced Scope defined by project

Factors to consider: core

functionality, templates

QA validates & Dev addresses all

issues

Targeted QA tests only specific content

related to known differences for a

particular combination

Only used before launch.

On

Demand

No QA activity before launch QA validates all issues. Remedial

action taken by Dev only where code

does not conform to WCAG and where

feasible.

None No QA activity prior to release. No QA or Dev activity, but

Customer Service does support user.

Levels of Support Defined

14

Page 15: Accessibility Support Baseline: Balancing User Needs Against Test Effort

WCAG &

the baseline

It SHOULD work It DOES work

Page 16: Accessibility Support Baseline: Balancing User Needs Against Test Effort

Accessibility Support Baseline

MOBILE APPS

Page 17: Accessibility Support Baseline: Balancing User Needs Against Test Effort

OS OS Version Assistive

Technolog

y (AT)

Device Level of

Test/ QA

Level of

user

support

iOS Latest major

version VoiceOver

Late-

model Full Full

Android

Latest major

version

with > 10% share

TalkBack

Late-

model,

minimal

bloatware

Full Full

Mobile App Baseline - Basics

Page 18: Accessibility Support Baseline: Balancing User Needs Against Test Effort

OS OS

Version

AT Device Level of

Test/ QA

Level of

user

support

iOS iOS 9.x VoiceOver iPhone 6 Full Full

Android Android

5.x TalkBack Nexus 6 Full Full

Mobile App Baseline – Basics w. specific versions

iOS versions stats:

https://developer.apple.com/support/app-store/

Android version stats:

https://developer.android.com/about/dashboards/index.html

Page 19: Accessibility Support Baseline: Balancing User Needs Against Test Effort

iOS 9 adoption – almost overnight

https://mixpanel.com/trends/#report/ios_9

Page 20: Accessibility Support Baseline: Balancing User Needs Against Test Effort

Android adoption – a different story

https://mixpanel.com/trends/#report/android_os_adoption

Page 21: Accessibility Support Baseline: Balancing User Needs Against Test Effort

OS OS Version AT Device Level of

Test/QA

Level of

user

support

iOS Latest major version VO Late-model Full Full

iOS Prior major version VO Different,

late-model Reduced Full

iOS All other versions the

app supports None None None On Demand

iOS

Future version,

if expected soon after

launch

VO Late-model Reduced Full

Android Latest major version

with > 10% share TB

Late-model,

minimal bloatware Full Full

Android Prior Android version TB

Most popular

Android device

(if known)

Reduced Full

Android Other versions None None None None

Mobile App Baseline - Generic

Page 22: Accessibility Support Baseline: Balancing User Needs Against Test Effort

Accessibility Support Baseline

WEB/DESKTOP

Page 23: Accessibility Support Baseline: Balancing User Needs Against Test Effort

OS AT/ mode Browser Level of

Test/QA

Level of

user

support

Windows JAWS (n-1) IE 11 Full? Full?

Windows JAWS (n, n-

2) IE 11 None On Demand

Windows NVDA FF (latest) Full? Full?

Windows

WindowEyes

ZoomText?

Other AT?

OSX VoiceOver Safari

Web/Desktop Baseline – fill in the blanks

Page 24: Accessibility Support Baseline: Balancing User Needs Against Test Effort

MOBILE WEB & RESPONSIVE WEB

Page 25: Accessibility Support Baseline: Balancing User Needs Against Test Effort

Responsive web

• Browser based

• Smartphone, Tablet, Desktop

• Breakpoints:

– May be more than 3

– Portrait vs. Landscape

– Interface components change

– Include targeted testing for changes

Page 26: Accessibility Support Baseline: Balancing User Needs Against Test Effort

OS OS Version AT Browser Device Level of

Test/QA

Level of

user

support

iOS iOS 9.2.x VO Safari Late-model iPad

–landscape view Full Full

iOS iOS 9.2.x VO Safari Late-model iPad

–portrait view Targeted Full

iOS Other versions

site supports VO Safari Late-model None On Demand

Android Android 5.1.x TB Chrome?

Firefox?

Nexus 10 –

landscape view Full Full

Android Android 5.1.x TB Chrome?

Firefox?

Nexus 10

portrait view Targeted Full

Android Other versions

site supports TB Any Late-model None On Demand

Responsive Web for Tablet Baseline - Specific

Page 27: Accessibility Support Baseline: Balancing User Needs Against Test Effort

Each organization or team needs

to make its own call on what is

the right baseline.

Page 28: Accessibility Support Baseline: Balancing User Needs Against Test Effort
Page 29: Accessibility Support Baseline: Balancing User Needs Against Test Effort

THE ACCESSIBILITY SUPPORT BASELINE

Aidan Tierney

@AidanA11y

CSUN March 25, 2016