Software Usability Course notes for CSI 5122 - University of Ottawa

Software UsabilityCourse notes for CSI 5122 - University of Ottawa

Section 4:

Usability in the Software Engineering Process

Timothy C. Lethbridge

<[email protected]>

http://www.eecs.uottawa.ca/~tcl/csi5122

Lethbridge - CSI 5122 2017 Section 4: Usability and the SE Process 2

I assert: Lack of usability is the most critical problem facing software engineering

In other words, the biggest gains in

• software quality

• productivity

• cost-reduction

• user satisfaction

• profitability, etc.

… would come from focusing on usability and related issues.


Evidence for the assertionFailures of projects due in significant part to usability

• E.g. FAA air traffic control systems

Observations of ‘great’ software vs. not-so-great with respect to usability• Great:

—Google (many of its products, but some getting worse)—Microsoft Office (some of it)—Mac-OS (much of it, but weaknesses creep in)

• Not-so-great:—Much in-house custom software—Software development environments (many aspects)—Web sites (far too many of them)

• I have observed many users formally and informally

My survey of practitioners (next slide)

Analysis of benefits and cost savings


Practitioner survey about 75 topics taught to computer science studentsLethbridge, T.C. (2000), “What Knowledge is Important to a Software Engineer?”, IEEE Computer, May, pp. 44-50.


More on the survey

HCI was second in terms of “knowledge gap”• Where importance most exceeds current knowledge

The top 5 (out of 75)• Negotiation• HCI/user interfaces• Leadership• Real-time system design• Management

I believe not much has changed since 1998


Sources of resistance to adopting usability practices

Primarily: Inadequate education of students, practitioners and managers

Which leads to:• Lack of introspection ability of software developers

—Unable to think like users• Lack of integration of usability into core development

processes• Persistent beliefs

—Most software is usable enough—Usability can be fixed at the “end”—Usability should be left to the HCI experts


Causes of the problems in education

Faculty don’t have the background either

The field of HCI is seen as distinct and separate

The field is perceived as too soft:

• There is a tendency to focus teaching and research on hard deterministic areas

There is little industry push for research in this area

There is little employer pull for greater education


Some partial solutionsThe SE and HCI communities need to work more closely together

• Core UI development and evaluation topics should be considered jointly part of the two fields

Improve education by ensuring HCI permeates curriculum models, certifications, accreditation, etc.

Enable corporations to be certified for their usability capabilities

• Would help “pull” education up by creating a need for professionals

• Clients might come to learn that better software comes from such companies


Should aspects of HCI be considered also integral to Software Engineering?

I argue yes:

• Usability should have no special status as compared to reliability, efficiency, maintainability, etc.

• Design involving users and their needs must drive software development


Should aspects of HCI be considered also integral to Software Engineering?

Yes, but this does not preclude HCI specialists• Analogies:

—A software engineer must be capable of designing architectural elements for storing data and information

- But there will always need to be database specialists

—Similar argument can be made for- Security experts, performance experts, requirements

experts, etc.

Core UCD techniques (coming slides) can be part of the SE field and part of a broader HCI field

User-Centred DesignRevisited



User centered design (UCD)

A term encompassing a range of software engineering approaches that focus on the user

We have talked about many of the techniques


UCD main stages

Pre-design

Design stage

Post-design


UCD Pre-design techniques:

Understand classes of user

Understand tasks

Perform a competitive analysis

• i.e. usability testing of competing products

Set usability goals

• using the metrics we discussed earlier


UCD Design-stage techniques

Parallel design

• Many developers generate UI ideas in parallel on their own

• They meet together and pick the best ideas

• Has maximum chance of discovering good ideas, without constraining the possibilities too soon


UCD Design-stage techniques - 2

Involving users

• Critiquing design

• Discovering problems

—with the UI

—with task descriptions

—through user testing

• Contributing ideas

• But users are not designers so don't ask them what do you want?



Coordinating the interfaces of all parts of the system

• Including documentation, training etc.

Applying guidelines and heuristic rules

Prototyping



Performing empirical testing using simulations and prototypes

• To discover problems

Measuring usability of the simulations and prototypes

Performing iterative design

• Design

• Evaluate

• Redesign


UCD Post-Design stage

Gather on-going feedback about potential improvements

Another term related to UCD

“Outside-in development”


Economics of Usability



An example of economic analysis in UCD: Economics of heuristic evaluation

We will look at

• Number of people needed

• Cost-benefit analysis


How many evaluators should you use in a heuristic evaluation?

Given:

• N = total number of problems to be found

—Clearly depends on:- Quality of the original design or work performed so far

- Size and complexity of the system

λ= proportion of all usability problems found by a single evaluator:

—Studies have shown this averages 34% (range 19% to 51%)

The percent of the total number of different usability problems found by aggregating reports from i evaluators is:

• PercentProblemsFound(i) = 1-(1-λ)i


We can plot the effect of this formula for different λ


We can also look at specific figures for the average case where λ=34%

Evaluators Percent Problems found1 342 563 714 815 876 927 95

Given a desired percentage of problems to be found, we can determine the number of evaluators needed

• by inverting the formula:• i = ln(1-P)/ln(1-λ)

It is not economical to try to find every problem but where should we stop?


How much cost savings can we achieve by finding a given problem?

This depends on many factors, but

• The worst problems are the easiest to find and will be found first

• As we put more effort into finding problems, we will find problems that are less and less severe

We can model the cost savings from fixing a problem as seconds of wasted time per user session

• Very approximate, but useful

—Some problems will only occur once, then never again because the user will learn to avoid them


Visualizing the diminishing returnsEstimated number of seconds per session saved by fixing the nth worst problem found in a particular project.


Deciding what to fix and the resulting savings

From the previous chart, find the total savings if we just solve the first n problems

We can divide this by the n to find the average savings per problem had we just found the first n

• A typical answer for this would be 2 seconds


Deciding what to fix and the resulting savings

Some data from the previous chart, where there were about 200 problems eventually found

nth problem s/session saved average savings had we justfound found the first n problems1 10.6s 10.6s10 7.4s 8.9s20 4.9s 7.4s40 2.2s 5.4s60 1.0s 4.1s80 0.45s 3.2s100 0.20s 2.6s In a large organization, we can gather data to calibrate our own charts and tables like the above.


Deciding how many evaluators to include:Costs

• Fixed costs to organize the evaluation: (no matter how many evaluators):

—Planning

—Preparing materials

—Writing report and communicating results to developers

—Typical estimate for a small system- = 75 hours work

- = $4500


Costs continued

• Variable costs for each evaluator

—Weighted salary for time spent

—Cost of analyzing the evaluator's report

—Typical estimate for an average system- 8 hours work for each evaluator ($480)

- 2 hours work analyzing the report ($120)

- Total = $600

• Total costs = $4500 + 600i

—Where i is the number of evaluators


Deciding how many evaluators to include:Benefits

Total benefits vary substantially depending on nature of system.

In some published studies, the benefit averaged $15000 per problem

On the following slides are two benefit calculation examples


Benefits example 1:small mass-market system

Three factors to consider:

1. Time saved per session per problem

• Will obviously vary from problem to problem and session to session

• Reasonable average 2s saved per session per problem (similar to data shown earlier)

2. Typical number of sessions of use per system

• Assume an average of 50 sessions

3. Number of users (licenses)

• Assume an average of 3000 users


Benefits example 1 - continuedTherefore: total time = 2s * 50 sessions * 3000 users

= 300000 seconds per problem!= 83.3 hours of saved time

If weighted at $60 per hour:• Potential benefit is $5000 per problem

Assuming only half the problems will get fixed:• Benefit = $2500 per problem found

Or added value to a license per problem fixed= 2s * 50 sessions= 100s= $1.66


Benefits example 2:small in-house system

The same three factors to consider:

1. Same average time savings per session (2s)

2. More average sessions over lifetime (200)

3. Fewer users (40)

Therefore: total time

= 2s * 200 sessions * 40 users

= 16000 s / problem

= 4.4 hours of saved time

Using the same weighted value of $60 per hour:

• Potential benefit is $266 per problem

• or $133 per problem actually fixed.


Cost benefit analysis

So will a usability study be economical?

• Let’s put the costs and benefits together


Example 1: Mass-market system

Insp Total benefit % probs total ben/cost

cost /prob found benefits ratio

2 5700 2500 56 280000 49

3 6300 2500 71 355000 56

4 6900 2500 81 405000 59

5 7500 2500 87 435000 58

6 8100 2500 92 460000 57

7 8700 2500 95 475000 55


Example 2: In-house system

Insp Total benefit % probs total ben/cost

cost /prob found benefits ratio

1 5098 132 34 8976 1.8

2 5700 133 56 14896 2.6

3 6300 133 71 18886 3.0

4 6900 133 81 21546 3.1

5 7500 133 87 23142 3.1

6 8100 133 92 24472 3.0

7 8700 133 95 25270 2.9

Odds and Ends



Prototyping

A prototyping tool should:

• Be easy to learn and use

• Possess good visual capabilities

• Allow easy extension

• Give full support for the types of interface being developed

• Support modular programming


Types of prototyping

Horizontal prototyping

• Complete UI, but no underlying functionality

Vertical prototyping

• In-depth functionality of only certain parts

Scenario prototyping

• Selected paths only


The design rationale document

Allows you to keep control of the various alternatives considered and decisions made

Explain your usability decisions so others know the reasoning you followed


Discount usability engineering

User and task observation• Cheap prototyping and scenarios• Simplified thinking aloud• Heuristic evaluation

Key goal, communicate qualitative information to developers in order to improve interfaces

• as opposed to qualitative measures

A study was performed of iterative design involving discount usability engineering

• Improvements of 38% after each stage


Consumability - Closely allied to usability

Ability for customers to get rapid return on investment.

Factors include

• all aspects of usability

• installability / uninstallability

• lack of inter-product dependency/interaction

• low footprint: lack of run-time resource consumption

• backward-compatibility

• integratability

• appropriate prioritization of iterations and features to yield business value

• low purchase cost

Rating corporate maturity with respect to usability



The Model for Usability Maturity (MUM)(proposed by Lethbridge!)

A 5-level CMM-like categorization of organizations

—Based on their interaction with human end-users in software development

—(Each level beyond 1 builds on the previous levels)

• Level 1: Haphazard

• Level 2: Defined input from users / usability awareness

• Level 3: Iterative interaction with users / design for usability

• Level 4: Controlled and measured involvement of users

• Level 5: Continually improving usability


Level 1: Haphazard

If you can get users to use the system, then it is considered good enough!


Level 2: Defined input from users / usability awareness

Users involved in requirements reviews• Feedback from users at reviews is incorporated into the

next stage

Design team members have basic training in usability

Design team adheres to usability standards for look and feel

Reuse of well-understood controls, styles etc.


Level 3: Iterative interaction with users/ design for usability

Users actively involved in decision making

Use case / task analysis

Competitive analysis

Design with careful attention to usability guidelines

Usability design decisions are carefully analyzed and the decisions are recorded

• E.g. options analysis, tables of pros and cons


Level 3: Continued

User input and feedback with repeated prototypes

• From paper prototypes to functional prototypes

Informal qualitative observations of users or heuristic evaluations

Discount usability engineering


Level 4: Controlled and measured involvement of users

User input and feedback at all stages

Design team has trained usability experts or cognitive scientists

Careful cost-benefit optimization including usability

Setting of quantitative usability objectives

• For all parts of the system that will be subject to regular or critical-situation use

• For learnability, efficiency of use, etc.


Level 4: Continued

User-centered design with formal usability studies

Measurement of usability so as to determine progress towards goals


Level 5: Continually improving usability

Active development of new UI understanding and innovations

Formal experiments to validate new UI modalities• metaphors, controls, widgets, styles etc.

Anthropological studies of human tasks• to enable optimized UI design

Scientific study of• Users and their work practices • Usability and the software engineering process


Potential benefits of MUM

It provides a visible framework that companies can use to incrementally improve their practices

If even a few companies adopted it and were certified, they might have a competitive advantage in some situations

Software Usability Course notes for CSI 5122 - University of Ottawa

Documents

Transcript of Software Usability Course notes for CSI 5122 - University of Ottawa