Usability Evaluation of Presentation Tools

A Concept-based Approach

Michael Schumann 205/27/12

Content

Presentation tools User-Task-System triad and concepts CASSM Evaluation experiments Conclusion and findings

Michael Schumann 305/27/12

Presentation tools

Black board White board Overhead projector Media projector Active board (SmartBoard)

Michael Schumann 405/27/12

eBoard screen shot

Michael Schumann 505/27/12

PowerPoint screen shot

Michael Schumann 605/27/12

Notebook screen shot

Michael Schumann 705/27/12

What is a concept?

A unit of thought. The semantic content of a concept can be re-expressed by a combination of other and different concepts, which may vary from one language or culture to another. (ISO 5963:1985)

A concept is a stable mental representation of objects, classes, properties, and relations. When we encounter a new object or event for the first time, we draw upon our mental store of concepts in order to identify it. One of the most important parts of the human learning process is concept-formation, where, after a number of distinct experiences of the ‘same' object or event, we acquire, by a process of induction, a concept for it. (Univ. of Sussex)

Michael Schumann 805/27/12

The U-T-S Diagram

User

System Task

Conceptual Gap

Physical Interfa

ce

Conceptual Interface

Michael Schumann 905/27/12

CASSM

Concept-based Analysis of Surface and Structural Misfits

Two parts Concept comparison between user,

interface and system Misfit investigation in relationships (based

on Cognitive Dimensions)

Michael Schumann 1005/27/12

The U-T-S Diagram

User

System Task

Conceptual Gap

Physical Interfa

ce

Conceptual Interface

Surface and Structural Misfits

Michael Schumann 1105/27/12

Surface analysis

Investigation on concepts System: manuals, documentation Interface: general HCI skills, heuristic

evaluation User: interview, observation Concepts (entities) have attributes Entities and attributes can, …, cannot be

changed or manipulated

Michael Schumann 1205/27/12

Limitations and aim of CASSM

Show potential usability problems What concepts does the user have to

learn? Where are interface deficiencies? No user performance or cognitive

knowledge tested No system redundancy, error handling

etc.

Michael Schumann 1305/27/12

Evaluation experiments based on Surface Analysis

System and interface analysis User concept analysis

User profile (persona): lecturer Questionnaire Interview Observation

Concept matching Further, more detailed analysis possible

Michael Schumann 1405/27/12

Evaluation experiment topic

“Explain the principle of a pendulum” Should invoke

Text, labels Graphics Movement, gestures

Can invoke Pictures Animations

Michael Schumann 1505/27/12

Evaluation sessions I

3 student groups: subject and observers Observers study system and note down

system/interface concepts Subject completes questionnaire Subject fulfils task while observers note

problems and subject’s user concepts Interview to clarify concepts

Michael Schumann 1605/27/12

Evaluation sessions II

Lecturer and observer Questionnaire filling Subject fulfils task and is being observed Observer checks concepts against given

list Interview to clarify concepts

Michael Schumann 1705/27/12

Cognitive Walkthrough eBoard

Exploratory evaluation of user interfaces One analyst creates user stories Representative tasks eBoard analysis

Menu labelling changed to object-oriented “Object” vs. “Graphical Object” “Eraser | Activate” > “Eraser | Start

erasing”

Michael Schumann 1805/27/12

Conclusions and findings

CASSM Presentation tool concepts Usability issues in presentation tools Evaluation experiments

Michael Schumann 1905/27/12

Conclusion: CASSM

No handling of abstract concepts Expert analysis skills required No view on procedures; complementary

use walkthroughs or task analysis recommended

Addresses concepts sufficiently Discussion of concept implementation

Michael Schumann 2005/27/12

Conclusion: PT concepts I

Presentation tools: complex Preparation and delivery stage Learning/Teaching styles

Adaptability is a challenge Frequency of discovered concepts point

at essential functionality

Michael Schumann 2105/27/12

Conclusion: PT concepts II

Presentation tool concept frequency Pre-preparation Point and writing device Text and drawing most essential Animations desired, but time-demanding Annotations and remarks rather oral than

written down Page size, page amount extendable

Michael Schumann 2205/27/12

Conclusion: Usability issues IPowerPoint

Tremendous amount of features Focus on textual elements Drawings secondary handling

problems with drawing tools No thorough analysis conducted!

Michael Schumann 2305/27/12

Conclusion: Usability issues INotebook

Toolbar items confusing Style and format changes difficult Eraser not applicable on vector graphics Licensing policy makes it uninteresting

for use in preparation

Michael Schumann 2405/27/12

Conclusion: Usability issues IeBoard

Testing unveiled bugs and implementation lacks

Menu labelling Formatting needs to be easy accessed Copy and paste strategies Slide sorter view missing

Michael Schumann 2505/27/12

Conclusion: Evaluation experiments

Different approaches on task solving Target group and non-target group

experiments which concepts known? Observations are time and money

consuming surplus for analysis? Concept-finding is problematic Video recording, screen capturing,

annotations

Michael Schumann 2605/27/12

Thank you for your attention!

Please feel free to question me…