Keeping things in context a comparative evaluation of focus plus context screens, overviews, and...

Keeping Things in Context: A Comparative Evaluation

of Focus Plus Context Screens, Overviews, and Zooming

Patrick Baudisch, Nathaniel Good, Victoria Bellotti, & Pamela SchraedleyInformation Sciences and Technologies Lab/Computer Science Lab

Xerox Palo Alto Research Center

Presented By Debaleena ChattopadhyayI590, Summer 2012

2

Revisiting Definitions

7/10/2012

Multi-scale Documents Static

Dynamic

Visual documents that are too large and detailed to fit on user’s computer screen at a time.E.g. maps, design diagrams.

Documents that remainunchanged unless the user modifies them.E.g. maps.

Information streams that are changing irrespective of user involvement.E.g. games, air traffic control.

3

Revisiting Definitions (contd..)

7/10/2012

f + c

o + d

z + pFocus + Context is a visualization technique for very large and detailed documents.

They contain wall-size low-res displays as context screens and an embedded high-res display region called focus.

Overview + Detail is a multi-window (usually two) visualization technique.

They contain one window (overview) which always displays the entire document, while another window (detail) shows a certain close-up.

Zooming and panning is a technique to display required information sequentially in terms of different views.

4

Legends

7/10/2012

Breakdown or Critique

Reflection

Idea or a plausible alternative

5

Research QuestionsRQ1:For users working with static documents too large and detailed to fit on their screen, controlling for the interfaces used, what is the relationship between display techniques, type of task and user experience (efficiency, effectiveness & satisfaction)?

7/10/2012

RQ2:For users working with dynamic information stream too complex to be perceived at a single resolution of the screen, controlling for the interfaces used, what is the relationship between display techniques, type of task and user experience (efficiency, effectiveness & satisfaction)?

6

Laboratory Study on Static

Documents

• 12 participants• 3 display

techniques ( f + c, o + d, z + p)

• 2 tasks

Laboratory Study on

Dynamic Views

• 8 participants• 2 display

techniques ( f + c, o + d)

• 1 task

The Workflow

7/10/2012

Field Work

Interview 14 multi-scale

document users covering all five

classes of activity.

Derive realistic tasks for controlled experiments

• Board task– verifying connections on a circuit board

• Map task – finding a closest hotel on a map

• Driving simulation task– avoid nails on the road and falling rocks while driving.

Adds to the face

validity and content

validity of the tasks

and hence task

performance

measures

Suffers from

external validity

due to controlled

settings.

7

Study 1: Hypotheses

• H1: Subjects would complete each task faster with the f + c interface than with the z + p interface. (one-tailed hypothesis)

• H2: Subjects would complete each task faster with the f + c interface than with the o + d interface. (one-tailed hypothesis)

• H3: Subjects would report higher satisfaction with the f + c interface than z + p and o + d interfaces. (one-tailed hypothesis)

7/10/2012

8

Study 1: Method details

Participants– 12 Xerox PRAC employees.

7/10/2012

No descriptive statistics of the sample? Was gender controlled? How about subjects’ experience with any of the display techniques or any of the tasks?

Independent Variables– Display Techniques (3 levels) and type of tasks (2 levels)

Dependent Variables– User Experience operationalized as time on task, accuracy in performing tasks and self reported questionnaire (adapted from QUIS)

No clear definition of the measures other than the questionnaire. How was accuracy in performing tasks measured? What constitutes an error?

9

Study 1: Method details (contd..)

7/10/2012

Research Design– 3 (Display techniques) X 2 (task types)

Map Task Board Task

z + p

f + c

o + d

Data Analysis– Two-way repeated measures ANOVAWhen we carry out our experimental manipulation with same people, the within-participant variance is made up of = the effect of manipulation + individual differences in performance.

We have three experimental conditions for display techniques. But since no corrected F-value is reported it is assumed that the data did not violate the assumption of sphericity (Mauchly’s test was not significant).

Sphericity refers to the equality of variances of the differences between treatment levels.

Two Factors

10

Study 1: Results

7/10/2012

Average Task completion times in seconds

z + p f + c o + d0

100

200

300

400

500

600

700

Map TaskBoard Task

Display Technique

Mea

n tim

e on

task

Graphing the means

• Effect of display type• No visible effect of

task type• No visible interaction

effect

11

Study 1: Results (contd..)

7/10/2012

Source Type III sum of squares

df Mean Square

F Sig.

Display Type Sphericity assumed 2 19.78 .000

Error (Display Type) Sphericity assumed 22

Task type Sphericity assumed 1 2.63 > .05

Error (Task type) Sphericity assumed 11

Display * Task Sphericity assumed 2 1.76 > .05

Error (Display * Task) Sphericity assumed 22

Test of within subject effects (Plausible table)Two factors: display type and task type

One effect: Completion timeDisplay type has a significant effect on task completion times.

dfM

dfR

12


7/10/2012


df Mean Square

F Sig.

Display Type Sphericity assumed 2 2.23 >.05


Task type Sphericity assumed 1 3547 .000


Display * Task Sphericity assumed 2 .60 > .05


Test of within subject effects (Plausible table)Two factors: Display type and task type

One effect: Task accuracy

Task type has a significant effect on task accuracy.

And how do we measure that?

dfM

dfR

13


7/10/2012


df Mean Square

F Sig.



Task type Sphericity assumed 1 0.011 >.05


Display * Task Sphericity assumed 2 0.97 > .05


Test of within subject effects (Plausible table)Two factors: Display type and task type

One effect: User Satisfaction

Display type has a significant effect on user satisfaction.

dfM

dfR

14


7/10/2012

Source Display type Task type Type III sum of squares

df Mean Square

F Sig.

Display Type o + d vs. f + c z + p vs. f + c

11

19.5436.52

.000

.000Error o + d vs. f + c z + p vs. f + c

1111

Test of within subject contrasts (Plausible table)Two factors: display type and task type

One effect: Completion timef + c yielded faster completion times than both the other display types.

Planned contrasts also suggested: f + c was favored for user satisfaction over other two displays as well as for easier orientation to the displays.

dfM

dfR

15

Study 1: Discussion• Switching effort was found to be the major factor to influence the experience

of an interface.• Problems rose like blurriness of projection and shadow casting.

7/10/2012

• Multivariate analysis with all the dependent variables.• Control for gender.• Better operationalization of the measures.• Report Effect Size: (More so, when it is this large)

• ro + d vs. f + c = 0.79 and rz + p vs. f + c = 0.88 (Very Large Effect Sizes)• Given the options, maybe a between-group design is not a good

idea. With N = 12 in each of the 6 (3 X 2) conditions, we get dfR as 6 X 11 = 66. The critical F-value required for significance (p < .001) also decreases, but at the loss of the effect size.

16

Study 2: Hypotheses

• H1: Subjects in the car task would produce lower error rates when using f + c interface compared to o + d interface. (one-tailed hypothesis)

• H2: Subjects would ne more satisfied using the f + c interface for the car task than the o + d interface. (one-tailed hypothesis)

7/10/2012

Study 2: Method details

7/10/2012 17

Participants– 8 Xerox PRAC employees. (subset of previous participants)

Won’t using the same people cater to additional change for the previous experimental manipulation? No descriptive statistics of the sample? Was gender controlled?

Independent Variables– Display Techniques (2 levels) and collision type (2 levels)

Dependent Variables– User Experience operationalized as accuracy in performing tasks and self reported ranking for the preferences of the interfaces.

Why user efficiency was not measured? Time on task?

18

Study 2: Method details (contd..)

7/10/2012

Research Design– 2 (Display techniques) X 2 (collision types)

f + c o + d

Run-over nails

Rocks hit

Data Analysis– Two-way repeated measures ANOVA

Two Factors

19

Study 2: Results

7/10/2012

Mean number of collisions subjects caused in the car task

o + d f + c0

5

10

15

20

25

Run over nailsRocks hit

Display type

Num

ber o

f err

ors

Graphing the means

• Effect of display type• More number of errors

for collision with nails, so effect of collision type.

• No visible interaction effect

20


7/10/2012


df Mean Square

F Sig.



Collision type Sphericity assumed 1 19.71 <.01


Display * Collision Sphericity assumed 1

Error (Display * Collision) Sphericity assumed 7

Test of within subject effects (Plausible table)Two factors: display type and collision type

One effect: ErrorsBoth collision type and display type has a significant effect on task accuracy.

dfM

dfR

21

Study 2: Discussion

• Also users reported higher satisfaction (significant) with f + c interface than o + d interface.

• Better performance in f + c may be because of the use of the peripheral vision.

• Did not report effect size

• Is it because o + d interface had two different screens that hindered peripheral vision? Was the overview screen in the optimal range of peripheral vision for the users? The f + c interface was overall larger in size. Was it creating an immersion effect on playing the game?

7/10/2012

22

Connecting the dots..• Two controlled experiments provided evidence of better

performance of f + c interfaces for navigating static and dynamic multi-scale documents.

• Authors claim that these results suggest that f +c screens can enable individuals to carry out monitoring + interaction tasks that are now typically done by a team of at least two. Sounds a little ambitious. No baseline performance evaluation with group of users were done in the study.

• No questions were asked to users to understand the effects of low-res context screen (f + c) compared to high-res overview (o + d). Did it predict the errors in f + c interfaces?

7/10/2012

23

Thank you

Questions?

7/10/2012

Keeping things in context a comparative evaluation of focus plus context screens, overviews, and...

Education

Transcript of Keeping things in context a comparative evaluation of focus plus context screens, overviews, and...