The “Map Trap”? An evaluation of map versus text-based interfaces for location-based mobile search services

International World Wide Web Conference, 2010Session: Visual interfaces

Karen Church, Joachim Neumann, Mauro Cherubini and Nuria OliverTelefonica Research, Barcelona, Spain

2010.7.2Presented by Seunghun Ok, IDB

2

Outline Introduction

The SSB Prototypes

Evaluation

Results

Discussion & Implications

Conclusions

3

Introduction (1/2) Users of super-powered mobile handsets tend to use

the Web more heavily than users of simpler devices– Such as iPhone

The world of mobile information access is evolving– Investing mobile version of standard Web services

The interface design of mobile Web services display information– According to which it refers

Geographical

– Based on some order or ranking Time or search engine ranking

4

Introduction (2/2) The most important concept to consider when design-

ing mobile interfaces is “context”– Where an application is used– How information is entered and interacted with

SocialSearchBrowser (SSB), mobile search proto-type– SSB gives users the ability to connect with other users while

on-the-go and ask them geo-located questions

The goal of this paper– To analyze the impact that the type of user interface has on

the search and information discovery experience of mobile users

5

The SSB Prototype (1/4) To enhance the search and information discovery ex-

perience of mobile users– By pro-actively displaying what other users have been

searching for in a particular location

SSB presents the users with a view of evolving search activities– That is sensitive to their context

Two core interfaces: SSBmap and SSBtext

6

The SSB Prototype (2/4) The software architecture consists of two components

– An iPhone application allows users to Issue queries Browse existing queries and their answers Answer other people’s queries

– A server Synchronizes and stores the queries Answers from both application in a common database

Difference between SSBmap and SSBtext

– Representation of user’s location, location of queries and an-swers

– SSBmap

Represents visually with a map

– SSBtext

Represents as textual addresses arranged in list format

7

The SSB Prototype (3/4)

SSBmap

– Map-based interface Provides users with a sense of place at a glance

SSBtext

– Text-based interface

8

The SSB Prototype (4/4) Two interactive filters

– Time filter Enables selective display of queries based on time

– Query similarity filter Enables users to limit the queries to those that overlap with the

queries that have been previously entered by the user him/herself

Query details– Header

Original query string

– Answers Human generated answers

– Local search results A set of localized search results extracted from Google’s local search

service

– Event search results A set of localized event listings

9

Outline Introduction The SSB Prototypes

Evaluation– Participants– Procedure– Resolving Locations via Wizard-of-Oz

Results Discussion & Implications Conclusions

10

Participants Participants are required to own an iPhone or iPod

Touch

34 users take part in and complete the live field study– 32 users with an iPhone– 2 users with an iPod Touch– 31 males, 3 females– Ranged in age between 20 and 55 (avg 32.2)

– Lived in various counties in Ireland– Worked in a wide range of employment sectors

Including IT, Accountancy, Banking, Healthcare, Construction, Public

– They used Internet and mobile phone every day

11

Procedure

Each participant was required to install the SSBmap and SSBtext on their personal iPhone or iPod Touch de-vice

The live field study ran for a period of 27 days during September 2009

Participants were asked to complete a post-study survey to gather subjective information on their ex-periences with the two applications

12

Resolving Locations via Wizard-of-Oz The location is manually resolved using a Wizard-of-

Oz (WoZ) approach– Instead of relying on automatic geo-coding

Would fail in cases like “at the Temple Bar side to the Ha’Penny Bridge”

– Wizard of Oz experiment is a research experiment in which subjects interact with a computer system that subjects be-lieve to be autonomous, but which is actually being operated or partially operated by an unseen human being

Employ 3 mechanical turks – Resolve the textual locations of queries and answers to phys-

ical latitude/longitude values– Mechanical turks means a fake chess-playing machine

13

Outline Introduction The SSB Prototypes Evaluation

Results– Basic Usage Patterns– Experience Samples– Content Classification: Queries & Answers

Query Classification Answer Classification

– Location Precision

Discussion & Implications Conclusions

14

Basic Usage Patterns The 34 participants generated 1266 interactions in total

– 236 queries, 835 query look-ups, 195 answers

Conducts an independent samples t-test– Participants produce more queries through the map interface than

through the text interface (t[34, 66] = 2.60, p < .05)

– Participants retrieved content more often through the text interface than through the map interface

(t[34, 66] = -3.35, p < .05)

– Participants answered queries more often through the text interface than through the map interface

(t[34, 66] = -1.66, p < .05)

15

Experience Samples Collects 94 samples throughout the 1 month period

– 41 via SSBmap and 53 via SSBtext

Samples via SSBmap

– Definite visual and location-specific aspect– Easier mechanism to look at different streets– Better visual overview and works well when attempting to pin-

point “local” queries

Samples via SSBtext

– Accessing a query– Viewing an answer submitted to a query– Seeing if there were any new queries that need to be answered– Quick and easy– Enable a more efficient means of looking up the details of a

query

16

Content Classification: Queries & An-swers

17

Query Classification 1. General queries: Focus on finding an answer to a particular

question– 1.1 Business / Service– 1.2 Other queries

2. Location explicit queries: Describe a query in which the user’s current location has a definite impact on the information need and the answer expected– 2.1 Addresses / directions– 2.2 Business / services– 2.3 Recommendation / opinion

3. Location implicit queries: Describe needs in which the user is searching for a physical location either directly or indirectly– 3.1 Businesses / services– 3.2 Recommendations

4. Misc queries: All queries that could not be classified into one of the other types

18

Answer Classification 1. General Answers: Describe a non location-specific answer

– 1.1 Business / service– 1.2 Recommendation / opinion– 1.3 Other

2. Location explicit answers: Describe an answer that includes an explicit location cue– 2.1 Address / directions– 2.2 Business / service– 2.3 Recommendation / opinion

3. Conversational Answers: Are probes for additional details or statements that appear to be motivated by the desire to chat

4. Application Related

5. Miscellaneous Answers

19

Location Precision (1/2) In SSBtext, users can choose to enter a location in free text form

– Manually classified the locations into one of five types based on their geo-graphical precision

1. Precise: Locations refer to very specific places

2. Street-level: Locations list a specific street name but no ex-act street number is provided

3. Neighborhood: Refer to a small area or borough within a city

4. City / county: Refer to a particular city or county within Ire-land

5. Imprecise: Do not provide the user with any relevant loca-tion details

20

Location Precision (2/2)

21

Outline Introduction The SSB Prototypes Evaluation Results

Discussion & Implications– Choice of interface

Personal Preferences Situational Context Information Need

– Location precision– Hybrid Interface ≠ Text + Map

Conclusions

22

Choice of interface The choice of user interface depends on three factors

– Personal preferences– Situational context– Information need

23

Personal Preferences Hypothesis 1 - Gender affects to the choice of user in-

terface– Men tend to have better spatial awareness skills than women– Men tend to orientate themselves more easily– The 3 women who took part in our user study indicated that

they preferred SSBtext

But, imbalance in gender exists

Hypothesis 2 – Users’ past experiences with similar applications will also have an impact– Users who preferred SSBmap rated their experiences and

knowledge of mapping services more highly than users who preferred SSBtext

Take-away message 1: Track the application usage/behavior of their users

24

Situational Context

Maps are a useful interface when trying to understand one’s surroundings or to visualize a physical area

Take-away message 2: Infer the situational context of the end-user automatically

25

Information Need The participants’ information need had a strong influ-

ence on the preferred interface– Participants seeking information related to a specific address

had a strong preference for SSBmap

– Participants preferred SSBtext when answering queries from other users

Take-away message 3: Automatically determining the intent of the user would allow designers to present the most appropriate interface type

26

Location precision (1/2)

SSBtext allowed users to specify the location of both queries and answers in more vague terms

Participants were more inclined to choose SSBtext when answering a query

– The effort required to submit an answer and its location via SSBtext was lower than the effort required to accomplish the same task via SSBmap

– We, as human-beings, often do not need exact locations to orientate ourselves and locate items of interest

27

Location precision (2/2) High-level location details are probably sufficient in

many circumstances– Such as “around the corner” or “down that street”– Precise locations are not necessarily needed or desired at all

times

Providing support for fuzzy or vague locations is impor-tant from a privacy perspective– UI perspective should provide users with more control in specify-

ing vague or ambiguous locations

Take-away message 4: Mobile search and information access tools should provide support for users to specify fuzzy or vague locations in order to address– Growing privacy concerns of mobile users– Increasing desires for ambiguous locations

28

Hybrid Interface ≠ Text + Map The user interface that is solely based on a map vi-

sualization is not optimal

However, an ideal hybrid solution is not a simple par-allel implementation of two interface but rather a smart mix

Take-away message 5: Location-based search tools should support both text-based and map-based inter-face modalities. However, the integration of the two modalities in a single hybrid application should in-volve a mash-up that supports users’ interactions and intentions while on-the-move.

29

Conclusions The majority of existing mobile location based ser-

vices are built on top of a map-based visualiztion

The choice of mobile interface depends on a range of factors– Including the user’s personal references, their information

need, their situational context, their need/desire

Hybrid solution that considers each of five take away messages is the way forward in terms of providing useful mobile information access services