Exploring the Cognitive Consequences of Social Search

April 9, 2009

EXPLORING THE COGNITIVE CONSEQUENCES OF SOCIAL SEARCH

CHI ’09

UC San Diego

PARCPARC

Brynn M. Evans

Sanjay KairamPeter Pirolli

Tuesday, April 14, 2009

MotivationResearch Goal

How do people make use of social resources during search tasks?

In what ways do social interactions cognitively benefit search?


As a study of social search, we are particularly interested in HOW people make use of social resources during search tasks.

Motivation

Traditional search fails with exploratory queries

Motivation


This is important because Google sometimes fails us.

In fact, exploratory queries are difficult to answer with traditional search alone (there were several sessions at CHI that have looked at this issue).

Social interactions are common during search

Motivation

Photo Credit: David Wild

MORRIS 2008; EVANS & CHI 2008


Yet prior work has shown that people do ask friends for help during search.

Social interactions are actually quite common during search (between half and two-thirds of searches may involve social interactions).

Motivation

Social interactions play an important cognitive role

Photo Credit: http://www.infodiv.unimelb.edu.au/tss/archive/history.html

HATCH & GARDNER 1993; KARASAVVIDIS 2002


Additionally, social interactions play an important cognitive role in problem solving, brainstorming, and learning tasks. People help each other think through problems and reframe issues.

This has been documented in classrooms, organizations, libraries, and other physical environments.

http://www.infodiv.unimelb.edu.au/tss/archive/history.html

http://www.infodiv.unimelb.edu.au/tss/archive/history.html

Study design

selected in pre-test survey for being:

• expert searchers

• highly social ‣ 55-1000 friends on primary social network

PARTICIPANTS:

PROTOCOL: • 2 Google-hard search tasks• Talk-aloud protocol• Video protocol analysis

(N=8)


To study the benefit of social interactions in search, we recruited 8 subjects who were expert searchers, highly social.

They performed 2 Google-hard search tasks in two seach conditions. We video recorded all interactions and asked them to talk-aloud while they were searching.

Study design: tasks

“If we lowered the speed limit nationally to 55 mph, how many fewer barrels of oil would the U.S. consume every year?”

55 MPH

“What role does pyrolytic oil (or pyrolysis) play in the debate over carbon emissions?”

PYROLYTIC OIL


These were our task questions.

Study design: conditions

NON-SOCIAL CONDITION

• search engines (Google, Yahoo)

•Wikipedia


Subjects received each task in one of two conditions.

In the non-social condition, they performed an otherwise typical web search...

Study design: conditions

SOCIAL CONDITION

• friends (email, phone, IM, etc.)

• social networks

•blogs

•Question-Answer sites


In the Social Condition, they were restricted to social resources only: friends, social networks, blogs, question-answer sites...

Three primary social strategies

TARGETED ASKING

NETWORK ASKING

SEARCHING

Photo Credit: Timothy Morgan


Our analysis is restricted to the social condition right now, since we are primarily interested in HOW people exploit their social enviroments.

We found that people used one of three primary social tactics to answer our task problems

!"

#"

$"

%"

&"

'"

("

)"

*"

+"

#!"

##"

#$"

#%"

!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"

-./012"345" 607/18"9:;<=" 607/18"9:/.>=" ?8.@5.@A"

!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"

Task performance

SS03

TIME



You can see these strategies reflected in the color coding of behaviors on one subject’s timeline.This person begins by searching over MetaFilter; then posts questions on MetaFilter & Twitter; and finally IMs with a friend to get additional help.But as you can also see, these activities are interspersed with many other behaviors (as expected) -- such as the gray bars: thinking/synthesizing the information he’s come across

!"

#"

$"

%"

&"

'"

("

)"

*"

+"

#!"

##"

#$"

#%"

!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"

-./012"345" 607/18"9:;<=" 607/18"9:/.>=" ?8.@5.@A"

!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"

Task performance

SS03

TIME

SEARCHING

[ ]

SEARCHING




!"

#"

$"

%"

&"

'"

("

)"

*"

+"

#!"

##"

#$"

#%"

!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"

-./012"345" 607/18"9:;<=" 607/18"9:/.>=" ?8.@5.@A"

!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"

Task performance

SS03

TIME

NETWORK ASKING

[ ]

NETWORK ASKING




!"

#"

$"

%"

&"

'"

("

)"

*"

+"

#!"

##"

#$"

#%"

!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"

-./012"345" 607/18"9:;<=" 607/18"9:/.>=" ?8.@5.@A"

!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"

Task performance

SS03

TIME

TARGETED ASKING

[ ]

TARGETED ASKING




!"

#"

$"

%"

&"

'"

("

)"

*"

+"

#!"

##"

#$"

#%"

!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"

-./012"345" 607/18"9:;<=" 607/18"9:/.>=" ?8.@5.@A"

!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"

Task performance

SS03

TIME

THINKINGRESPONSES?OTHER





Of course, subjects were highly diverse in the strategies they employed, as can be seen by the various color coding patterns in their search episodes. This shows patterns for all 8 subjects (in the social condition).Some didn’t have an expert friend on hand -- (they didn’t have the “lifeline” to call.) Or they didn’t usually post questions to their friends on Facebook, for example. For this reason, not everyone performed all three social strategies.

!"

#"

$"

%"

&"

'"

("

)"

*"

+"

#!"

##"

#$"

#%"

!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"

-./012"345" 607/18"9:;<=" 607/18"9:/.>=" ?8.@5.@A"

!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"

Task performance

!"

#"

$"

%"

&"

'"

("

)"

*"

+"

#!"

##"

#$"

#%"

!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"

-./012"345" 607/18"9:;<=" 607/18"9:/.>=" ?8.@5.@A"

!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"

!"

#"

$"

%"

&"

'"

("

)"

*"

+"

#!"

##"

#$"

#%"

!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"

-./012"345" 607/18"9:;<=" 607/18"9:/.>=" ?8.@5.@A"

!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"

each

lin

e =

= o

ne

fact

SS03


Next, we mapped the facts subjects found to the type of behavior they were engaged in.

...the size of the bubble represents the depth of processing of that one fact. And you can see that some facts do become synthesized more over time. [see fact #2, #3, #4, #5, #6]

Results

ID#social tactics

#facts discovered

S01 3 6

S02 1 2

S03 3 8

S04 1 1

S05 2 5

S06 2 1

S07 1 1

S08 3 3

Spearman R: 0.77

combining social tactics correlates better to performance on our tasks than:

• social network size• network diversity

(as measured by the position generator)

• background knowledge / intrinsic interest in topic


We found that any one social tactic used on its own didn’t produce as good performance outcomes as combining social tactics together.

This is reflected in both the total number of facts discovered in the session and how deeply users actually processed information (which isn’t elaborated on here).

This suggests that accessing people with different technologies makes a difference in their benefit to you.

Results

deeper cognitive processing of information while:

“Now, what could I say?”

“Let’s see...what do I really want to be asking?”

Composing the question Receiving Information

Network (Asking)

More thinking, contemplation while crafting query (3/5 users)

No pondering or mulling over of network responses (0/4 users)

Targeted (Asking)

Little thinking or reformulation of problem statement

(2/7 users)

More synthesis of information from friends’ replies (4/5 users)


And looking at the cognitive processing of information, this is absolutely what we saw.

However the story is more nuanced than that:

If we divided the search phase into the question and answering components:We found that users had more thinking and contemplation of their search problem while they were composing the question (e.g., when posting it to a social network...)

Composing the question Receiving Information

Network (Asking)

More thinking, contemplation while crafting query (3/5 users)

No pondering or mulling over of network responses (0/4 users)

Targeted (Asking)

Little thinking or reformulation of problem statement

(2/7 users)

More synthesis of information from friends’ replies (4/5 users)

Results



In contrast, users more deeply integrated and synthesized information from replies shared from friends who they interacted with one-on-one.

This could be due to a number of different things. For one, replies on social networks tended to be goofy, silly, or off-target. They didn’t contain any facts that helped subjects advance their understanding of the search question. (In fact, when we followed up with these respondents, they reported wanting to start a conversation with our users rather than help with a substantive reply).

Replies from targeted friends, on the other hand, were content-heavy. Even replies shared through asynchronous channels (an email, a single IM) caused subjects to pause and really think about the information they just received. Respondents here reported wanting to help our users develop content; they were less interested in the “social” aspect of the communication.

Results


!"

#"

$"

%"

&"

'"

("

)"

*"

+"

#!"

##"

#$"

#%"

!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"

-./012"345" 607/18"9:;<=" 607/18"9:/.>=" ?8.@5.@A"

!,!!,!!" !,!),#$" !,#&,$&" !,$#,%(" !,$*,&*" !,%(,!!"

SS03

More synthesis of information from friends’

replies (4/5 users)


We can see that users paused to think about shared information from targeted asking in one subject’s timeline: The green (asking friends) and gray bands (thinking episodes) pattern each other. This is only illustrated for one subject, but was true for most of the subjects who engaged in targeted asking.

Conclusions & implications

• People serve as more than information sources; they also provide cognitive support.

• But different types of social engagement benefit the search process in different ways:

• Problem reformulation while posting questions on social networks

• Integration of information from friends’ replies.

IMPLICATIONS: Can we design new tools that leverage these cognitive benefits during search tasks?

CONCLUSIONS:


So we have seen how different types of social engagement benefit the search process.

People serve as more than information resources; they also provide cognitive benefits!

In particular we saw problem reformulation while posting questions on social networks and when integrating information from friends’ replies.

Could we design an integrated search tool that takes this into account? For example, if users posted questions in social networking sites, could we route their question to domain experts from their personal network who then might feel more compelled to reply? Could a model like this help maximize the cognitive benefits for searchers during both questioning and answering phases of the search query?

At the same time, we should learn more about why people are motivated to respond appropriately to targeted questions, but seem to provide less relevant replies on social networks.

Acknowledgements

our participants

SRC judges (CHI ’09)

Augmented Social Cognition group at PARC!

BRYNN M. EVANS [email protected] http://brynnevans.com

A GREAT THANKS TO:


mailto:[email protected]




Exploring the Cognitive Consequences of Social Search

Technology

Transcript of Exploring the Cognitive Consequences of Social Search