Post on 16-Aug-2015
A Presenta*on from the NewMR “Social Media Research” Event
9 October, 2012
All copyright owned by The Future Place and the presenters of the material For more informa:on about NewMR events visit newmr.org
Enriching Social Data for Market Research Jasper Snyder, Converseon
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2
Enriching Social Data for Market Research
#NewMR
Jasper Snyder 9 October 2012
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 3
Questions for Today’s Session
• What sorts of questions can
social media research and
analysis answer?
• How do you move from millions
and billions of un- and semi-
structured documents to
actionable market research
insights?
• How can social media-based
market research be tailored to
different client use cases?
• How does machine learning
work, and why does it matter for
market research?
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 4
Social-media research can support both traditional market research goals and PR functions.
Communications Functions through Social Media Monitoring
Traditional Market Research through Social Media Listening
• Thoughts and opinions about products and brands
• Market awareness of products or brands
• Purchase triggers
• Consumer complaints and product malfunctions
• Crisis monitoring and
response
• Reputation management
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2
Social Media Monitoring
5
These two use cases – market research and communications – closely align with two services.
Social Listening
When what matters most is understanding a consumer segment or market.
Goal is to understand a population “out there” in the world. Higher tolerance for missing content. Lower tolerance for irrelevant content.
When what matters most is delivering customer service, navigating a crisis situation or detecting reputation threats. Goal is comprehensive, real time coverage. Higher tolerance for irrelevant content. Lower tolerance for missing content.
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 6
The Social Media Research Process: From Raw Data to Insights
2. Data Enrichment
3. Analysis & Insight
Generation
1. Data Collection
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2
Stage 1: Social Data Collection
2. Data Enrichment
3. Analysis & Insight
Generation
1. Data Collection
Primary Challenges:
1. Pull in relevant data and metadata
2. Coverage of appropriate social media channels
3. Eliminate spam and irrelevant content.
Primary Goal:
Identify and acquire the data that can answer your business questions.
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 8
Stage 2: Data Enrichment
2. Data Enrichment
3. Analysis & Insight
Generation
1. Data Collection
Primary Challenges:
1. Data normalization
2. Classification
3. Scalability
Primary Goal:
Implement document- and sub-document-level enrichments like topic, consumer segment, emotion and sentiment.
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 9
Stage 3: Analysis & Insight Generation
2. Data Enrichment
3. Analysis & Insight
Generation
1. Data Collection
Primary Challenges:
1. Reliability
2. Strategic Value
Primary Goal:
Connect the dots between a suite of metrics and data points in order to reach sound strategic conclusions.
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2
Social Media Channel Approx. Monthly Volume Furthermore…
Blogs 30 million new posts On-site comments and social cues and sharing
Facebook 1.8 billion status updates Social cues (e.g., “likes”) and comments
Twitter 4 billion tweets Social cues like favoriting and flagging other users
YouTube 400 million social actions 240 years of video content uploaded each month
10
Social media represent a massive compendium of documents…
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 11
…and these documents are all of different types.
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 12
Harvesting Data and Metadata from Social Media Documents: A Tweet Dissected
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 13
Harvesting Data and Metadata from Social Media Documents: A Tweet Dissected
• Author Name • Text • Publication Date • Some hashtags
Datapoints:
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 14
Harvesting Data and Metadata from Social Media Documents: A Tweet Dissected
• Person or tweet that a tweet is in reply to
• Follower count of author • Times retweeted • Times favorited • Author description
Metadata:
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 15
Sorting Social Metadata
Tweets that contain #Ford in the text.
A
B
C
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 16
Relevancy as a Sorting Task…
Relevant Documents
Irrelevant Documents
All Documents Containing Your Boolean Query
All Social Media Documents • Spam
• Documents not in target language (e.g., not English)
• Contain keyword but not relevant to client question
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 17
Data Enrichment: What Should We Measure?
Metric Explanation Sentiment Does the author make a negative or positive
point about a product or brand? Topics What topic is the author talking about the
product or brand in relation to? Purchase Stage Has the author of a document already
purchased the product when writing about it online?
Consumer Segmentation What segment is the document’s author from?
Emotions What emotions do authors express toward the target brand or product?
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 18
Data Enrichment: What Should We Measure?
Metric Sorting Categories Sentiment Positive, negative, neutral Topics Pre-selected topic and unexpected topics Purchase Stage Before making a purchase or after. Consumer Segmentation Young male, middle-aged woman, etc. Emotions Joy, anticipation, surprise, fear, etc.
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 19
How can we implement the sorting tasks we’ve discussed so far?
Sorting Tasks
Human Sorters Machine Sorters
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 20
Q: How do you know when a computer is correct?
A: The same way you know that a human is correct:
“I know it when I see it…”
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 21
Establishing A Basis for How Well Humans Agree With One Another
Tweet Coder 1 Coder 2
I do not like the cats with thumbs “advert”
Disgust Anger
I say that video is real, definitely.
Trust No Emotion Expressed
Item Coder 1 Coder 2
1 Positive Positive
2 Positive Neutral
3 Neutral Neutral
4 Negative Positive
etc. … …
Example 1: Inter-Coder Agreement on Sentiment Example 2: Inter-Coder Agreement on Emotion
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 22
Using Human Parallel Coding to Establish Gold Standards
Confusion Matrix: Human as Gold Standard
Raw Accuracy: 61.5%
POSITIVE NEGATIVE NEUTRAL TOTAL POSITIVE 365 24 159 548
NEGATIVE 57 81 65 203 NEUTRAL 274 60 415 749
TOTAL 696 165 639 1500
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 23
Using A Credit Matrix to Create Improved Measurement
POSITIVE NEGATIVE NEUTRAL POSITIVE 100% 0% 50%
NEGATIVE 0% 100% 50% NEUTRAL 50% 50% 100%
Credit Matrix
POSITIVE NEGATIVE NEUTRAL POSITIVE 365 24 159
NEGATIVE 57 81 65 NEUTRAL 274 60 415
Confusion Matrix: Human 1 as Gold Standard
Partial Credit Figure of Merit: 82.3%
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 24
But how does the machine learn?
1. Collection of Human Annotated Data
2. Machine ingests coded data and finds patterns in each category classification
3. Machine applies model from step two on raw data. Results are compared to human coding of same material.
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 25
In conclusion….
• Get specific about your use case
– is it market research or
monitoring?
• Select metrics that speak to your
business questions
• Take advantage of data (e.g., the
text of what people say), but also
meta-data (e.g., the additional
information that most social
media documents contain).
• Understand that machines will
rarely make “perfect” decisions,
because classifications and
judgments are inherently
subjective, but we can help them.
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2
Ray Poynter VCU, Vision Critical
Q & A
Jasper Snyder Converseon
© 2011 Converseon Inc. Proprietary and Confidential Speaker: Jasper Snyder, Converseon
NewMR Social Media Research Event, 9 October 2012, Session 2 27
Jasper Snyder, VP, Converseon jsnyder@converseon.com
Thank you