Extracting meaning from Social Media Monitoring

or…

12/04/2023

Keith Woods-Holder@socialitekwh

2

The tools used often present you with this…

1st Generation 2nd Generation 3rd Generation 4th Generation Automated identification of

keywords Manual sentiment mark-up

with article level values Simple totals positive, neutral,

negative

Concept of 'brand equity /reputation' as a commercialasset.

Integration of measurementwith advertising / marketingevolution of 'corporatescorecard metrics'

Metrics integrated into writtenreports, access to underlyingdata difficult / impossible, blackbox methodologies used toprotect tool vendors IP atexpense of client's needs

Business Drivers Business Drivers Business Drivers Business Drivers

Deliverables Deliverables Deliverables Deliverables

Era - 1980s to present 1995 to present 2003 to present 2010

Tool providers include: Cymfony Autonomy Lexalytics• Brandwatch

Tool providers include: Echo Research Biz360 Carma Metrica

First attempts at measuring sentiment and messaging withproducts and brands

Metrics still mostly in printedreports which eliminateadvantages in speed of analysis

'Black Box' methods makeprogress / refinement slow

Shortening business cycles andglobalisation of operationspush leading brands into moresophisticated monitoring /control metrics

Increasing use of desktopcomputing in operationscreating the ability to usemore timely information

Identification of keywords andBoolean logic for product,names and keywords

Automated sentiment forkeywords using dictionaries

Simple totals

Identification of keywords and textstrings in context using Boolean logicfor products, names and keywords

Automated sentiment forkeywords/strings using dictionaries

Article level values but basedon keyword / string values

The arrival of the Internet in mainstream business collapses the time-lines for reporting and business decision making, creating need for accurate tools with online search and refinement options

The first web based tools allowing interactive use of data and the first drill down access to underlying materials but adaptation without direct involvement of the client still not possible due to 'Black Box' methods and limitations

Tool providers include: Cymfony KWHR Biz360 Factiva/IBM• Radian6

Natural language tracking forphrases, names and equivalents(antonyms and synonyms) incontext with ‘entity level’ analysis

'Phrase level' analysis includinganaphora and collectives usingcomplex logic and semantic rules

Social media multiplies mediaoutlets into millions

Business reaction time reducedto real-time responses

MarCom and press officemerges

Fully interactive 'right time' delivery of filtered and categorised materialsInnovation in removal of 'Black Box' allows brands to obtain 'best fit' results for individual job functions and sub brands

Tool providers include: Glide Technologies Crimson Hexagon (USA only)

Understanding the meaning of results and how to use monitoring tools so they work for you

Filtering the information•Using a top down approach to filtering often wastes time, money and effort

•Try a bottom up approach to define filters – it’s quicker and it will help understand the nature of the social media and how your brand interacts with it better – particularly useful when you need to ‘expand’ the search

•Use as many specifics as you can (that may be none, but usually isn’t)

•Be prepared to spend some time planning how you will use filters to categorize and quantify data (every hour you spend at the planning stage can save you days in execution – and can be especially valuable when you are briefing vendors)

•Make sure the filters make sense to your business and monitoring objectives

5

The perfect tool?

6

How much do you already know about your audience?

•Take the time to establish which social media channels it

makes sense for your audience to use

•Better still, choose a tool which allows you to check your

assumptions

•Experiment and test your assumptions before you go setting up a whole lot of details in specifications and filtering

•Use ‘exceptions’ to exclude unwanted audiences

•Make sure you understand the vocabulary your audiences use

7

Data cleanup

Customizing the sample

9

10

11

12

A matter of perspective

14

15

Context

16

Framing questions

•The most important rule is to make sure the answers are

answering YOUR questions

•Either make sure you test ‘logical’ questions ahead of time

or…

17

An end to keywords?

•Look at the new generation of analyzers which combine natural language and context to generate results which ‘mean’ what you ‘said’ in your question

•The benefit is that missed ‘hits’ are minimised simply because your vocabulary missed a word or variation, or the time tense is different, or one or more of the key elements isn’t language (such as an emoticon, or slang expressions)

•This approach allows you to work in a more natural way in both framing questions and evaluating outcomes

18

The trouble with automated sentiment analysis

•Is often what it is expected to accomplish

•If you have designed in the filtering, contextual mapping and the end user of the information it is perfectly practical to expect a result in the 90-95% accuracy range

•Beware of claims over 95% (in a ‘live’ language, subject to fashion and the re-use of terms in new ways there are very real and tangible barriers

•But don’t make the mistake of assuming a human reader can do any better – a 2009 study of 2,000 people saw them score an average of 84% – and 88% was the top mark

19

The trouble with automated sentiment analysis

•There is a ‘semantic’ trap’ in most analysers in that they use language as it is defined – not as it is actually used – and most rely on words (social media posters sometimes don’t) – a lot of language isn’t traditional linguistics!

•Black box solutions, however good they appear, are hiding things from you – insist on transparency

20

Death to all humans?

21

Why automated analysis is NOT about replacing people

•The role of automation is not about replacing people from the process – rather it should be about allowing you more time to think about what is important rather than speed reading

•Take the Apple iPhone 4 – social media DISCUSSION THREADS not posts were ruining at 30/second during the launch day – leaving anyone trying to read and make sense of the sentiment in a passive, or reactive state.

•Context-based analysis can reduce the results to actions and insights which are both manageable and insightful – without increasing the errors from volume – without getting tired and it will read every blog or post all the way through.

Making results count

23

And finally…

•Feedback loops – both in the human and automated sense are a valuable mechanism to ‘shrink wrap’ results ever closer to your brand objectives – use them!

24

1st Generation 2nd Generation 3rd Generation 4th Generation Automated identification of keywords Manual sentiment mark-up with article level values Simple totals positive, neutral, negative

Business Drivers

Deliverables

2010

Identification of keywords and Boolean logic for product, names and keywords Automated sentiment for keywords using dictionaries

• Identification of keywords and text strings in context using Boolean logic for products, names and keywords • Automated sentiment for keywords/strings using dictionaries • Article level values but based on keyword / string values

• Natural language tracking for phrases, names and equivalents (antonyms and synonyms) in contact entity• 'Phrase level' analysis including anaphora.

• Social media multiplies media outlets into millions• Business reaction time reduced to real-time responses• MarCom and press office merges

• Fully interactive 'right time' delivery of filtered and categorised materials• Innovation in removal of 'Black Box' allows brands to obtain 'best fit' results

Tool providers include:Glide Technologies

• We don’t use humans for analysis or dictionaries

• Ability to dynamically handle high volume up to 10,000 per

hour

• Sentiment Analysis is computer generated using Natural

Language Processing (NLP),able to score slang, irony and

even symbolic language

• Accuracy levels are high, transactions to provide overall

article scoring with ‘transparent’ sentiment scoring.

• Use of anaphora to identify “its”, “their”, “the company”

and maintain reference

• Fully integrated with all other media types – no silos – full

cause and effect mapping

26