Who Matters Online: Measuring influence, evaluating content and countering violent extremism in...

7/28/2019 Who Matters Online: Measuring influence, evaluating content and countering violent extremism in online social ne

1/56

Who Matters Online:Measuring infuence,evaluating content andcountering violentextremism in online

social networks

J.M. Berger & Bill Strathearn

March 2013

Developments in Radicalisationand Political Violence


2/56

Developments in Radicalisationand Political Violence

Developments in Radicalisation and Political Violenceis a series o papers published by the International Centre

or the Study o Radicalisation and Political Violence (ICSR).

It eatures papers by leading experts, providing reviews o

existing knowledge and sources and/or novel arguments

and insights which are likely to advance our understanding

o radicalisation and political violence. The papers are written

in plain English. Authors are encouraged to spell out policy

implications where appropriate.

EditorsAlexander Meleagrou-Hitchens

Dr John Bew

ICSR, Kings College London

Editorial AssistantKatie Rothman

ICSR, Kings College London

Editorial BoardProf. Sir Lawrence Freedman

Kings College London

Dr. Boaz Ganor

Interdisciplinary Center Herzliya

Dr. Peter Neumann


Dr. Hasan Al Momani

Jordan Institute o Diplomacy

ContactAll papers can be downloaded ree o charge at www.icsr.

ino. To order hardcopies, please write to [email protected].

For all matters related to the paper series, please write to:

ICSR


138142 Strand

London WC2R 1HHUnited Kingdom

ICSR 2013


3/56

1

Contents

Introduction and Key Findings 3

Overview 7

Methodology and terms 9

Collected Data 11

Analysis o Inuence 15

Identifcation o Engaged Extremists 17

Content o Tweets 21

Links 21

Top Sites 23

Hashtags 25

Political Context Considerations 27

Comparison Group 29

Identication Rate 30

Content Analysis 31

CVE Implications and Recommendations 35

Measurement 37

Targeting Infuence 38

Targeting Content 41Intervening with At-Risk Individuals 43

Countering Extremist Narratives 44

Reactive Opportunities to Counter Extremist Narratives 45

Data at a glance 47

Appendices 48

A: Seed accounts 48

B: Scoring ormula and glossary o metrics 48

C: Top 10 most infuential 50

D: Top 10 most exposed 51E: Top 20 most interactive 51

F: Glossary o Twitter Terms 52

G: Disclosures 53


4/56

2

About the Authors

J.M. Berger is a researcher and consultant working on

problems related to extremism, with a particular ocus on theuse o the Internet by extremist movements. He has written

about extremism or Foreign Policy, The New York Times,

The Daily Beast, and the Sentinel, a peer-reviewed journal

published by the Combating Terrorism Center at West Point.

Berger is the author o Jihad Joe: Americans Who Go to War

in the Name o Islam (Potomac Books, 2011).

Bill Strathearn is a sotware engineer with Google and Google.

org with an interest in creating technology that improves theinterace between government and citizens. He has 13 years

o experience in a broad range o sotware systems including

social graph analysis pipelines used to improve search

results. Strathearn studied computer science at the University

o Caliornia at Santa Barbara where he graduated with a

Masters degree.

AcknowledgementThe authors would like to acknowledge and thank

Google Ideas or its support in commissioning this paper.

Views expressed in the paper are those o the authors.


5/56

3

Introduction and

Key FindingsIt is relatively easy to identiy tens o thousands o social

media users who have an interest in violent ideologies, but

very dicult to gure out which users are worth watching.

For students o extremist movements and those working to

counter violent extremism online, deciphering the signal amid

the noise can prove incredibly daunting.

This paper sets out a rst step in solving that problem. We

have devised a scoring system to nd out which social

media accounts within a specic extremist circle were most

infuential and most prone to be infuenced (a tendency we

called exposure).

Our starting data centered on ollowers o 12 American white

nationalist/white supremacist seed accounts on Twitter. We

discovered that by limiting our analysis to interactions with

this set, the metrics also identied the users who were highly

engaged with extremist ideology.

Within our total dataset o 3,542 users, only 44 percent

overtly identied themselves as white nationalists online. By

measuring interactions alonewithout analyzing user content

related to the ideologywe narrowed the starting set down

to 100 top-scoring accounts, o which 95 percent overtly sel-

identied as white nationalist. Among the top 200, 83 percent

sel-identied, and or the top 400, the sel-identication rate

was 74 percent. A comparison analysis run on ollowers o

anarchist Twitter accounts suggests the methodology can be

used without modication on any number o ideologies.

Because this approach is entirely new (at least in the

public sphere), the paper spends some time discussing the

methodology and ndings in some detail, beore concluding

with a series o recommendations or countering violent

extremism (CVE) based on the ndings. The key terms or

understanding the recommendations are:


6/56

4

Inuence: The tendency o a user to inspire a

measurable reaction rom other users (such as a replies

or retweets).

Exposure: The fip side o infuence, this is the

tendency o a user to respond to another user in a

measurable way.

Interactivity: The sum o infuence and exposure scores,

roughly representing how oten a user interacts with the

content o other users.

Our key ndings include:

Infuence is highly concentrated among the top 1 percent

o users in the set.

High scores in both infuence and exposure showed a

strong correlation to engagement with the seed ideology

(white nationalism in our primary analysis, and anarchism

in a secondary analysis).

Interactivity, the sum o infuence and exposure scores,

was even more accurate at identiying users highly

engaged with the seed ideology.

In the course o collecting the data needed to measure

infuence and exposure, we incidentally collected a large

amount o data on hashtags and links used by people who

ollow known white nationalists on Twitter. When we examined

this data, we discovered that members o the dataset were

highly engaged with partisan Republican and mainstream

conservative politics. The paper presents a signicant amount

o context needed to properly evaluate this nding.

Working rom these ndings, the paper makes several

recommendations or new CVE initiatives with a ocus

on NGO eorts, which was the purpose o this research,

although we recognize our ndings will likely have utility

or government eorts in this sphere as well.

Our recommendations include:


7/56

5

We believe these metrics oer ways to concretely

measure which types o CVE approaches are eective

and which are not, bringing some clarity to a realm where

strategies are oten wishul and based on assumptions,while conclusions are oten anecdotal and inconclusive.

The concentration o infuence among a very ew users

suggests that disruptive approaches and counter-

messaging should be targeted to the top o the ood

chain, rather than working with the larger base o users.

Our analysis ound that the seed accountsall well-

known white nationalist ideologues and activistswerenot necessarily producing the most popular content and

links to external Web sites. The collected data can be

used to nd the most important external content sources,

and target them or disruption through terms-o-service

violation reporting, or through counter-messaging.

By tracking these metrics on an ongoing basis, NGO

eorts to counterprogram against extremist narratives

can be evaluated to measure how many users adopt

or respond to counter-messaging content, and how

much infuence accrues to dierent kinds o positive

messaging.

Since the data suggests white nationalists are actively

seeking dialogue with conservatives, CVE activists should

enlist the help o mainstream conservatives, who may be

considerably more successul than NGOs at engaging

extremists with positive messaging. Further research may

also suggest avenues or engagement between other

kinds o extremists and other mainstream political and

religious movements.

Finally, we believe that these metrics are only a starting point

or the study o extremist use o social media. We believe the

metrics and approaches here can be urther rened, and we

believe that additional research may yield substantial new

techniques or monitoring and countering the promotion o

violent ideologies online.


8/56

6


9/56

7

Overview

As social media grows more important to organizational

activity o all types, its role in enabling radicalization and

extremism has come under intense scrutiny. This paper

examines one type o extremism, white nationalism, on one

social network, Twitter,1 in order to learn more about the

nature o social media interactions.

We used a numerical scoring system to analyze the number

and type o interactions among 3,542 Twitter users whoollowed the accounts o well-known white nationalists.

Not all members o the overall dataset were committed white

nationalists, but we ound that 95 o the 100 users with the

highest scores or interactivity were also observed to be highly

engaged with white nationalist (WN) ideology in their tweets

and how they described themselves in their Twitter account

proles. For the top 200, the metrics accuracy in identiying

users overtly engaged with WN ideology was 83 percent, and

or the top 400 the accuracy rate was 74 percent.

Importantly, the scoring system did not include analysis o

any terms or conditions specic to white nationalism, dened

here as any ideology that prioritizes white racial identity as a

undamental component o how society should be organized.

Instead, results were derived solely by measuring interactions

within the dataset, regardless o content, which suggests

the approach can be applied without modication to other

extremist ideologies. To test that theory, we replicated the

analysis o the initial dataset on a group o Twitter users who

ollowed anarchist accounts, with encouraging but more

ambiguous results.

As one component o the scoring system, we analyzed the

dataset to identiy the most infuential users. We ound that

nearly 50 percent o all infuence in the system (as dened by

1 A glossary o Twitter terms is included as Appendix F. For more explanation o

how Twitter works see https://support.twitter.com/articles/215585-twitter-101-

how-should-i-get-started-using-twitter


10/56

8

a weighted measurement o interaction described below)

was concentrated among the top 1.5 percent o users.

We believe these results oer signicant opportunities toimprove initiatives aimed at countering violent extremism

(CVE) in the online arena, including improved selection

o targets or counter-messaging and disruptive tactics

and the ability to quantitatively measure the success o

dierent approaches.

In a secondary analysis, we also examined links and

hashtags tweeted by users in the dataset and ound a

signicant portion o the content related to mainstreamconservative and partisan Republican politics. This nding

requires a signicant amount o context to be interpreted

airly, and it is important to reiterate that not all users in

the dataset were committed to white nationalism. Most

importantly, the timing o the data collection, during the

political convention season with a major election

approaching, should also be taken into consideration as

it likely infuenced the choice o both links and hashtags,

as discussed in greater detail below.

With a ull consideration o the context, we believe this

nding suggests that social conservatives are in a very

strong position to conduct eective CVE programs aimed

at neutralizing the messages and narratives promulgated

by infuential white nationalists.

Our comparison analysis o ollowers o anarchist

Twitter accounts ound that while those users posted a

large amount o material pertaining to liberal views, they

posted much less content related to mainstream and

partisan politics.


11/56

9

Methodology and terms

Methodology

A short glossary o Twitter-specic terms used herein is

attached as Appendix F.

Sampled Twitter accounts: 12 seed Twitter

accounts or users identied as prominent individuals or

organizations within the white nationalist movement

Followers dataset: 3,542 Twitter accounts that ollow

one or more seed accounts

Total dataset: The 200 most recent tweets or each seed

and ollower, or all available tweets i the account had less

than 200. Total tweets analyzed: 342,807.

Sample period: Automated collection was completed

25 August 2012. Manual inspection o accounts began

25 August 2012 and continued through 31 October 2012.

Metrics: A complete explanation o all metrics may be

ound in Appendix B.

Key Terms

White nationalist: Any ideology that prioritizes white

racial identity as a undamental component o how a

given society or nation should be organized.

Inuence: A metric measuring a Twitter users ability to

publicly create and distribute content that is consumed,

armed and redistributed by other users.

Exposure: A metric measuring a Twitter users tendency

to publicly consume, arm and redistribute content

created by other users.

Interactivity: A metric measuring a Twitter users

combined infuence and exposure based on their

public activity.


12/56

10


13/56

11

Collected Data

Collection began with 12 Twitter accounts belonging to

prominent individuals and organizations that openly and

unambiguously promote organized, ideological white

nationalism, including the ocial accounts o David Duke, the

Aryan Nations and the White Aryan Resistance. A ull list is

ound in Appendix A.

These seed accounts were chosen based on the current

and historical signicance o the people and organizations towhite nationalist ideology, as well as or their unambiguous

status as white nationalists. We collected inormation that

the ollowers o the seed accounts made publicly available

between August and October 2012, using a Web application

designed by co-author Bill Strathearn.

No one o the 12 seed accounts attracted a notably large

number o ollowers. All 12 accounts together had 3,542

ollowers with publicly viewable proles, many o whom

ollowed more than one seed. For each o the 12 seeds

and their 3,542 ollowers, we collected the 200 most recent

tweets. In some cases, accounts had not yet posted 200

tweets, in which case all available tweets were collected. A

total o 342,807 tweets were analyzed statistically. Tens o

thousands o tweets were also manually reviewed.

Not all the ollower accounts collected belonged to users

who openly embraced or supported white nationalist views.

A manual inspection o 350 randomly selected accounts

rom the ollower dataset showed 44 percent were overtly

engaged with white nationalism in observed tweets or by

sel-identication in their user prole (margin o error +/- 5.4

percent). The sample was selected using a random number

unction to assign a random number value to each user.

The dataset was then sorted according to the random

value, and the rst 350 accounts were analyzed.


14/56

12

Sixteen percent o the dataset unambiguously sel-identied

with white nationalism in their proles or usernames by using

clear terms and phrases to describe themselves, such as

Aryan, racist, racialist or white nationalist. We took such

primary sel-identication to indicate a very high level o

engagement with WN ideology.

Close to 50 percent o the random sample o ollowers

o seed accounts displayed no overt sign o positive

engagement with white nationalism in their tweets or proles

other than the act o ollowing a seed account. The vast

majority o these simply provided no clear evidence o their

attitudes toward white nationalism, either or or against.

A tiny handul overtly expressed views incompatible with

white nationalism, such as belie in racial harmony or non-

discrimination. Only one account was observed to be an

activist opposed to white nationalism.

44%36%

6%

64%50%

Overtly engagedwith white

nationalism

No overt white

nationalism

Random Sample

N = 350

Overtly Engaged

N = 154

Tweets Only

Self-description


15/56

13

Twelve percent o the random sample ollowed more than

500 other accounts. We judged these individuals to be

indiscriminately ollowing others, oten in hopes o gaining

more ollowers themselves (a common Twitter strategy).Twenty-two percent o the indiscriminate ollowers were

overtly engaged with WN ideology on Twitter, about hal the

rate o the overall set, while 70 percent showed no overt

engagement.

All o the seed accounts were based in the United States.

Followers came rom a number o locations. Because Twitters

prole parameters are unstructured, we could not statistically

analyze the locations, but users were anecdotally observedto live primarily in the United States, the United Kingdom,

Sweden and Canada. Users were observed rom elsewhere in

Europe, including France and Germany, Latin America, Asia

and the Middle East.

Only public inormation was actored into this analysis. No

data was collected on accounts marked as private at the time

o collection. The status o some accounts changed between

the start o collection and the end o analysis, including a

handul o voluntary and involuntary suspensions, and some

accounts which switched rom public to private.


16/56

14


17/56

15

Analysis o Infuence

We designed a scoring system to measure the infuence

and exposure o the seed accounts and their ollowers

using their patterns o interaction with other users in the

ollowers dataset. Interactions with the seeds and users

outside the dataset o ollowers are not scored in the

metrics presented here.

Infuence and exposure are dened as Twitter interactions

that mirror the real lie denitions o the terms infuencedened as the ability to spread and promote content or ideas,

and exposure as a tendency to consume content or ideas

generated by others.

In terms o Twitter activity specically, infuence was dened

as when a user did something that inspired a reaction rom a

second user (such as prompting the second user to reply to a

tweet, or to retweet content initially tweeted by the rst user).

In the context o this paper, a users infuence score may be

said to determine how infuential that user is.

Exposure was simply the other side o the coin, dened

as when a user perormed such an action in response to

another user. A user who retweeted a second user, or replied

to something the second user tweeted, received points or

exposure. Throughout the paper, a users exposure score may

be said to determine how exposed that user is to content

generated by other users within the dataset. For instance,

that a user with a high exposure score may be designated as

more exposed than one with a low score.

Although infuence and exposure represent opposite types o

interactions, an account could have high or low scores in one

or both categories, depending on the nature and quantity o

its interactions. The complete scoring ormula, with examples,

is attached as Appendix B.

Interactions were weighted to emphasize keywords in tweets

indicating they might be linked to more signicant types o


18/56

16

communication such as call, email or DM (reerring to

Twitters direct messaging system or private communications

between users).

Ater analysis o these initial results, we created a third score,

which was simply the sum o the infuence and exposure

scores, a measurement we reer to as total interactivity,

representing the overall quantity o meaningul interactions

conducted by the account holder.

Importantly, the scoring system did not consider or score

terms relevant to white nationalism. Our initial goal was

simply to analyze the social network structure to evaluatewhich ollowers o seed accounts were extremely infuential

or extremely vulnerable to infuence. The scoring systems

eectiveness at identiying people overtly engaged with

the ideology that united the seed accounts was an

unexpected nding.


19/56

17

Identication o Engaged

Extremists

The rankings discussed throughout this paper apply to

the ollowers o the seed accounts and exclude the seeds

themselves, as our goal was to evaluate the less obvious

participants in the network. Within the ollower dataset, the

number one most-infuential account, by a wide margin, had

been identied as a potentially important user in the dataset

through a manual examination o the seeds and their ollowers

prior to data collection, suggesting that the scoring systems

results were not out o line with manual analysis. Manual

inspection o the most infuential and exposed accounts ound

that the ranked accounts generally conormed to the qualities

we sought to identiy (i.e., infuence and exposure).

0

20

40

60

80

100

PERCENT

Random

sample

Top 400 Top 200 Top 100 Top 50

RANK ACCORDING TO INTERACTIVITY SCORE

USERS OVERTLY ENGAGING WITH WHITE NATIONALISM

Not overtly engaged

Overtly engaged


20/56

18

Among top 100 most infuential, 86 percent were observed to

be overtly engaged with white nationalism beyond the simple

act o ollowing a seed account (as previously dened in the

dataset section, overtly engaged means users sel-identiedas white nationalists or tweeted consistently about white

nationalism in a non-adversarial light). For the top 200, the

percentages dropped to 73.5 percent. Ninety-three o the

100 most exposed accounts were overtly engaged with white

nationalism. O the top 200, 83 percent were overtly engaged.

Ater examining these results in detail and observing their

unanticipated utility at identiying engaged extremists, we

posed the question o whether it was possible to improve theidentication process. We experimented with variations on the

original scores and ound that adding infuence and exposure,

creating a measurement o overall interactivity, produced

improved identications.

Manual examination showed 95 o the 100 accounts with

the highest interactivity scores were overtly engaged with

TOP 100

MOST INTERACTIVE

USERS

5%

24%

71%

Overtly engaged in tweets only

Sel-identied as white nationalist

Not overtly engaged


21/56

19

white nationalism, and 71 sel-identied as white nationalist

in their usernames or proles, showing extremely high levels

o engagement. This compared very avorably to the random

sample, where 44 percent were overtly engaged and 16percent sel-identied.

For users with the top 200 interactivity scores, 83 percent

were overtly engaged and 57.5 percent sel-identied. O the

top 400, 74 percent were overtly engaged.

There was substantial overlap among the highest-scoring

accounts in each category, with a number o accounts

scoring high in all three measurements.

Within the top 50 o all three categories, only one account

appeared totally irrelevant, a person who generically

interacted with accounts that began ollowing him. Another

user sel-identied using neo-Nazi terminology and interacted

with engaged white nationalists, but did not hersel

obsessively tweet on the subject.

The remaining top-50 users were observed to be highly

engaged with white nationalism and spent the vast majority

o their time on Twitter discussing WN ideology.

Based on these ndings, we believe the scoring ormula may

have a number o useul applications in CVE eorts, which

will be discussed below.


22/56

20


23/56

21

Content o Tweets

Although we set out with the primary goal o analyzing the

network interactions o the ollowers o white nationalist

seed accounts, the nature o our process also resulted in the

collection o other data, including links to external content and

hashtags (sel-conscious content identiers) used in tweets

by members o the dataset. A casual examination o this

collected data produced unexpected insights into the content

being promoted and consumed by the ollowers dataset, and

so we carried out a more detailed analysis.

Links

We detected 676 distinct domains in a total o 84,825 tweets

that included links rom the seed accounts and their ollowers.

2%

SOURCES BY

NUMBER OF LINKS

Identifiable content

domain names

4%

5%

16%

7%

12%

46%

8%

Alternative

Mainstream news

Extremist

Youtube

Facebook

Blogspot

Wordpress

All other content-neutral, includinglink shorteners

CONTENT-NEUTRAL

DOMAIN NAMES


24/56

22

We classied the links into our major categories: mainstream

news, alternative sources, extremist sources, and content-

neutral. Sites that generally adhered to journalistic standards

were classied as mainstream news, even i they had apolitical slant, such as Fox News.

The alternative category included sites that explicitly

articulated a political slant and sites that did not consistently

conorm to mainstream journalistic standards.

Extremist sources consisted o sites that openly advocated

white nationalism or very closely related themes.

The content-neutral category included links where the

specic content could not be resolved, as well as sports,

entertainment, health and science news sources deemed

o minimal interest to the study.

Mainstream news accounted or 12 percent o all links;

alternative and extremist sources combined represented

another 15 percent.

The categorization process necessarily involved a number

o subjective judgments. For instance, a handul o sites

included in the extremist category did not explicitly endorse

white nationalism, but provided narrative support to themes

white nationalists consider to be crucial. The most important

o these, in terms o the number o links tweeted by users in

the dataset, was Inowars.com, a conspiracy-oriented Web

site. Other subjective calls included the classication o state-

sponsored media and oreign nationalist sites. Overall, we

estimate around 6 to 8 percent o these subjective judgments

could be considered open or debate.

The single most-linked domain by an overwhelming margin

was YouTube, which accounted or about 21 percent o

all links detected. The content-neutral category included

a number o other service providers, such as Blogger and

Wordpress, where the nature o the content could not be

determined. Finally, generic URLs generated by various

popular Twitter clients represented 15.9 percent o all links.


25/56

23

We suspect many o these links would be classied as

alternative or extremist content, to an extent that could

substantially shit the percentages given above.

Top Sites

The top 10 sites linked in each category were:

Extremist Alternative Mainstream Content-neutral

whiteresister.com rt.com dailymail.co.uk youtube.com

cocc.org examiner.com oxnews.com bit.ly

realisten.se theblaze.com bbc.co.uk acebook.com

inowars.com wnd.com nytimes.com youtu.be

vnnorum.com breitbart.com guardian.co.uk blogspot.com

amren.com presstv.ir hungtonpost.com twitter.com

northwestront.org dailycaller.com telegraph.co.uk tumblr.com

malevolentreedom.org hotair.com thehill.com wordpress.com

thenewamerican.com globalresearch.ca cnn.com b.me

vdare.com thegatewaypundit.com reuters.com instagram.com

Top 10 as a percentage o all links

46.3 46.8 36.1 58.5

Mainstream conservative inormation sources were ound

throughout the links, mostly in the alternative category, which

by denition included political content.

The top 10 alternative URLs accounted or nearly hal o

all links in the alternative category. Among the top 10, 54

percent o links went to sources espousing clear partisan

Republican viewpoints (as opposed to simply expressing

conservative views).

The number o l inks per site dropped sharply in the lower

rankings. Many sites in the lower 112 URLs were oriented

so ambiguously among mainstream conservative, ringe

conservative and libertarian views as to deeat easy

classication.


26/56

24

Among the top 10 mainstream news sites, 31 percent o

links went to the top two sources, the Daily Mail (UK) and

Fox News (US), both o which are strongly oriented toward

mainstream political conservatism. The remainder went to

a mix o top mainstream news sources, including sites that

were reasonably considered apolitical, such as the New

York Times, or had a liberal orientation, such as the

Hungton Post.

The most-linked extremist site was WhiteResister.com,

but more than hal the links to that site originated with two

Twitter accounts, both aliated with the site. Discounting

links rom those two accounts, WhiteResister.com would

have dropped to sixth.

Signicantly, the accounts promoting WhiteResister.com

were the two most ollowed by other users in the dataset,

pointing to infuence above and beyond the bias introduced

by their sel-promotional tweets (or at least highlighting the

eectiveness o sel-promotion at garnering links).

ALL OTHER

ALTERNATIVE

LINKS

53%

PartisanConservative(54%oftop 10)

OtherAlternative

DISTRIBUTION

OF ALTERNATIVE

LINKS

TOP 10

47%

TOP 10

0%


27/56

25

The top 10 list also il lustrates an important point. While

the ollowers o white nationalist Twitter accounts are clearly

engaged with the ideology, that doesnt mean they are

involved in promoting the seed accounts o leading WNgures and organizations.

O the top 10 most-linked extremist sites, only two amren.

com and northwestront.org were associated with seed

accounts. Our infuence rankings excluded the seed

accounts. I they had been included in the dataset, three o

the seed accounts would have ranked among the top 10

most infuential. But the Web sites associated with those three

accounts did not appear among the 100 sites most-linked bythe ollowers dataset.

This demonstrates that ollowers were ar more likely to

promote their own related agendas than to promote well-

known leaders and groups and suggests that these well-

known leaders o white nationalism in the United States may

be losing touch with their constituents. While they still carry

signicant weight in the movement, they are not generating

daily buzz, on Twitter at least.

Connecting the most infuential Twitter users with online

content hosted elsewhere also expands the universe o

options or CVE practitioners, which will be discussed in

more detail in the CVE Implications section o this paper.

Hashtags

Hashtags included in tweets by the seed accounts and

their ollowers pointed more clearly to engagement with

mainstream politics. The top 10 hashtags used by the seeds

and ollowers combined were:

1. tcot (top conservatives on Twitter)

2. svpol (Swedish police)

3. teaparty

4. p2 (used to address comments to progressives)

5. gop

6. tlot (top libertarians on Twitter)


28/56

26

7. (used to recommend Twitter accounts

to

other ollowers)

8. Obama9. news

10. ows (Occupy Wall Street)

Given that not all accounts overtly engaged with white

nationalism, we analyzed the top 10 hashtags tweeted by the

200 and 50 most infuential ollowers o seed accounts, who

were more highly engaged with white nationalism.

For hashtags, narrowing the dataset down to the mostinfuential and extreme users suraced specic tags relating

to extremism, including #wpww (White Power World Wide),

#nwo (New World Order) and #edl (English Deense League).

However, the mainstream political tags remained signicant

and in many cases represented ahigherpercentage as the

user base became more extreme. The tags #tcot, #gop and

#teaparty remained in the top 10, in the same order as they

appeared or the whole dataset. #tcot was the most-used

hashtag in all subsets examined.

All ollowers 200 mostinuential

50 mostinuential

TCOT 10% TCOT 18% TCOT 20%

SVPOL 3% SVPOL 11% SVPOL 18%

TEAPARTY 3% TEAPARTY 6% WPWW 14%

P2 3% GOP 5% EDL 11%

GOP 2% P2 5% TEAPARTY 8%

TLOT 2% WPWW 4% FF 6%

FF 2% TLOT 4% GOP 5%

OBAMA 1% OCRA 3% OBAMA 5%

NEWS 1% NATPOL 3% NWO 4%

OWS 1% NOW 3% P2 3%

TOP 10 HASHTAGS, PERCENTAGE OF ALL TAGS


29/56

27

Political Context Considerations

In our random sample, 16 percent o users sel-identied

as white nationalists, but only 4 percent sel-identied as

mainstream conservatives. That suggests the link and

hashtag results are driven more by white nationalists eeling

an anity or conservatism than by conservatives eeling an

anity or white nationalism.

The timing o the data collection, during the polit ical

convention season with a major election approaching, likely

infuenced the choice o both links and hashtags. President

Obamas race almost certainly motivated white nationalists to

take an enhanced interest in mainstream Republican politics.

A number o the 10 hashtags most oten tweeted by theollower dataset (including those most oriented toward

Republicans) were recommended or use in an organized

0

5

10

15

20

PERCENTAGEOFALLHASHTAGS

All followers Top 200 Top 50

LESS EXTREME USERS MORE EXTREME USERS

#TCOT

#TEAPARTY

#GOP


30/56

28

program o tweets by a social media group outside the

dataset with a stated intention to discredit the Obama

campaign. This eort began no later than August and may

have tilted the results.2

Many o the same tags were recommended on a non-racial

militia movement Web site in September 2012 as a way to

broadcast that movements message to those in the political

mainstream.3

This tactic could also have been adopted by white

nationalists, although explicit evidence o such an eort was

not ound. These online campaigns were ound by searchingor several o the hashtags together in a database o extremist

Web sites routinely maintained by co-author J.M. Berger.

Finally, it is important to note that engagement does not

necessarily mean positive or two-way engagement. For

instance, the ranking o #p2 and #ows among the top

10 suggests the mere presence o a hashtag does not

necessarily represent an endorsement.

In light o these actors, we recommend revisiting this

research ater the election and caution against drawing

overbroad conclusions about the relationship between

mainstream conservative politics and white nationalism.

Despite all this, we believe these results are neither invalid

nor insignicant. The context and timing may have infated

numbers regarding mainstream political engagement, but

the social networks involvement in electoral politics is

itsel interesting and signicant.

Additional research over time may help illuminate the

strength and persistence o these patterns, but we suspect

it would not erase them. We discuss the implications o

these ndings urther in the CVE recommendations below.

2 https://www.acebook.com/groups/TwitterAttack/, retrieved October 20123 HASHTAGS # or Waging Political Warare: Use these tags i youre using

twitter posted on a militia Web site monitored by Intelwire.com on September

15, 2012


31/56

29

Comparison Group

Ater reviewing the rst set o results, we perormed

supplemental data collection in October using a seed group

o eight well-known and overtly anarchist Twitter accounts in

order to urther test the scoring system and try to capture a

better picture o how mainstream political discourse relates

to extremist movements aside rom white nationalism.

Anarchism is by denition less leadership-oriented, which may

skew comparisons in various ways. It is also, by its nature,less clearly dened, making the categorization o accounts

more dicult. For instance, the Occupy movement has

strong anarchistic components, but Occupy is not generally

considered anarchist by denition.

We collected rom ewer accounts than we did WNs, but the

eight anarchist seeds had more ollowers than the 12 white

national accounts (5,409 versus 3,542). In part, this likely

refects the lesser social stigma associated with anarchism

compared to white nationalism, as well as an anecdotally

observed tendency o anarchists to be younger and more

technologically procient than white nationalists.

Based on anecdotal observation, we would also argue it is

easier to be casually involved with anarchism than it is to be

casually involved with ideological white nationalism. Users

who sel-identied with a variation o the word anarchist oten

included several other interests in their sel-identication.

Users who sel-identied as white nationalist were much more

prone to identiy solely with that ideology.

For all o these reasons, we believe that anarchism likely

represents the most dicult ideological extremism challenge

or metrics-based identication.


32/56

30

Identifcation Rate

O the 100 ollowers o anarchist seeds with the highest

interactivity scores, 70 percent overtly engaged on

Twitter with anarchism; engaged with very closely related

movements, including the Black Bloc segment o the Occupy

movement and Anonymous; or sel-described as radical or

revolutionary.

Another 20 percent overtly engaged in tweets with Occupy

but not Black Bloc, or overtly engaged with strongly anti-

government letist movements, making their presence clearly

relevant, while not precisely matching the denition o the

seeds. Depending on how you classiy that 20 percent, the

identication rate alls between 70 and 90 percent or the top

100, compared to 95 percent or white nationalists.

Fity-eight o the top 100 accounts sel-identied as anarchist,

Black Bloc, Anonymous or something clearly within the same

ideological space. Another 18 sel-identied with Occupy,

communism or socialism, but did not sel-identiy

as anarchist.

100

80

60

40

20

PERCENT

Anarchists AnarchistsWhite Nationalists White Nationalists

SELF-IDENTIFICATION TWEETS

TOP 100 MOST INTERACTIVE, COMPARISON

Not overtly engaged

Engaged with relatedideology

Overtly engaged


33/56

31

This sel-identication rate o 58 percent to 76 percent

(depending how tightly one denes the target category)

compared to a sel-identication rate o 71 percent or the

top 100 most interactive members o the white nationalistollower dataset.

Given that anarchism is extremely diverse and resistant to

consistent labeling and movement cohesion, and given the

lower threshold or casual involvement, we believe these

results are extremely encouraging, especially given that only

about 10 percent o the entire dataset sel-identied with a

variation o the word anarchist.

In light o these ndings, we believe the scoring system can

be eective at identiying ollowers who are highly engaged

in ideological extremism in social networks revolving around

seed accounts with a clear ideological orientation, regardless

o what specic ideology is being examined.

Content o Tweets

3%

SOURCES BY

NUMBER OF LINKS

Identifiable content

links

4%

8%

13%

9%

15%

45%

3%

Alternative

Mainstream news

Extremist

Youtube

Facebook

Wordpress

Blogspot

All other content-neutral, includinglink shorteners

CONTENT-NEUTRAL

LINKS


34/56

32

Links and hashtags used by the anarchist seed accounts

and their ollowers somewhat predictably skewed to the

let side o the political spectrum. However, they tweeted

considerably less content related to partisan politics thanWN seeds and ollowers.

In addition to being a larger dataset, the anarchist ollowers

were also prolic tweeters. We analyzed 188,573 links

(the top 85 percent o the 220,214 collected) and ound

that ollowers o anarchist seeds relied somewhat more

on mainstream news than white nationalists (15 percent

compared to 13 percent), while relying much less on clearly

extremist sites (3 percent versus 8 percent). The numbero content neutral links was the same, 73 percent, with the

remaining links going to alternative sources.

Categorizations were even more subjective or this dataset

than or the original. For instance, it was dicult to determine

whether Occupy, Wikileaks, Anonymous and hacker sites

should be considered as extremist sites relevant to the seed

ideology.

For purposes o the analysis above, all o these were counted

as extremist. I they were discounted, extremist sites would

have represented only 1.4 percent o the total mix, ar less

than the 7 percent tweeted by the white nationalist dataset.

Alternative sites were overwhelmingly liberal in their

orientation, although debatably less partisan. For instance,

Democracy Now, Mother Jones and Salon were among the

top 10 alternative sites, but the latter two especially have

requently and sharply criticized the Obama administration

on topics such as law enorcement, surveillance, oreign

policy and the use o drones.

The hashtags oer a better window or comparing the

political proclivities o ollowers o white nationalist seeds

versus ollowers o anarchist seeds. The top 10 hashtags

used in the anarchist dataset were:

1. ows (Occupy Wall Street)

2. occupy

3. oo (Occupy Oakland)


35/56

33

4. occupydenver

5. anonymous

6. p2 (used to engaged with progressives)

7. (Follow Friday)8. s17 (September 17, the anniversary o the ounding

o Occupy Wall Street)

9. occupywallstreet

10. occupyoakland

Compared to the top 10 hashtags used by white nationalists,

these tags suggest a much higher disenranchisement rom

mainstream politics. Again, this may relate to the basic nature

o anarchism, which is undamentally opposed to politicalinstitutions, compared to white nationalism, which is not

opposed to institutions per se.

Overall, we interpreted these results to indicate anarchists,

while clearly embracing many liberal values, were ar less

engaged with partisan liberal politics than white nationalists

with partisan conservative politics.


36/56

34


37/56

35

CVE Implications and

Recommendations

The rst challenge in conducting any type o Countering

Violent Extremism (CVE) program is nding the target

audience. Given the ease o access to extremism online and

the allure o large numbers, many CVE eorts have ocused

on social media as a hunting ground or extremists.But while

it is relatively simple to collect large datasets o potential

extremists online, it is much more challenging to identiy

people who are suciently engaged to be suitable targets

or CVE. Online activity oten involves a lot o bad behavior

and indiscriminate sampling o ideas. Shielded by physical

0

20

40

60

80

100

PERCENT

Random

sample

Top 400 Top 200 Top 100 Top 50

RANK ACCORDING TO INTERACTIVITY SCORE

USERS OVERTLY ENGAGING WITH WHITE NATIONALISM

Not overtly engaged

Overtly engaged


38/56

36

distance and anonymity, people oten express views online

that they would rarely, i ever, express in their ofine lives.

In short, the vast majority o people taking part in extremisttalk online are unimportant. They are casually involved,

dabbling in extremism, and their rhetoric has a relatively

minimal relationship to the spread o pernicious ideologies

and their eventual metastasization into real-world violence.

Any CVE program must begin by siting the wheat rom

this cha.

While this study is not an end-state solution to the sorting

problem, we believe it represents a signicant step orward.As shown in the graph above, a higher interactivity score

clearly corresponded to a higher probability that a user was

overtly engaged with the ideology espoused by the seed

accounts. Ranked by the interactivity metric, we ound that

95 o the 100 highest-scoring members o the ollowers

dataset were highly engaged with white nationalist ideology.

These results are not a substitute or manual examination o

user activity, but they provide a way to take a snapshot o a

large user base and almost instantly prioritize the dataset,

isolating ruitul avenues o investigation. For instance, a

manual examination o the ollowers o seed white nationalist

accounts took several days and identied only two or three

ollower accounts that subjectively appeared to be very

infuential. The metrics identied the same accounts and

many more while requiring much less direct investment o

time and eort by a human analyst.

CVE initiatives are not limited to one type o extremism.

They can cover diverse ideologies such as jihadism, white

nationalism, black nationalism, anti-Muslim, sovereign citizen,

anarchism, eco-terrorism and more.

Based on the comparison results or anarchists, we believe

this is an o-the-shel approach that can be applied to any

ideology without customization. However we do believe the

metrics and analysis can be improved, perhaps substantially,

with urther research and testing.


39/56

37

We urther believe that the paradigm o counting and

weighting interactions can be adapted to any social network

where an adequate amount o data can be scraped using

automated tools.

In addition to identication, we believe the scoring

system applied in this study has several other quantitative

applications to CVE programs.

Measurement

CVE takes various orms. The most-discussed avenuesinvolve persuasion (convincing potential extremists not to

pursue violent ideologies) and disruption (preventing networks

that promote radicalization rom unctioning eectively).

These eorts can be conducted by government or by non-

governmental organizations.

Critics o current CVE initiatives, including co-author J.M.

Berger, have questioned the eectiveness o many CVE

programs, pointing to the act that there are no meaningul

methods to measure success, especially as it pertains to

persuasion programs. CVE strategies to date have been

guided largely by intuition and anecdotal observation rather

than by clearly relevant metrics.4

The infuence, exposure and interactivity scores showed

remarkable success at identiying highly engaged white

nationalists within the starting set. Manual inspection o

the top-ranked accounts also suggests that the scores

themselves can be interpreted as they were intended as

quantitative measures o engagement-related activity.

Furthermore, the scores are not proportions or percentages;

they measure the temperature o a given community, which

means that a drop in score represents an absolute drop in

the property (such as infuence) being measured. So i the

sum o all infuence scores in the dataset drops, it suggests

4 J.M. Berger, Intelwire.com, Finding a Way Forward or CVE, 19 August 2011,http://www.oreignpolicy.com/articles/2011/12/09/monsters_and_children; Will

McCants and Clint Watts, Foreign Policy Research Institute, U.S. Strategy or

Countering Violent Extremism: An Assessment, December 2012


40/56

38

that the leaders o the dataset have become less infuential.

I interactivity scores drop, it suggests that engagement

with the target ideology by members o the dataset has

been reduced.

Put simply, we believe the scoring system can be used to

measure the absolute amount o engagement within a targeted

online community in real time and over extended periods.

This opens the door to quantitative testing o dierent CVE

approaches. For instance, the scoring system can measure

eorts to diminish the infuence o key leaders, to inoculate

a target audience against exposure, or simply to depress all

interaction in the dataset.

Several CVE approaches are suggested by these conclusions.

Targeting Inuence

INFLUENCESCORE

USERS RANKED BY INFLUENCE

12000

10000

8000

6000

4000

2000

140

79

118

157

196

235

274

313

352

391

430

469

508

547

586

625

664

703

742

781

820

859

898

937

976

1015

1054

1093

1132

1171

1210

1249

1288

1327

1366

Infuence was disproportionately distributed throughout users with a nonzero score, asshown above. More than 50 percent o all the infuence in the system was attributed to just

54 accounts, or 1.5 percent o the total number o accounts. The chart above shows the

curve or all users with nonzero infuence scores rom highest to lowest.


41/56

39

Our understanding o infuence in the real world is airly

simple. A relative ew individuals wield a tremendous amount

o infuence over much larger numbers o people. Our ndings

put some specicity on that intuitive concept.

The vast majority o infuence in the WN dataset was

disproportionately concentrated at the top, as seen in the

chart above. The top 50 most-infuential users, looking this

time at both ollowers and seeds just 1.4 percent o the

total accounted or nearly 46 percent o all the infuence in

the system. (Because o their prominence, the seeds become

more relevant when targeting accounts or CVE eorts.)

These ndings generally conorm to other studies o online

activity, including the well-known 90-9-1 rule, which states 90

percent o users in most online social networks are passive, 9

percent are somewhat engaged and 1 percent drive most o

the engagement and discussion.5

We attempted to enhance measures o simple engagement

by weighting interactions to emphasize users with greater

reach and who provide content that is widely redistributed

and deemphasize users whose engagement carried less

weight in group dynamics. We expect the scoring system

can be rened to improve these measurements urther, but it

perormed very well out o the gate.

The top 10 most infuential accounts (less than 0.3 percent

o all sampled users) accounted or 24.8 percent o the total

infuence in an ecosystem o 3,554 Twitter accounts (ollowers

plus seeds) a massively disproportionate distribution. This

suggests the system is extremely vulnerable to disruption by

ocusing CVE eorts at the top o the ood chain.

Possible methods or disruption include submitting complaints

over violations o terms o service, where applicable;

challenging and discrediting infuencers through investigative

5 Jakob Nielsen, Nielsen Norman Group, Participation Inequality: Encouraging

More Users to Contribute, 9 October 2006, http://www.useit.com/alertbox/participation_inequality.html; Ben McConnell, The Church o the Customer

Blog, The 1% Rule: Charting citizen participation, 3 May 2006, http://www.

churchothecustomer.com/blog/2006/05/charting_wiki_p.html


42/56

40

reporting and research; or challenging and discrediting

infuencers through open debate in the online sphere.

Infuence was disproportionately distributed even aterremoving the top 50 users. The chart above shows the curve

or all remaining users with nonzero infuence scores.

Ater the top 50, a sharp curve remained even among the

users who contributed the lower 50 percent o infuence to

the system. While this suggests there would be a diminishing

benet to removing infuencers over time, the distribution

remains top-heavy, with the second 50 (1.4 percent o the

remaining sampled users) comprising slightly more than 27percent o the remaining infuence. This suggests targeting

the top o the curve can be eective even ater the ecosystem

has suered signicant attrition.6

While it is impossible to know or certain without direct

experimentation, we believe these curves suggest disruptive

CVE aimed at the top can be very eective over the course o

a long engagement.

Another CVE option is to aim disruptive techniques at those

with the highest exposure or interactivity scores. We originally

suspected that high exposure scores might correlate to

people at risk o radicalization who could be targeted or

personalized interventions by nonprot organizations. What

we discovered instead was that users with high exposure

scores were extremely committed to white nationalism and

very active participants in the network.

While interventions seem unlikely to be eective with such

users, disruption might be eective. The distribution o both

exposure and interactivity was less concentrated at the

top than infuence (with interactivity slightly outperorming

exposure), but the distribution remained disproportionately

top-heavy or both metrics.

6 These calculations do not account or the redistribution o infuence whenusers are removed rom the system, with some o the remaining accounts

becoming more prominent and thus more infuential. Nor do they account or

the reemergence o removed users under new Twitter handles.


43/56

41

Our evaluation o users with high exposure but medium to

low infuence is that they were mainly reactive, unctioning

as a supportive cheering section but not necessarily directly

driving the content o discussions (which is not the same assaying their role in the system is insignicant). However, the

interactivity metric was signicantly more eective than any

other method at identiying highly engaged users. Since users

cannot be infuential without other users who are exposed

to them, we suspect that targeting the highest interactivity

scores might have benets that are not obvious but could

become visible when scores are tracked over time.

Finally, we should note that the infuence distribution wasdierent or anarchists than or white nationalists. The

distribution o infuence or the anarchists was much fatter,

but still quite top heavy, with hal the infuence in the system

residing among the top 106 accounts. This suggests the

approach may need to be ne-tuned or dierent ideologies,

but that the basic principles will remain eective.

Targeting ContentDisruptive CVE approaches on Twitter are limited by broadly

permissive Terms o Service. Social media platorms in

general are biased towards reedom o speech, although it

should be noted that reedom o speech is not synonymous

with the reedom to use a specic companys services.

Providers o Internet publishing platorms are not obligated

to host content in violation o their published rules, and they

have wide latitude in deciding what those rules should be.

Twitters TOS as o October 2012 banned direct, specic

threats o violence against others and the use o Twitter to

conduct or promote illegal activities and loosely dened

abusive behavior.

These terms are suciently ambiguous to allow designated

global terrorist groups like Al Shabab to maintain active

accounts on Twitter (although Shababs account was recently

suspended or a direct threat o violence, it has since opened


44/56

42

a new account).7 The Taliban and many other extremists also

use Twitter extensively. However several prominent extremist

accounts have been suspended or unknown reasons as o

December 2012. This may refect a shit in policies, or it may

be due to other unknown reasons.

Regardless, disruption need not be limited to Twitter, thanks

in part to the content analysis, which shows the most

signicant external sources o content. A large part o Twitters

value as a social media tool involves the ability to provide links

and steer users to more detailed material on other Web sites.

For instance, YouTube emerged as the most-linked source o

content in the dataset, representing more than 21 percent o

all links and 49 percent o the top 10 links. Facebook, Blogger

and Wordpress also ranked among the top 10 sources, even

7 BBC News, Somalias al-Shabab opens new Twitter account, 4 Feburary

2013, http://www.bbc.co.uk/news/world-arica-21321687

2%

TOP 10 USER-

GENERATED

CONTENT SOURCES

BY NUMBER

OF LINKS

3%

3%

5%

5%

9%

15%

49%

6%

3%

acebook.com

youtube.com

blogspot.com

twitter.com

tumblr.com

wordpress.com

instagram.com

oursquare.com

twitpic.com

yahoo.com


45/56

43

without accounting or sites that use those services under a

custom domain. As noted previously, we could not determine

the nature o the content being linked to under these

domains, but its likely a signicant percentage o these linkspoint to user-generated content related to white nationalism.

Terms o service vary or each o these online publishing

platorms. YouTube has relatively robust tools or reporting

and removing hate speech, including the ability to report

users or violent or repulsive, and hateul or abusive

content, with specic subcategories or promoting terrorism

and violence and hatred.

CVE practitioners can also make note o which subsets o a

sampled dataset are linking to content, or instance targeting

links provided by the most infuential, exposed or interactive

users, and report TOS violations, i any exist, with the

content publisher.

Intervening with At-Risk Individuals

Manual inspection o the highest infuence, exposure and

interactivity scores ound that almost all o the top 50 in each

category were highly committed white nationalists unlikely to

be swayed by intervention.

The percentages dropped steadily and consistently as the

view expanded to the top 100 and 200 accounts. We believe

manually examining users whose interactivity rank alls

between 200 and 500 might be ruitul in identiying people

vulnerable to radicalization or in its early stages, whose

interactions indicate an interest in an extremist ideology but

not a single-minded obsession with it.

Additional research is required to evaluate these ndings

in order to capture the characteristics o at-risk users

and evaluate possible approaches to online intervention.

Approaches can be eld-tested and the scoring system

can be used to help evaluate the results, although manual

inspection and contextualization will likely be required. For

instance, i an at-risk user is questioning the wisdom o

pursuing white nationalism as a personal ideology, he or she


46/56

44

might temporarily become more interactive while seeking to

test assumptions and challenge infuencers.

Since radicalization is a process, the rankings are onlyrelevant to this dataset during the time period o collection,

so scores should be recalculated prior to experimentation.

Additional research might eventually point to ranges o

infuence, exposure and interactivity where the highest

percentage o at-risk individuals can be ound. Some

adjustments to the scoring system would be necessary

in order to support a universal range-based approach.

CVE partners who undertake this research should keep

detailed notes on the scores, ranks and proles o individuals

they approach and the results o their intervention in order to

urther rene the most eective target ranges.

Countering Extremist Narratives

A popular approach to countering violent extremism involves

creating and promoting narratives that oppose extremist

ideologies, in the hopes o undermining existing extremist

messages and competing or the hearts and minds o

potential recruits.

Although such eorts can be evaluated by how many

participants they attract, their utility in actually countering

extremism remains unclear. For instance, accumulating

1,000 social media users who promote messages opposed to

racism is an achievable goal, but its not clear how that eort

lands in at-risk communities and whether people already

attracted to white nationalism are positively infuenced by

such eorts or i they may even be urther alienated by them.

The metrics developed in this paper oer the possibil ity o

perorming quantitative evaluations or online messaging

eorts. For instance, i the eort is centered around a Web

site, CVE practitioners can track whether the links to the site

are being tweeted by members o a targeted dataset, how the

site is being characterized and whether infuential users are

discussing it.


47/56

45

I the eort is being promoted by specic Twitter accounts,

those accounts can be introduced into the dataset and their

infuence can be tracked. Similarly, eorts to sway sentiment

among specic subsets o users (or even individuals) canbe tracked and scored to measure their success or ailure

in the online context. For instance, i the goal is to diminish

the reach o the seed accounts, it is possible to track the

infuence o those accounts, whether users have exited the

dataset by unollowing them, and so orth.

It should be noted that it is possible to game the system, so

CVE practitioners must be coached on how to achieve honest

results, although it is also possible that gaming the systemcould in some cases actually achieve the desired result.

Further study and rigorous analysis o uture outcomes will

be required to evaluate how the scoring system can best be

applied to such initiatives.

Reactive Opportunities to Counter

Extremist Narratives

The political content observed during data collection strongly

suggests that white nationalist seed accounts, their Twitter

ollowers and the most engaged subsets o their ollowers

believe that mainstream political conservatives represent

potential recruits, potential allies, or at minimum, persons

with whom they share some interests.

Regardless o exactly why, the data indisputably shows

that during the collection period, people engaged with

WN ideology consumed and distributed content related to

mainstream political conservatism and tried to engage with

mainstream conservatives.

Many mainstream conservatives likely nd these approaches

oensive and choose to ignore them, but this outreach

represents opportunities to intervene with people at various

stages in the radicalization process. I someone reaches

out to infuence you, you gain an opportunity to infuence

them back.


48/56

46

Conservatives who are politically active online may

thereore be better positioned than anyone to attempt CVE

interventions with white nationalists. We strongly recommend

that politically active conservatives consider ways they canspearhead or aggressively contribute to such CVE eorts by

directly challenging white nationalists who attempt to get their

attention through dierent styles o interaction online.


49/56

47

Data at a glance

Data collected

Seed accounts 12

Follower dataset 3,542

Tweets collected 342,807

Total number o links 84,825

Distinct domains linked to 676

Total number o hashtags 69,476

Unique hashtags 1,240

Identifcation rates byscore based on randomsample o 350 accounts(margin o error+/- 5.4 percent)

% overtly engagedwith WN as observedin manual inspectiono accounts

All ollowers 44

Infuence (Top 200) 73.5

Infuence (Top 100) 86

Infuence (Top 50) 98

Exposure (Top 200) 83



Interactivity (Top 200) 83



Link content by percentage

Mainstream news 12

Alternative 7

Extremist 8

Content neutral 73 15.9 (could not bedetermined)


50/56

48

Appendices

A: Seed accounts

The ollowing seed accounts were manually identied or

collection based on co-author J.M. Bergers previous research

on white nationalist movements. Each account was associated

with a well-known ideological white nationalist organization or

individual. The organizational aliation or each account is

given in parentheses.

1. @HeimdallsGhost (White Aryan Resistance)

2. @American3rdP (American 3rd Position)

3. @ANP14 (American Nazi Party)

4. @AryanNations (Aryan Nations, Morris Gullett branch)

5. @david_duke (David Duke)

6. @jartaylor (Jared Taylor, American Renaissance)

7. @TheNewNaziParty (American Nazi Party)

8. @nwront (Northwest Front)

9. @UKANW (Northwest chapter o The United Klans o

America)

10. @UKASOUTHFLA (Southern chapter o The United Klans

o America)

11. @TheHoodedone88 (Aliated with the United Klans)

12. @NSFM_Commander (Edward McBride, National Socialist

Freedom Movement)

The number o seed accounts was designed to be large

enough to produce substantial amounts o data, but small

enough to keep collection rom becoming prohibitively time-

and resource-consuming.

B: Scoring ormula and glossaryo metrics

For each interaction, points were awarded or infuence and

exposure. For instance, i Account A retweeted a post by

Account B, Account B received points or infuencing Account

A, while Account A received points or exposure to Account B.


51/56

49

I account C tweeted many links that were requently

retweeted, but never directly replied to or retweeted other

users, Account C would accrue a high infuence score but a

low exposure score. I Account D sent replies to many users,but only a ew responded, Account D would accrue a high

exposure score, but a low infuence score.

I Account E oten engaged in two-way conversations with

other users, Account E would register high scores in both

infuence and exposure.

Interactions were scored as ollows:

at-replies: 10

retweets (RT): 5

hat-tip (HT): 20

modifed tweet (MT): 15

non-reply mention: 2

any o the above actions in a tweet with a high-

value keyword: 25. The list o high-value keywords

included reerences to private communications, and

ofine and real-lie interactions such as call, email,

or DM (direct message). These terms were chosen

on the theory that the most infuential members o the

network would be those who extend their infuence

beyond Twitter.

ollowing: For Account A ollowing Account B, scores

were awarded between one and 10 based on the

average number o tweets per day. For instance, i

Account B tweeted an average o one or ewer times

per day, Account B received one point or infuence and

Account A received one point or exposure. I Account

B tweeted an average o 10 times per day or more,

the score was capped at 10 in order to avoid unduly

weighting scores toward users who were simply prolic.


52/56

50

We experimented with several dierent parameters or the

social network we would measure infuence and exposure

to the seeds, infuence and exposure among ollowers o

the seeds, and infuence and exposure to ollowers o theollowers. Based on manual examination, the second category

was easily the most interesting and eective. Except where

explicitly noted, all reerences to infuence and exposure are

limited by the parameter, interactions only among ollowers

o the seeds.

A quick overview o the measurements produced:

Infuence: A score that measures the ability o one twitteruser to prompt other Twitter users to redistribute or

engage with content.

Infuential: A Twitter user who has a high infuence score

and prompts many other users to engage with content.

Exposure: A score that measures the tendency o one

Twitter user to engage with or redistribute content

created by other users.

Exposed: The measured tendency o a given Twitter user

to engage with or redistribute content.

Interactivity: The sum o a users infuence and exposure

score, refecting quantity and quality o the users overall

interactions with other users.

Interactive: The tendency o a user to interact with

other users.

C: Top 10 most inuential

1. Eaglesnest1488

2. MAreedom

3. GILLY1488

4. MarmiteMan4

5. WARRIOR33_6

6. Aryanliving


53/56

51

7. Par_Sjogren

8. emilyscreams18

9. johnnywhiter

10. whiteunity14

D: Top 10 most exposed

1. WARRIOR33_6

2. danielb1714

3. Eaglesnest1488

4. karlhanke32

5. whiteunity146. HellWar88

7. DogoWarN

8. xdtac

9. craig25081989

10. viking14ranger

E: Top 20 most interactive

1. Eaglesnest1488

2. MAreedom

3. WARRIOR33_6

4. GILLY1488

5. whiteunity14

6. Aryanliving

7. MarmiteMan4

8. johnnywhiter

9. danielb1714

10. WhiteResister

11. KevinGoudreau

12. Par_Sjogren

13. emilyscreams18

14. IvanaWAU

15. HVRabbit

16. DogoWarN

17. B14USA

18. xdtac

19. PaulinaForslund

20. craig25081989


54/56

52

F: Glossary o Twitter Terms

Account: An account registered with Twitter. These are

usually, but not always, managed by a single user. Accountsmay also be congured automatically tweet content rom a

Web site.

Handle: The username chosen by a Twitter account holder.

Tweet: A short message posted on Twitter, limited to 140

characters in length. Tweets may be simple comments or

replies to other users, and may include links to content rom

other Web sites.

Follow: When a Twitter user subscribes to another

users tweets.

Followers: A list o all users that ollower a specic

Twitter account.

Followed: A list o all users ollowed by a specic Twitter

account, sometimes reerred to as riends.

Timeline: A users view o all tweets posted by the people

they ollow.

Reply or @reply: When a user directs a comment publicly to

another user. This is done by placing an @ sign in ront o

the target users handle.

Direct Message: When two users ollow each other, Twitter

allows them to communicate privately through this service.

Retweet or RT: When a user republishes another users

tweet, with attribution.

Modifed Tweet or MT: A retweet in which the user modies

the text o the original tweet.

Hat-Tip or HT: A tweet that acknowledges another user

as the source o a link.


55/56

53

Non-Reply Mention: Including another users handle in a

tweet that is not a response to a previous tweet by that user.

This oten unctions as the equivalent o CCing someone on

an e-mail message.

Hashtag: When a Twitter users highlights a term in a tweet

by placing the # sign in ront o a word or phrase. Hashtags

can be used or emphasis but are more oten intended to

make the tweet show up in thematic searches by other users.

G: Disclosures

This paper was commissioned by Google Ideas, but the

opinions expressed are those o the authors.

A minimum o 11 accounts in the set were noted in the

data to be ollowing co-author J.M. Bergers Twitter account

(@intelwire) at the time o collection. Bergers Twitter eed

ocuses on violent extremism and national security.

The most-linked URL in the white nationalist dataset was

YouTube.com, which is owned by Google, o which Google

Ideas is a part. The sixth most-linked URL was Blogspot.com,

another Google property. We considered a number o

online platorms or this study, including YouTube, but

Twitter was ultimately chosen due to its more accessible

API or developers, and the relative transparency o its

social transactions.


56/56

About ICSR

ICSR is a unique partnership o

Kings College London, the University

o Pennsylvania, the InterdisciplinaryCenter Herzliya (Israel) and the

Regional Center or Confict Prevention

at the Jordan Institute o Diplomacy.

Its aim and mission is to bring together

knowledge and leadership to counter

the growth o radicalisation and

political violence. For more inormation,

see www.icsr.ino

Who Matters Online: Measuring influence, evaluating content and countering violent extremism in...

Documents

Transcript of Who Matters Online: Measuring influence, evaluating content and countering violent extremism in...