Using Big Data and Network Centrality Metrics to Identify Influencers Caitlin Rowe, Jed Lee, Max...
-
Upload
thomas-powell -
Category
Documents
-
view
216 -
download
0
Transcript of Using Big Data and Network Centrality Metrics to Identify Influencers Caitlin Rowe, Jed Lee, Max...
Using Big Data and Network Centrality Metrics to Identify Influencers
Caitlin Rowe, Jed Lee, Max Nugmanov & Ruben Vargas
Advisor: David Beskow
Agenda
• Introduce Regionally Aligned Units
• Problem Statement
• Framing the Problem
• Data
• Model
• Results2
Recent Ebola Virus Outbreak
October 2014—U.S. Army Africa deployed 3,200 soldiers to Monrovia, Liberia as Joint Force Command for Operation United Assistance– support interagency
humanitarian efforts– supervise construction of Ebola
treatment units
• These Soldiers/Leaders deployed into an austere and unfamiliar environment and problem set
• Important that soldiers understand operating environment – Influential actors– Key infrastructure 3
Difficult Environments
4
Humanitarian Assistance
Natural Disaster Regional Conflict
Symmetric & Asymmetric Threats
Problem Statement
5
We will fuse open-source data to identify influential
state and non-state actors and infrastructure in
order to help tactical leaders of regionally aligned
units understand and engage their areas of interest
before and during temporary deployments to
austere environments.
Influential People? Important Places?
Framing the Problem:The Operations Process
6
Build and Maintain Situational Understanding
Visualize
Take from ADRP 5.0
Take from ADRP 5.0
Framing the Problem:OSINT – Open Source Intelligence
• OSINT – intelligence that is produced from publicly available information and is collected, exploited, and disseminated in a timely manner to an appropriate audience for the purpose of addressing a specific intelligence requirement – Congress findings
• Open source data contains a wealth of information that can help military leaders understand these complex environments.
• Creation of OSC under CIA • Working to get OSINT on equal ground
with all other intelligence sources
7
• Global Knowledge Graph is part of GDELT** Project
• GDELT is built using an enhanced TABARI algorithm
• GKG is an open source database that “connects the world's people, organizations, locations, themes, counts, and emotions into a single holistic network over the entire planet” 8
Data:Global Knowledge Graph
**GDELT is the Global Database of Events, Language, and Tone
9
Data:TABARI Algorithm
• Text Analysis By Augmented Replacement Instructions
• Machine coding of international event data using pattern recognition and simple grammatical parsing
• Designed to extract information from short news articles– Applied to ~80k English articles daily
Date Counts Themes Locations Persons Organizations Tone Source URLs
john kerry;barack obama;chris christie;george w bush
barack obama
john kerry
Persons found in the same article are listed in the Persons field, separated by a semicolon
The Global
Network
We assumed persons found in the same article are “connected”
GKG Data
11
GDELT
from by Kalev Leetaru in “Foreign Policy”
12
Identifying Influential People
Step 1:
Unix Code queries the GKG data set in order to find a geographic or topical subset (i.e. Bangladesh subset).
Linux Query
Step 2:
Use Network Centrality Models in R to identify influential individual names
Network Centrality Wiki Merge
Step 3:
Use Python to programmatically search wikipedia for 2 sentence descriptions
13
Measuring Network Centrality
Betweenness
Closeness
Degree
Eigenvector
j
ijiiD XXndC )(
A1
1
),()(
g
jjiic nndnC
kj
jkijkiB gngnC /)()(
14
Degree Centrality
• Number of neighbors, or edges, for a node
• Counts connections with other nodes
• Ignores any directions on the edges
• Measures local centrality• Computationally fast
j
ijiiD XXndC )(
• Closeness is a measure of a nodes distance to all other nodes.
• Closeness measures how easy it is for information to spread from a node to all other nodes sequentially
15
Closeness Centrality
1
1
),()(
g
jjiic nndnC
16
Betweenness Centrality
• Measures the number of shortest paths that cross a node
• Shows when one particular person is located in a strategically central position of the group
kj
jkijkiB gngnC /)()(
gjk = the number of geodesics connecting jk;gjk(ni) = the number that actor i is on.
17
Eigenvector Centrality
• Measures the importance of a node in a network, using adjacency matrix
• Assigns relative scores to nodes by giving “points” to nodes that are more influential than others
• Nodes depend on number of connections and relative scores of nodes
• Connections to more influential people will contribute to their own influence
A = adjacency matrix of graphλ = constant, eigenvalueν = eigenvector
18
Our Primary Algorithm
UNIX grep command for query
Bootstrap for Large Datasets
Combination of 2
Calculate Network Centrality
19
Nigeria Case Study (1 of 3)
• Boko Haram – Extremist group formed in 2009 that seeks establishment of an Islamic state in Nigeria.• In addition to Nigeria, they also
operate in Chad, Cameroon, and Niger
• Opposes Westernization of Nigerian society
• Pledged allegiance to Islamic State of Iraq and Syria (ISIS) on March 7, 2015.
• They have destabilized Nigeria, killing over 13,000 civilians and causing 1.5 million people to flee
What if US Regionally Aligned Forces deployed to Nigeria to advise and assist in the fight against Boko Haram?
20
Nigeria Case Study (2 of 3)
Abuja, Nigeria13,527 People in Network
321,206 Connections
• Given data for entire world for the last 30 days, we selected only the data related to Abuja, Nigeria
• This gives us a network with: • 13,527 People• 321,206 Connections• Density of 0.004
• This is a disconnected
network
21
Nigeria Case Study (3 of 3)
Rank Name Description
1 goodluck jonathan Goodluck Ebele Azikiwe Jonathan, GCFR, BNER, GCON (born 20 November 1957) is a Nigerian politician who has been President of Nigeria since 2010. Prior to his role as President, he served as Governor of Bayelsa State from 2005 to 2007 and as Vice-President of Nigeria from 2007 to 2010. Jonathan is a member of the ruling People's Democratic Party (PDP).
2 muhammadu buhari Muhammadu Buhari (born 17 December 1942) is a Nigerian politician and a retired Major General in the Nigerian Army who ruled Nigeria from 31 December 1983 to 27 August 1985, after taking power in a military coup d'état. The term Buharism is ascribed to the Buhari military government.
3 attahiru jegaProfessor Attahiru Muhammadu Jega is a Nigerian academic and Vice-Chancellor of Bayero University, Kano.
4 olusegun obasanjo Oluc¹gun Mathew OkikiÍla ; (born circa 5 March 1938) is a former Nigerian Army general who was President of Nigeria from 1999 to 2007. A Nigerian of Yoruba descent, Obasanjo was a career soldier before serving twice as his nation's head of state, as a military ruler from 13 February 1976 to 1 October 1979 and as a democratically elected president from 29 May 1999 to 29 May 2007.
5 sambo dasuki Sambo Dasuki, a retired Colonel of the Nigerian Army, is the current National Security Adviser (NSA) to President Goodluck Jonathan of Nigeria. He was appointed NSA on June 22, 2012 following the removal of General Owoye Andrew Azazi, retired.
6 rotimi amaechi Chibuike Rotimi Amaechi (born 27 May 1965) is the current Governor of Rivers State, Nigeria, since 2007. He was re-elected for a second term on 26 April 2011. Amaechi was a member of the People's Democratic Party before defecting to the All Progressives Congress on 27 November 2013.
7 garba shehu Kabir Garba Marafa is a Nigerian politician who was elected Senator for Zamfara Central in Zamfara State, Nigeria in the April 2011 elections, running on the All Nigeria People's Party (ANPP) ticket.Engineer Kabir Garba Marafa was formerly the Commissioner for Water Resources in Zamfara State.
8 ayo fayose Peter Ayodele Fayose (born 15 November 1960) is the current governor of Ekiti State in Nigeria .
9 ibrahim babangida General Ibrahim Badamasi Babangida (born August 17, 1941), also known as IBB, is a retired Nigerian Army officer who was the military ruler of Nigeria. He ruled Nigeria from 27 August 1985, when he overthrew Major General Muhammadu Buhari in a coup, until his departure from office on 27 August 1993, having annulled the elections held on June 12 that year.
10 olisa metuhOlisa Metuh is a Nigerian Lawyer, Politician and the National Publicity Secretary of the People's Democratic Party.
Rank Name
1 goodluck jonathan
2 muhammadu buhari
3 attahiru jega
4 olusegun obasanjo
5 sambo dasuki
6 rotimi amaechi
7 garba shehu
8 ayo fayose
9 ibrahim babangida
10 olisa metuh
23
Current Status and Way Forward
• Tactical units and leaders are testing the model and web-app
• 95th Civil Affairs Brigade and the Communications-Electronics Research, Development and Engineering Center will continue to develop the tool next year
• The model has sparked interest in the OSINT Community at the Intelligence and Security Command at Fort Belvoir
24
Feedback from Users
“This is an awesome concept! It would definitely be helpful in preparing to visit an country or conduct an engagement”
“This also broadens the lens beyond just military considerations to a fuller geo-political/social spectrum.”
“This could be used for: a) Prepping Senior leader for country travel and engagements, b) Regional Engagement Directorates keeping tabs on country development, c) Intel analyst looking at open source trends”
Questions?
The authors would like to thank the Data Tactics Corporation for their support and collaboration throughout this project.
25
Back-up Slides
26
27
Local Networks