Open source intelligence analysis
30
Open Source Intelligence Analysis Petr Špiřík “True genius resides in the capacity for evaluation of uncertain, hazardous, and conflicting information.” Winston Churchill
Transcript of Open source intelligence analysis
Open Source Intelligence Analysis
Petr Špiřík
“True genius resides in the capacity for evaluation of uncertain, hazardous, and conflicting information.”
Winston Churchill
About
The course
„You never know.“
„Truth is in the middle.“
„Can’t trust anything THEY say.“
„Don’t even try to understand.”
O RLY?
Petr Špiřík
Enterprise security, incident response, security architecture and design. This is what I do.
Cyber security, privacy, counter-surveillance and threat intelligence. This is what I like.
Education and the power of knowledge. This is what I believe.
CC-BY-SA • Petr Špiřík
Method in the Madness
Open Source
Public domain
Internet centric
Unstructured
Unreliable
Overwhelming
Intelligence Analysis
Define problem
Collect data
Analyze information
Report conclusion
Check with reality
CC-BY-SA • Petr Špiřík
Problem definition
“If one does not know to which port one is sailing, no wind is favorable.”
Seneca
CC-BY-SA • Petr Špiřík
Time Flow
Prediction
Forward looking
Limited assurance
Consistency is key factor
Am I the target of surveillance?
Explanation
What lead to current situation?
Which of these stories is true?
Opinions and behavior forming
How do vaccines cause autism?
CC-BY-SA • Petr Špiřík
Tangible problem
Right questions
True or false
Selection from menu
Realistic
Expected results
Ability to decide and act
Gather evidence
Debunk lie
CC-BY-SA • Petr Špiřík
Examples
Good
What is the root cause of Ukraine crisis?
Should higher education be free of charge?
Are government owned media biased?
Bad
Learn something about Ukraine and stuff.
Kids are unhappy at our schools, this must change!
Is this whole world just an illusion?
CC-BY-SA • Petr Špiřík
Collection
“Facts do not cease to exist because they are ignored.”
Aldous Huxley
CC-BY-SA • Petr Špiřík
Pick one
Data Driven
Holistic, mosaic, immersive
Information channels required
Establishes model
Hypothesis Driven
Problem focused
Hypothesis generation required
Solves one problem only
CC-BY-SA • Petr Špiřík
Data Driven
Sources
Validation of sources
credibility, accuracy, speed
Source management
review, update, remove
Typology
academic, research, news
Channels
Real time
RSS, Twitter & TweetDeck
Regular Google queries
weekly, monthly
Knowledge management system
notepad, wiki, Evernote
CC-BY-SA • Petr Špiřík
Hypothesis Driven
Google (Hacking)
Google operators provide powerful tool
-site:bbc.co.uk (Germany OR France) AND (Russia OR Putin OR “Russian Federation”) filetype:pdf
Investigation with Maltego
Open source intelligence, investigation and forensic tool
Community edition free of charge
Requires focus and dedication
Starting point and goal are absolute must
CC-BY-SA • Petr Špiřík
Evidence Evaluation
Weight
Relative
Can change based on subject of analysis
0% - not relevant
100% - critical evidence
Credibility
More stable
Function of source selection and management
0% - aeronet.cz
100% - your mother
CC-BY-SA • Petr Špiřík
How much Information you Really need?
Incomplete information
We make incomplete information decisions all the time
We will never have complete information
Consistency beats superstar intuitive guesses in the long run
Beware of indecision paralysis
Information Overload
You can always look for more information
There is critical mass of information that is “enough”
Additional information provided beyond this point do not change the result significantly
CC-BY-SA • Petr Špiřík
Analysis
“War is 90% information.”
Napoleon Bonaparte
CC-BY-SA • Petr Špiřík
Mind
Memory
Human mind is prone to errors
Tool is not important – the process is
Think about thinking – some errors can’t be avoided but can be compensated
Record everything
Separation
Do one step at a time
Do not mix idea generation with analysis
Do not make final judgment after first hypothesis evaluation, disregarding how strong it looks
Record everything
CC-BY-SA • Petr Špiřík
Situational vs. Theory Driven Analysis
Situational
Focus on specific situation
Location, culture, company
Understand the environment
Seek for issues present in given context
Judgment prioritizes the situation assessment and include issues identified
Theory Driven
Focus on issue investigated
Abuse, espionage, conflict
Understand the issue
Seek for shared symptoms of the issues in given context
Judgment prioritizes the issues and assess how these are affecting the situation
CC-BY-SA • Petr Špiřík
Problem deconstruction
Method
Useful for decision making
Define factors first
Assign them weight (up to 100%)
Define options
Quantify options (up to 100%)
Calculate the result
Sample Matrix
CC-BY-SA • Petr Špiřík
Should I go to Erasmus?
Weight ErasmusCzech Republic
Cost 40 20 (8) 80 (32)
Timing 10 70 (7) 30 (3)
Experiencegained
50 60 (30) 40 (20)
Total 100 45 55
Competing Hypothesis I
Hypothesis Generation
Brainstorming and recording
No evaluation
Clear definition required
Identify key differences
Remove redundant or unclear hypothesis
Evidence Gathering
Gather and validate evidence
Check evidence to each option
Strong/weak
Supporting/Disproving
Remove irrelevant evidence
CC-BY-SA • Petr Špiřík
Competing Hypothesis II
Review and conclusion
Identify promising hypothesis
Look for invalidation
Review evidence weight and credibility
Review hypothesis
Make tentative judgments
Identify game changing evidence
Example
War in Europe?
Yes, please.Cold war only.
Everything is good
Crimean crisis
+ ++ --
Greece vs. EU talks
-- + -
ISIS expansion
-- - -
CC-BY-SA • Petr Špiřík
Biases I
Evaluation of Evidence
Vividness
Absence of data
Thrive for consistency
Unassessed evidence
Confirmation bias
Cause and Effect
Favoring casual explanation
Favoring central scheme
Cause and effect
Internal vs. External drivers
Overestimating our importance
Mirror image/Projection
CC-BY-SA • Petr Špiřík
Biases II
Probabilities estimates
Availability rule
Anchoring
Verbal expressions
Complex scenarios
Base rate fallacy
Hindsight biases
“Everyone knew how this was destined to end. I am surprised you did not see it coming.”
Not problem of analysis itself
Problem of target audience
Can be discouraging
CC-BY-SA • Petr Špiřík
Reporting
“If you can’t explain it simply, you don’t understand it well enough.”
Albert Einstein
CC-BY-SA • Petr Špiřík
Audience
Formal
Professional assignment
Academic research
Reporting up
Focus on form
Credibility is at stake
Informal
Your own use
Circle of friends
Informing down
Don’t overdo it
Shoot early, update often
CC-BY-SA • Petr Špiřík
Structure
Top Down Approach
“Let the train crash. People want to see the train crash.”
Lead with key judgment first
Do not start with data
Make a statement, do not ask questions.
Length
One sentence for key message
One paragraph for executive summary
One page for overview report
Anything above one page – nice, but no one is going to read it.
CC-BY-SA • Petr Špiřík
Content
Be clear
Report is finished product
State the result
Provide estimates
Offer alternative conclusion
Be consistent
Create templates and use them
Align with problem statement
Keep the estimates consistent
Highlight game changing factors
CC-BY-SA • Petr Špiřík
Reality Check
If you know the enemy and know yourself you need not fear the results of a hundred battles.”
Sun Tzu
CC-BY-SA • Petr Špiřík
Close the loop
Look forward
Note breaking points in advance
Prepare the paths
Follow up if triggered
Update your system
Did any evidence source changed its reliability?
What was the feedback on the report?
What tasks were waste of time?
Learn, adapt, improve.
CC-BY-SA • Petr Špiřík
Tips & Tricks
DO
Trust in your analysis
Aim for constant improvement
Train. Intelligence analysis is a skill
Make this count
Do not
Become overconfident
Expect to read the future
Lose focus on problem
Raise unrealistic expectations
CC-BY-SA • Petr Špiřík
Key Judgments
“Hope is not a strategy.
Fear is not an option.
Luck is not a factor.”
James Cameron
CC-BY-SA • Petr Špiřík
Thanks!
Petr Špiřík
www.slideshare.net/zapp0/
Resources
Richards J. Heuer: Psychology of Intelligence Analysis
Michael Bazzell: Open Source Intelligence Techniques
Daniel Kahneman: Thinking, Fast and Slow
CC-BY-SA • Petr Špiřík
