Post on 12-Jan-2017
Big Data Shouldn’t be Big
ZoneFox – Who Are We ?
• Spin-out from Edinburgh Napier University
• Endpoint Detection and Response
ZoneFox as Big Data
• ZoneFox streams and collects endpoint data
• On average we write more than 300 million events per
day, per client
• 1 Trillion events on average for a 500-endpoint
customer
How do our Customers use ZoneFox?
What we learned
• Theoretical
• Practical
In Theory…
“Big data is a term for data sets that are so large or
complex that traditional data processing
applications are inadequate.”
Vs.
• 332 million active users
• 300-500 million tweets per-day
• 1.65 billion active users
• Every 60 Seconds:
– 510 Comments
– 293,000 statuses
– 136,000 photos
Source: Pew Research Center Source: Cowen & Company
$250 Billion$10 Billion
What does Facebook get right?
• Meaning
• Focus
Meaning
• Data is not Information
– Relevant vs Irrelevant
– Signal to Noise
Focus
• Volume
• Velocity
• Variety
= Value
In Practice…
ZoneFox – Architecture
Endpoint Agents [Windows, Linux &
SQL Server]
Collector & Presentation
Server(s)[Microsoft Windows]
Database, Reporting & Search Server(s)
[Ubuntu]
Horizontally Scalable. Proven to handle 10,000+ endpoint agents
Secure, Highly Compressed
Messages
Lightweight, Zero-Configuration
Agent
ZoneFox Data
• Simple Event Model
– Machine
– User
– Process
– Object
– Behaviour
– Time
What Does this Mean?
• We’re able to tell when
– Bob in HR starts to act like Alice in Engineering
– Alice is thinking of leaving the organisation
– Eve uses shadow IT out of frustration
Rolling your own
• Technology is often immature, resulting in frequent release
cycles with breaking changes.
• Usually, 1 big data solution solves 1 niche problem
• Expect to trade some speed for consistency: getting results
faster means you have to compromise on data "freshness"
Rolling your own
• Very small talent pool, as most devs won't have much
experience with the latest technology X
• Security in big data is an actively developing area
• Any additional features are usually not available out of the
box.
Thanks – Any Questions?
Jamie Graves, CEOj.graves@zonefox.com