Real-Time Data Security Webinar | SQLstream | July 2013 Series

If you haven’t dialed into the audio portion, please do so now:

U.S.A +1909.259.0025 | code 986.247.708

T H A N K YO U F O R J O I N I N G ! T H E W E B I N A R I S A B O U T TO S TA R T

Predict and Avert: Using Log File Data to Prepare Cybersecurity and Prevent Fraud Attacks in Real Time

| 2 Copyright © 2013 | +1 877 571 5775 | [email protected]

¤  Explain real-time Big Data and Operational Intelligence

¤  The principles of streaming data management

¤ Share our thoughts, experiences and use cases

¤ Audience Q&A

PROGRAM MISS ION


¤  Introduction (5 min)

¤ Presentation (35 min)

o  Internet, Theft and Cybersecurity – Emerging Issues

o  Streaming Operational Intelligence for Real-time Log Analyics

o  SQLstream at InfoArmor

o  Demonstration

¤ Q | A (20 min)

JULY 16 AGENDA


¤  July 9 2013 |10:00am PST- recording available upon request Analytics, Predictive Analytics, Prescriptive Analytics: The Anatomy of Operational Intelligence

¤  July 16 2013 |11:00am PST- recording available upon request Listen to your Sensors: A Tale of Managing Large Scale Sensor Networks in Real-time

¤  July 23 2013 |11:00am PST Predict and Avert: Using Log File Data to Prevent Cybersecurity and Fraud Attacks in Real Time

¤  July 30 2013 |10:00am PST No more CPR for your CDRs: Meet Real-time Traffic Utilization, Billing and Fraud Detection

T he Opera t i ona l I n t e l l i gen ce Se r i e s


OPERAT IONAL INTEL L IGENCE I n t eg ra t i ng Opera t i on s and Ana ly t i c s i n Rea l - t ime

As we move toward a real-time business environment, the capability to process data flows swiftly and flexibly will become increasingly important. SQLstream leads the industry in this kind of capability. ”

Robin Bloor Chief Analyst for Bloor Group

Business Intelligence

Operations

Real-time Operational Intelligence Continuous monitoring and analytics

Improve decision-making

Automate operational processes

Cybersecurity Fraud Monitoring

Identity Theft Alerts Compliance

Log Monitoring

”


T he I n fo r ma t i on Va l ue Cha i n

What is happening?

What might happen?

What just happened?

Make stuff happen!


ABOUT SQLs t ream

facts

o  Launched 2009 o  Multiple deployments

across many industries

o  Real world benchmarks

capabilities

o  Unstructured and structured data

o  Accelerates and extends Hadoop & RDBMS

o  Not only SQL

innovations

o  Massively scalable streaming data platform

o  Only standard SQL streaming engine

o  Five patents for stream processing

A Streaming Big Data Management Platform for real-time Operational Intelligence

from high-velocity machine data

Recent awards:


¤  Highest-scoring computer science graduate from the University of Manchester

¤  Highly successful career in the high tech, real-time software sector, with senior management positions at HP, XACCT and Followap.

¤  Holds 11 US patents

¤  Finalist in the 1995 International Management Challenge

Today ’ s P re sen te r : Dam ian B la c k SQL s t ream Co- founde r & CEO


¤  Over 15 years of information security, network security and intrusion detection experience

¤  CTO of InfoArmor, with previous experience at Level 3 Communications, Trustwave and owner of Sage Technologies.

Today ’ s Gue s t : Ch r i s t i an Lee s I n foAr mor CTO

¤  TWEET: during and after the webinar, please

use #cybersecurity for live discussions

¤  DIRECT QUESTIONS: please use the box to the

right of your screen

¤  RECORDINGS: an edited version of the webinar

recording will be emailed upon request

Cyber se cu r i t y - t he i s s ue s


Cybe r se cu r i t y | A GROWING MARKET

¤  No longer an unorganized hacker world ¤  Innovation and technology ¤  Global economy ¤  Political support

$207 Billion

Entrepreneur.com

In 2012, U.S. Navy databases were hacked and 200,000 sailors’ information was put at risk.


Cybe r A t ta c k s | DAMAGES

î  12.6 Million Americans were ID Theft victims last year

î  608,271,950 and growing records have been compromised due to security breaches since 2005

î  94% of healthcare organizations surveyed had at least one data breach in the past 2 years

î  1 in 4 data breach notification recipients became a victim of identity fraud

î  5 times more likely to be a fraud victim if your Social Security Number has been compromised in a data breach


Types of At tacks

¤  Credit card numbers ¤  Personally identifiable information ¤  Protected health information ¤  Social Security Numbers ¤  Intellectual Property or trade secrets

data breach

security incident in which sensitive, protected or confidential data is copied, transmitted, viewed, stolen or used by an unauthorized individual.


ABOUT InfoAr mor

¤  Founded by Washington Mutual to protect 10M credit card holders

¤  Focused on employee benefits and value-added subscriber services

¤  70% employee owned

¤  Growing at triple digit rates

¤  Engaged, satisfied subscribers

¤  Overall Satisfaction:

>90% “Strongly Agree” or “Agree”

Fortune 500 Clients

information protection experts


INTERNET SURVEILLANCE

InfoArmor Internet Surveillance uses bots to continuously monitor the Underground Economy to uncover compromised,

sensitive information. Whether it is personal identifying data or a medical insurance card, Internet Surveillance

uncovers breached data and alerts in real time.

What We Monitor:

¤  Malicious Command & Control Networks

¤  Black Market Forums

What is the Underground

Economy?

An ever-evolving complex of compromised machines, networks and web services identified by InfoArmor and leading cyber security firms.

¤  Phishing Networks

¤  Exploited Websites

¤  Known Compromised Machines & Servers


INTERNET SURVEILLANCE

How We Monitor:

¤  Proprietary hardware and software solution

¤  Unparalleled alert accuracy (minimized false positives)

¤  Secure: separate reconnaissance and analysis efforts, plus no refined search queries

What We Monitor:

¤  SSNs, names, addresses, emails and DOBs

¤  Wallet items (i.e. credit cards, medical insurance card)

INFOARMOR BOTS monitor UNDERGROUND ECONOMY

COMPROMISED DATA sent back to INFOARMOR

SENSOR compares compromised to subscriber data in secure environment, creating

ALERTS with 100% accuracy

X


KNOW YOUR DATA | S I EM CHALLENGES

SIEM Tools ¤  ‘Black Box’ tools – Easy start but rigid ¤ Limited to supported rules ¤ No repurposing of tools ¤ High cost of real-time performance

one size does NOT fit all

You as a product owner know your data best, not the SIEM provider.

S t r eam i ng Ope ra t i o na l I n t e l l i ge n ce


H igh - ve lo c i t y B ig Da ta Ana ly t i c s

Historical queries and data

enrichment

Storing valuable derived streams for future access

Ope

ratio

nal I

ntel

ligen

ce

Logs

Sensors

GPS

Networks

Social media

RFIDs

Servers

Telecom

Smart grid

Oil & Gas

Manufacturing

Logistics

M2M

Telematics

Retail

Internet

Banking

Data centers

Automotive

¤  Continuous Queries over Sliding Time Windows ¤  Analysis and Integration of Unstructured and Structured data ¤  Prescriptive Analytics drives Automated Actions


L og and Mac h i ne Da ta W h e r e i s t h e i n t e l l i ge n c e ?

TRANS,2013-02-17-15:30:22,3458783,2347897953,128.56.0.253,STATUS:-15, DE69975, 4157588342 Transaction Log Details

Web Server Logs

Network Logs

Device Locations

Twitter {"created_at:Thu Feb 17 15:30:55 +0000 2013,id:304612775055998976,id_str:304612775055998976,text:@MyServiceProvider today sucks, keeps dropped!,source:u006ca href=http:www.url.com rel=nofollow,followers_count:147,friends_count:10142, location: San Francisco, time_zone: Pacific, geo_enabled:true, location:u00dcT: -6.1987552,106.8661953, screen_name:APerson

<id>1597831220</id><deviceid>0198873465</deviceid><lat>lat=47.643957</lat><lon>lon= -122.3269</lon><time>2013-02-17T15:37:26Z</time><bearing>223.4535</bearing>

<id>1597865781</id><deviceid>0198873465</deviceid><lat>lat=47.645982</lat><lon>lon=-122.327500</lon><time>2013-02-17T15:37:26Z</time><bearing>200.6138</bearing>

<id>1597940125</id><deviceid>0198873465</deviceid><lat>lat=47.647381</lat><lon>lon=-122.326501</lon><time>2013-02-17T15:37:26Z</time><bearing>87.4357</bearing>

[Sun Feb 17 15:30:49 2013] [notice] srv-sfo-08 caught SIGTERM, shutting down [Sun Feb 17 15:30:49 2013] [notice] Apache/2.2.21 -- resuming normal operations

TERMINATE,ctl09gsx,01299796304,GMT-08:00,02-17-13,15:21:00,9,387,64ms,02-17-13,15:30:55,0005, IP-TO-IP,4157588342,8775715775,1,0,4157588342,RD_AXY_NN0_001,SFR01AAG34,40.50.245.60, 234.234.60.75,65678,411,399,SIP,SANFRANCISCO,0x4B1698,0x0005E,0x49768,4157588342,0198873465

Timestamp

Timestamp

Timestamp

Timestamp

Timestamp

Mobile # Customer

Mobile # Device ID Term Reason

Device ID Location

Location

Service Provider

Fail Code

Server


SELECT STREAM "SuspectLoginFailures"."accountNumber”,

"webFail" + "phoneFail" AS "loginFailureCount”, "transactionType", "amount”,"city", "region", "lat", "lon” FROM "SuspectLoginFailures" OVER "lastFew” JOIN "Transactions" OVER "now" USING ("accountNumber”) WHERE "isDebit” WINDOW "lastFew" AS (RANGE INTERVAL '1' MINUTE PRECEDING), "now" AS (RANGE CURRENT ROW);

The Power Of S t ream ing SQL A n E x a m p l e o f A P i p e l i n e P r o c e s s i n g N o d e

BUSINESS NEED: Detecting suspect account

debits following unusual login behavior

BLAZING SPEED: Processing millions of records

per second on low-end servers.

Identifies accounts where transactions have taken place concurrent with

suspect login activity over a 1 minute moving time window


CLEANING & FILTERING

STREAMING ANALYTICS

STREAMING AGGREGATION

CONTINUOUS INTEGRATION

Internet Security Fraud

Prevention Network

Monitoring

CyberAttack Monitoring

Compliance Monitoring

An Opera t i ona l I n t e l l i gen ce P la t fo r m

Log Files Databases Locations Networks Social Media Servers M2M Feeds


Rea l - t ime A r c h i t e c t u re Streaming Analysis and Integration for Infinite Flows of Unstructured Data in Real Time

Streaming Agent & Adapter Layer + JDBC API Hadoop Streaming

Query Planner & Optimizer for MPP Execution SQL

Developer Tools

Platform Administration

Streaming SQL Real-time Applications

Real-time Dashboards & Visualization

Logs

Sensors

GPS

Networks Social Media Servers

M2M Telematics

Impala SQL

HBase

HDFS / MR

Hadoop for Stream Persistence, Enrichment & Replay (Optional)

Any external data warehouse, operational system and

enterprise platform


T he SQL s t ream s -S t ream ing P roduc t Po r t fo l i o

s-Server Data Management Platform for Streaming Big Data

s-Analyzer Real-Time Visualization for Streaming

Operational Intelligence

s-Transport Geo-Analytics for Location-based

Applications

s-Visualizer Advanced

Visualization

s-Cloud s-Server EC2 AMI Deployment

s-St

udio

D

evel

oper

& A

dmin

Con

sole

Case s t udy


New cards New cell phones New accounts New addresses

offline

online Authentications Social Media

KNOW YOUR DATA | Da ta Sou r ce s


Continuous integration with existing data warehouses

Alerts, real-time dashboards and integration

with operational systems

Real-time syslog data collection using remote

lightweight agents

SQL s t ream| Rea l - t ime Secu r i t y I n t e l l i gen ce

Syslog data collection ¤  Agents turn log data into real-time streams ¤  Data from firewalls, authentication logs, databases, intrusion detection, user activity logs

Correlation and rules execution ¤  Dynamic rules (streaming SQL) ¤  Evaluate any combinations of log

streams any number of times ¤  Real-time and time window-based

analytics

Alert logging and data warehouse updates


Nex t s t ep s | COMPL IANCE

è Monitor system commands for employees, vendors and contractors

Continuous integration with existing data warehouses

Alerts, real-time dashboards and integration

with operational systems

Real-time syslog data collection using remote

lightweight agents


KNOW YOUR DATA | S I EM CHALLENGES

one size does NOT fit all

The real-time advantage ¤ Consumes any data source ¤ Scales to millions of events per second ¤ Streaming SQL analytics – tailored to our rules and data ¤ Multipurpose with low cost of ownership ¤ Scales the business for real-time operations

Demon s t ra t i o n Rea l - t ime Se c u r i t y I n t e l l i ge n ce


¤ Security Intelligence Demo

¤  Overview

o  Fraud analytics

o  Log monitoring

o  Alerts

¤  Demonstration

o  Walkthrough the visual demo

o  Illustrate SQL rules

o  Discuss ease of update and deployment of additional rules

Demo: Real - t ime Secur i ty In te l l igence


REAL-T IME FRAUD DETECT ION | Demons t rat ion

Remote Log Agent

Remote Log Agent

Remote Log Agent

Web Login Servers (Data Center 1)

Phone Login Servers

(Data Center 2)

Transaction Servers

(Data Center 3)

LoginEvents (STREAM JOIN: UNION ALL)

SuspectLoginFails (STREAM ANALYSIS, 1 MINUTE WINDOW)

SuspectDebits (STREAM JOIN, 1 MINUTE WINDOW)

RealTimeAlerts

SMTP Adapter

Account API

Data Warehouse

Email Alerts

Account Freeze (Auto)

Continuous ETL

Adapter & Agents

Streaming SQL Query

Data Stream


CREATE OR REPLACE VIEW “SuspectDebits” AS SELECT STREAM "SuspectLoginFailures"."accountNumber”, "webFail" + "phoneFail" AS "loginFailureCount”, "transactionType", "amount”,"city", "region", "lat", "lon” FROM "SuspectLoginFailures" OVER "lastFew” JOIN "Transactions" OVER "now" USING ("accountNumber”) WHERE "isDebit” WINDOW "lastFew" AS (RANGE INTERVAL '1' MINUTE PRECEDING), "now" AS (RANGE CURRENT ROW);


CREATE OR REPLACE VIEW "LoginEvents” AS SELECT STREAM "recNo", 'PHONE' AS "accessType", "accountNumber", "loginSuccessful", "callerId"

AS "client”, "directDial" AS "server", "customerId”, "countryCode", "countryName", "city", "region", "lat", "lon”

FROM "PhoneLoginEvents” UNION ALL SELECT STREAM "recNo", 'WEB' AS "accessType", "accountNumber", "loginSuccessful", "sourceIP”

AS "client", "destIP" AS "server", "customerId", "countryCode", "countryName", "city", "region”, "lat", "lon”

FROM "WebLoginEvents";

Opera t i o na l I n t e l l i ge n ce a nd To ta l Co s t o f Pe r fo r man ce


R E C O R D S P E R S E C O N D

To ta l Co s t Of Pe r fo r mance ( t o ta l COP ) T h e H i g h - Ve l o c i t y, L ow - L a t e n c y T i p p i n g Po i n t f o r B i g D a t a

Patterns Trends Mining Connections

Searches Inventory Reports Statistics Billing

SOCIAL E-COMM SECURITY TELEMATICS TELECOM

Trading Advertising Alerts Detection Signal

Intelligence

TO

TA

L C

OS

T


Intelligence

TELECOM

Patterns Trends Mining Connections

Searches Inventory Reports Statistics Billing

Trading Advertising Alerts Detection Signal

SOCIAL E-COMM SECURITY TELEMATICS

R E C O R D S P E R S E C O N D

TO

TA

L C

OS

T To ta l Co s t Of Pe r fo r mance ( t o ta l COP ) T h e H i g h - Ve l o c i t y, L ow - L a t e n c y T i p p i n g Po i n t f o r B i g D a t a


DATA EXPLOSION

COMPLEXITY

BUSINESS AGILITY

S t ream ing Opera t i ona l I n t e l l i gen ce

Eliminates the development risk •  Simplifies development, rapid time to market

Lowest Cost of Performance for Real-time Apps •  Efficient scale-out for high velocity data

Adding new applications on the fly •  With dynamic sharing of data streams across Apps

Real-Time Data Security Webinar | SQLstream | July 2013 Series

Technology

Transcript of Real-Time Data Security Webinar | SQLstream | July 2013 Series