Defending Against Advanced Persistent Threats …...2010/03/31 · Many people making changes....
Transcript of Defending Against Advanced Persistent Threats …...2010/03/31 · Many people making changes....
Copyright 20091
Defending Against Advanced Persistent Threats(Building Defensible Networks)
Ron Trunk, CCIE, CISSPSr. Consultant
Chesapeake Netcraftsmen
Copyright 20092
The FBI's Point of View
"The cyber threat can be an existential threat -- meaning it can challenge our country's very existence, or significantly alter our nation's potential. How we rise to the cybersecurity challenge will determine whether our nation's best days are ahead of us or behind us.”March 23, 2010
Steven ChabinskyDeputy Assistant Director
Cyber division, Federal Bureau of Investigation
Copyright 2009
Agenda
• A Few Case Studies• Threat Summary• What We Do Wrong• Building Defensive Networks• Security Monitoring• Detection• Response
Copyright 20094
Once Upon a Time......
Copyright 20095
A High Technology Company
• Analysts noticed high volumes of data being sent to multiple destinations in US and overseas.
• By the time blocks were put in place, significant amounts of data were sent.
• Forensic investigators reviewed logs and other evidence.
• Created a profile of attackers and timeline of events.
Copyright 20096
Who Were They?
• Can't attribute to specific people, but:• Attackers have habits and characteristics
– Individual “signature”– Allows investigators to create a profile of attacker
Copyright 20097
Who were They?
• Two teams of attackers
• A “Penetration Team”
• A “Exfiltration Team”
•
Copyright 20098
Who Were They?
• Similar attacker profiles at other companies came from addresses in PRC
• This company had many “minor” attacks in the past from PRC.
• Malware discussed on Chinese language sites and blogs
Copyright 20099
Characteristics of Attackers
• Preparation had likely been underway for months.
• Attackers understood company's network architecture and system details
• Compiling map of network, servers, groups, etc
Copyright 200910
Multiple Entry Points
• Stole user credentials, and used them to establish VPN access
• Tunneled RDP through compromised hosts• Piggybacked sessions to defeat two-factor
authentication• Pass-The-Hash techniques to compromise
other workstations and domain controllers
Copyright 200911
Timeline (Team1)
• Attackers enumerated group memberships to identify “interesting” employee accounts.
• Identified high performance servers as staging servers
• Copied target data from shares to staging servers using standard tools (copy & paste)
• Compressed and renamed target files to innocuous Windows names
• Established multiple RDP sessions to staging servers from internal C2
• Tested connections many times
Copyright 200912
Timeline (Team 2)
• Rehearsed exfiltration operations several times
• Tested connections, control, files, etc several times.– First attempts with custom FTP didn't work well
– Downloaded standard FTP software
• Began exfiltration in evening local time.
• Internal C2 had connection to local DSL customer– DSL customer had multiple connections to Hong Kong
• Used seven email servers in tandem to exfiltrate data
Copyright 200913
Response
• Company detected exfiltration in progress.– Blocked access, but not before large data loss
• Tuned IDS sensors– Saw attempts continuing for several hours.
Copyright 200914
Another Story...
• Small promotions company in New York
• Owner discovered her PC wouldn't boot Monday morning
• Used other computer for banking transactions
• Discovered that five wire transfers were made previous Friday totaling $164K– No prior business with recipients
• Bank originally said would cover losses– Audit revealed valid credentials used to transfer money.
– Bank now saying they're not responsible.
– No fraud protection on commercial accounts
– Case still pending
– Company facing bankruptcy
Copyright 200915
More
• Analysis of computer showed Zeus trojan• Designed specifically to steal bank credentials• Uses “screen scrapers” to capture mouse
keyboards• Can modify transactions on fly, and shows
you false response.• Has “kill OS” feature to delay discovery.
Copyright 200916
One More Story
• Small local non-profit• Involved with human rights issues around the
world• FBI informed them that copies of their emails
were being sent to PRC government.• Fewer than 50 people• No full time IT staff
Copyright 200917
Two Major Trends
• No longer defacements or bragging rights• International Organized Crime
– Stealing Money
• International Espionage– Stealing Technology, Intellectual Property– Strategic Intelligence
Copyright 200918
Espionage
• “China and other countries are engaged in global competition with us.”
– Tony Sanger, Dir of information Assurance, National Security Agency
•
• For China, gaining strategic advantage through the use of cyber attacks is a matter of official policy.
•
• Not James Bond stuff but:
– Stealing technology, intellectual property
– Why develop it when you can steal it from your adversary?
– Learning how we do things, how we operate, what our weakness are.
– Foreign military, government, patriotic groups (with or without gov't approval
Copyright 200919
Threat Summary
Copyright 200920
Advanced Persistent Threats
Copyright 200921
Advanced Persistent Threats
• Threats– Goal is to steal information and/or money
Copyright 200922
Advanced Persistent Threats
• Threats– Goal is to steal information and/or money
• Advanced– Sophistication of tools and techniques is every bit
as sophisticated as anything you might use.
Copyright 200923
Advanced Persistent Threats
• Threats– Goal is to steal information and/or money
• Advanced– Sophistication of tools and techniques is every bit
as sophisticated as anything you might use.
• Persistent– Attackers have specific goal. They will continue
their operations until they succeed.
Copyright 200924
The Malware Industry
• All the organization and sophistication of commercial software industry
• Specialties in every feature in the chain from crook to your wallet.
• Clever, well educated programmers.• Open market to buy and sell malware
Copyright 200925
Malware Specialization
• R & D in Vulnerabilities
• Locating victims
• Creating trojans and other attacks
• Phishing operations
• Operating botnets
• Generating spam
• Extortion sites
• Gathering hosts for botnets
• Fencing credit cards or other info
• Password dictionary brokers
• Software with activation keys
• Hardware tokens
• Customized malware editions
Copyright 200926
Who are the Targets?
• Anyone with money– Small to medium businesses are especially at risk
• Any government agency– The closer you are to defense or technology the
greater the risk
• Any organization with global operations– Contract Negotiations
Only amateurs attack machines. Professionals attack people. – Bruce Schneier
Copyright 200927
Who Are The Targets?
• Executives– Have access to financial info– Often have privileged accounts– Used to getting their own way
• “Whaddya mean I can't visit those sites?”– Not always the most tech savvy– Execs feel invincible and don't listen to awareness.
• That's for employees.
Copyright 200928
Who Are the Targets?
• Why, it's you!• You are the admins• You have the passwords• You can circumvent every security control
Copyright 200929
You Are Human
• You can be fooled by a phishing attack– Maybe not an email from a Nigerian official with a
banking problem, but
– An email from a colleague, or someone you know, regarding work, your interests, hobbies, etc
Copyright 200930
How Do They Know So Much About Me?
Copyright 200931
What We Do Wrong
Copyright 200932
Process
• Focused on compliance, not defense– Spend money and effort on wrong things– Security monitoring? What's that?
“We're not so good at defending against network attacks. But we're very good at defending against network auditors”
Copyright 200933
Security Management Becoming Unmanageable
• Increasing number of vulnerabilities• Average user has 22 applications on her PC• On the average, requires a security patch
every 5 days
Copyright 200934
People
• We tend to blame users (victims)
• Users act rationally, based on perceived risk
• Users tend to underestimate risks
• Security professionals tend to overestimate risks
• Concentrate on wrong things
• Cost to user for breach is low
• Cost to be secure is high
•
Copyright 200935
No Sanctions, No Change
• What is the response if users don't follow security advice?
• What is response if they don't get the contract out on time?
Copyright 200936
Designed to Fail
• We concentrate on what technology can do– It's our job, after all
• We don't think about how it can fail or be abused
• We lack imagination because we are basically (I assume) honest people
• We are not police detectives– Not trained in criminology– Used to be viruses and annoyances– Now real crime, real theft, real espionage– We're first line of defense, but we're not even playing
the game
Copyright 200937
Market Forces
• Deploy more, faster• Everything, everywhere, all the time!• Ooooh! Shiny new application!
Copyright 200938
Self Defeating Network
• Unknown– No control over devices on the network. Little or no documentation. No
inventory
• Unmonitored– The most common monitoring system? Users.
– If the phone ain't ringing, everything must be OK
• Uncontrolled– No change control. Many people making changes.
– Authentication systems weak or nonexistent
• Unmanaged– No control over software or systems.
– React to problems rather than anticipating them
• Trusted– Assumes everything is a trusted agent
Copyright 200939
The Result
• “We're the most vulnerable, we're the most connected, we have the most to lose, so if we went to war today in a cyber war, we would lose.”– Mike McConnell, former Director of National
Intelligence
Copyright 200940
Building Defensible Networks
Copyright 200941
Security as an Ecosystem
• All components are involved– Hardware– Networks– Operating Systems– Applications
• If you concentrate on one thing, your adversaries will just work around them
• You can no longer just be the “Network Guy”• Defensible Networks require that everything
play together.
Copyright 200942
Design Techniques – 3 Main Goals
• Make your network hostile to attacks– Not prevent, or necessarily stop– Slow down attackers– Make presence known– Give you time to react
• Make attacks detectable– Add capabilities to identify malicious behavior
• Make effective response– Make intelligent decision on how to react
Copyright 200943
Compartmentalization
• Think of fire doors• Fire doors don't prevent
fires• But they slow fire's
spread• Give time to evacuate• Give time to respond• Limit damage
Copyright 200944
Understand Your Network
Copyright 200945
Copyright 200946
Logical Compartmentalization
• Use addressing to define functional areas•
Users
Mgmt
Servers
Copyright 200947
Logical Function Provides Context
• Addressing provides who and what•• Context can define an event•
– Long duration connection to Chinese web site• Someone in sales• How about a server in R&D?
Copyright 200948
System Segregation
• Separate VLANs (subnets) for• Servers• Users• Administrators• Help Desk• Address indicates context• Context distinguishes between normal and
something bad
Copyright 200949
Device Segregation
• Separate VLANs for devices with different security policy requirements
Copyright 200950
Apply Policy To Devices
• No access from PCs to Phones• Limit access to printers• Limit access to other devices (cameras, etc.)
Copyright 200951
Block Inter-workstation Traffic
• Most common entry point– Cuz they're attached to users!
• Allow workstations to talk to servers
• Allow workstations to talk to gateways/proxies
• Allow workstations to talk to management devices
• Block everything else with host firewall
Copyright 200952
Control Access to Servers
• Allow only needed ports• Similar servers on VLANs
– Web, DB, file/print
• Block RDP or VNC from user workstations and other servers
Copyright 200953
Compartmentalization Results
• Reduces attack surface• Fewer places for attacker to go• Fewer places to hide
Copyright 200954
Control Outbound Access
• How many of you have an outbound firewall policy like this:
Permit ip any any
Copyright 200955
Control Outbound Access
• Only allow necessary ports (www, http, dns, etc)
• Block all others, esp dangerous ports (tcp 6667)
• Limits damage you can do to others– You may be the proxy network to China!
• Limit ports to those you proxy (more about that later)
Copyright 200956
Control Outbound Access
• Exceptions – Everybody's got 'em• Document them
– Who talks to whom– Which ports
• If you know what normal behavior is, you can detect abnormal behavior
• No workstation DNS, SMTP.• VPN with caution
Copyright 200957
Control Outbound Access
• Do your servers have Internet access?– Go right now and stop that.
• Easy path for attackers to exfiltrate• Document servers and ports for legitimate
access• Need a patch?
– Download it to a management workstation first, then install it.
Copyright 200958
Control Administrative Access to Servers
• Allow admin protocols like– RDP (tcp 3389)– VNC (tcp 5900)
• Only from defined hosts/networks– Management Workstations
Copyright 200959
Proxy External Connections
• Allows logging/inspection of URLs– Filtering URLs– Blacklists
• Proxy SSL connections too!– Install certificate on proxy
• Squid open source proxy– Well established, many commercial products based
on it
• Netronome• Cisco Ironport Web Gateway
Copyright 200960
How Proxies Work
Copyright 200961
Not Just For HTTP
• FTP• SSH• Telnet
Copyright 200962
Reputation Filtering
• Uses multiple criteria to “rate” websites• Much like a credit report• Information is gathered from web• Shared among participants in real time•• Cisco Ironport Web Gateway• McAfee Sidewinder•
Copyright 200963
Create A Sinkhole
• Replaces your default route• Only devices that need a default route are your
proxies• All other unknown traffic goes into bit bucket
(drop).
Copyright 200964
Sinkhole Operation
Copyright 200965
Sinkhole Operation
Copyright 200966
Sinkhole Operation
Copyright 200967
Sinkhole Advantages
• Drops any unknown traffic• Traps and identifies unknown traffic• Alerts and logs unknown traffic
– Bot phoning home– Malware C2 channel– Malware scanning for other hosts
Copyright 200968
Configuring a Sinkhole
• Use a PC running Wireshark– Configure static default route pointing to PC
• Use a Router– Create access-list to generate syslog messages
Copyright 200969
Layer 2 Security
• Prevents layer 2 attacks that bypass all the stuff we've talked about
• Insider attack, but malware is on the inside
Copyright 2005
Lower Levels Effect Higher Levels
• OSI Was Built to Allow Different Layers to Work Without the Knowledge of Each Other
• Unfortunately this means if one layer is hacked, communications are compromised without the other layers being aware of the problem
Physical LinksPhysical Links
MAC AddressesMAC Addresses
IP AddressesIP Addresses
Protocols/PortsProtocols/Ports
Application StreamApplication StreamApplicationApplication
PresentationPresentation
SessionSession
TransportTransport
NetworkNetwork
Data LinkData Link
PhysicalPhysical
ApplicationApplication
PresentationPresentation
SessionSession
TransportTransport
NetworkNetwork
Data LinkData Link
PhysicalPhysical
Co
mp
r om
ise
dC
om
pr o
mis
ed
Initial CompromiseInitial Compromise
Copyright 2005
Normal CAM Behavior 1/3
MAC A
MAC B
MAC C
Port 1
Port 2
Port 3
MAC Port
A 1
C 3
MAC Port
A 1
C 3
ARP for BARP for B
ARP for B
ARP for B
ARP fo
r B
ARP for B
B Is Unknown— Flood the Frame
Copyright 2005
Normal CAM Behavior 2/3
MAC A
MAC B
MAC C
Port 1
Port 2
Port 3
A Is on Port 1Learn:
B Is on Port 2
I Am
MAC B
I Am
MAC B
I Am MAC BI Am MAC B
B 2B 2
MAC Port
A 1
C 3
MAC Port
A 1
C 3
Copyright 2005
Normal CAM Behavior 3/3
MAC A
MAC B
MAC C
Port 1
Port 2
Port 3
Traffic A -> BTraffic A -> B
B Is on Port 2
Does Not See Traffic to B
Traffi
c A ->
B
Traffi
c A ->
B
MAC Port
A 1
B 2
C 3
MAC Port
A 1
B 2
C 3
Copyright 2005
CAM Overflow
MAC A
MAC B
MAC C
Port 1
Port 2
Port 3
MAC Port
A 1
B 2
C 3
MAC Port
A 1
B 2
C 3 Y Is on Port 3
Z Is on Port 3
I am M
AC Y
I am M
AC Y
Y 3Z 3
I am M
AC Z
I am M
AC Z
Traffic A -> BTraffic A -> B
Traffic A -> B
Traffic A -> B
Tra
ffic
A -> B
Traffi
c A ->
B
I See Traffic to B!
Assume CAM Table Now Full
Copyright 2005
Countermeasures for MAC Attacks
00:0e:00:aa:aa:aa00:0e:00:bb:bb:bb
00:0e:00:aa:aa:aa00:0e:00:bb:bb:bb
132,000 Bogus MACs
Only 3 MAC Only 3 MAC Addresses Addresses Allowed on Allowed on
the Port: the Port: ShutdownShutdown
Solution:
• Port security limits MAC flooding attack and locks down port and sends an SNMP trap
Limit the number of of MAC’s on an interfaceBy using Port Security
Copyright 2005
Port Security: Example Config
• 3 MAC addresses encompass the phone, the switch in the phone, and the PC• “Restrict” rather then “error disable” to allow only 3, and log more then 3
• Aging time of 2 and aging type inactivity to allow for phone CDP of 1 minute
Switch (config-if)#switchport port-security switchport port-security maximum 3 switchport port-security violation restrict switchport port-security aging time 2 switchport port-security aging type inactivity
If violation error–disable, the following log message will be produced:4w6d: %PM-4-ERR_DISABLE: psecure-violation error detected on Gi3/2, putting Gi3/2 in err-disable state
Copyright 2005
ARP Function Review
• Before a station can talk to another station it must do an ARP request to map the IP address to the MAC address
– This ARP request is broadcast using protocol 0806
• All computers on the subnet will receive and process the ARP request; the station that matches the IP address in the request will send an ARP reply
Who Is Who Is 10.1.1.4?10.1.1.4?
I Am I Am 10.1.1.410.1.1.4MAC AMAC A
Copyright 2005
ARP Function Review
• According to the ARP RFC, a client is allowed to send an unsolicited ARP reply; this is called a gratuitous ARP; other hosts on the same subnet can store this information in their ARP tables
• Anyone can claim to be the owner of any IP/MAC address they like
• ARP attacks use this to redirect traffic
I Am I Am 10.1.1.110.1.1.1MAC AMAC A
You Are You Are 10.1.1.110.1.1.1MAC AMAC A
You Are You Are 10.1.1.110.1.1.1MAC AMAC A
You Are You Are 10.1.1.110.1.1.1MAC AMAC A
Copyright 2005
ARP Attack in Action
• Attacker “poisons” the ARP tables 10.1.1.
1MAC A
10.1.1.2MAC B
10.1.1.3MAC C
10.1.1.2 Is Now 10.1.1.2 Is Now MAC CMAC C
10.1.1.1 Is Now 10.1.1.1 Is Now MAC CMAC C
ARP 10.1.1.1 ARP 10.1.1.1 Saying Saying
10.1.1.2 is MAC C10.1.1.2 is MAC CARP 10.1.1.2 ARP 10.1.1.2
Saying Saying 10.1.1.1 is MAC C10.1.1.1 is MAC C
Copyright 2005
ARP Attack in Action
• All traffic flows through the attacker 10.1.1.3 Is Now 10.1.1.3 Is Now
MAC CMAC C
10.1.1.1 Is Now 10.1.1.1 Is Now MAC CMAC C
10.1.1.1
MAC A
10.1.1.2MAC B
10.1.1.3MAC C
Transmit/ReceiveTransmit/ReceiveTraffic toTraffic to
10.1.1.1 MAC C10.1.1.1 MAC C
Transmit/Receive Transmit/Receive Traffic toTraffic to
10.1.1.2 MAC C10.1.1.2 MAC C
Copyright 2005
ARP Attack Clean Up
• Attacker corrects ARP tables entries
• Traffic flows return to normal10.1.1.
1MAC A
10.1.1.2MAC B
10.1.1.3MAC C
10.1.1.2 Is Now 10.1.1.2 Is Now MAC BMAC B
10.1.1.1 Is Now 10.1.1.1 Is Now MAC AMAC A
ARP 10.1.1.1 ARP 10.1.1.1 Saying Saying
10.1.1.2 Is MAC B10.1.1.2 Is MAC BARP 10.1.1.2 ARP 10.1.1.2
Saying Saying 10.1.1.1 Is MAC A10.1.1.1 Is MAC A
Copyright 2005
Dynamic ARP Inspection
• Uses the DHCP Snooping Binding Table Information
• Dynamic ARP Inspection– All ARP packets must match the
IP/MAC Binding table entries– If the entries do not match,
throw them in the bit bucket
10.1.1.1MAC A
10.1.1.2MAC B
10.1.1.3MAC C
ARP 10.1.1.1 ARP 10.1.1.1 Saying Saying
10.1.1.2 is MAC C10.1.1.2 is MAC C
ARP 10.1.1.2 ARP 10.1.1.2 Saying Saying
10.1.1.1 is MAC C10.1.1.1 is MAC C
Is this is my Binding Table?NO!NO!
None Matching
ARP’s in the Bit Bucket
DHCP Snooping EnabledDynamic ARP inspection Enabled
Copyright 2005
Summary of ARP Attacks
• Port Security prevents CAM table attacks by limiting the number of MAC addresses per port
• Dynamic ARP inspection prevents ARP Attacks by intercepting all ARP requests and responses
• For more information:– http://www.cisco.com/web/CA/events/pdfs/L2-security-
Bootcamp-final.pdf
Copyright 200984
Operational Changes
• The hardest to do• Requires changes in behavior• Affects worst users
– Better than everyone else– Don't need rules
Copyright 200985
Administrative Passwords
• Individual administrative passwords– Used only for administrative tasks
• Domain administrative passwords different than all others – Use only for domain controllers
• Local administrator for servers must be different than user PCs
• If user must be local administrator, create two accounts
Copyright 200986
Don't Cache Passwords
• Stops Pass-The-Hash attacks• Set credential cache to 0
– HKEY_LOCAL_MACHINE\Software\Microsoft\ Windows NT\Current Version\Winlogon\ CachedLogonsCount
– Default is 10
• For Laptops, set to 1•
Copyright 200987
No LM or NTLM hashes
Copyright 200988
Remove Debug Rights
Copyright 200989
Admin Accounts
• Are you local admin on your PC?• Are you domain admin?•
Copyright 200990
Don't Surf and Manage From Same PC
• Best: 2 PCs• Also good: Use virtualization
– Host has no Internet– Guest has proxy configured– Create snapshot (safe config)
• Separate admin and user accounts for you.• Split subnets for admin filtering• Download patches on user machine, store on
share.
Copyright 200991
Why Go Through All This?
• You have admin credentials• You can bypass security features• You have “master keys”
Copyright 200992
Miscellaneous
• Make /tmp directory non-executable• Yes, your users will complain• Must move files to another directory.
Copyright 200993
Security Monitoring
Copyright 200994
Security Monitoring
• Defensible Network is built for monitoring• Required for secure networks• All the tactics we've talked about will not stop
attacks• Secure buildings still need security guards• (Very) Limited Forensics
Copyright 200995
Why Monitor?
• Are your hosts compromised?• How do you know?• Are your protections working?
Copyright 200996
Monitoring Infrastructure
• Secure Access• Data Storage• Analysis Tools
Copyright 200997
Secure Access
• Monitoring tools on separate subnets• Protect with access-lists (no firewalls)• Use SSH/SSL for access• Consider OOB for admin access
Copyright 200998
Storage Requirements
• Can generate lots of data• Need for archive, retrieval• Integrity checking
Copyright 200999
What data to monitor
• Syslog• Netflow• Other logs• IDS/IPS alerts
– “Smoking packet” unlikely
• Packet Capture
Copyright 2009100
Syslog
• Everybody does it• Nobody looks at it• Retrieval as important as collection• Syslog-NG• Consider second server
Copyright 2009101
Log Analysis Tools
• Splunk• Log Logic• Sawmill•• Indexes all kinds of logs, not just syslog:
proxies, dhcp, web servers, etc• Search, graph, charts
Copyright 2009102
Netflow
• Router collects info on data flows• Data flow = phone call• Netflow = phone bill•• Shows source and destination addresses and
ports, protocol, number of bytes, time, duration.
Copyright 2009103
Netflow Data Collection
NetFlow Data
NetFlowCollector
Query
TRAFFIC
FLOw
• When configured, router sends a Netflow record for every flow to the collector.
• Netflow collector stores Netflow records for later query and analysis
Copyright 2009104
Configuring Netflow on IOS
• MyRouter(config)# ip flow-export version 5• MyRouter(config)# ip flow-export destination 192.168.0.1 2055• MyRouter(config)# ip flow-export source interface Loopback0• MyRouter(config)# Interface FastEthernet 0/1• MyRouter(config-int)# ip route-cache flow• MyRouter(config-int)# end• MyRouter#
Copyright 2009105
Reading Netflow
• Can display on CLI– Show ip cache flow
• Send data to a collector– Keep records for later use– Search for specific flows
Copyright 2009106
Netflow Can Answer Questions
• Did host A talk to server B?• Did host make a connection to a particular
website?• Why did A ftp 200Mb of data to a host in
china?• What other hosts did A connect to?• Why is our top destination a foreign address?• Why is host A using a TOR or anon proxy?•
Copyright 2009107
Netflow Collectors
• Scrutinizer (www.plixer.com)• Solarwinds• OSU Flow-tools (open source)
Copyright 2009108
Other Logs
• Tacacs– Who made what changes to routers
• Proxy – What sites we visited. What URLs
• DHCP – Which host had which address
• DNS– What domains were resolved. What ones failed?
Copyright 2009109
Full Packet Capture
• Capture and store every packet for later analysis• No, I’m not crazy
– Well, maybe a little bit, but that’s between me and my therapist.
SpanPort
Snort orTethereal or
Tcpdump
Internet
Copyright 2009110
Full Packet Capture
Data Mb GB
1 54
5 270
10 540
25 1,350
45 2,430
50 2,700
75 4,050
100 5,400
• Not as difficult as you might imagine• 1. Compute your average data rate• 2. Multiply by storage time
• Example: 5 day retention– 10Mb average rate– 540GB of storage– Less if compression used
– 1TB disks drives < $150
Copyright 2009111
With Full Packet Capture You Can
• Analyze suspicious activity– What did the user FTP to an external host?
• Confirm alarms– Was that really a Web attack?
• Identify attack vectors– Did host A download a trojan?
• Provide evidence for legal action– Capture complete conversation between hosts
• Identify who talked to whom– What other hosts did this host talk to?
• Verify IDS or patch effectiveness– After patching, replay attack. Did the patch work?
Copyright 2009112
Full Packet Capture
• Capture data with– tethereal– tcpdump
– Snort
• Snort can write directly to MySQL database• Report using
– any SQL reporting tool
– Wireshark
• Commercial Products
– Netwitness
– Solera
Copyright 2009113
Intrusion Detection/Prevention Sensors
• Labor intensive• Requires lots of tuning to be effective• Use for specific, known threats• Need to correlate with other information to be
effective
Copyright 2009114
Cisco Global Correlation
• Reputation filtering for IPS sensors• Uses worldwide monitoring on Internet traffic
to determine threats, bots, malware sites, etc• Takes advantage of “wisdom of crowds”• Your sensor is updated every 5 minutes• Bad sites are blocked
– Increases sensor “risk rating”–
• www.cisco.com/go/ips
Copyright 2009115
Where Do I Put IDS/IPS Sensors?
• Edge of network– Border Gateways
• Protect Sensitive Data– Data Center Distribution
Copyright 2009116
Aggregation and Filtering
• Aggregation switches for multiple links• Multi 10G inputs• Filter Outputs to multiple sensors• Mfrs:
– Gigamon– Netoptic– Apcon
Copyright 2009117
Copyright 2009118
Aggregation and Filtering
• Filter Traffic to Sensors – Http– SMB– POP/SMTP– Specific addresses– Others for specific forensic needs
• Use multiple sensors for high data rates
Copyright 2009119
Log Maintenance
• Long term storage• Evidence Requirements
•Disclaimer: I am not an attorneyNone of my friends are attorneysI actively discourage my children from
attending law school.
Copyright 2009120
Evidence Requirements
• Chain of Custody– Who had possession of files– Keep a handwritten log, sign and date entries
• Integrity– Demonstrate that files have not changed– Create checksums of files– Log rotation script can generate checksum– Burn to CD
Copyright 2009121
Detection
Copyright 2009122
Detection
• What do you look for?• Monitor policy violations
– Actual or attempted
• ACL logging• Firewall logs
– Denied AND allowed (outbound)
• IDS on sinkhole• Root or administrator login• Database auditing
Copyright 2009123
Detection
• Windows Logs– New accounts– Logon/Logoff– Failure Audit– Success audit on sensitive files/directories– Event 552 Explicit credentials–
• Snare– Converts Windows events to Syslog
Copyright 2009124
What Do You Monitor?
• Systems that– Have Sensitive Data– Are High Risk (legacy)– Generate Revenue– Monitor Security
Copyright 2009125
What Do You Monitor?
• Actionable Events– Don't alert if you aren't going to do anything about
it.
• Enforceable Events– Can you enforce policy ?––
Copyright 2009126
Anomaly Detection
• Mostly a manual process
• Serious attacks try to stay under the radar
• Netflow Anomalies– High traffic
– Foreign addresses
– Failed attempts
• DNS Logs– Failed resolutions
• Bots looking for C&C
• Windows logs– Failed logins
– Disallowed RDP Sessions
Copyright 2009127
Security Event Monitor (SEM)
• Security Incident Monitor (SIM)
• Security Incident Event Monitor (SIEM)
• Correlates logs
• Logic rules
• If this and then that...
•
• Cisco MARS
• Nitrosecurity
• LogRythym
• Arcsight
Copyright 2009128
Response
Copyright 2009129
Incident Response
• Planning is essential• Contact lists
Operations– Legal– Public contact
• Define response team• Tools – HW & SW
– Cables, adapters, media
• Practice makes perfect
Copyright 2009130
Incident Response
• Detect Anomaly• Investigate• Declare Incident• Contain• Clean-up• Review
Copyright 2009131
What Are Your Organization's Priorities?
• Prevent Exfiltration• Prevent Public Disclosure• Keep Systems in Production• Restore Integrity• Prosecute Attackers
Copyright 2009132
Should You Involve Law Enforcement?
• Can be a tough call• Involve senior staff and legal dept• What you should know
– Publicity– Confiscation of evidence
• Regulatory Requirements• If foreign attacker, odds of prosecution go
waaaaay down
Copyright 2009133
Response
• Resist temptation to remediate immediately– You'll just drive them deeper underground
• Try to identify all compromised machines– Detect patterns, URLs, hosts– Remediate all hosts at once
Copyright 2009134
In the meantime...
• Slow them down• Policing filters (9600 baud?)••
Copyright 2009135
Should you clean up compromised hosts?
• If you really know what you're doing.• IMHO, it's too easy to miss something• Best to reimage
–
Copyright 2009136
Post Mortem
• Understand chain of event• Improve detection filters• Improve processes• Prepare for next time
– There will always be a next time
Copyright 2009137
Summary
Copyright 2009138
Summary
• Internet attacks can pose a serious threat to your organization
• Sophisticated intruders use stealth to defeat antivirus and other detection software
• Defensible networks help detect and thwart attacks
• Use compartmentalization, segregation to defend
• Monitor your network for signs of compromise
Copyright 2009139
Questions?
•Ron Trunk•rtrunk at netcraftsmen.net
•