Outsourcing Security Analysis with Anonymized Logs

Jianqing Zhang, Nikita Borisov, William Yurcik

2nd International Workshop on the Value of Security through CollaborationFriday, September 1, 2006

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

2

Motivation Managed Security Service Providers: Security

outsourcing is a trend Security monitoring is getting more complicated and

sophisticated Economical: assemble skilled security professionals Effective: shared security infrastructure across

organizational boundaries

Challenges Sensitive data is shared

Data protected by privacy laws Valuable information to competitors Useful information to adversaries

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

3

Managed Security Service Provider

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

4

Problem Statement

What are the criteria for log anonymization that sufficiently protect privacy and guarantee MSSP’s efficiency?

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

5

Contributions Case studies of common attack types based on classic logs

Derive a common set of anonymization criteria Retain time interval dependence between records Pseudonymize the external IP addresses re-identifiably Pseudonymize the internal IP addresses re-identifiably an

d preserve some network topology information

First step for privacy-preserving MSSPs

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

6

NetFlows and Syslogs NetFlows: network-based log

Timestamps IP address pairs (source/destination) Port pairs (source/destination) …

Syslog: host-based log Application level critical events

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

7

Which Data is Sensitive? Identity information

External (source) IP Partner, common guest and adversary

Internal (destination) IP Internal user

System privacy & security Timestamp

When the transactions happen Destination port number

Services and applications hosted on the system Subnet number

Internal network structure Records number

Overall resource usage

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

8

Log Anonymization Mechanisms Timestamp anonymization

Time unit annihilation Random time shifts Enumeration

IP address anonymization Truncation Random permutation Prefix-preserving pseudonymization

Port number anonymization Bilateral Classification Black Marker Anonymization Random permutation

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

9

Traffic Traces Logs: Port Scan

Start time SrcIPaddr SrcPort DstIPaddr DstPort P Pkts18:56:23.916 130.241.53.23 902 128.146.38.15 4138 6 1

18:56:23.924 130.241.53.23 900 128.146.38.15 4139 6 1

18:56:23.936 130.241.53.23 893 128.146.38.15 4140 6 1

18:56:23.944 130.241.53.23 891 128.146.38.15 4141 6 1

…

Scan all ports of a single host: Source: same address, different port numbers Destination:

Same addresses Different ports (sequentially)

In a short time

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

10

Traffic Traces Logs: DoS/DDoS SYN Flood

Source: same addresses, same (or different) port numbers Destination:

Same addresses Same port (intended to a particular protocol or application)

Protocol / Packets/ Packet size In a short time

Start time SrcIPaddr SrcPort DstIPaddr DstPort P Pkts B/Pk

21:47:11.670 165.132.86.201 514 128.146.97.7 80 6 1 40

21:47:11.854 165.132.86.201 514 128.146.97.7 80 6 1 40

21:47:12.198 165.132.86.201 514 128.146.97.7 80 6 1 40

21:47:12.338 165.132.86.201 514 128.146.97.7 80 6 1 40

…

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

11

Anonymization Constraints on Traffic Traces Logs Timestamp (Start Time)

Events interval and time dependence should be retained

Anonymization Time unit annihilation Random time shifts Enumeration

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

12

Anonymization Constraints on Traffic Traces Logs (cont.) Source/Destination IP address

Anonymized and re-identifiable Retain virtual network topology (dest.)

Anonymization Truncation Random permutation (pseudonyms)

Source (external) IP address Prefix-preserving pseudonymization

Destination (internal) IP address

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

13

Anonymization Constraints on Traffic Traces Logs (cont.) Source/Destination port number

Contain sensitive information More efficient if retained

Anonymization Bilateral Classification Black Marker Anonymization Random permutation

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

14

Active Operating System Fingerprinting

Syslog

Syslog + Tcplog

Time Stamp Host Name (IP) MessageSource Port Dest. Port

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

15

Anonymization Constraints on Syslog

Attributes List Anonymization Constraints Recommended Anonymization

Start TimeRetain events interval and time dependence

Random Time Shifts

Source IP Address Anonymized and Re-identifiable Pseudonyms

Source Port More efficient if retained Pseudonyms

Dest. IP Address Retain virtual network topology Re-identifiable if anonymized

Pseudonyms + Prefix-preserving

Dest. Port More efficient if retained Re-identifiable if anonymized Pseudonyms

Msg. Retained --

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

16

Sensitive Data After Anonymization Traffic volumes

Batched upload Aggregate volumes

Dummy log records Sacrifice the efficiency at MSSP False positives and false negatives

Size of customer base; customer retention Change the pseudonym mappings periodically

Structure of the internal network Simple pseudonyms Periodic rotation of pseudonyms

Policy dependent

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

17

Conclusion Sensitive data should be anonymized for

security monitoring

Constraints on log anonymization

Sensitive data leakage after anonymization and countermeasures

Privacy and efficiency is a trade-off

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

18

Future Work

Analyze other attacks Anonymization strategies for wide range of a

ttacks Patterns of attack detection and general prin

ciples Study other log formats and types Analyze correlation of different logs acro

ss different organizations

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

19

Q & A

Jianqing Zhangjzhang24@cs.uiuc.edu

Nikita Borisovnikita@uiuc.edu

William Yurcikbyurcik@ncsa.uiuc.edu

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

20

Anonymization Constraints on Traffic Traces Logs

Attributes List

Anonymization Constraints Recommended Anonymization

Start TimeRetain events interval and time dependence

Random Time Shifts

Source IP Address Anonymized and Re-identifiable Pseudonyms

Source Port More efficient if retained Pseudonyms

Dest. IP Address Retain virtual network topology Anonymized and Re-identifiable

Pseudonyms + Prefix-preserving

Dest. Port More efficient if retained --

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

21

Port Scan (cont.)

Portmap scan: Source: same address, different port numbers Destination: various addresses, same port (portmap

daemon) In a short time

Start time SrcIPaddr SrcPort DstIPaddr DstPort P Pkts10:53:42.54 165.132.86.201 9781 128.146.0.16 111 6 1

10:53:42.54 165.132.86.201 9788 128.146.0.71 111 6 1

10:53:42.54 165.132.86.201 9791 128.146.0.11 111 6 1

10:53:42.54 165.132.86.201 9381 128.146.0.51 111 6 1

…

Sep. 1, 2006 Outsourcing Security Analysis with Anonymized Logs

22

DoS/DDoS (cont.) Distributed SYN Flood

Source: different addresses, different port numbers Destination:

Same addresses Same ports (intended for a particular protocol)

Protocol / Packets/ Packet size In a short time

Start time SrcIPaddr SrcPort DstIPaddr DstPort P Pkts B/Pk

19:08:40.492 192.1.6.69 77 194.20.2.2 1308 6 1 40

19:08:40.532 192.1.6.222 1243 194.20.2.2 1774 6 1 40

19:08:40.720 192.1.6.108 114 194.20.2.2 1869 6 1 40

19:08:40.764 192.1.6.159 804 194.20.2.2 1050 6 1 40

…