Intrusion Detection Systems (IDS)

Adli Wahid

Role of Detection in Security

• Part of security monitoringoViolation of security policieso Indicators of compromiseo Threat drive or Vulnerability driven oWhat’s happening on the network?

• RulesoDetection is based on rules

• Action • What do we do when detection happens? • Alert and Investigate • Drop / Block

Perspective – Adversary Tactics and Techniques • Mitre Att&ck Framework• Tactics – what are the goals of the adversary?• Technique – how do they do it? • Subject to:

o Resourceso Platforms

• Can we used this knowledge for detection?o Observe Adversaries Behaviouro Techniques, Tactics and Procedures (TTPs)o Deploy in prevention, detection, response

https://attack.mitre.org

Your Adversaries

Your Assets Your Systems

MotivesTargets

InfrastructureBehaviour

Reference: https://published-prd.lanyonevents.com/published/rsaus19/sessionsFiles/13884/AIR-T07-ATT%26CK-in-Practice-A-Primer-to-Improve-Your-Cyber-Defense-FINAL.pdf

Reference: https://published-prd.lanyonevents.com/published/rsaus19/sessionsFiles/13884/AIR-T07-ATT%26CK-in-Practice-A-Primer-to-Improve-Your-Cyber-Defense-FINAL.pdf

Making Your Infrastructure Forensics Ready

• Detecting known or potentially malicious activities • Part of the incident response plan • If your infrastructure is compromised

oCan you answer the questions: what happened and since when?oCan we ‘go back in time’ and how far back?

• What information you you need to collect and secure? • Centralized logging

Intrusion Detection Systems

• An intrusion detection system (IDS) is a device or software application that monitors a network or systems for malicious activity or policy violations. Any malicious activity or violation is typically reported either to an administrator or collected centrally using a security information and event management (SIEM) system

Different types of Intrusion Detection Systems

• Host Based • Network Based

IDS Technology landscape

Preventive Real Time

Host Based IDS• A host-based IDS is capable of monitoring all or parts of the dynamic

behavior and the state of a computer system, based on how it is configured.owhich program accesses what resourceso state of a systemonot been changed by intruders

• Monitoring Dynamic Behaviour • Who is doing what in a system

• Monitoring State• Detect modifications

Host Based IDS (2)

• Techniques o System Integrity Check oAlertingoVulnerability Detection oConfiguration assessment oRootkit detection o Security PolicyoActive Response

• OpenSCAP• OpenSCAP is an OVAL (Open Vulnerability Assessment Language) and XCCDF

(Extensible Configuration Checklist Description Format) interpreter used to check system configurations and to detect vulnerable applications.

Source: https://wazuh.com

Examples • OSSEC

ohttps://www.ossec.net

• Wazuhohttps://www.wazuh.com

• Some other interesting projectsoOSQuery - https://www.osquery.io/o Loki - https://github.com/Neo23x0/Lokio Sysmon - https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon

oKey component – agent or log/data shipper

Network Based IDS • Network intrusion detection systems (NIDS) are placed at a strategic

point or points within the network to monitor traffic to and from all devices on the network.operforms an analysis of passing traffic on the entire subnet, and matches the

traffic that is passed on the subnets to the library of known attacks.

oDetection Method o Signature basedoAnomaly based

• Examples (Free / Open Source) o SNORTo Suricata o Zeek (Bro)

Limitations

• Noise • False Positives• Signature management

oOutdated o0-days

• Can’t compensate for weak authentication / identification • Encrypted packets

How to monitor the network?

• Network TAPso A network tap is a hardware device which provides a way to access

the data flowing across a computer network o The network tap has (at least) three ports: an A port, a B port, and a

monitor porto Network Taps are fully passive device

§ Pros§ Passive / Fail Safe§ Exact duplicate of network traffic

§ Cons§ Expensive § Require physical infrastructure

Port Mirroring / SPAN Port• Also known as SPAN (Switch Port Analyzer)• A SPAN is a dedicated port on a managed switch that takes a mirrored copy of network

traffic off the switch to be sent to a monitoring device • Port mirroring is used on a network switch to send a copy of network packets seen on

one switch port (or an entire VLAN) to a network monitoring connection on another switch port

• Pros• Low cost, easy to deploy • Feature available in most switch

• Cons• Potential packet loss • Utilise switch resources • Attacker can disable SPAN/Mirror Port

Caveats of IDS

• "Alert Fatigue", • can be a daunting task and quickly fill an analysts plate combing through false positives

trying to find that one good alert.• Administrators fail to keep alerts relevant

• IDS is seen as a system with many of false positives• No maintenance is devoted towards managing it, can be spotty coverage• Rules/signatures are not up to date

• Analysts fail to understand rules• Don't have proper training on how to validate rules• Are not kept in the loop on specific rules that are of high importance

• Organization can't respond to problems generated by IDS• Response policies are not in place• System administrators don't know where to look for issues• Security organization isn't empowered to respond to issues

Suricata

Suricata Intrusion Detection System

• Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine.

• It is open source and owned by a community-run non-profit foundation, the Open Information Security Foundation (OISF).

• Suricata is developed by the OISF• The Suricata source code is licensed under version 2 of the GNU

General Public License

Suricata - History

• Beta release – Dec 2009• First standard release – July 2010• Features

oMulti-threadingoAutomatic protocol detection o JSON standard outputso file matching, logging, extraction, md5 checksum calculationoDNS loggero etc

In a nutshell

• The Suricata engine is capable of real time intrusion detection (IDS), inline intrusion prevention (IPS), network security monitoring (NSM) and offline pcap processing

• Suricata inspects the network traffic using a powerful and extensive rules and signature language, and has powerful Lua scripting support for detection of complex threats

• With standard input and output formats like YAML and JSON integrations with tools like existing SIEMs, Splunk, Logstash/Elasticsearch, Kibana, and other database become effortless

Rules Management

• It is important to have rules that are up-to-date • Management of rules is being done by suricata-update• Within the configuration file there are variables for default-rules-path

and rule-files: • By default all rules are merged into a single file suricata.rules• Rules can be enabled and disabled

• /etc/suricata/enabled.conf• /etc/suricata/disabled.conf

Rules/Suricata

• Actions (i.e. alert or drop) are decided by rules • In most occasions people are using existing rulesets

• Emerging Threats • Talos/Cisco • https://github.com/suricata-rules/suricata-rules

Rules Format

• A rule/signature consists of the following:o The action, that determines what happens when the signature matcheso The header, defining the protocol, IP addresses, ports and direction of the

rule.o The rule options, defining the specifics of the rule

drop tcp $HOME_NET any -> $EXTERNAL_NET any (msg:”ET TROJAN Likely Bot Nick in IRC (USA +..)”; flow:established,to_server; flowbits:isset,is_proto_irc; content:”NICK “; pcre:”/NICK .*USA.*[0-9]{3,}/i”; reference:url,doc.emergingthreats.net/2008124; classtype:trojan-activity; sid:2008124; rev:2;)

Rules – Action

• What happens if signature matches• Options

o Passo Drop (IPS mode)o Reject o Alert

alert tcp $HOME_NET any -> $EXTERNAL_NET any (msg:”ET TROJAN Likely Bot Nick in IRC (USA +..)”; flow:established,to_server; flowbits:isset,is_proto_irc; content:”NICK “; pcre:”/NICK .*USA.*[0-9]{3,}/i”; reference:url,doc.emergingthreats.net/2008124; classtype:trojan-activity; sid:2008124; rev:2;)

Rules - Protocol

drop tcp $HOME_NET any -> $EXTERNAL_NET any (msg:”ET TROJAN Likely Bot Nick in IRC (USA +..)”; flow:established,to_server; flowbits:isset,is_proto_irc; content:”NICK “; pcre:”/NICK .*USA.*[0-9]{3,}/i”; reference:url,doc.emergingthreats.net/2008124; classtype:trojan-activity; sid:2008124; rev:2;)• 4 protocols

o tcp (for tcp-traffic)oudpo icmpo ip (ip stands for ‘all’ or ‘any’)

• And some application layer protocols*o Dns, http, smb, ssh, smtp, imap, tls , etc

Rules - Source and destination

• drop tcp $HOME_NET any -> $EXTERNAL_NET any (msg:”ET TROJAN Likely Bot Nick in IRC (USA +..)”; flow:established,to_server; flowbits:isset,is_proto_irc; content:”NICK “; pcre:”/NICK .*USA.*[0-9]{3,}/i”; reference:url,doc.emergingthreats.net/2008124; classtype:trojan-activity; sid:2008124; rev:2;)

• Source and Destination of traffic • IP address / Block• Domain names

• Can be set as: • Variables – defined in /etc/suricata.yaml• IP address (v4/v6) format • ‘any’• Negation i.e. ! can be used as well

Rules - Ports (source and destination)

• drop tcp $HOME_NET any -> $EXTERNAL_NET any (msg:”ET TROJAN Likely Bot Nick in IRC (USA +..)”; flow:established,to_server; flowbits:isset,is_proto_irc; content:”NICK “; pcre:”/NICK .*USA.*[0-9]{3,}/i”; reference:url,doc.emergingthreats.net/2008124; classtype:trojan-activity; sid:2008124; rev:2;)

• Port number(s) can be applied to source and destination traffic • Port helps to determine which application is receiving data • Application that sends packet tend to be assigned random port

numbers • alert http $HOME_NET any -> $EXTERNAL_NET 80

• Variables can be set: • WEB_PORTs = [80, 443, 8080]

Rules – direction

drop tcp $HOME_NET any -> $EXTERNAL_NET any (msg:”ET TROJAN Likely Bot Nick in IRC (USA +..)”; flow:established,to_server; flowbits:isset,is_proto_irc; content:”NICK “; pcre:”/NICK .*USA.*[0-9]{3,}/i”; reference:url,doc.emergingthreats.net/2008124; classtype:trojan-activity; sid:2008124; rev:2;)

• The direction tells in which way the signature has to match• Possible values:

-> : source -> destination<> : both directions

Rules – options

drop tcp $HOME_NET any -> $EXTERNAL_NET any (msg:”ET TROJAN Likely Bot Nick in IRC (USA +..)”; flow:established,to_server; flowbits:isset,is_proto_irc; content:”NICK “; pcre:”/NICK .*USA.*[0-9]{3,}/i”; reference:url,doc.emergingthreats.net/2008124; classtype:trojan-activity; sid:2008124; rev:2;)

• Enclosed by parantheses and separated by • Format:

• keyword: settings; • keyword;

Rules – Keywords

• Meta keywords o They affect how Suricata report events o Many keywords!

• Examples: o msg: “some description related to alert”; o sid:1; <- signature ido rev:2; <- revision information o gid:1; <- group id for a set of rules o classtype: “information about classification of rules” (classification.config)o reference: type, ref

o reference:url, www.info.com or reference: cve, CVE-2014-1234o priority:1; highest priority will be examined first (1)

Rules – keywords (2)• The are also keywords for different categories i.e.

o IPo TCPo UDPo ICMPo Payloado HTTP o DNSo TLS o And many more!

alert ip $EXTERNAL_NET any -> $HOME_NET any (msg:”GPL MISC 0 ttl”; ttl:0; reference:url,support.microsoft.com/default.aspx?scid=kb#-#-EN-US#-#-q138268; reference:url,www.isi.edu/in-notes/rfc1122.txt; classtype:misc-activity; sid:2101321; rev:9;)o More information here:

o https://suricata.readthedocs.io/en/latest/rules/header-keywords.html

Rules – payload keywords

• Contento Content:”content/payload here”; o Take note of special characters, need to be represented in heximal notation i.e. ”

|22|o alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"Outdated Firefox on

Windows"; content:"User-Agent|3A| Mozilla/5.0 |28|Windows|3B| "; content:"Firefox/3."; distance:0; content:!"Firefox/3.6.13"; distance:-10; sid:9000000; rev:1;)

• Others: • nocase; • depth:• offset:• More here: https://suricata.readthedocs.io/en/latest/rules/payload-keywords.html

Thinking about rules

• Detect outbound SSH connections• Compare the two alerts

alert tcp $HOME_NET any -> $EXTERNAL_NET 22 (msg: ”Outbound SSH Detected"; sid:10; rev:1;)

alert ssh $HOME_NET any -> $EXTERNAL_NET any (msg: “Outbound SSH Detected”; sid:11; rev:1.1;)

Alerts Output

• Line based alerts logo fast.log10/05/10-10:08:59.667372 [**] [1:2009187:4] ET WEB_CLIENT ACTIVEX iDefense COMRaiderActiveX Control Arbitrary File Deletion [**] [Classification: Web Application Attack] [Priority: 3] {TCP} xx.xx.232.144:80 -> 192.168.1.4:56068

• Extensible Event Format (EVE) JSON Outputo Alerts, metadata, file info and protocol specific records through JSONo Enabled by default, very verbose

• Binary format, compatible with unified2o Can be processed by tool such as Barnyard2

• Other line based alerts – http.log, dns.log, • Pcap - full packet capture format

{"timestamp": "2017-11-

09T15:35:16.120665+0000","flow_id": 1736687033191953,"pcap_cnt": 55,"event_type": "alert","src_ip": "192.185.57.176","src_port": 80,"dest_ip": "10.11.9.101","dest_port": 49167,"proto": "TCP","metadata": {

"flowbits": ["min.gethttp","ET.http.binary"

]},

{"tx_id": 0,"alert": {"action": "allowed","gid": 1,"signature_id": 2018959,"rev": 4,"signature": "ET POLICY PE EXE or DLL

Windows file download HTTP","category": "Potential Corporate

Privacy Violation","severity": 1,"metadata": {"updated_at": ["2017_02_01"

],"created_at": ["2014_08_19"

],"former_category": ["POLICY"

]}}

eve.json

Deployment Considerations

• Distributed sensor for visibilityo Where should you tap?

• Playbooks? o Policyo Threat Model o Rulesets

• Collect and store logs for analysis & visualization (SIEM)o Space for storage o Sending logs securely to centralised locationo Suporting datasets

• Incident Response Plan o What to do when malicious activities / attacks are detected?

• Privacy o Handling user privacy / trust

o Managing alertso Check out threshold, eent_filter and suppress in threshold.conf and can also be defined within rules

• Tuning

Viewing alerts • A couple of GUI options

o Snorbyo Eveboxo SguiloMoloch (with full packet capture)

• SIEM integration • Elasticstack and Splunk• Prepackaged in distributions like

SecurityOnion or SELKS

Snorby

Links: 1. Snorby: https://github.com/Snorby/snorby2. Evebox: https://evebox.org/ 3. https://bammv.github.io/sguil/index.html4. SecurityOnion: https://securityonion.net/

5. SELKS: https://www.stamus-networks.com/open-source/

Evebox

Where Should We Place our sensors?

Source: The Practice of Network Security Monitoring, Richard Bejtlich (2013)

• Consider different sets of IP block for the network

• Considerations • IP Blocks • Network Address Translations

• B,G,HoAbility to see true source and

destination IP address

Other features

• Extract file within traffic to disk • http, smtp, imap

• File extraction o - file-store:o enabled: no # set to yes to enableo log-dir: files # directory to store the fileso force-magic: no # force logging magic on all stored files

JA3 fingerprinting

• Monitoring / detection based on TLS fingerprinting ohttps://github.com/salesforce/ja3o app-layer.protocols.tls.ja3-fingerprints’ to ‘yes’

• Parses multiple fields set in TLS client hello packet sent over during SSL handshake

• Exampleo SSLVersion,Cipher,SSLExtension,EllipticCurve,EllipticCurvePointFormato769,49172-49171-53-47-49162-49161-56-50-10-19-5-4,0-5-10-11-65281,23-

24-25,0o1eede9d19dc45c2cb66d2f5c6849e843

ja3 -j Poseidon.pcap*[ { "destination_ip": "185.67.0.108", "destination_port": 443, "ja3": "769,49172-49171-53-47-49162-49161-56-50-10-19-5-4,0-5-10-11-65281,23-24-25,0", "ja3_digest": "1eede9d19dc45c2cb66d2f5c6849e843", "source_ip": "192.168.56.101", "source_port": 49161, "timestamp": 1527008276.377147 }

https://www.trustwave.com/en-us/resources/blogs/spiderlabs-blog/inspecting-encrypted-network-traffic-with-ja3/

* Poseidon point-of-sales malware

ja3 - rules

alert tls $HOME_NET any -> $EXTERNAL_NET any (msg:"match JA3 hash"; \flow:established,to_server; \ ja3_hash; content:"1eede9d19dc45c2cb66d2f5c6849e843 "; \ sid:100001; rev:1; )

Labs

1. Setup and run Suricata on the honeypot 2. Analyze Suricata Logs with jq

a. JSON log format

3. Cowrie log with jq

sensoX.honeynet.asia

cowrie

Suricata

Discussion & Recap

• Firewall• ID• Security Monitoring