DETECTION ENGINEERING: Passive TLS Fingerprinting · DETECTION ENGINEERING: Passive TLS...

DETECTION ENGINEERING:

Passive TLS Fingerprinting

Experience from adopting JA3

Kjell Tore Fossbakk

HelseCERT

TLS Fingerprinting

a technique to identify a clientapplication (or library) based onparameters in the TLS traffic,without decrypting

TLS Fingerprinting

detect malicious activity by how it communicates, rather than what it communicates to: IP, domain, URI, SNI

Source: David Bianco

Hash values

IP Address

Domain Names

Artifacts

Tools

TTPs

Network: uri, c2 info, HTTP User-Agent

crafted software, distinctive keep-alive, behavior

file md5/sha256

PYRAMID OF PAIN: IoCs

TA modus operandi, TA training, distinctive TA traits

Host: files, registry, mutex, memory

SNI


PYRAMID OF PAIN: network security monitoring

Hash values

IP Address

Domain Names

Artifacts

Tools

TTPs

Network Security

Monitoring


Hash values

IP Address

Domain Names

Artifacts

Tools

TTPs

TLS ENCRYPTED

TRAFFIC

TLS Fingerprinting??

PYRAMID OF PAIN: challenge

DNS over HTTPS/TLS, esni

HTTPS (HTTP over TLS)

VPN (tls), Tor, open proxies, CDN

Cisco 2018 Annual Cybersecurity Report (acr)

INCREASED VOLUME OF ENCRYPTED TRAFFIC

letsencrypt.org/stats/Source: https://transparencyreport.google.com/https

30 %

70 %

HTTP VS HTTPS

HTTP HTTPS

Source: HelseCERT sensordata may 2019

Unencryptedtraffic

Encryptedtraffic

ENCRYPTED VS UNENCRYPTED WEB TRAFFIC


85 %

5 %

4 %4 %2 %0 %0 %0 %

TLS VERSION

TLS1.2 TLS1.3 TLSv1 Failure TLS1.3draft TLS1.1 SSLv2 SSLv3

TLS VERSIONS

2016: 10-12%

2017: >= 70%

Source: Cisco 2018 Annual Cybersecurity Report (acr2018)

MALWARE USING ENCRYPTED TRAFFIC

Akamai ThreatResearch Team

Majority (~82%) of malicioustraffic (…) is carried out usingsecure connections over SSL/TLS

Source: blogs.akamai.com/sitr/2019/05/bots-tampering-with-tls-to-avoid-detection.html

Client Server

ClientHello

ServerHello

Certificate

ServerHelloDone

ClientKeyExchange

ChangeCipherSpec

Finished

ChangeCipherSpec

Finished

ApplicationData ApplicationData

3-WH: SYN, SYN-ACK, ACK

Ciphertext, encrypted

Plaintext, unencrypted

TLS 1.2 HANDSHAKE

2009: mod_sslhaf

Source: https://blog.ivanristic.com/2009/07/analysis-of-googlebots-frugal-cipher-suite-list.html

2012: p0f

Source: https://idea.popcount.org/2012-06-17-ssl-fingerprinting-for-p0f

2015: FingerprinTLS

{

"id": 0,

"desc": "Adium 1.5.10 (a)",

"record_tls_version": "0x0301",

"tls_version": "0x0303",

"ciphersuite_length": "0x0048",

"ciphersuite": "0x00FF 0xC024 0xC023 0xC00A 0xC009 0xC008

0xC028 0xC027 0xC014 0xC013 0xC012 0xC026 0xC025 0xC005 0xC004

0xC003 0xC02A 0xC029 0xC00F 0xC00E 0xC00D 0x006B 0x0067 0x0039

0x0033 0x0016 0x003D 0x003C 0x0035 0x002F 0x000A 0xC007 0xC011

0xC002 0xC00C 0x0005",

"compression_length": "1",

"compression": "0x00",

"extensions": "0x0000 0x000A 0x000B 0x000D",

"e_curves": "0x0017 0x0018 0x0019",

"sig_alg": "0x0501 0x0401 0x0201 0x0403 0x0203",

"ec_point_fmt": "0x00"

}

Source: https://blog.squarelemon.com/tls-fingerprinting

2017: JA3

Source: https://engineering.salesforce.com/open-sourcing-ja3-92c9e53c3c41

2018: JA3S

Source: https://engineering.salesforce.com/tls-fingerprinting-with-ja3-and-ja3s-247362855967

2019: (Cisco) Joy

Source: https://blogs.cisco.com/security/tls-fingerprinting-in-the-real-world

• Requirements:• Work with existing tools

• Unique client identification

• Easy to create, share and use

• Salesforce contribution (new stuff!)• Hash a concatenated string

• Hash is the signature

WHY JA3?

{

"id": 0,

"desc": "Adium 1.5.10 (a)",

"record_tls_version": "0x0301",

"tls_version": "0x0303",

"ciphersuite_length": "0x0048",

"ciphersuite": "0x00FF 0xC024 0xC023 0xC00A 0xC009

0xC008 0xC028 0xC027 0xC014 0xC013 0xC012 0xC026

0xC025 0xC005 0xC004 0xC003 0xC02A 0xC029 0xC00F

0xC00E 0xC00D 0x006B 0x0067 0x0039 0x0033 0x0016

0x003D 0x003C 0x0035 0x002F 0x000A 0xC007 0xC011

0xC002 0xC00C 0x0005",

"compression_length": "1",

"compression": "0x00",

"extensions": "0x0000 0x000A 0x000B 0x000D",

"e_curves": "0x0017 0x0018 0x0019",

"sig_alg": "0x0501 0x0401 0x0201 0x0403 0x0203",

"ec_point_fmt": "0x00"

}

Client Server

ClientHello

• TLSVersion

• List of Ciphers

• List of Extensions

• EllipticCurves

• EllipticCurvesPointFormat

Ciphertext, encrypted

Plaintext, unencrypted

JA3


TLSVersion,Ciphers,Extensions,EllipticCurves,EllipticCurvePointFormats

JA3

JA3 string: 771,47,5-10-11-65281,23-24-25,0

JA3 hash(md5): b16ab4d4897ed1192362cb0dbda28f86

47

771

0

23-24-25

5-10-11-65281

771,47,5-10-11-65281,23-24-25,0

Client Server

ServerHello

• TLSVersion

• Accepted Ciphers

• List of Extensions

TLSVersion,Ciphers,Extensions

JA3S string: 771,47,65281

JA3S hash(md5): 573a9f3f80037fb40d481e2054def5bb

ClientHello

JA3S


771,47,65281

Client Server

ServerHello

Tickbot malware:

JA3 = 6734f37431670b3ab4292b8f60f29984

JA3S = 623de93db17d313345d7ea481e7443cf

Emotet malware:

JA3 = 4d7a28d6f2263ed61de88ca66eb011e3

JA3S = 80b3a14bccc8598a1f3bbe83e71f735f

ClientHello

JA3 + JA3S


0

5

10

15

20

25

Supported

JA3 Bro + Python

md5 MISP

JA3 Moloch,

nginx, trisul

JA3S Moloch

JA3 MISP

JA3 Suricata

JA3 @abuse_ch

JA3 @joe4security

JA3S release

JA3S Bro + Python

JA3 release

JA3 SUPPORT IN TOOLS

Sandboxes:

https://twitter.com/abuse_ch

Bugs!• https://github.com/aol/moloch

capture - fixed ja3s mishandling of 10/11 extension types (thanks Norwegian Healthcare CERT)capture - fixed ja3 mishandling of 11 extension types (thanks Norwegian Healthcare CERT)

• https://github.com/dreadl0ck/ja3/issues/3• https://github.com/D4-project/sensor-d4-tls-

fingerprinting/issues/13

JA3 string > JA3 hash

ReproducableTwin (with/without SNI)Verifiable

COLLISIONS

Client App

Client lib 623de93db17d313345d7ea481e7443cf

623de93db17d313345d7ea481e7443cf

OS API

Client App

Client lib

OS API

???

COLLISIONS

Firefox

OpenSSL 623de93db17d313345d7ea481e7443cf

623de93db17d313345d7ea481e7443cf

Win10 socket

Firefox

OpenSSL

Win10 socket

???

Multiple fingerprints

One client app generate severalfingerprints, depends on the TLS implementations

Fingerprinting database

Fingerprint database to match fingerprints = client apps

Laboratory <tool> JA3 hash

Burp 2.0.20Beta TP!

Detected live pentest{

"ja3": "d9e47f0ebed131ce3c9c998d65abc0fc",

"ja3_string": "771,4865-4866-49196-49195-49200-157-49198-49202-159-163-49199-156-

49197-49201-158-162-49188-49192-61-49190-49194-107-106-49162-49172-53-49157-49167-57-

56-49187-49191-60-49189-49193-103-64-49161-49171-47-49156-49166-51-50-49160-49170-10-

49155-49165-22-19-255,0-5-10-11-13-50-17-23-43-45-51,23-24-25-9-10-11-12-13-14-22-256-

257-258-259-260,0"

}

Fingerprinting database

Manual labour doesn’t scale

• Solution: Continuously and automatically fuse network and endpoint data.

CISCO (JOY): NETWORK & ENDPOINT

?VM

?

Network Data

Endpoint Data

Long-Term

Storage

Source: Cisco: The Generation and Use of TLS Fingerprints, Blake Anderson.

CISCO (JOY): NETWORK & ENDPOINT

Source: Cisco: The Generation and Use of TLS Fingerprints, Blake Anderson.

{

"str_repr":

"(0303)(0a0a130113021303c02bc02fc02cc030cca9cca8c013c01400

9c009d002f0035000a)((0a0a)(0000)(0017)(ff01)(000a000a00080

a0a001d00170018)(000b00020100)(0023)(0010000e000c026832086

87474702f312e31)(000500050100000000)(000d00140012040308040

401050308050501080606010201)(0012)(0033)(002d00020101)(002

b000b0a0a0a0304030303020301)(001b0003020002)(0a0a000100)(0

015))",

"md5_repr": "d417ee3f0512f88b29dd9c28b52c02e4",

"source": [

"Cisco"

],

"max_implementation_date": "2018-10",

"min_implementation_date": "1999-01",

"tls_features": {

"version": "TLS 1.2",

"cipher_suites": [

"GREASE",

"TLS_AES_128_GCM_SHA256",

"TLS_AES_256_GCM_SHA384",

"TLS_CHACHA20_POLY1305_SHA256",

"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",

"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",

"TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",

"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",

"TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",

"TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",

"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA",

"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA",

"TLS_RSA_WITH_AES_128_GCM_SHA256",

"TLS_RSA_WITH_AES_256_GCM_SHA384",

"TLS_RSA_WITH_AES_128_CBC_SHA",

"TLS_RSA_WITH_AES_256_CBC_SHA",

"TLS_RSA_WITH_3DES_EDE_CBC_SHA"

],

"process_info": [

{

"process": "chrome.exe",

"application_category": "browser",

"prevalence": 0.24,

"sha256":

"1FA5A6C8438A4E6D373D39C96B77C0C84540D38B80628EFFDEC89E77D

02D7E57",

"os_info": {

"(WinNT)(Windows 10 Enterprise)(10.0.17134)":

0.71,

"(WinNT)(Windows 7 Enterprise)(6.1.7601)": 0.2,

"(WinNT)(Windows 10 Enterprise)(10.0.15063)": 0.06

}

},

{

"process": "chrome.exe",


"prevalence": 0.15,

"sha256":

"072228D83FEB4E4A9D0C16191E09B57D0CA66C483B1B81BB68AC4C274

46EF172",

"os_info": {



"(WinNT)(Windows 10 Enterprise)(10.0.15063)": 0.03

}

},

{

"process": "Google Chrome",


"prevalence": 0.08,

"sha256":

"2074E9B822A4AF37A4677D8B1DD0534EA96CE9B042F8FCD7FA83045C2

B8AE635",

"os_info": {

"(Mac OS X)(Unknown)(10.14.2)": 0.37,

"(Mac OS X)(High Sierra)(10.13.6)": 0.22,

"(Mac OS X)(Unknown)(10.14.3)": 0.13

"extensions": [

…

{

"signature_algorithms": {

"signature_hash_algorithms_length":

18,

"algorithms": [

"ecdsa_sha256",

"rsa_pss_sha256",

"rsa_sha256",

"ecdsa_sha384",

"rsa_pss_sha384",

"rsa_sha384",

"rsa_pss_sha512",

"rsa_sha512",

"rsa_sha1"

]

}

},

{

"supported_versions": {

"supported_versions_list_length":

10,

"supported_versions": [

"GREASE",

"TLS 1.3",

"TLS 1.2",

"TLS 1.1",

"TLS 1.0"

]

}

},

SALESFORCE: SYSMON + BRO

Source: Salesorce Engineering, Jeff Atkinson 2018/2019

• Win Sysmon ID 3 netcon with the SSL/TLS analyzer of Bro to generate logging which contains the process ID, path to executable, and JA3 fingerprint.

• Signature:• Blacklist: find known bad

• Whitelist: allow known good

• Totally unique fingerprint for one spesific app (Burp?)

• Pivot point• source IPs with high percentage of unknown JA3 hashes

• source IPs deviate from its neighbours

• JA3 database• manual analysis of «first seen» prints

STRATEGIES FOR USING JA3

Challenges

JULY 2018: AVOID TLS FINGERPRINTING

Cipher Stunting

Technique to evade TLS fingerprinting by randomizingcipher suite list in ClientHello

Source: blogs.akamai.com/sitr/2019/05/bots-tampering-with-tls-to-avoid-detection.html

Mimic

Send a crafted ClientHello to deliberately produce a spesificJA3 hash, and be detected as Burp 2.0.20Beta

# Forged TLS Client Hello

ciphers =

list([4865,4866,49196,49195,49200,157,49198,49202,159,163,49199,156,49197,49201,158

,162,49188,49192,61,49190,49194,107,106,49162,49172,53,49157,49167,57,56,49187,4919

1,60,49189,49193,103,64,49161,49171,47,49156,49166,51,50

,49160,49170,10,49155,49165,22,19,255])

named_groups = TLSExtension() /

TLSExtSupportedGroups(named_group_list=[23,24,25,9,10,11,12,13,14,22,256,257,258,25

9,260])

p = TLSRecord(version='TLS_1_2') / TLSHandshakes(handshakes=[TLSHandshake() /

TLSClientHello(

cipher_suites=ciphers,

extensions=[

TLSExtension() /

TLSExtServerNameIndication(server_names=[TLSServerName(data="meh")]),

TLSExtension() / TLSExtStatusRequest(),

named_groups,

TLSExtension() / TLSExtECPointsFormat(),

TLSExtension() / TLSExtSignatureAlgorithms(),

TLSExtension() / TLSExtSignatureAlgorithmsCert(),

TLSExtension() / TLSExtStatusRequestV2(),

TLSExtension() / TLSExtExtendedMasterSecret(),

TLSExtension() / TLSExtSupportedVersions(),

TLSExtension() / TLSExtPSKKeyExchangeModes(),

TLSExtension() / TLSExt51KeyShare(),

],)])

Burp 2.0.20Beta:

{

"ja3": "d9e47f0ebed131ce3c9c998d65abc0fc",

"ja3_string": "771,4865-4866-49196-49195-49200-157-

49198-49202-159-163-49199-156-49197-49201-158-162-

49188-49192-61-49190-49194-107-106-49162-49172-53-

49157-49167-57-56-49187-49191-60-49189-49193-103-64-

49161-49171-47-49156-49166-51-50-49160-49170-10-

49155-49165-22-19-255,0-5-10-11-13-50-17-23-43-45-

51,23-24-25-9-10-11-12-13-14-22-256-257-258-259-

260,0"

}

d9e47f0ebed131ce3c9c998d65abc0fc d9e47f0ebed131ce3c9c998d65abc0fc

PYTHON JA3 FORGERY

• Purpose built TLS library for mimicry to protect tools from TLS fingerprinting

• Mimic popular fingerprints

• Generate randomized fingerprints to defeat blacklists

UTLS: ANTI-CENSORSHIP

Source: https://tlsfingerprint.io/static/frolov2019.pdfhttps://github.com/refraction-networking/utls

TLS 1.3 + JA3/JA3S

Deprecated TLS Version field5 ciphersDeprecated ECPF

Encrypted CertificateEncryptedExtensionsEncrypted SNI

SNI

Different fingerprint with or withoutext: server_name (0x0000)

• Explore JA3 / JA3S combination

• Automate (JA3) fingerprint database• Convert Cisco Joy DB JA3?

• Explore other fingerprint techs• Cisco Joy (ClientHello)

• Cisco Joy for ServerHello?

• Explore other parts of TLS handshake / session

Endpoints!

FUTURE WORK

$ dig +short www.nhn.no

{

"timestamp": "2019-05-27T10:51:50.191336+0200",

"flow_id": 1378618553329428,

"in_iface": "em1",

"event_type": "dns",

"src_ip": "....",

"src_port": 53,

"dest_ip": "....",

"dest_port": 51628,

"proto": "017",

"dns": {

"type": "answer",

"id": 9,

"flags": "8180",

"qr": true,

"rd": true,

"ra": true,

"rcode": "NOERROR",

"rrname": "www.nhn.no",

"rrtype": "A",

"ttl": 180,

"rdata": "52.174.150.24"

}

}

PLAINTEXT PDNS

$ curl https://dns.google.com/resolve?name=www.nhn.no

DNS over HTTPS (DoH) + TLS 1.2

{

"timestamp": "2019-05-27T10:48:11.198920+0200",

"flow_id": 2242512555804940,

"in_iface": "em1",

"event_type": "tls",

"src_ip": "....",

"src_port": 61483,

"dest_ip": "....",

"dest_port": 443,

"proto": "006",

"tls": {

"subject": "C=US, ST=California, L=Mountain View, O=Google LLC, CN=*.google.com",

"issuerdn": "C=US, O=Google Trust Services, CN=Google Internet Authority G3",

"serial": "07:17:11:25:2B:C1:BA:E5:D4:3E:22:3E:89:51:1E:A9",

"sni": "dns.google.com",


"notbefore": "2019-05-07T11:29:56",

"notafter": "2019-07-30T10:54:00"

}

}

$ curl https://dns.google.com/resolve?name=www.nhn.no

DNS over HTTPS (DoH) + TLS 1.3

{

"timestamp": "2019-05-27T10:48:11.198920+0200",

"flow_id": 2242512555804940,

"in_iface": "em1",

"event_type": "tls",

"src_ip": "....",

"src_port": 61483,

"dest_ip": "....",

"dest_port": 443,

"proto": "006",

"tls": {


}

}

* DNS over HTTPS (DoH)* DNS over TLS (DoT)

100 %

0 %

DNS VS DOT/HOH

DNS DoT/DoH


Unencryptedtraffic

ENCRYPTED VS UNENCRYPTED DNS TRAFFIC

DETECTION ENGINEERING: Passive TLS Fingerprinting · DETECTION ENGINEERING: Passive TLS...

Documents

Transcript of DETECTION ENGINEERING: Passive TLS Fingerprinting · DETECTION ENGINEERING: Passive TLS...