Download - Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Transcript
Page 1: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Automated cryptanalysis using statistical testing

Karel Kubıcek

Centre for Research on Cryptography and Security (CRoCS), Masaryk University, Brno, Czech Republic

June 21, 2018

Page 2: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Outline

1 Statistical testing in cryptanalysis

2 CryptoStreams and Randomness Testing Toolkit

Current research

3 Statistical testing tool EACirc

Master thesis

4 Statistical testing tool BoolTest

Current research of our group

2 / 33

Page 3: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Motivation for cryptanalysis

AES competition

announced in 1997Rijndael selected in 2000FIPS approved in 2001 (November)2018 – no sign of a successor

DES – used more than 25 years

3 / 33

Page 4: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Motivation for cryptanalysis

AES competition

announced in 1997Rijndael selected in 2000FIPS approved in 2001 (November)2018 – no sign of a successor

DES – used more than 25 years

3 / 33

Page 5: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Motivation for cryptanalysis

AES competition

announced in 1997Rijndael selected in 2000FIPS approved in 2001 (November)2018 – no sign of a successor

DES – used more than 25 years

3 / 33

Page 6: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Classical cryptanalysis

Brute-force

Differential cryptanalysis

Linear cryptanalysis

Algebraic cryptanalysis

Meet-in-the-middle, key-scheduling, sliding attack, cube attack...

Manual and skill-demanding

Goals of statistical testing as cryptanalysis:

Black-box methodEasy to use for cipher designersQuick security margin estimateTest wide set of properties

4 / 33

Page 7: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Classical cryptanalysis

Brute-force

Differential cryptanalysis

Linear cryptanalysis

Algebraic cryptanalysis

Meet-in-the-middle, key-scheduling, sliding attack, cube attack...

Manual and skill-demanding

Goals of statistical testing as cryptanalysis:

Black-box methodEasy to use for cipher designersQuick security margin estimateTest wide set of properties

4 / 33

Page 8: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Classical cryptanalysis

Brute-force

Differential cryptanalysis

Linear cryptanalysis

Algebraic cryptanalysis

Meet-in-the-middle, key-scheduling, sliding attack, cube attack...

Manual and skill-demanding

Goals of statistical testing as cryptanalysis:

Black-box methodEasy to use for cipher designersQuick security margin estimateTest wide set of properties

4 / 33

Page 9: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Statistical testing

Example: Monobit test

Set of statistical tests – batteries:

Donald Knuth’s tests in TAoCP 2 (1969)NIST STS (FIPS 140-2) (1998)Dieharder (2004)TestU01 (2007)

NIST STS used on AES competition (Soto, Juan. Randomness testing of the AEScandidate algorithms. NIST (1999))

5 / 33

Page 10: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Statistical testing

Example: Monobit test

Set of statistical tests – batteries:

Donald Knuth’s tests in TAoCP 2 (1969)NIST STS (FIPS 140-2) (1998)Dieharder (2004)TestU01 (2007)

NIST STS used on AES competition (Soto, Juan. Randomness testing of the AEScandidate algorithms. NIST (1999))

5 / 33

Page 11: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Statistical testing

Example: Monobit test

Set of statistical tests – batteries:

Donald Knuth’s tests in TAoCP 2 (1969)NIST STS (FIPS 140-2) (1998)Dieharder (2004)TestU01 (2007)

NIST STS used on AES competition (Soto, Juan. Randomness testing of the AEScandidate algorithms. NIST (1999))

5 / 33

Page 12: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Statistical testing

Example: Monobit test

Set of statistical tests – batteries:

Donald Knuth’s tests in TAoCP 2 (1969)NIST STS (FIPS 140-2) (1998)Dieharder (2004)TestU01 (2007)

NIST STS used on AES competition (Soto, Juan. Randomness testing of the AEScandidate algorithms. NIST (1999))

5 / 33

Page 13: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Statistical testing as cryptanalysis

Analysis of cryptoprimitive’s output, but for what input?

PRNG, stream ciphers → stream, keystream

Block ciphers, hash functions → ?

Test confusion → use low entropy inputs

Test diffusion → use strict avalanche criterion, linear cryptanalysis scenario

6 / 33

Page 14: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Statistical testing as cryptanalysis

Analysis of cryptoprimitive’s output, but for what input?

PRNG, stream ciphers → stream, keystreamBlock ciphers, hash functions → ?

Test confusion → use low entropy inputs

Test diffusion → use strict avalanche criterion, linear cryptanalysis scenario

6 / 33

Page 15: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Statistical testing as cryptanalysis

Analysis of cryptoprimitive’s output, but for what input?

PRNG, stream ciphers → stream, keystreamBlock ciphers, hash functions → ?

Test confusion → use low entropy inputs

Test diffusion → use strict avalanche criterion, linear cryptanalysis scenario

6 / 33

Page 16: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

CryptoStreams

Generalized crypto-data generator

Page 17: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

CryptoStreams

https://github.com/crocs-muni/CryptoStreams

> 100 cryptoprimitives

22 block ciphers (AES competition, TLS suite)57 hash functions (SHA-3 competition)32 stream ciphers (eSTREAM competition)53 schemes of authenticated encryption (CAESAR)6+ PRNGs (Master thesis in progress)

Round-reduced

10+ input strategies

Output postprocessing

8 / 33

Page 18: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

RTT – Randomness Testing Toolkit

https://github.com/crocs-muni/randomness-testing-toolkit

http://rtt.ics.muni.cz

RTT is a unification of statistical batteries

9 / 33

Page 19: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

10 / 33

Page 20: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

11 / 33

Page 21: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

12 / 33

Page 22: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

13 / 33

Page 23: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

14 / 33

Page 24: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

15 / 33

Page 25: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

16 / 33

Page 26: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Results

17 / 33

Page 27: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Results

17 / 33

Page 28: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Results

18 / 33

Page 29: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

EACirc

Analyzing randomness using supervised learning

https://github.com/crocs-muni/eacirc

Page 30: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Motivation

Statistical batteries = fixed set of tests

We can construct data passing all batteries

Create tests while analyzing the data

Incremental improving using supervised learning

Individual (=statistical test) representationNeighbourhoodFitness

20 / 33

Page 31: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Motivation

Statistical batteries = fixed set of tests

We can construct data passing all batteries

Create tests while analyzing the data

Incremental improving using supervised learning

Individual (=statistical test) representationNeighbourhoodFitness

20 / 33

Page 32: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

EACirc test representation

IN0

IN1

IN2

IN3

IN4

IN5

IN6

IN7

MASK34

SHIR11

NOT106

CONS122

AND19

SHIL181

AND152

NOR52

AND10

OR188

SHIL22

OR246

SHIR82

MASK107

NOP176

NOP73

ROTR98

NOP79

XOR30

NOP228

NOR160

SHIR221

NOP32

XOR187

SHIL190

ROTR146

ROTL109

NOT247

CONS229

ROTL24

OR210

SHIL191

NAND244

21 / 33

Page 33: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Problem optimization

Individual (=statistical test) representation

Simulated electronic circuit

Neighborhood

Changes of connectors and node functions

Fitness

Evaluate on 500 QRND reference vectors and 500 analyzed vectorsFitness = # correct guesses / 1000

Optimization methods:

Evolutionary algorithmsSingle-solution heuristics – iterated local search, neighborhood search, guided localsearch, simulated annealing. . .

22 / 33

Page 34: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Problem optimization

Individual (=statistical test) representation

Simulated electronic circuit

Neighborhood

Changes of connectors and node functions

Fitness

Evaluate on 500 QRND reference vectors and 500 analyzed vectorsFitness = # correct guesses / 1000

Optimization methods:

Evolutionary algorithmsSingle-solution heuristics – iterated local search, neighborhood search, guided localsearch, simulated annealing. . .

22 / 33

Page 35: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Approach advantages and limitations

+ Automatic

+ Better interpretation

+ Adapts to learning data

+ Simple distinguisher after learning

− Only byte level bias

− Only local bias

− Huge solution space

+ Surpasses NIST STS

− Dieharder and TestU01 are still superior

23 / 33

Page 36: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Approach advantages and limitations

+ Automatic

+ Better interpretation

+ Adapts to learning data

+ Simple distinguisher after learning

− Only byte level bias

− Only local bias

− Huge solution space

+ Surpasses NIST STS

− Dieharder and TestU01 are still superior

23 / 33

Page 37: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

BoolTest

Testing randomness with polynomials

https://github.com/crocs-muni/booltest

Page 38: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Motivation

EACirc cannot detect bit-level bias

EACirc needs reference data

Bias is often correlation of some bits

Polynomials in algebraic normal form: f : {0, 1}b → {0, 1} (b for block length).E.g., f (x0, x1, . . . , xb) = x2 · x34 + x7 · x15

25 / 33

Page 39: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Motivation

EACirc cannot detect bit-level bias

EACirc needs reference data

Bias is often correlation of some bits

Polynomials in algebraic normal form: f : {0, 1}b → {0, 1} (b for block length).E.g., f (x0, x1, . . . , xb) = x2 · x34 + x7 · x15

25 / 33

Page 40: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Approach advantages and limitations

+ Superior test for < 100 MB to all batteries

+ Found practical distinguishers in glibc rand() and Java Random (LCG variants)

+ Direct interpretation – what bits are correlated

+ Single run in order of seconds

Comparable with batteries for 100 MB, weaker than TestU01 for GBs

− Limited polynomial complexity by the estimate phase

− Will find correlations only in close bits

≤1024-bits blocks for practical reasons

26 / 33

Page 41: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

BoolTest results

27 / 33

Page 42: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

BoolTest heatmap visualisation – AES 10 rounds – ”random” reference

Source: Mecko, Vladimır. Interpretation and speedup of a randomness testing via the booleanfunctions, Master thesis, Faculty of Informatics, Masaryk University (2018) 28 / 33

Page 43: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

BoolTest heatmap visualisation – AES 3 rounds – structure

29 / 33

Page 44: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

BoolTest heatmap visualisation – RC4 – extreme distinguisher

f = x7x135

30 / 33

Page 45: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Overall conclusion

Statistical testing in a comprehensive study

Security margins for 40 cryptoprimitives using four input strategies

Novel practical results on Rabbit stream cipher

Advancements in test interpretation

EACirc and BoolTestRTT’s visualisation and test interpretation

31 / 33

Page 46: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Project (and involvement) overview

CryptoStreams – generator of cryptographic material

Main author – initial idea, leading developer

Randomness Testing Toolkit

Tester and user

EACirc – statistical test using evolutionary circuits

Project started in 2008Kubıcek, Novotny, Svenda, and Martin Ukrop. New results on reduced-round TinyEncryption Algorithm using genetic programming, IEEE Infocommunications (2016)Master thesis on optimisation methods

BoolTest – generator of cryptographic materialSys, Klinec, Kubıcek, and Svenda. BoolTest: The fast randomness testing strategy basedon boolean functions with application to DES, 3-DES, MD5, MD6 and SHA-256,forthcoming in Communications in computer and information science, Springer, 2018Experiment design and execution

32 / 33

Page 47: Automated cryptanalysis using statistical testing...Automated cryptanalysis using statistical testing Karel Kub cek Centre for Research on Cryptography and Security (CRoCS), Masaryk

Questions?