I am a legend: Hacking Hearthstone with machine learning - Defcon 22 talk slides

I am a legendCeline & Elie Bursztein

https://www.elie.net/hs

Celine & Elie Bursztein I am a legend - Defcon 2014 https://www.elie.net/hs

Digital Collectible Card Game

Released by Blizzard in 2014

Based of World of Warcraft universe

Sometimes too interesting leads to un-intended consequences

Painting by http://www.jason-w.com/blog/the-fall-of-lordaeron

Game complexity generates exploitable biases

Outline

1. Finding undervalued cards

Outline

2. Predicting opponent deck

Outline

3. Predicting the game outcome

Outline

3. Predicting the game outcome

4. Incoming alien invasion (or not)

You hero

Opponent hero

Hero health

mana pool

Weapon

Minions

video of a turn

Attack

Attack Health

Card special abilities is what makes the game complex and interesting

Finding undervalued cards

Model assumptions

1. Mana cost is proportional to card power

Model assumptions

2. The power of cards roughly increase linearly

Model assumptions

3. Card effects have constant price

Model assumptions

4. A card have an intrinsic value

Model assumptions

4. A card have an intrinsic value

5. The value of the card is the sum of its attribute

Model assumptions

= +mana attack +health intrinsic value

= +mana attack +health intrinsic value4 4a + i= + 5h

6 = 6a + 7h + i

6 = 6a + 7h + i 4 = 4a + 5h + i

6 = 6a + 7h + i 4 = 4a + 5h + i/6

6 = 6a + 7h + i

1 = 1a + 1.16h + i

4 = 4a + 5h + i/6

6 = 6a + 7h + i

1 = 1a + 1.16h + i

4 = 4a + 5h + i/6 /4

6 = 6a + 7h + i

1 = 1a + 1.16h + i

4 = 4a + 5h + i

1 = 1a + 1.25h + i

4 = 6d

4 = 6d 1 mana = 1.5 dmg

4 = 6d

10 = 10d

1 mana = 1.5 dmg

4 = 6d

10 = 10d 1 mana = 1 dmg

1 mana = 1.5 dmg

4 = 6d

10 = 10d 1 mana = 1 dmg

Pre nerf (8 mana)

1 mana = 1.5 dmg

4 = 6d

10 = 10d 1 mana = 1 dmg

8 = 10dPre nerf (8 mana)

1 mana = 1.5 dmg

4 = 6d

10 = 10d 1 mana = 1 dmg

8 = 10d 1 mana = 1.25 dmgPre nerf (8 mana)

1 mana = 1.5 dmg

10 damages 4 damagesimply

15 damages 6 damages

10 damages 4 damagesimply

Hunting for under-valued cards

How to find undervalued cards?

Model cards

Reverse coefficients

Model cards

Compute cards real value

Model cards

Compute cards real value

Profit :)

Simple 5 cards example

Use simpler coefficientsApproach illustrated

Charge

Charge Divine shield

4 = 4a + 3h + c + i

6 = 5a + 2h + c + i

6 = 4a + 2h + c + d + i

3 = 3a + 1h + d + i

1 = 1a + 1h + d + i

Reversing attribute cost

mana Atk Health Charge Divine Intrinsic

4 4 3 1 0 1

6 5 2 1 0 1

4 4 3 1 0 1

6 5 2 1 0 1

6 4 2 1 1 1

4 4 3 1 0 1

6 5 2 1 0 1

6 4 2 1 1 1

3 3 1 0 1 1

4 4 3 1 0 1

6 5 2 1 0 1

6 4 2 1 1 1

3 3 1 0 1 1

1 1 1 0 1 1

4 4 3 1 0 1

6 5 2 1 0 1

6 4 2 1 1 1

3 3 1 0 1 1

1 1 1 0 1 1

Least square

4 4 3 1 0 1

6 5 2 1 0 1

6 4 2 1 1 1

3 3 1 0 1 1

1 1 1 0 1 1

Health

Charge

Divine

Intrinsic

Least square

4 4 3 1 0 1

6 5 2 1 0 1

6 4 2 1 1 1

3 3 1 0 1 1

1 1 1 0 1 1

Health

Charge

Divine

Intrinsic

Warning these example attribute costs are bogus as we didn’t use enough cards

Least square

Finding card real price using reversed coefficients

Atk = 1 Health = -1Charge = 2 Divine = 1 Intrinsic = 1Coeffs:

4a + 2h + c + d + i

4a + 2h + c + d + i4*1 + 2*-1 + 2 + 1 + 1 = 6

1a + 1h + d + i

4a + 2h + c + d + i4*1 + 2*-1 + 2 + 1 + 1 = 6

1a + 1h + d + i1*1 + 1*-1 + 1 + 1 = 2

4a + 2h + c + d + i4*1 + 2*-1 + 2 + 1 + 1 = 6

1a + 1h + d + i1*1 + 1*-1 + 1 + 1 = 2

Under-valued card!

Redit story Thanks you for the feedback!

https://www.elie.net/hs

Modeling dependance between characteristics

Thanks to Niels for the idea

Charge = Atk * charge coeff

Windfury = Atk * windfury coeff

Divine = ? (health related?)

Charge = Atk * charge coeff

Windfury = Atk * windfury coeff

Divine = ? (health related?)

Model also use a card budget: 2*mana + 1

Et voila!

Reversed budget coefficientsEffect cost per pointDestroy minion 10.63Board damage 3.69Draw card 3.68Divine Shield 2.74Freeze 2.04Silence 1.66Damage 1.64Durability 1.22Stealth 1.21Attack 1.14Taunt 1.02WindFury 0.96SpellPower 0.93Health 0.81Battlecry heal 0.69Battlecry self hero heal 0.68Charge 0.65Intrinsic value 0.32

2 coeff point ~ 1 mana point

Effect Cost per pointOpponent draw card -3.97Discard cards -2.67Overload -1.68Self hero damage -0.54

Cost assigned by Blizzard (in mana)0 1 2 3 4 5 6 7 8 9 10

undervalued cards

overpriced cards

Model found a couple of clearly undervalued cards :)

undervalued cards

overpriced cards

Most under-valued cards (~130 cards)

Full data https://www.elie.net/tools/hearthstone/cards_value

Taking it to the next level

• 100 000 games from May to June

• Thanks to … for it :)

• Need a longer term solution

Game replays

Pricing cards with unique effects

Cards in hand

Health Real Value

Cards in hand

1.31.9

5.35.9

OverpricedFairUndervalued

Drake Health1 2 3 4 5 6 7 8 9

Average real value 3.7

Twilight drake price is fair

1.31.9

5.35.9

OverpricedFairUndervalued

Drake Health1 2 3 4 5 6 7 8 9

Cards played

this turn

Edwin size Real value2/2 1.094/4 3.046/6 4.988/8 6.9210/10 8.8712/12 10.5114/14 12.7516/16 14.7018/18 16.6420/20 18.5822/22 20.53

Cards played

this turn

Over-pricedFairUndervalued

5.06.9

16.6 18.620.5 22.5 24.4 26.4

VanCleef size (atk/hp)2/2 4/4 6/6 8/8 10/10 12/12 14/14 16/16 18/18 20/20 22/22 24/24 26/26 28/28

5.06.9

16.6 18.620.5 22.5 24.4 26.4

VanCleef is undervalued, a fair price is probably between 5 and 7 mana

5.06.9

16.6 18.620.5 22.5 24.4 26.4

Num Minions

Damage Value

8 13.9

12 21.3

16 28.6

20 36.0

24 43.4

28 50.7

Board damage coeff

Num Minions

Normal damage coeff

Damage Real Value

4 2.43835

8 5.71835

12 8.99835

16 12.27835

20 15.55835

24 18.83835

28 22.11835

Flamestrike price is fair Don’t split board damage and single damage coeff

18.822.1

Over-pricedUndervalued

Potential damage4 8 12 16 20 24 28

Predicting cards

Predicting opponent deck

Celine

Our tool :)

My deck with card tracking

Game metrics

Opponent cards played so far

Opponent next cards prediction

Real time dashboard

Game metrics

MeOpponent

My deck

T = TotalP = PlayedD = Dead

My opponent

T = TotalP = PlayedD = Dead

Predictions

Game data from

Sniff packets

Game data from

Sniff packets

OCRGame data from

Sniff packets

Debug log

Game data from

Sniff packets

Debug log

Game data from

Real logs from Blizzard like in WoW ?

Demo video

https://www.youtube.com/watch?v=--byrlBQLCY

Demo video

https://www.youtube.com/watch?v=--byrlBQLCY

Turn by Turn History

Available on Github LightWind/hearthstone-dashboard

Naxx new cards made the meta to unstable to be predicted accurately for now

Anatomy of our prediction system

Model card affinities

Evaluate affinities

Learn from replays

Evaluate affinities

Learn from replays

Evaluate affinities

Profit :)

Modeling cards affinities

Cards bigrams

Cards un-ordered bigrams

Evaluate cards affinities

Played

Played Bi-grams

Played

Bi-grams

Played

Bi-grams

Played

Bi-grams

Played

Bi-grams

Played

Bi-grams Ranked Predictions

Played

Bi-grams

Ranked Predictions

Played

Bi-grams

Ranked Predictions

Played

Bi-grams

Ranked Predictions

Training: 45000 replays

Testing : 5000 replays

1 model per class

Training and evaluation

Suspense is killing me

97% success rate for best prediction by turn 3

Best predictionAverage10th prediction

abilit

Turn2 3 4 5 6 7 8 9 10

Best predictionAverage10th prediction

abilit

Turn2 3 4 5 6 7 8 9 10

Predicting game outcome

How to optimize deck for mana-throughput

Hero powers comparison

Comparing decks typesWhat’s next?

Thank you! https://www.elie.net/hs & @elie/@cealtea on Twitter

http://swiatgry.pl/tapety/pc/5488,hearthstone_heroes_of_warcraft

I am a legend: Hacking Hearthstone with machine learning - Defcon 22 talk slides

Data & Analytics

Transcript of I am a legend: Hacking Hearthstone with machine learning - Defcon 22 talk slides

hearthstone Herald - Trilogy Health Servicestrilogyhs.com/sites/default/files/campus-newsletters/Hearthstone... · HearthStone Herald June 2018. Greetings ... George K. June 03 ...

DEFCON 22 Glenn Wilkinson Practical Aerial Hacking and Surveillance

Hearthstone tutorial

Hacking in a Foreign Language - media.defcon.org CON 13/DEF CON 13...Hacking in a Foreign Language: A Network Security Guide to Russia Kenneth Geers CISSP DEFCON 13. Briefing Outline

Hardware Hacking for us dumb software guys - Defcon

Hacking from the Palm of your Hand - DEF CON...Hacking from the Palm of your Hand Paul Clip DEFCON - August 01, 2003

Could Your Security Vulnerabilities Cause Network Gridlock? · PDF file6 Car hacking tricks revealed at Vegas hacking conference At Defcon 21, Charlie Miller, a security engineer at

Hearthstone Ranch - Colony

I am a legend: Hacking Hearthstone with machine learning - Defcon 22 talk slides

Hearthstone education plan

The Art of VoIP Hacking - Defcon 23 Workshop

DEFCON 19 Barnett Deceptive Hacking WP

DEFCON 25 Voting Machine Hacking Village CON 25 voting village... · DEFCON 25 Voting Machine Hacking Village Report on Cyber Vulnerabilities in U.S. Election Equipment, Databases,

BURPKIT Presented by - Paper Conf/Defcon/2015/DEFCON-23-Nadeem-Douba...• Building hacking tools • Prior work: • Sploitego (presented at DEF CON XX) • Canari (used by Fortune

RFID Hacking: Live Free or RFID Hard - Defcon

Making (and Hacking) the DEFCON 17 Badge by Joe … · DEFCON 17 Haiku Joe Grand aka Kingpin Electronic badge Audio input Affects LED output Sound and light combined Upload new ﬁrmware

Les maths dans Hearthstone - apprendre-en-ligne.net · Les maths dans Hearthstone Hearthstone (anciennement Hearthstone : Heroes of Warcraft) est un jeu de cartes à collectionner

DEFCON 22 Cesar Cerrudo Hacking Traffic Control Systems UPDATED

DEFCON 18 - Octosavvioctosavvi.com/slides/WiMAX Hacking 2010 - DEFCON 18.pdf · WiMAX Hacking 2010 Pierce, Goldy, and aSmig feat. sanitybit DEFCON 18 Updated slides, code, and discussion

Hearthstone 1 Happenings 1 1 1 - Hearthstone Senior Living