Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility...
Transcript of Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility...
![Page 1: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/1.jpg)
ImprovingAI Decision Modeling
ThroughUtility Theory
ImprovingAI Decision Modeling
ThroughUtility Theory
Dave MarkPresident & Lead
DesignerIntrinsic Algorithm LLC
Dave MarkPresident & Lead
DesignerIntrinsic Algorithm LLC
Kevin DillAI Engineer
Lockheed Martin
Kevin DillAI Engineer
Lockheed Martin
![Page 2: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/2.jpg)
Dave Mark• President & Lead Designer of
Intrinsic Algorithm LLC, Omaha, NE• Independent Game Studio• AI Consulting Company
• Author of Behavioral Mathematics for Game AI
![Page 3: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/3.jpg)
AIGameDev.net:Trends for 2009 in Retrospect
What's new in 2009 is:1. There's now an agreed-upon name for this architecture: utility-
based, which is much more reflective of how it works. Previous names, such as "Goal-Based Architectures" that Kevin Dill used were particularly overloaded already.
2. A group of developers advocate building entire architectures around utility, and not only sprinkling these old-school scoring-systems around your AI as you need them.
The second point is probably the most controversial.
http://aigamedev.com/open/editorial/2009-retrospective/
![Page 4: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/4.jpg)
We do requests…
“Wow… you’ve got a lot of stuff on utility modeling in here…You should do a lecture on this stuff at the AI Summit.”
Daniel KlineOutside P. F. Chang’sStanford MallOctober 2009
![Page 5: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/5.jpg)
What is “Utility Theory”?In economics, utility is a measure of the relative satisfaction from, or desirability of, consumption of various goods and services.
Given this measure, one may speak meaningfully of increasing or decreasing utility, and thereby explain economic behavior in terms of attempts to increase one's utility.
http://en.wikipedia.org/wiki/Utility
![Page 6: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/6.jpg)
What is “Utility Theory”?
• How much is something worth to me?• Not necessarily equal to “value”
– E.g. $20 might mean more or less than $20• Allows comparisons between concepts• Allows decision analyses between
competing interests• “Maximization of expected utility”
![Page 7: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/7.jpg)
What is “Utility Theory”?
• Related to…– Game theory– Decision theory
• Used by…– Economics– Business– Psychology– Biology
John von Neumann
![Page 8: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/8.jpg)
Value Allows Analysis• Converting raw numbers to usable concepts
– Distance– Ammo– Health
• Converting raw numbers to useful concepts– Distance → Threat– Ammo → Reload Necessity– Health → Heal Necessity
![Page 9: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/9.jpg)
Value Allows Comparisons• By assigning value to a
selection, we can compare it to others
• Von Neumann and Morgenstern’s game theory
• Without value, comparisons are difficult…or even impossible!
![Page 10: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/10.jpg)
Marginal Utility
• Utility isn’t always the same
![Page 11: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/11.jpg)
Marginal Utility• Decreasing Marginal Utility
– Each additional unit is worth less than the one before– The rate of increase of the total utility decreases– Utility of 20 slices != 20 * Utility of 1 slice
Utility per Slice of Pizza
0
1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Total Utility
0
1
2
3
4
5
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
![Page 12: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/12.jpg)
Marginal Utility• Increasing Marginal Utility
– Each additional unit is worth more than the one before– The rate of increase of the total utility increases– Utility of 20 Lego != 20 * Utility of 1 Lego
![Page 13: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/13.jpg)
Converting Data to Concepts• What does the information say?• Raw data doesn’t mean much without context• If data is ambiguous, we can’t reason on it• Various techniques to make sense of raw data
– Conversion formulas– Response curves– Normalization (e.g. 0..1)
![Page 14: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/14.jpg)
Processing One Piece of Info
As the distance changes,how much anxiety do you
have?
![Page 15: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/15.jpg)
Simple Rule
Binary Threshold
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50 60 70 80 90 100
Distance
Anx
iety
Binary
If distance <= 30 then anxiety = 1
![Page 16: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/16.jpg)
Linear Threshold
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50 60 70 80 90 100
Distance
Anx
iety
Linear
Linear FormulaAnxiety = (100 – distance) / 100
![Page 17: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/17.jpg)
Exponential Formula
Exponential Threshold
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50 60 70 80 90 100
Distance
Anxi
ety
Exponent ial
Anxiety = (100 – distance3) / (1003)
![Page 18: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/18.jpg)
Changing Exponents
Exponent Function Variations
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50 60 70 80 90 100
Distance
Anxi
ety
k = 2, 3, 4, 6Anxiety = (100 – distancek) / (100k)
![Page 19: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/19.jpg)
Shifting the Curve
Exponent Function Variations
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50 60 70 80 90 100
Distance
Anx
iety
![Page 20: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/20.jpg)
Threshold / Linear/ Exponential
Exponential Threshold
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50 60 70 80 90 100
Distance
Anxi
ety
Binary
Linear
Exponent ial
![Page 21: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/21.jpg)
Logistic Function
xey −+=
11
(One of the sigmoid – or “s-shaped” – functions)
![Page 22: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/22.jpg)
Logistic Function
Logistic Function Threshold
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50 60 70 80 90 100
Distance
Anxi
ety
Logist ic
Anxiety = 1/(1+(2.718 x 0.45)distance+40 )
Soft threshold
![Page 23: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/23.jpg)
Variations on the Logistic Curve
Logistic Function Variations
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50 60 70 80 90 100
Distance
Anx
iety
Anxiety = 1/(1+(2.718 x 0.45)distance+40 )
![Page 24: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/24.jpg)
Shifting the Logistic Function
Logistic Function Variations
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50 60 70 80 90 100
Distance
Anx
iety
Anxiety = 1/(1+(2.718 x 0.45)distance+40 )
![Page 25: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/25.jpg)
Curve Comparison
Exponential Threshold
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50 60 70 80 90 100
Distance
Anx
iety Binary
Linear
Exponent ial
Logist ic
![Page 26: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/26.jpg)
xy e −=
11log
Logit Function
![Page 27: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/27.jpg)
Logit Functiony = loge(x/(1- x ))
Logit Function
-5
-4
-3
-2
-1
0
1
2
3
4
5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
![Page 28: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/28.jpg)
Logit Function
Logit Function Variations
-5-4-3-2-1012345
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Distance
Anx
iety
y = log?(x/(1- x ))
![Page 29: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/29.jpg)
Logit Functiony = loge(x/(1- x ))+5
Logit Function Shifted +5
0
1
2
3
4
5
6
7
8
9
10
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
![Page 30: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/30.jpg)
Logit Function Shifted +5 and Divided by 10
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Logit Functiony = (loge(x/(1- x ))+5)/10
![Page 31: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/31.jpg)
How Do We Model Our Information?
• Increasing or Decreasing?• Rates of change
– Steady or Variable?– Inflection Point?
• Amount of change– Constrained or Infinite?– Asymptotic?
![Page 32: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/32.jpg)
But What Good Is It?
Exponential Threshold
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50 60 70 80 90 100
Distance
Anx
iety Binary
Linear
Exponent ial
Logist ic
When Anxiety > n then…
![Page 33: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/33.jpg)
Comparing Apples and Ammo
• By using normalized utility values, we can define relationships and comparisons that otherwise would have been obscure– Risk vs. Reward (game theory)– Fear vs. Hate– Ammo vs. Health
![Page 34: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/34.jpg)
Comparing Apples and Ammo• 100 Health (Max)• 75 Health• 50 Health• 25 Health (??)• 5 Health (!!!)
• 100 Ammo (Max)• 75 Ammo• 50 Ammo• 25 Ammo• 5 Ammo
Normalized Importance of Taking Action
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50 60 70 80 90 100
Value
Impo
rtan
ce
HealReload
![Page 35: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/35.jpg)
Comparing Apples and Ammo
• As health decreases,urgency to heal increases
• Make sure we don’t get too low on health!
• As ammo decreases,urgency to reload increases
• Urgency hits maximum when we are out of ammo
Normalized Importance of Taking Action
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50 60 70 80 90 100
Value
Impo
rtanc
e
HealReload
![Page 36: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/36.jpg)
Comparing Apples and Ammo
• Collect current states of independent variables
• Normalize using response curves• (Combine as necessary)• Compare normalized values and select:
– Highest scoring selection– Weighted random from all choices– Weighted random from top n choices
![Page 37: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/37.jpg)
Comparing Apples and Ammo
Normalized Importance of Taking Action
0.00
0.25
0.50
0.75
1.00
0 5 10 15 20Enemy Strength
Thre
at L
evel
Threat
0.684
![Page 38: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/38.jpg)
Comparing Apples and Ammo
Utility
Value
0.1250.1180.684
50505
HealthAmmoEnemiesNormalized Importance of Taking Action
0.00
0.25
0.50
0.75
1.00
0 5 10 15 20Enemy Strength
Thre
at L
evel
Threat
0.684
Normalized Importance of Taking Action
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50 60 70 80 90 100
Value
Impo
rtanc
e
Heal
Normalized Importance of Taking Action
0.00
0.25
0.50
0.75
1.00
1 11 21 31 41 51 61 71 81 91 101
Value
Impo
rtanc
e
Reload
0.118 0.125
![Page 39: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/39.jpg)
Comparing Apples and Ammo
Utility
Value
0.7320.1180.684
35505
HealthAmmoEnemiesNormalized Importance of Taking Action
0.00
0.25
0.50
0.75
1.00
0 5 10 15 20Enemy Strength
Thre
at L
evel
Threat
0.684
Normalized Importance of Taking Action
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50 60 70 80 90 100
Value
Impo
rtanc
e
Heal
Normalized Importance of Taking Action
0.00
0.25
0.50
0.75
1.00
1 11 21 31 41 51 61 71 81 91 101
Value
Impo
rtanc
e
Reload
0.118
0.732
![Page 40: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/40.jpg)
Comparing Apples and Ammo
Utility
Value
0.0030.7290.684
85105
HealthAmmoEnemiesNormalized Importance of Taking Action
0.00
0.25
0.50
0.75
1.00
0 5 10 15 20Enemy Strength
Thre
at L
evel
Threat
0.684
Normalized Importance of Taking Action
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50 60 70 80 90 100
Value
Impo
rtanc
e
Heal
Normalized Importance of Taking Action
0.00
0.25
0.50
0.75
1.00
1 11 21 31 41 51 61 71 81 91 101
Value
Impo
rtanc
e
Reload
0.729
0.003
![Page 41: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/41.jpg)
Comparing Apples and Ammo
Utility
Value
0.1250.0160.000
50750
HealthAmmoEnemiesNormalized Importance of Taking Action
0.00
0.25
0.50
0.75
1.00
0 5 10 15 20Enemy Strength
Thre
at L
evel
Threat
Normalized Importance of Taking Action
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50 60 70 80 90 100
Value
Impo
rtanc
e
Heal
Normalized Importance of Taking Action
0.00
0.25
0.50
0.75
1.00
1 11 21 31 41 51 61 71 81 91 101
Value
Impo
rtanc
e
Reload
0.0160.125
![Page 42: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/42.jpg)
Comparing Apples and Ammo• Don’t simply process 1 potential action at a time
– Should I attack?– Should I reload?– Should I heal?– Should I have a beer?
• Compare all potential actions to each other– Of all of the things I could do, which is the most
important at this moment?
![Page 43: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/43.jpg)
Beyond Apples and Ammo
• Utility measurements can model more than simply tangible data
• They can model abstract concepts:– Threat– Safety– Morale– Emotions
![Page 44: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/44.jpg)
Stacking Apples and Ammo
• Individual utility value can be combined to form new conceptual utilities
• “Need to take cover”– Amount of fire being taken (Threat)?– Is it almost time to reload?– Is it almost time to heal?
Cover = (0.2 + Reload + (Heal x 1.5)) x (Threat x 1.3)Cover = (0.2 + Reload + (Heal x 1.5)) x (Threat x 1.3)Cover = (0.2 + Reload + (Heal x 1.5)) x (Threat x 1.3)
![Page 45: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/45.jpg)
Stacking Apples and Ammo
Relative Utilities
0.000 0.000
0.600
0.156
0.000
0.250
0.500
0.750
1.000
Reload Heal Threat Cover
Nor
mal
ized
Util
ity
Cover = (0.2 + Reload + (Heal x 1.5)) x (Threat x 1.3)
![Page 46: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/46.jpg)
Stacking Apples and AmmoCover = (0.2 + Reload + (Heal x 1.5)) x (Threat x 1.3)
Relative Utilities
0.200
0.100
0.600
0.429
0.000
0.250
0.500
0.750
1.000
Reload Heal Threat Cover
Nor
mal
ized
Util
ity
![Page 47: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/47.jpg)
Stacking Apples and AmmoCover = (0.2 + Reload + (Heal x 1.5)) x (Threat x 1.3)
Relative Utilities
0.400
0.300
0.600
0.819
0.000
0.250
0.500
0.750
1.000
Reload Heal Threat Cover
Nor
mal
ized
Util
ity
![Page 48: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/48.jpg)
Stacking Apples and AmmoCover = (0.2 + Reload + (Heal x 1.5)) x (Threat x 1.3)
Relative Utilities
0.200
0.300
0.800
0.884
0.000
0.250
0.500
0.750
1.000
Reload Heal Threat Cover
Nor
mal
ized
Util
ity
![Page 49: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/49.jpg)
Stacking Apples and AmmoCover = (0.2 + Reload + (Heal x 1.5)) x (Threat x 1.3)
Relative Utilities
0.400
0.300
0.1000.137
0.000
0.250
0.500
0.750
1.000
Reload Heal Threat Cover
Nor
mal
ized
Util
ity
![Page 50: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/50.jpg)
Stacking Apples and AmmoCover = (0.2 + Reload + (Heal x 1.5)) x (Threat x 1.3)
Relative Utilities
0.900 0.900
0.050
0.159
0.000
0.250
0.500
0.750
1.000
Reload Heal Threat Cover
Nor
mal
ized
Util
ity
![Page 51: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/51.jpg)
Utility of Time
• Time can be converted into a utility value– Time to travel over distance– Time to complete something
• Utility of time can be used for comparisons• Utility of time can modify other utilities
![Page 52: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/52.jpg)
Utility of Time
10 10
n 2n
=
All other things being equal,select the closest goal
![Page 53: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/53.jpg)
Utility of Time
10 20
n 2n???
How does the2x difference in the relative utility of
the goals compare to the2x difference in the distances?
<
Exponential Threshold
0.00
0.25
0.50
0.75
1.00
0 10 20 30 40 50 60 70 80 90 100
Distance
Anx
iety Binary
Linear
Exponent ial
Logist ic
![Page 54: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/54.jpg)
Utility of Time
New goal
Main goal
Backtrack!
Main goal
By taking the keys out of consideration as a potential
action, we neglect to get them as we pass right by them.
![Page 55: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/55.jpg)
Utility of Time
• Normalized distance utility as inverse of distance• Use as coefficient to modify base utility of getting keys
Utility of Grabbing Keys
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
20 18 16 14 12 10 8 6 4 2 0 2 4 6 8 10 12 14 16 18 20
Distance
Util
ity
keysDist1
![Page 56: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/56.jpg)
Utility of Time
Main goal
By keeping the keys inconsideration at all times, andfactoring in the utility of time,we get them as we pass by
them.
Utility of Grabbing Keys
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
20 18 16 14 12 10 8 6 4 2 0 2 4 6 8 10 12 14 16 18 20
Distance
Util
ity
![Page 57: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/57.jpg)
Stacking It All UpNumberof Allies
Numberof Enemies
Threat Ratio
My HealthProximityto Leader
Strengthof Allies
Strengthof Enemies
Allied Strength Enemy Strength
My Morale
Retreat Score
Proximityto Base
Urgency“Compartmentalized
Confidence”
![Page 58: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/58.jpg)
???????
Spreading It All OutNumberof Allies
Numberof Enemies
Threat Ratio
My HealthProximityto Leader
Strengthof Allies
Strengthof Enemies
Allied Strength Enemy Strength
My Morale
Retreat Score
Proximityto Base
Urgency???????
??? Score
?????????
?????????
??? Score
???????
?????????
?????????
Data processing != Decision processing
![Page 59: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/59.jpg)
Managing Scalability• Don’t perform all calculations every frame
– Every n frames– Use triggered updates
• Split data calculation off into separate processes– Used by multiple utility calculations for same agent– Used by decision calculations for multiple agents– Blackboard architecture to manage and store
• Lends itself well to multi-threading
![Page 60: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/60.jpg)
Everything is Relative• Many AI decision processes (BTs, FSMs):
– Examine one choice at a time and ask “should I do this one thing?”
– Are certain parameters met to justify that choice?– If not, move on to the next one in a pre-specified
order• What happens if no options meet their criteria?
– Fall back (idle) behavior may not be appropriate– Very susceptible to edge cases
![Page 61: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/61.jpg)
Everything is Relative• Utility-based architectures:
– Continuously analyze all options (rather than just one)– Rate all options based on their respective factors– Select the option that is most appropriate at the time
• Not based on arbitrary, independent thresholds• Handles situational edge cases better• Easier to manage potentially conflicting logic
![Page 62: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/62.jpg)
Dave MarkIntrinsic Algorithm LLC
Dave MarkIntrinsic Algorithm LLC
www.IntrinsicAlgorithm.com
(402) 208–7054
IADaveMarkon:
Yahoo – AIM – Skype – Twitter
IADaveMarkon:
Yahoo – AIM – Skype – Twitter
![Page 63: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/63.jpg)
Kevin Dill
• 9 year industry veteran• Staff Software Engineer, Lockheed Martin• Lecturer, Boston University• Technical Editor, Charles River Media
![Page 64: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/64.jpg)
Example: Apartment Shopping
- Close to work- No off-street parking- Convenient shopping district- Great view… of a used car lot- Beautiful loft apartment
ConsPros606 Automobile Way, Apt 316
- Landlady lives upstairs- Beautiful wooded lot- Nearby parks, bike trails
- No shopping nearby- Electricity & water included- 45 minute commute- Low rent
ConsPros10-B Placid Avenue
![Page 65: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/65.jpg)
Key Insight• We have 12 distinct pros and cons, but only 4
types of considerations:– Cost– Distance to __________
• Could be the distance to work, shopping, parking, etc.– Aesthetic (i.e. how nice looking is the place)
• Could be interior or exterior• Obviously, there is a lot of variability in what people
consider to be “nice”– Noise restrictions
• Many of these are reusable in other contexts!!
![Page 66: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/66.jpg)
Example: Apartment Shopping
- Close to work- No off-street parking- Convenient shopping district- Great view… of a used car lot- Beautiful loft apartment
ConsPros606 Automobile Way, Apt 316
- Landlady lives upstairs- Beautiful wooded lot- Nearby parks, bike trails
- No shopping nearby- Electricity & water included- 45 minute commute- Low rent
ConsPros10-B Placid Avenue
![Page 67: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/67.jpg)
Executive Joe• Joe is a high-powered executive at a big bank• Joe makes lots of money
– Cost doesn’t matter• Joe works late most every night
– Exterior aesthetics don’t matter when the sun is down– Distance to recreation doesn’t matter – who has time?– Distance to work, shopping, and parking matter a lot
• Joe likes to throw big parties– Interior aesthetics are very important– Joe is not fond of noise restrictions
![Page 68: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/68.jpg)
Example: Apartment Shopping
- Close to work- No off-street parking- Convenient shopping
district
- Great view… of a used car lot
- Beautiful loft apartmentConsPros
606 Automobile Way, Apt 316
- Landlady lives upstairs- Beautiful wooded lot- Nearby parks, bike trails
- No shopping nearby- Electricity & water included- 45 minute commute- Low rent
ConsPros10-B Placid Avenue
![Page 69: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/69.jpg)
Stan the Family Man• Stan goes to work to put in his time and get home
to his family• Stan’s wife wants a nice place with lots of
recreation for the kids… Stan wants something he can afford
• The apartment needs to be kid-friendly• Stan likes to drive – it gives him some quiet time• Stan’s family is in bed by 10:00
![Page 70: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/70.jpg)
Example: Apartment Shopping
- No off-street parking- Great view… of a used car
lot- Close to work
- Beautiful loft apartment- Convenient shopping district
ConsPros606 Automobile Way, Apt 316
- Landlady lives upstairs- Beautiful wooded lot- Nearby parks, bike trails
- No shopping nearby- Electricity & water included- 45 minute commute- Low rent
ConsPros10-B Placid Avenue
![Page 71: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/71.jpg)
High Principles• Modular
– A decision is made up of atomic pieces of logic, called “considerations”
– We can easily add and remove considerations• Extensible
– We can easily create new types of considerations• Reusable
– From decision to decision– From project to project
![Page 72: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/72.jpg)
Terminology & Architecture• Reasoner has a list of possible choices
– E.G. play with a ball, build a swordsman unit, select a particular weapon, play a particular animation, etc.
• Each choice has a list of considerations– Considerations evaluate one aspect of the situation
• Considerations generate appraisals• Appraisals inform our final selection
![Page 73: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/73.jpg)
Consideration• Encapsulates one aspect of a larger
decision– Distance – Selection History– Cost – Benefit– Etc.
• Parameterized for easy customization– For this decision and for this character
![Page 74: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/74.jpg)
Consideration Interfaceclass IConsideration{public:
// Load the data that controls our decisionsvoid Load(const DataNode& node) = 0;
// Evaluate this consideration Appraisal Evaluate(const Context& c) = 0;
}
Relies on:– DataNode– Context– Appraisal
![Page 75: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/75.jpg)
DataNode• XML (or equivalent) that contains the
parameterization• May be tool-generated (not hand-generated)• May be part of a larger AI specification
![Page 76: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/76.jpg)
Context• Contains all of the information the AI needs to
make a decision• Provides an abstraction layer between the AI
and the game– If well implemented, can facilitate porting your
considerations from game to game
![Page 77: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/77.jpg)
Appraisal• Generated by the Evaluate() function• Drives our final decision• Common techniques include:
– Boolean logic (e.g. all appraisals must return TRUE)– Highest score– Weight-based random– Optimize resource allocation to maximize utility
• Experience has taught me to start as simple as possible, extend only when necessary
![Page 78: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/78.jpg)
Simple Utility-Based Appraisals• Each appraisal contains two components:
– Base Score: a floating point indicating how good we think this choice is (based on our one consideration)
– Veto: a Boolean allowing each consideration to prevent us from selecting the associated choice
• Calculating total utility for a choice:– If any consideration sets Veto to false, utility is 0– Otherwise, add all of the base scores together
![Page 79: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/79.jpg)
Example: Weapon Selection• “Tuning consideration” provides a base score
– A tuning consideration always returns the values specified in data, regardless of the situation
• “Range consideration” can add utility or veto as needed– Pistols are better at short ranges, sniper rifles at long
• “Inertia consideration” adds utility to current choice– So we don’t change without a good reason
• “Random noise consideration” has a random base score– So we don’t always pick the same thing
• “Ammo consideration” checks if we have ammo• “Indoors consideration” prevents grenade use indoors• Select the weapon with the best total score
![Page 80: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/80.jpg)
Appraisal With A Multiplier*• Replace Veto parameter with a “Final Multiplier”
– Add all base scores together, then multiply by each of the final multipliers
– A multiplier of 0 is still a veto• Allows you to scale utility more smoothly/cleanly
– For example, scale sniper rifle utility at short range• Other things you could add:
– Exponents – Polynomials– Etc.
* (This is Kevin’s preferred approach.)
![Page 81: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/81.jpg)
Multi-Utility Appraisals• Add a Priority attribute to the appraisal• When combining appraisals, take the max Priority
– In other words, if one consideration sets the priority to be high, keep that priority
• Only consider choices with max priority – Allows you to say “If X is true, only consider this small
set of options.”– For example, force the use of a melee weapon at short
range, a ranged weapon at long range
![Page 82: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/82.jpg)
Summary• Modular• Extensible• Reusable• Applicable to a wide variety of game genres and
reasoner architectures– Kohan 2: Kings of War and Axis & Allies– Prototype dog AI– Iron Man boss AI– Red Dead Redemption
• Weapon Selection• Dialog Selection
– Event selection in All Heroes Die
![Page 83: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/83.jpg)
ReferencesDill, Kevin, “Prioritizing Actions in a Goal-Based RTS AI,” AI Game
Programming Wisdom 3Dill, Kevin, “A Pattern-Based Approach to Modular AI for Games,” AI
Game Programming Gems 8Mark, Dave, Behavioral Mathematics for Game AIAlexander, Bob, “The Beauty of Response Curves,” AI Game
Programming WisdomHarmon, Vernon, “An Economic Approach to Goal-Directed Reasoning in
an RTS,” AI Game Programming WisdomOrkin, Jeff, “Constraining Autonomous Character Behavior with Human
Concepts,” AI Game Programming Wisdom 2Garces, Sergio, “Extending Simple Weighted Sums,” AI Game
Programming Wisdom 3
![Page 84: Improving AI Decision Modeling Through Utility Theory · AI Decision Modeling Through Utility Theory Dave Mark President & Lead Designer Intrinsic Algorithm LLC Dave Mark President](https://reader034.fdocuments.net/reader034/viewer/2022042212/5eb4b2eac8ef7b4f9e31819c/html5/thumbnails/84.jpg)
Questions?Dave Mark
Intrinsic Algorithm LLCDave Mark
Intrinsic Algorithm LLC
www.IntrinsicAlgorithm.com
(402) 208–7054
IADaveMarkon:
Yahoo – AIMSkype – Twitter
IADaveMarkon:
Yahoo – AIMSkype – Twitter
Kevin DillLockheed MartinKevin DillLockheed Martin