CS 480: GAME AI
Transcript of CS 480: GAME AI
![Page 1: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/1.jpg)
CS 480: GAME AI MACHINE LEARNING IN GAMES
5/31/2012 Santiago Ontañón [email protected] https://www.cs.drexel.edu/~santi/teaching/2012/CS480/intro.html
![Page 2: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/2.jpg)
Reminders • Check BBVista site for the course regularly • Also: https://www.cs.drexel.edu/~santi/teaching/2012/CS480/intro.html
• Project 4 due June 7th • Final exam, next week: June 5th
• Tactic & Strategy • Board Games • Learning
![Page 3: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/3.jpg)
Outline • Student Presentations:
• “Game AI as Storytelling” • “Computational Approaches to Story-telling and Creativity” • “Procedural Level Design for Platform Games”
• Very Brief Introduction to Machine Learning • Machine Learning in Games • Course Summary
![Page 4: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/4.jpg)
Outline • Student Presentations:
• “Game AI as Storytelling” • “Computational Approaches to Story-telling and Creativity” • “Procedural Level Design for Platform Games”
• Very Brief Introduction to Machine Learning • Machine Learning in Games • Course Summary
![Page 5: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/5.jpg)
Machine Learning • Branch of AI that studies how to:
• Infer general knowledge from data: supervised learning • Infer behavior from data: learning from demonstration • Find hidden structure in data: unsupervised learning • Infer behavior from trial an error (data): reinforcement learning
• Underlying principle is inductive inference: • E.g. after seeing 100 times that objects fall down, we can infer by
induction the general law of gravity. • We cannot deduce gravity from observation, we can only induce it.
![Page 6: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/6.jpg)
Supervised Learning • Given labeled (supervised) training data:
(x1,y1) (x2,y2) (x3,y3) … (xn,yn)
• Find the general function that maps x to y: • f(x) = y
Examples
![Page 7: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/7.jpg)
Supervised Learning Example • Learning when to cross the street:
• X is defined as a list of features: (“cars coming?”, “pedestrian traffic light color”, “time”)
• Y defined as: cross, stay • Examples:
• x1 = (no, green, 3pm) y1 = cross • x2 = (no, red, 8am) y2 = don’t cross • x3 = (yes, red, 5pm) y3 = don’t cross • x4 = (yes, green, 9pm) y4 = don’t cross • x5 = (no, green, 10am) y5 = cross
• Outcome of machine learning: • f(x) = “cross if no cars coming and light color green”
![Page 8: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/8.jpg)
Supervised Learning • Examples can be:
• Fixed-length vector of attributes: • Numerical • Categorical
• Structured representations (graphs, trees, logical clauses)
• Machine learning works better with: • Few representative attributes • Lots of data
![Page 9: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/9.jpg)
Simplest Method: Nearest Neighbor • Examples are fixed vectors of real numbers • To predict f(x):
• f(x) = yi where (xi,yi) is the example with xi most similar to x (nearest neighbor)
![Page 10: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/10.jpg)
Simplest Method: Nearest Neighbor • Examples are fixed vectors of real numbers • To predict f(x):
• f(x) = yi where (xi,yi) is the example with xi most similar to x (nearest neighbor)
x
![Page 11: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/11.jpg)
Simplest Method: Nearest Neighbor • Examples are fixed vectors of real numbers • To predict f(x):
• f(x) = yi where (xi,yi) is the example with xi most similar to x (nearest neighbor)
x Nearest neighbor f(x) = gray
![Page 12: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/12.jpg)
Machine Learning Methods: • K-Nearest Neighbor • Decision Trees (ID3, C4.5) • Linear Regression • Bayesian Models (Naïve Bayes) • Boosting (AdaBoost) • Kernel Methods (SVM) • Neural Networks (MLP) • Inductive Logic Programming • etc.
![Page 13: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/13.jpg)
Supervised Learning Methods: • K-Nearest Neighbor • Decision Trees (ID3, C4.5) • Linear Regression • Bayesian Models (Naïve Bayes) • Boosting (AdaBoost) • Kernel Methods (SVM) • Neural Networks (MLP) • Inductive Logic Programming • etc.
Which one is the best?
![Page 14: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/14.jpg)
Supervised Learning Methods: • K-Nearest Neighbor • Decision Trees (ID3, C4.5) • Linear Regression • Bayesian Models (Naïve Bayes) • Boosting (AdaBoost) • Kernel Methods (SVM) • Neural Networks (MLP) • Inductive Logic Programming • etc.
Which one is the best?
NONE! Each one has its own biases
![Page 15: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/15.jpg)
Outline • Student Presentations:
• “Game AI as Storytelling” • “Computational Approaches to Story-telling and Creativity” • “Procedural Level Design for Platform Games”
• Very Brief Introduction to Machine Learning • Machine Learning in Games • Course Summary
![Page 16: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/16.jpg)
Why Learning in Games?
Benefit for Games:
• Adaptive, believable Game AI • Find good player matches • Realistic player movement • Reduce development cost (e.g.
fine-tuning parameters)
Benefit for AI:
• Games are great test beds (perfect instrumentation and measurements)
• Reduced cost • Extremely challenging domains • Great way to motivate
students!
![Page 17: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/17.jpg)
Uses of Learning in Games • Driving Games:
• Learn good driving behavior from observing humans • Fine-tune parameters of physics simulation, or of car handling
parameters
• RTS Games: • Automatically balance the game
• RPG/FPS Games: • Believable movements
• Others: • Specific game genres possible only with learning (training games) • Automatically adapt to player strategies • Learning player models
![Page 18: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/18.jpg)
Example Usages: Black & White
![Page 19: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/19.jpg)
Learning in Black & White
Desires
Opinions
Beliefs
Intention: Abstract
Plan
Specific Plan
Action List Belief-Desire-Intention Architecture
![Page 20: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/20.jpg)
Learning in Black & White
Desires
Opinions
Beliefs
Intention: Abstract
Plan
Specific Plan
Action List Belief-Desire-Intention Architecture
Represented as hand-crafted perceptrons (single
layer neural networks)
Given the current situation, they activate more or less, triggering certain desires.
Example: hunger
The structure of the
perceptrons is hardcoded, but the parameters are
learned
![Page 21: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/21.jpg)
Learning in Black & White
Desires
Opinions
Beliefs
Intention: Abstract
Plan
Specific Plan
Action List Belief-Desire-Intention Architecture
Represented as learned decision trees, one per
desire.
They capture towards which object the creature should
express each desire.
Example: hunger The creature will learn a
decision tree from player’s feedback of what can be
eaten
![Page 22: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/22.jpg)
Learning in Black & White
Desires
Opinions
Beliefs
Intention: Abstract
Plan
Specific Plan
Action List Belief-Desire-Intention Architecture
Beliefs are just lists of properties of the objects in the game, used for learning
opinions.
![Page 23: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/23.jpg)
Learning in Black & White
Desires
Opinions
Beliefs
Intention: Abstract
Plan
Specific Plan
Action List Belief-Desire-Intention Architecture
Which desire will be attempted, and towards
which object: e.g. “destroy town” or “eat human”
![Page 24: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/24.jpg)
Learning in Black & White
Desires
Opinions
Beliefs
Intention: Abstract
Plan
Specific Plan
Action List Belief-Desire-Intention Architecture
Desire, and list of objects that will be used. E.g.
“Destroy town by throwing a stone”
![Page 25: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/25.jpg)
Learning in Black & White
Desires
Opinions
Beliefs
Intention: Abstract
Plan
Specific Plan
Action List Belief-Desire-Intention Architecture
Specific list of actions to execute the plan
![Page 26: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/26.jpg)
Learning in Black & White • BDI model to simulate character personality
• Learning is deployed at specific players to adapt the BDI model to player preferences
• Quite advanced, for commercial game standards!
![Page 27: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/27.jpg)
Example Usages: Forza Motorsport
![Page 28: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/28.jpg)
Example Usages: Forza Motorsport • Forza Motorsport lets you create “Drivatars”
• Drivatars use machine learning to “learn” to drive like the player.
• Supervised machine learning: • Input: the track just ahead of the car • Output: car trajectory
![Page 29: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/29.jpg)
Track Segments
a4 a2 a3 a1 Segments
• Define a set of “track segments”. All the tracks are made up of these • For each segment, define the different ways in which it can be traversed
![Page 30: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/30.jpg)
Learning from the Player • When the player plays:
1. For each segment the player traverses, Drivatar remembers the trajectory used (out of the ones it has stored), speed, and other parameters.
2. Drivatar only remembers the last data seen, and forgets old data (to adapt to newer driving styles of the player)
3. When playing the game, it selects the trajectories learned form the player for each track segment, and combines them into a minimal curvature line, which is the path taken by the AI
![Page 31: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/31.jpg)
Minimal Curvature Line
![Page 32: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/32.jpg)
Learning in Drivatars: Key Ideas • Learning to drive is very complex! Too Many parameters!
• Solution: simplify the problem!
• Simplifications: • Divide the track in segments • Identify a finite set of possible trajectories for each segment (this
defines a discrete and finite vocabulary for the machine learning method)
• The goal of AI in games is to make the game fun, not to create advanced AI! (at least from a game company perspective J)
![Page 33: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/33.jpg)
Other Games Using Learning • Colin McRae Rally 2.0: Neural networks to learn how to
drive a rally car
![Page 34: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/34.jpg)
Other Games Using Learning • Command & Conquer Renegade: learning new paths by
observing the player (e.g. player breaks a window and creates a new path)
![Page 35: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/35.jpg)
Other Games Using Learning • Re-Volt: genetic algorithm tunes the parameters of the
steering behaviors for cars, to optimize lap time
![Page 36: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/36.jpg)
Ways to Fake “Learning” in Games • Preprogram different levels of difficulty and change
between them at run time
• Rule-based system: have a large pool of rules, and add them bit by bit to the AI
• Change parameters during game play: • Increase the accuracy of shooting • Decrease reaction time of enemies • Etc.
![Page 37: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/37.jpg)
Outline • Student Presentations:
• “Game AI as Storytelling” • “Computational Approaches to Story-telling and Creativity” • “Procedural Level Design for Platform Games”
• Very Brief Introduction to Machine Learning • Machine Learning in Games • Course Summary
![Page 38: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/38.jpg)
Game AI • Intelligent behavior for characters in games
• The goal is not to make characters smart, but to make the game believable, engaging and fun
• There is a lot of design in game AI
• Commercial games use a collection of very simple, but highly customizable AI techniques (FSMs, Behavior Trees, Influence Maps, etc.)
![Page 39: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/39.jpg)
Game AI
AI
World Interface
(perception)
Strategy
Decision Making
Movement
![Page 40: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/40.jpg)
Game AI • Perception: interfacing the AI with the game
• Movement: jumping, firing, steering behaviors
• Pathfinding: quantization, navmeshes, A*, TBA*, LRTA*, D*-Lite
• Decision Making: FSMs, Decision Trees, Behavior Trees
• Tactics/Strategy: Rule-Based systems, waypoints, influence maps
• Board Games: Minimax, alpha-beta, Monte-Carlo search
• Learning in games: supervised learning, reinforcement learning
![Page 41: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/41.jpg)
Game AI • Perception: interfacing the AI with the game
• Movement: jumping, firing, steering behaviors
• Pathfinding: quantization, navmeshes, A*, TBA*, LRTA*, D*-Lite
• Decision Making: FSMs, Decision Trees, Behavior Trees
• Tactics/Strategy: Rule-Based systems, waypoints, influence maps
• Board Games: Minimax, alpha-beta, Monte-Carlo search
• Learning in games: supervised learning, reinforcement learning
Projects 1, 2, 3, 4
![Page 42: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/42.jpg)
Game AI • Perception: interfacing the AI with the game
• Movement: jumping, firing, steering behaviors
• Pathfinding: quantization, navmeshes, A*, TBA*, LRTA*, D*-Lite
• Decision Making: FSMs, Decision Trees, Behavior Trees
• Tactics/Strategy: Rule-Based systems, waypoints, influence maps
• Board Games: Minimax, alpha-beta, Monte-Carlo search
• Learning in games: supervised learning, reinforcement learning
Midterm
![Page 43: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/43.jpg)
Game AI • Perception: interfacing the AI with the game
• Movement: jumping, firing, steering behaviors
• Pathfinding: quantization, navmeshes, A*, TBA*, LRTA*, D*-Lite
• Decision Making: FSMs, Decision Trees, Behavior Trees
• Tactics/Strategy: Rule-Based systems, waypoints, influence maps
• Board Games: Minimax, alpha-beta, Monte-Carlo search
• Learning in games: supervised learning, reinforcement learning
Final
![Page 44: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/44.jpg)
Project 4 • Project 4 (and last): Rule-based Strategy for RTS Game (S3)
• Idea: • Create a perception layer that creates a simple knowledge base (logical
terms) • Create a simple unification algorithm with variable bindings • Define a set of actions the rule-based system can execute • Define a small set of rules (do not overdo it! J) • RETE is optional (extra credit) • See how well it plays and how easy is it to make the AI play well!
• Anyone wants to do a different project 4? Any ideas?
![Page 45: CS 480: GAME AI](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285dd59ab6a1914513787fc/html5/thumbnails/45.jpg)
Next Week • Final Exam • Project 4 presentation