10.3 Understanding Pattern Recognition Methods

Chris Kramer

Pattern Recognition is ...

Abstracting relevant information from game world

Constructing concepts or models and deducing patterns for higher-level reasoning and decision-making systems.

necessary especially when game world is not deterministic

built-in randomness

player actions.

Approaches to Understanding

Functional

Derived from the roles Pattern Recognition has in games.

Methodological

Applying computer science concepts to PR in games

Functional Approach

Think about role of PR in an actual game

Sports game: “read” the match

RTS: identify threats; strategize

1 on 1: react to opponent's favored moves

3 Aspects determine role of a PR system

Level of decision making

Stance toward player

Use of modeled knowledge

Level of Decision Making

3 classic military levels: Strategic, Tactical, Operational

Strategic: Lots of data and time; Many speculative decisions; cost of single poor decision is high

Tactical: Mediator between Strategic and Operational; Accomplish strategic plan; Coordinate groups of entities; More real-time

Operational: Very concrete; Many short-term, reactive decisions; individual unit actions

Stance toward player

Enemy: provide a challenge; be purposeful, if not intelligent

Ally: assist player; communicate in an accessible format; be consistent

Neutral: fairness; especially in case of observer – AI which governs camera movement in sports game

Use of Modeled Knowledge

Generators and Symbols – game world seen as a machine generating a series of different states or actions

Symbols are the fundamental reasoning unit resulting from PR

Sequence of symbols used in 2 ways: Prediction and Production

Prediction: model player as generator and predict next symbol

Production: AI models self as generator and determines best symbol to execute

Methodological Approach

Apply general computer science innovations to pattern recognition in games

3 aspects: Optimization, Adaptation, Uncertainty

Optimization: mathematical maximization of some objective function

Adaptation: create model based on known results of previous models

Uncertainty: applying methods which account for uncertainty

Optimization

3 mathematical elements: objective function, variables, constraints

Techniques are iterative, time-consuming and usually offline

Problem: like any math function, there are local optimums in addition to the global optimum.

Many approaches to find global optimum – most focus on multiple traces of search space

Genetic algorithms – best when variables are independent

Swarm algorithms – iterators “fly” in the search space

Optimization Issues

Usually multi-dimensional (many variables) – impossible to visually represent

Computational difficulty can be eased with heuristic weakening of optimality criteria

Real-time usage must be linear heuristic search with few variables.

Optimization Example:Age of Empires

AoE: RTS game with many varied units and teams

Genetic algorithm used to balance units

Set up mock battles, use results of battles to guide selection

Objective function: want both sides to have equal numbers of wins and losses

Adaptation

Recognizing changing circumstances

Best for indeterminate or unknown factors

Methods include: Reinforcement Learning, Influence Maps, Neural Networks

Uncertainty

Soft Computing refers to methods that account for uncertainty: probabalistic reasoning, fuzzy logic

Fuzzy Sets: items have partial or probabalistic membership in a set.

Fuzziness can be incorporated with other methods to better deal with uncertainty: Fuzzy genetic algorithm, Fuzzy Neural Nets

10.5 Getting Around the Limits of Machine Learning

Method of Analysis: 4 Questions

Analysis of Machine Learning in 3 actual games

Specific Limits to learning

Analysis Method

4 Questions:

1. Cheap to recognize what to learn from?

2. Cheap to store the knowledge?

3. Cheap to use the knowledge?

4. Does game benefit from learning?

Major divisions of the problem:

implicit vs. explicit

online vs. offline

Black & White

Player dictates good or bad creature behavior

Explicit Online learning

Answer to all 4 Questions: YES!

#2 might have been problem in past

Testing started early and was integral to project

Innate programmed behaviors constrained learning

Command and Conquer: Renegade

Unimplimented feature: if player goes between areas by an unkown path, AI will learn that path

Online Implicit Learning

4 Questions: yes

Path is already processed, Storage of new pathnodes is negligible

Re-Volt

Genetic algorithm used during development to tune parameters of car AIs

Offline Implicit Learning

Lap Times provided simple criteria for selection

Limits to Learning

Context of first 3 questions

Recognizing something to learn from

More realism in games results in problems similar to machine learning in the real world

Noisy Inputs: Lots of data, Little useful data

Time Dependencies

Some algorithms to costly for online learning

Use of knowledge can be expensive also

Good Knowledge Representation Helps Everywhere

Preprocessing input signals and boiling them down to easier-to-use units

High level of representation is easier to work with

Beware loss of information

MIDI vs. Raw audio

Seeing More Clearly

Learning from the present: modern games are good at this

Learning from past: Causal Chains

Recognizing key precursors to learning event

Keep detailed history to learn from

Never been implemented

Seeing the Wrong Thing in theRight Examples

Instance-based learning models often learn the wrong things

Military System: detect if a tank is present in an aerial photograph

Performed well on training and test sets

But very poorly on real data

All training photos of tanks were on sunny days, and photos of nothing were on cloudy days

Must have “difficult” data

Storing New Knowledge

Find ways to incorporate new knowledge online

Complex algorithms – recalculation unfeasible

Cannot lose performance against original data set

Over-fitting

Existing algorithm for neural nets: Temporal Differences

Conclusion

Question #4 is the most important!