Agenda

• Refreshment : Problems and Goals• Answering the why–Why we’ve used Case-Based Reasoning.–Why we’ve used Reinforcement

Learning.

• System Architecture.• Project Testing Strategy– Turing Test.– NPC (Static AI).

Problems and Goals

Problems and Goals

Adaptive

Problems and Goals

AdaptiveIntelligent

Problems and Goals

AdaptiveIntelligent Agent

Machines rely on static scripting techniques.

Problems and Goals

Problems and Goals

Mobile

Problems and Goals

Mobile Experience

The Absence of sharing experience costs a lot.

Case Based Reasoning- a Brief

Why Case-Based Reasoning

Why Case-Based Reasoning

Plan Learning

Why Case-Based Reasoning

Plan Learning

FailureLearning

Why Case-Based Reasoning

Plan Learning

CriticLearning

FailureLearning

Why Case-Based Reasoning

Plan Learning

CriticLearning

FailureLearning

Prediction

Reinforcement Learning – A Brief

Why Reinforcement Learning

Requires No Model

Used in the Revising Phase Sub-optimal policies

Balance Exploration- Exploitation

Applies Bootstrapping

Why Reinforcement Learning

Used in the Revising Phase

Why Reinforcement Learning

Requires No Model

Why Reinforcement Learning

Applies Bootstrapping

Why Reinforcement Learning

Learn Sub-Optimal Policies

Why Reinforcement Learning

Balance Exploration-Exploitation

System ArchitectureI-Strategizer AI Engine : Online Case Based Planner I-StrategizerToWargus

Wargus (G

ame)

Expansion Module

Execution Module

Case Based Reasoner EE Module

Plan Retriever

Case (Plan) Base

Plan Adaptor

Plan Reviser(RL Techniques)

Plan Retainer

Perception Module

Actions Executor

Game State

Actions

Goal

Plan to be adapted

Retained Plan

Revised Plan

Retrieved Plan

Plan

Adapted Plan

Plan to be adapted

Game State

Game State

Executed Plan

Game State

Game Specific Actions

FeedbackGame Specific Feedback

Case Representation : An Example

Interleaved Expansion and Execution

Testing Strategy – Turing Test

Testing Strategy –Playing Static AI

References

• Santiago Ontanon, Ashwin Ram - On-Line Case based Planning – 2010

• Kristian J.Hammond - Case-Based Planning - A Framework for planning from Experience - 1994

• Book: Reinforcement Learning An Introduction – 1998

• Matthew Molineaux, David W. Aha, & Philip Moore - Learning continuous action models in a real-time strategy environment - 2008

• Book: AI Game Engine Programming - 2009

Thanks