Post on 29-Dec-2015
Knowledge acquisition for ada-tive game AI
Marc Ponsen et al.
Science of Computer programming
vol. 67, pp. 59-75, 2007
장수형
Outline
• Introduction• Related work• Adaptive Script of Wargus• Experiment• Result• Alternative method
2 /24
Introduction
• Game– Become increasingly realistic– Graphical presentation – Capabilities of characters ‘living’
• Game AI– Game developers
• Encompass techniques such as pathfinding, animation, collision physics
– Academic researchers• Intelligent behavior
– Inferior quality• Benefit from academic research into commercial games
3 /24
Introduction
• Adaptive game AI– Behavior of computer-controlled opponents– Potentially increase the quality of game AI– Incorporate a sufficient amount of correct prior domain
knowledge• Dynamic scripting
– Offline reinforcement learning technique– Dynamic scripting in a real-time strategy game called
Wargus– Ambitious performance task– The quality of the knowledge base is essential
4 /24
Introduction
• Knowledge base– Manually encode
• Take a long time• Sub-optimal due to analysis • Not generate satisfying result
– Semi-automatically• Increase the performance• Machine learning• Added to knowledge bases• Evolution algorithm
– Automatically• Evolutionary algorithm• Automatically transfers the domain knowledge
5 /24
Related work
• Few studies exist on learning to win complex strategy games
• Focusing on simpler tasks– Relational Markov decision process model to some lim-
ited Wargus scenarios(Guestrin et al.)– Case-bases plan recognition approach for assisting War-
gus player(Cheng and Thawonmas)• Manual knowledge acquisition
– Typical RTS games(Age of Empires and Command & Cun-quer)
• Semi-automatic knowledge acquisition– Pattern recognition technique(Street et al.)
• Automatic knowledge acquisition– Neural network for Backgammon, GO, Chess(Kirby)
6 /24
RTS games
• Usually focus on military combat• Control armies and defeat all opposing forces that are situated in a virtual battlefiled(often called a map) in real-time
• Collecting and managing resources• Determines all decision for a computer opponent over the course of the whole game
– Form of scripts which are list of game action that are ex-ecuted sequentially
– Constricting buildings, researching new technologies, and combat
7 /24
Wargus
• Clone of the popular RTS game Warcraft II
• Open source• Stratagus engine• Strategy
– Small Balanced Land Attack– Large Balanced Land Attack– Soldier’s Rush– Knight’s Rush
8 /24
Complexity of Wargus
• No single tactic dominates all others– The rock-paper-scissors principle
• Large action space– The set of possible actions that can be executed at a par-
ticular moment• In Wargus…
– A : number of assignments workers can perform– P : average number of workplace– T : number of troops– D : Average number of directions that a unit can moves– S : number of choices for a troop’s stance– B : number of buildings– R : average number of choices for research objectives at
a building– C : average number of choice of units to create at a build-
ing9 /24
Complexity of Wargus
• Decision complex of each state– – Higher than the average number of possible moves in
many board game such as chess(30)
11 /24
Dynamic Scripting for Wargus
• Game AI for complex games is mostly defines in scripts
– Contain weaknesses, which human players can exploit– Dynamic script
• Introduced by Spronck et al.• Ability to adapt to a human player’s behavior• The probability that a tactics is selected for a script is an increasing
function of its associated weight value
– Requirements• The game AI can be scripted• Domain knowledge on the characteristics of a successful script can be
collected• Evaluation function can be designed to assess the success of the func-
tion’s execution
12 /24
Dynamic Scripting for Wargus
• Divide the game into a small number of distinct game states
• Each state corresponds to a unique knowledge base
13 /24
Weight adaptation in Wargus
• F : The overall fitness• Fi : the stats fitness(state i)• Sd : the score for the dynamic player• So : the score for the player’s opponent
14 /24
Weight adaptation in Wargus
• Sx : the score of the dynamic player state x• Mx : the military points for player x• Bx : building points for player x
15 /24
EA(Fitness Function)
• Md : Military points for the dynamic player• Mo : Military points for the dynamic player’s opponent• b : break-even point• Ct : game cycle• Cmax : maximum game cycle(the longest time a game is allowed to continue)
16 /24
Performance evaluation
• Dynamic scripting under three condition– Manually acquired– Semi-automatically acquired– Automatically acquired
• The other is controlled by a static script• Four strategy
– SBLA, LBLA, SR, KR• Randomization turning point
– Number of the first game in which the dynamic player statistically outperforms the static player
– A low RTP value indicates good efficiency
18 /24
Conclusions
• Three alternative for acquiring high-quality domain knowledge used by adaptive game AI
– Manual, semi-automatic, automatic• Discussed dynamic scripting• Domain knowledge is crucial factor to the perfor-mance of dynamic scripting
• The automatic knowledge acquisition approach takes best performance
20 /24
Alternative method
• Alternative method of script handling– Bayesian Network
• Case study : StarCraft• ‘Adaptive Reasoning Mechanism with Uncertain Knowledge for Improving Performance of Artificial In-telligence in StarCraft
21 /24
베이지안 네트워크 설계
• 불확실한 지식정보– 상대방 진영으로의 정찰 시도– 지어진 건물들의 구성– 생산한 유닛의 구성– 건물과 유닛의 개수– 위의 정보들을 얻어낸 시각
• 거짓정보는 아니지만 완벽한 정보도 아니다– 숨겨진 유닛 , 숨겨진 건물 , 지어지다가 취소된 건물
23 /24