Artificial Intelligence in Video Games

Jason Fuller

Abstract

With today’s expanding industry of video games, Artificial Intelligence has become a core component of what makes or breaks a good game. From its humble beginnings as just a simple space invader to the strategic General that can lead thousands into battle, game AI has advanced immensely. Game AI started out as merely pattern based enemies without any decision making, but now AI has complete control over decisions and can learn based on pattern analysis. I look at different ways of implementing game AI, how they have evolved, and give examples of their use in the industry. The video game industry will not be slowing down any time soon and the advancement of game AI will continue.

What is Game AI?

Many think that video game artificial intelligence is just the enemy, just something to beat. Game AI is much more than that; its purpose is to imitate human behavior in non-player characters or known as NPCs. It also tries to make the game world feel real, make the game world come alive, and show that the NPCs really do live there [3].

The main goal of game AI is to be fun, to make the player come back. The AI also should be challenging but not overwhelmingly difficult as to make the player give up. Unless the player wants the AI to be overwhelming, there are players who like a seemingly impossible challenge because I am one of them. Another goal is to make sure the AI is not cheating or at least make so the player does not suspect the AI of cheating. This is different from the AI getting bonuses when the difficulty increases such as a damage boost or speed boost. This is more along the lines of always knowing where the player is located [3].

On a more technical side of game AI goals, it should not monopolize all the resources. On average, game AI gets about 30% of the CPU time. As for memory, it really depends on the game and what the developer wants out of it.

2

Game AI Types

There are many types of game AI but the two mostly used are Finite State Machines and Artificial Neural Networks. There is a third commonly used option called Fuzzy Logic, but it is usually used in conjunction with one of the other two types [3].

Finite State Machines are mainly used in Action and RPG (Role Playing Game) games. These games include Shooters, Racing, Sport, and RPGs. Technically, RPGs are considered different from Action games but they often have very common AI functionality. Finite State Machines are used in these because their AIs have much less information to process and their behavior is more limited and more state like such as attack, defend, and run away [3]. RPG games include the Final Fantasy series, the Elder Scrolls series with the newest being Skyrim, and the Mass Effect series.

Artificial Neural Networks are primarily used in RTS (Real Time Strategy) games. The RTS game type also includes Simulation games. Since the AI in RTS games have a large amount of information to process and their behavior is more open, Artificial Neural Networks combined with Fuzzy Logic is the choice currently. RTS AI did use Finite State Machines before Artificial Neural Networks come along though. Games only seriously started using Artificial Neural Networks in the early 2000s [3]. For reference, RTS games are Starcraft, Warcraft, and Age of Empires.

Separate from the AI types is a Path Finding component. Any AI type needs this component and there are many different types. Many game developers have made their own based on what types of games they make and how they want their AIs to move around the world [3].

AI Path Finding

There are two major algorithms in AI path finding today; one is Dijkstra’s Algorithm and the other is the A* Algorithm. Dijkstra’s Algorithm takes longer to process but it usually comes up with a better path than the A* method, but since time is the only thing that really matters the A* Algorithm is more widely used. The A* method is optimized to find the shortest path relative to distance, which is not always to the best path. The A* Algorithm also allows for high optimization so the AI can find the best path for the scenario such as shortest distance or shortest travel time. The method is optimized by changing the search algorithm or by changing how it store data so the prioritization of elements is more suited to the game [3].

3

Figure 1: This is a world in an Action game or RPG represented by the A* Algorithm. The white area is playable area and the gray area is non-playable area. The red lines are known paths

between the gridded playable areas.

How the A* Algorithm works is very similar to how people move between locations in a city they know. So when a person is traveling across town, they do not get out a map, sit down at their kitchen table and plan out their path to Wal-Mart. They just get on the street that will get them moving in the right direction and make decisions as they travel. The algorithm works similarly. Looking at Figure 1, the AI accesses that grid to move around the game world. If the AI is trying to move from the bottom left corner to the top right area, it will move directly towards that area until it cannot and then make a decision on which way to take from there.

Finite State Machines

Figure 2: This shows a simply Finite State Machine with labels of the different parts.

The first of the two most used game AI types is also the simplest. Finite State machines are made up of States and State Transitions. Like I mentioned before, it is mostly used in Action games and RPGs. Among the States and State Transitions there are four components. The States which

4

define behavior, the State Transitions which are the movement from one state to another, the Conditions which must be met for State Transitions, and the Events or Actions which are internally or externally generated which may lead to a Condition being met. The Events or Actions trigger a Condition being met and the Condition calls the State Transition [3].

Figure 3: This is a Finite State Machine used in Quake 2 for enemies.

Looking at Figure 3, the enemy AIs would always be in one of these states and they would start out in the Idle state. To get out of the Idle state, the AI would have to either see the enemy (the player) or to get shot. If the AI sees the players, it transitions from the Idle state to the Attack state because the event “see enemy” would trigger the condition “see enemy” and that would call the state transition. Similarly, if the enemy is in the Attack state but then loses sight of the player, it would transition into the Search state and look around for the player, but once a certain time is reached without spotting the player, the AI would transition to the Idle state once again.

A disadvantage of Finite State Machines is the AI becomes very predictable if there are not enough states. Once there are too many states it gets difficult to organize and to code the AI. The developer needs to find the right balance to make the AI good. Another disadvantage is that since there are such crisp rules to state transitions, the AI does not feel natural. For example, the AI could be running away from the player but then suddenly turn around and start attacking once again because the code tells the AI to run away if the player gets to close and to attack when the player to far enough away.

A common technique now is to have a Finite State Machine with a state of another Finite State Machine. This leads to better state behaviors and state transitions. Such as running away from the player once they get too close but still attacking at the same time.

History

So now that some topics behind game AI have been covered, here is a brief history. All the following games used Finite State Machines for their AI structure. In 1942, Nim was made and it was the first to implement an artificial opponent. I say opponent and not intelligence because

5

there was no decision making for it. All of its moves were laid out by a pattern [2]. Then in the 1960s and 70s, Spacewar! and Pong released single player versions. They started out as 2 player games but an AI was made for each. In 1978, Space Invaders was released and it was the first to use movement patterns for enemies, an increasing difficulty level system, and the first to have in-game events dependent on what the player did [1].

In the 1980s, many games expanded upon known AI concepts but very little new concepts were developed. The 80s did give us Simcity. It pioneered the complex simulation game type which is very common today in games such as the Roller Coaster Tycoon and the Sid Meier’s Civilization series [1].

As for the 1990s, many new concepts were developed. Dragon Quest IV, which is a turn based RPG, introduced the option to change friendly AI tactics, such as to offensive or defensive [1]. Warcraft, which is a RTS, released a cheap way to Path Finding as before this, it was a big resource hog. Half-Life, which is a FPS, was the first the implemented friendly AI effectively for the first time, such as working together and helping the player based on the commands they were receiving.

Artificial Neural Networks

In the late 1990s, Artificial Neural Networks started being used for game AI. Artificial Neural Networks is the second of the two major AI types. This expanded upon what game AI was capable of and what they could be capable of one day. There is no agreed definition of what it exactly is yet, but a commonly found one is “a network of simple processing elements, which can exhibit complex global behavior, determined by the connections between the processing elements and element parameters.” All that is really saying is it works very similarly to a brain as it is based on biological neural networks which are brains. The big upside of using Artificial Neural Networks is that it is an adaptive structure that can learn by pattern analysis [4].

Figure 4: An example of what an Artificial Neural Network would look like.

6

Being based on brains means it also operates similarly. It has input nodes which would act as our senses, a processing layer, which is called the hidden layer which would be the middle unlabeled nodes in Figure 4, and the output nodes which would lead in performing an action and even further inputs of another network. The web structure for the nodes allows for failure in complete decision making while other nodes do make a decision. The hidden layer contains mathematics and logic based operations which differ from game to game. It is called the hidden layer because its signals are never outside of the neural network. So the input layer’s signal comes from outside the network, the output layer’s signal leaves the network. The hidden layer is not just a couple of nodes as seen in Figure 4, but an extensive network of hundreds of nodes [4]. Individual nodes perform just a small part of the overall calculation such as just performing the “+ b” part of the equation y = m * x + b. This allows them to process very fast and to perform more processes a second. The bigger the web, the more CPU time is needed so finding the right balance for AI can be done by estimating but the best way would just be experimenting.

This structure learns by example which is basically pattern analysis. The entire network has connections between the nodes and these connections have certain weights. The weights are displayed as the thickness of the arrows in Figure 4. So when a node gets called by 2 different nodes, it pays more attention to the higher weighted node calling it. The learning is centered on those weights. When a network is first made, the weights are just randomly selected, but these weights can be changed to get different outputs. So to start “teaching” an Artificial Neural Network, inputs are selected and the desired outputs are selected too. Based on the initial weights, the network will calculate outputs based on the selected inputs but they will most likely be wrong because the initial weights are random. The desired output is compared to the calculated output and the difference is determined. Using that difference, the weights are then changed in the network based on an equation that uses the difference, the previous weight, the previous input value of that node, and the learning rate. The weights are changed throughout the network and it then calculates outputs again using the selected inputs. The process of letting the network calculate outputs, find the difference between the desired and calculated outputs, and changing the weight values to get closer to the desired output can happen millions of times before the desired output is calculated correctly [4].

There are games that feature Artificial Neural Networks and implemented them very well.

Black & White

Black & White was released in 2001 and was the first game to effectively use Artificial Neural Networks. The player was a “God” in the game and led the citizens of an island to riches or ruins. The player interacted with the citizens through a large animal creature which could learn by example and would eventually help or hurt the citizens based on the teachings from the player [1].

7

Halo Wars

Halo Wars came out in 2009. It is a RTS in which the player leads the UNSC (Humans) against an alien covenant focused on the destruction of the Human Race. It featured a difficulty level named “Custom” and this difficulty level used Artificial Neural Networks to learn how the player strategized and would get better the more the player played against it. The difficulty had a failsafe though because eventually the player wouldn’t be able to beat it and after so many losses for the player, the AI would automatically “dumb” itself down. This would allow the player a chance to win once again.

Fuzzy Logic

Fuzzy logic is another major AI type but it is usually used in conjunction with Finite State Machines or Artificial Neural Networks because of its superb decision making abilities. Fuzzy Logic was introduced in 1965 for being used in the research of artificial intelligence. It was not used in game AI until the 1990s. Fuzzy Logic is a way to present problems to computers in a way similar to how humans solve problems and that everything is a matter of degree or preference or context. A simple example is a problem that store owners have to solve frequently. A store owner needs to decide how much produce to order, elements to take into consideration are: the time of year, what the weather is like, and if there is a holiday coming up? Those can all lead to certain types to produce selling more than normal such as more fruit is eaten during the summer or more potatoes are sold around Thanksgiving on average [5].

Figure 5: This shows the comparison of height of basketball players using Fuzzy Logic.

So another example of decision making using Fuzzy Logic deals with basketball. The common way of classifying player height on a basketball team to a computer would be to have a limit. All the players above a certain height would be considered tall and all the rest would be considered not tall. Classifying the players this way does not tell the computer much. Using Fuzzy Logic, the computer can classify the players much more accurately. All the players would be considered tall to a certain degree. As seen in Figure 5, player 8 is now tall to a degree of 0.9 and player 23 who is the furthest left player is tall to a degree of 0.65. The computer can then make much better decisions based on the degrees of tallness instead of the players being tall or not tall.

8

Racing Game AI

Figure 6: The red segmented line represents the general path and the light blue line represents the optimal path.

Racing game AI uses a very basic Finite State Machine model. In most racing games, the AI already knows the entire track and the optimal path. It may feel like the AI is cheating by knowing all that already but it is a necessity even if it is unfair. Depending on the difficulty selected by the player, the AI will use the optimal path or not and will go full speed or not. All of the behavior of the AI is also determined before the start of the race dependent on difficulty. If the AI goes flying into a wall without any encouragement from the player, it was already decided. The AI is not dumb for doing that, it was forced [3]. So if the AI is set to the easy difficulty, it will take the general path as seen in Figure 6 which would give the player a chance to pass it on the inside of the corner but if the AI is set to the hard difficulty, it will take the optimal path which will prevent the player from passing it on the inside.

It is the most basic form of game AI but in some racing simulators, the AI gets much more complicated. For example, if the player is using the optimal path around the track, the AI will actively try to push them off of it. The AI will also use tricks such as spinning out opponents by making their back tires lose grip.

FPS (First Person Shooter) Game AI

FPS game AI is implemented with a layered structure. Bottom layers control path finding and animation selection. The animation selection would be similar to if running then do running animation, if reloading weapon then do reloading animation, etc. The higher layer is where the Finite State Machine would be located for tactical reasoning [3].

RPG Game AI

Most RPG AI structures are very similar to Action game AI structures. Usually just dumb AI populates the game world instead of enemy AI, for example the AI is just walking back and forth from two locations. Most encounters with the AI are unscripted, which means the player just

9

bumped into them or it is not part of the overall story of the game. The dumb AI will react either by just thinking nothing happened, running away, or maybe even fighting the player [3]. Grand Theft Auto 4 and Far Cry 2 made big changes to pedestrian AI. In the games, they operate like they have a goal or a task, not just aimlessly walking around [1].

Bioshock Infinite is another good example of AI advancement. The player is accompanied by Elizabeth, who is an AI. Except for when the player first meets her, she is entirely unscripted. How she interacts with the environment and with other AI is based on her decision making and not on hard code telling what to do. She also has emotions, such as if another AI tries to hurt her, she shows fear and if the player gets hurt, she shows concern and sadness. Over the course of the game, the more fighting she encounters, the more used to it she gets and the less fear she shows.

RTS Game AI

RTS AI started out as Finite State Machines but this limited them. The AI would always build up their base in a very strict way since the state machine controlled what it built. Also, once the player developed a strategy that worked against the AI, it would always work because the AI could not learn or evolve. This led to the switch to using Artificial Neural Networks mixed with Fuzzy Logic. By changing to Artificial Neural Networks and Fuzzy Logic, it led to a much smarter AI. The fuzzy logic would be used for short term decisions such as combat and Artificial Neural Networks would be used for long term decision making such as base build up and the overall strategy [5].

Maxis have led the way in many simulation AI elements. With the release of the new SimCity, the simulation landscape has changed once again. The SimCity franchise is a city simulator in which the player is the mayor of a city and is in charge of city design, public works, such as power and water, and safety utilities which are hospitals and police. In the new SimCity, every “Sim” or citizen of the player’s city is an individual AI. With every Sim being an AI, the city seems much more real. Each Sim has their own agenda and even their own wants and needs. Sims will have preferences on what types of jobs they like, what entertainment they like, and so on. Before in the older games, as the player, you would want to build the cheapest things that got the job done, but now you need to make sure you are fulfilling your Sims needs.

Future of Game AI

Game AI has made great leaps forward since they were first developed way back in the 1960s. Artificial Neural Networks has huge possibilities in every game type, an AI that can learn proves to be a great opponent. The challenge of a smarter AI is much better than the challenge of an “I do more damage than you” AI. When a multiplayer game is played on the same map multiple times it is not like each game is exactly the same. Every game feels different because different decisions are made and different outcomes are reached. Now move that kind of decision making of real people to AI for single player games. Every time a mission is played it plays out

10

differently because the AI acts like real people. That is the future of game AI has with the introduction of Neural Networks.

References

[1] Champandard, Alex. Top 10 Most Influential AI Games. Aigamedev.com. February 25, 2013. http://aigamedev.com/open/review/top-ai-games/

[2] Grant, Eugene, and Rex Lardner. The Talk of the Town. TheNewYorker.com. February 25, 2013. http://www.newyorker.com/archive/1952/08/02/1952_08_02_018_TNY_CARDS_000236053

[3] Grzyb, Janusz. Artificial Intelligence in Games. Software Developer's Journal. February 25, 2013.http://www.codeproject.com/Articles/14840/Artificial-Intelligence-in-Games

[4] Neural Networks: A Requirement for Intelligent Systems. February 25, 2013. http://www.learnartificialneuralnetworks.com/#training

[5] Short Term Decision Making with Fuzzy Logic And Long Term Decision Making with Neural Networks In Real-Time Strategy Games. Hevi.info. February 25, 2013. http://www.hevi.info/tag/artificial-intelligence-in-real-time-strategy-games/