Overview

For this project I developed an AI for a group of agents of a 3D third person shooter stealth game, in which groups of enemies patrol an area and react to the presence and actions of the player, shooting them, healing or getting cover.

To do so, I used an Hierarchical Finite State Machine and a Behavior Tree for decision making and Unity’s NavMesh for pathfinding.

In particular, the AI is able to:

• Patrol

• Investigate a position considered suspicious

• Shoot

• Take cover

• Reload the weapon

• Heal

• Retrieve medkits from dead bodies

A very simple level design and blockout was done to prototype and test the AI.

AI Design

Patrolling

When the player has not been spotted yet, the AI patrols the area, following a certain path and stopping at waypoints. When the AI is aware of the player's presence, the waypoints change and it enters into an alerted patrol.

Investigation

When the AI sees the dead body of another agent, or doesn’t see the player for a while after they were spotted, it searches for the player in the suspected position and the near possible hiding positions. The AI returns to an alerted patrol, together with other agents, if it doesn’t find them.

Shooting

Once the player is spotted, the AI can shoot them. This happens both if the player is uncovered or covered, to put pressure. The probability to shoot in the two cases is different: if the player is uncovered the probability to shoot is higher.

If the shot can’t reach the player's surrounding area, the agent moves towards him/her until the shot can reach them.

Take cover

The agent can take cover to flee from the player or to protect itself during reloading and healing.

To have a more dynamic behavior, the agent is able to take cover even after firing a random number of shots (between 2 and 5).

The covers the agent can choose depend on three things:

• If it is occupied by another agent.

• If the cover is visible to the player.

• The distance between the cover and the player.

Heal and reload

Each agent starts with 1 medikit to heal itself when its health is below 50%, but it can use it only when in cover, while the reload can be done both when it is in cover or after shooting.

Retrieving medikit

When agents don’t have medikits available and their health is below 50%, they can take the medikits of another dead agent, choosing it between those who still have a medikit and are not visible to the player.

Implementation

Perception

The perception module of the AI is used to retrieve the informations needed during the decision making. These data can concern both the internal knowledge of the agent and the external knowledge about the world state.

The world state is saved in the class Status, in particular it saves:

• The last time the player has been seen: Saved as seconds from the start of the application;

• If the player has been spotted;

• The last known position of the player: Saved as a placeholder, represented by an empty gameobject in the Unity hierarchy, with just a trigger collider attached to it;

• A list of agents that are actually shooting the player;

• The unseen dead agents;

• All the covers of the map;

• An array containing the living agents: Sorted based on the distance between him and the player, updated every 5 seconds.

During the game, the AI doesn’t refer directly to the player position, but always to the placeholder, to give the idea to the player that the AI doesn’t actually know where they are, except during the shoot if the player is visible.

AGENT'S SENSING

The class AISenses takes care of managing the agent’s sensing, checking if the agent is seeing the player or the dead body of another agent, through the method IsSomethingVisible:

This method is called in a Coroutine which is started in the Start method. The coroutine at each frame calls IsSomethingVisible passing the player as parameter.

If the player has already been spotted it would just update the state in the Status class, otherwise the agent will spot the player only if they have been visible for a certain time (inversely proportional to the distance) or if the distance is less than a certain threshold.

Other ways the player can be spotted are

• If they touch the agent, through the OnCollisionEnter method;

• If they shoot the agent and the shot hits it or passes within a certain distance from the agent.

To check if a shot passes near an agent, a trigger collider is attached to a child gameobject, and when the player shoots, a raycast is performed to check if it collides with the child gameobject’s collider. The raycast stops on the first occurrence of a solid object.

Decision making

For the decision making, a Hierarchical Finite State Machine was used, which at each step checks the conditions needed to go from one state to another and triggers the relative transition if the conditions are met. It has also been evaluated to use a Decision Tree during the early phase of the project, but the HFSM has been deemed more straightforward and easier to design and maintain.

The base layer FSM is made of 4 states:

• Patrol: The patrol state is the starting state of the FSM. When starting the game the agents start to patrol following a certain path, until a transition to another state is triggered. When the agent is aware of the player’s presence, the path of the patrolling and the speed of the agent change.

• Investigate: The agent enters this state when it sees a dead body of another agent (if player has not been spotted), if the player has not been seen for a certain time or if it sees that the player is no more in the last known position. The agent will investigate the position of the dead body or the one of the player, searching for him/her in the surrounding area.

• Search Medikit: When entering this state, the agent will go to take the medikit of one of the dead agents.

• Combat: The internal FSM which manages the agent’s behavior during the fight with the player.

The inner FSM Combat manages the agent’s behavior during the fight with the player, deciding if the agent has to take cover or shoot. The idle state is used as a transitory state for two reasons:

• To decide the right state for the agents based on the current world state and internal knowledge

• In order to add a bit of randomness for the first state of the combat FSM.

Healing and reloading happens in the Cover state, when the agent has reached the chosen cover.

Considering the kind of game, a value similar to the human’s reaction time (330ms) for the update time value of the FSM should be enough, so I have found a value of 0.5 seconds to be quite good considering both performances and behavior's believability.

COVER CHOICE

To choose the cover, tests on the possible covers are made, in particular, it should not be occupied by another agent, it should not be visible to the player and it should be quite far away from the player. So the method getPotentialCovers() is called.

Then between all the potential covers found, the agent chooses the closest one.

The visibility of a cover is evaluated through a coroutine present in the class Cover, which calls the method checkVisibility every 0.5 seconds.

Since the agent has to refer to the last known position and not to the current position of the player, an hypothetical position for the camera is computed, rather than the real position of the camera, and it is used to calculate the direction of the raycast. This mechanism simulates the perception that the player is still there and thus that they can see the cover.

No assumptions are made about the camera field of view and orientation, because the cover is considered visible even if the player is not directly looking at it.

SHOOTING

The fire state of the HFSM runs a Behavior Tree which evaluates what to do before shooting. In particular, since there is the possibility for the agent to not go to cover when the gun is empty, it first evaluates if it needs to be loaded, then it evaluates if the shot can reach the player and in case it shoots, otherwise it goes towards the player. The probability to go to cover when the gun is empty is calculated during the stay action of the Fire state, after the step of the behavior tree.

The probability to actually shoot the player during the fire action depends on if the player is behind a cover or they are actually visible to the agent. In particular after some testing I have considered the behavior believable using a probability of 85% in case the player is uncovered and 50% in case the player is covered. If the agent hasn’t shot for a certain time then it just shoots without considering the probability.

The precision of the shot depends on the distance between the agent and the player, with a maximum offset when the distance is the same as the agent’s range:

After testing I’ve found that using 2 as value for the maximum offsets is a good compromise.