Behavior trees and state machines: selection, transitions, and interruption

Start with decision responsibilities

An AI controller needs to choose work, maintain ongoing actions, and react when circumstances change. A diagram's appearance does not settle those responsibilities. Before choosing a behavior tree or state machine, write down what may interrupt each action and which system owns movement, targeting, and animation commitments.

Selection versus transition

Behavior trees organize task selection through composites and conditions. Execution and abort semantics vary; Unreal's implementation uses event-driven observation for relevant conditional aborts. [1] State machines express active states and transitions. Unreal's StateTree combines hierarchical selection with transitions that trigger reselection, illustrating that production systems can mix these ideas. [2]

Worked example

Design an illustrative guard with Patrol, Investigate, and Attack behavior, plus a high-priority Stunned interruption. A behavior-tree selector can prioritize eligible work; a state machine can express the same policy through guarded transitions. Neither representation should decide weapon damage inside an unrelated movement task.

Suppose the guard begins Attack, reserves a firing position, and then becomes Stunned. Define an abort contract: release the reservation, cancel the pending move, and prevent an unfinished attack from committing damage. If damage already committed, the abort must not reverse it accidentally. On recovery, select behavior from present knowledge rather than unconditionally resuming a stale target.

This example tests the meaningful distinction: how selection is reconsidered and how running work is cancelled, not which editor uses more boxes.

Authoring workflow

Give every task clear start, running, success, failure, and cancellation behavior. Keep condition checks observational; repeated evaluation should not spend ammunition or allocate reservations. Put durable world changes behind explicit action boundaries.

Document priorities and transition guards in plain language. When two rules become true together, specify which wins. Introduce deliberate hysteresis or minimum commitment only where the design needs it, and make emergency interruption rules explicit. A hierarchical parent can share common exit handling without copying it into every leaf.

Debugging and limits

Record the active action, the condition that selected it, and the reason for every interruption. Test target disappearance during movement, an animation completing after cancellation, and repeated state entry. Verify that asynchronous callbacks belong to the current action instance before applying results.

Trees can become tangled through hidden blackboard dependencies; state machines can accumulate difficult transition networks. Neither eliminates shared-state ownership problems. Prefer the structure that keeps this game's priorities and lifecycles inspectable, and consider combining a small high-level state machine with task-oriented sub-behaviors when their boundaries are explicit.

Sources

[1] Behavior Tree overview — Epic Games.

[2] StateTree overview — Epic Games.