[Paper Review] Hidden Markov Models derived from Behavior Trees

Blake Hannaford|arXiv (Cornell University)|Jul 23, 2019

Time Series Analysis and Forecasting27 references4 citations

TL;DR

This paper establishes a novel mathematical relationship between Behavior Trees (BTs) augmented with probabilistic information and Hidden Markov Models (HMMs), enabling the application of established HMM algorithms—such as Baum-Welch and Viterbi—for state estimation and parameter identification in noisy, task-based systems. The key contribution is a systematic method to derive a unique HMM from a probabilistically augmented BT, allowing robust analysis of human or robotic task performance from noisy observations.

ABSTRACT

Behavior trees are rapidly attracting interest in robotics and human task-related motion tracking. However no algorithms currently exist to track or identify parameters of BTs under noisy observations. We report a new relationship between BTs, augmented with statistical information, and Hidden Markov Models. Exploiting this relationship will allow application of many algorithms for HMMs (and dynamic Bayesian networks) to data acquired from BT-based systems.

Motivation & Objective

• To address the lack of algorithms for tracking or identifying parameters of Behavior Trees under noisy observations.
• To establish a formal relationship between probabilistically augmented Behavior Trees and Hidden Markov Models.
• To enable the use of well-established HMM algorithms—such as Baum-Welch and Viterbi—for state estimation and system identification in BT-based systems.
• To facilitate the analysis of time-series data from human or robotic task performance when observations are corrupted by noise.
• To extend the applicability of HMMs and Dynamic Bayesian Networks to systems modeled by BTs.

Proposed method

• Augment a Behavior Tree with statistical or prior probability information to form an Augmented Behavior Tree (ABT).
• Derive a unique Hidden Markov Model (HMM) from the ABT by mapping BT nodes and transitions to HMM states and transition probabilities.
• Use the ABT's emission probabilities (Bij) and transition probabilities (Aij) to define the HMM's emission and transition matrices.
• Apply standard HMM algorithms: Viterbi for state sequence decoding, Baum-Welch for parameter identification.
• Use divergence measures (e.g., Jensen-Shannon divergence) to assess the decodability of HMMs derived from ABTs.
• Validate the method using synthetic data generated from 6-state and 16-state ABTs with controlled perturbations and varying levels of state transparency.

Experimental results

Research questions

• RQ1 Can a Behavior Tree with probabilistic augmentation be formally mapped to a unique Hidden Markov Model?
• RQ2 How well do standard HMM algorithms—Viterbi and Baum-Welch—perform when applied to data generated from an augmented BT?
• RQ3 What is the impact of model size and state transparency (i.e., distinguishability of states) on the accuracy of parameter identification and state decoding?
• RQ4 How do divergence measures between emission distributions predict the feasibility of successful decoding in BT-derived HMMs?
• RQ5 Can the derived HMM framework support robust system identification and state estimation in noisy, real-world task performance data?

Key findings

• The Viterbi algorithm achieved near-perfect state sequence decoding when the Jensen-Shannon divergence (JSD) between emission distributions was high (JSD ≈ 1.0), even with significant mismatch between generative and decoding model parameters.
• Baum-Welch parameter identification showed higher error when the generative and decoding models differed, with RMS error ranging from 0.2 to 0.6 after convergence, compared to 0.1 for model perturbation alone.
• For the 6-state model, parameter estimation error was lower than for the 16-state model, especially with obscured states (R=0), indicating better performance on smaller, less complex models.
• Parameter estimation was most accurate when states were transparent (R=5), as unambiguous transitions reduced estimation uncertainty.
• Divergence measures computed from labeled data were effective in predicting decoding performance, with higher divergence correlating with better state sequence recovery.
• The method enables the use of HMM and Dynamic Bayesian Network algorithms on BT-based systems, significantly broadening the analytical toolkit for task modeling in robotics and human motion tracking.