[Paper Review] A Factor-Based Framework for Decision-Making Competency Self-Assessment

Brett Israelsen, Nisar Ahmed|arXiv (Cornell University)|Mar 22, 2022

Bayesian Modeling and Causal Inference4 citations

TL;DR

This paper proposes a factor-based framework, FaMSeC, for autonomous agents to self-assess decision-making competency using probabilistic meta-reasoning. It computes interpretable self-confidence scores—such as outcome difficulty and solver quality—based on task-specific reward distributions and solver performance, enabling reliable, human-understandable trust signals without expert input for each scenario.

ABSTRACT

We summarize our efforts to date in developing a framework for generating succinct human-understandable competency self-assessments in terms of machine self confidence, i.e. a robot's self-trust in its functional abilities to accomplish assigned tasks. Whereas early work explored machine self-confidence in ad hoc ways for niche applications, our Factorized Machine Self-Confidence framework introduces and combines several aspects of probabilistic meta reasoning for algorithmic planning and decision-making under uncertainty to arrive at a novel set of generalizable self-confidence factors, which can support competency assessment for a wide variety of problems.

Motivation & Objective

• To address the challenge of enabling autonomous agents to assess their own decision-making competency in a way that is interpretable and trustworthy for human supervisors.
• To develop a generalizable, algorithmic framework that computes self-confidence without requiring expert-annotated confidence values for every possible scenario.
• To support human supervisors in making better-informed decisions by providing succinct, quantifiable self-assessment reports from the agent.
• To extend machine self-trust beyond ad hoc methods by grounding it in probabilistic reasoning and stochastic meta-analysis of decision-making components.

Proposed method

• The framework computes self-confidence factors from core components of an autonomous agent’s decision-making pipeline, such as planning algorithms and models.
• It uses the upper partial moment to lower partial moment (UPM/LPM) ratio to quantify the probability that cumulative task rewards exceed a user-defined minimum acceptable threshold, forming the Outcome Assessment factor ($x_O$).
• The Solver Quality factor ($x_Q$) compares the performance of a deployed solver against a trusted, gold-standard solver using a surrogate model trained on historical performance data.
• Both $x_O$ and $x_Q$ are transformed and scaled to a standardized range (e.g., [-1, +1]) for interpretability and human comprehension.
• The framework supports generalization to non-reward-based outcomes and black-box world models through semantic outcome modeling and surrogate learning.
• It enables automated, real-time self-confidence reporting during task execution, reducing reliance on human experts for confidence assignment.

Experimental results

Research questions

• RQ1 Can a general-purpose, algorithmic framework compute interpretable self-confidence scores for autonomous decision-making agents without requiring expert input for each scenario?
• RQ2 How effectively can self-confidence factors like outcome difficulty ($x_O$) and solver quality ($x_Q$) improve human supervisory performance in task dispatching?
• RQ3 To what extent do self-confidence reports influence human trust and decision-making speed in multi-tasking environments?
• RQ4 Can the framework be extended to non-reward-based outcomes and learning-based world models?

Key findings

• The inclusion of either or both FaMSeC factors ($x_O$ and $x_Q$) significantly improved cumulative delivery scores in a human-in-the-loop study with 255 participants.
• Self-confidence reports had a weaker but still positive effect on self-reported trust levels among human supervisors.
• Decision-making time was negligibly affected by the presence of self-confidence reports, indicating minimal cognitive overhead.
• The $x_O$ factor successfully captured the likelihood of task success relative to a performance threshold, with values ranging from -1 (complete lack of confidence) to +1 (complete confidence).
• The $x_Q$ factor effectively quantified the relative performance of a candidate solver against a trusted gold-standard solver using surrogate modeling.
• The framework demonstrated feasibility and utility in real-world simulation scenarios involving autonomous delivery trucks navigating complex, uncertain environments.