[Paper Review] Predictable Artificial Intelligence

Lexin Zhou, Pablo A. Moreno-Casares+13 more|arXiv (Cornell University)|Oct 9, 2023

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

This paper introduces Predictable Artificial Intelligence (Predictable AI), a new research direction focused on anticipating key validity indicators—such as performance, safety, and alignment—of AI systems before deployment. It formalizes predictability as a core principle, arguing that prioritizing predictability over raw performance enhances trust, control, and safety in AI ecosystems, and outlines frameworks, predictors, and trade-offs for building reliably predictable AI systems across diverse configurations.

ABSTRACT

We introduce the fundamental ideas and challenges of Predictable AI, a nascent research area that explores the ways in which we can anticipate key validity indicators (e.g., performance, safety) of present and future AI ecosystems. We argue that achieving predictability is crucial for fostering trust, liability, control, alignment and safety of AI ecosystems, and thus should be prioritised over performance. We formally characterise predictability, explore its most relevant components, illustrate what can be predicted, describe alternative candidates for predictors, as well as the trade-offs between maximising validity and predictability. To illustrate these concepts, we bring an array of illustrative examples covering diverse ecosystem configurations. Predictable AI is related to other areas of technical and non-technical AI research, but have distinctive questions, hypotheses, techniques and challenges. This paper aims to elucidate them, calls for identifying paths towards a landscape of predictably valid AI systems and outlines the potential impact of this emergent field.

Motivation & Objective

• To establish predictability as a foundational principle in AI development, prioritizing it over performance to ensure trust and safety.
• To formally define predictability in AI systems and identify its core components and validity indicators.
• To explore trade-offs between maximizing AI system validity and ensuring its predictability across diverse ecosystem configurations.
• To distinguish Predictable AI from related fields like interpretability and alignment, highlighting unique research questions and methodological challenges.
• To map a landscape of predictably valid AI systems and chart pathways for future research.

Proposed method

• Formal characterization of predictability as the ability to anticipate key validity indicators (e.g., performance, safety) of AI systems.
• Identification of candidate predictors—such as model architecture, training data, and system monitoring mechanisms—that can inform predictions about future behavior.
• Development of a framework to evaluate trade-offs between maximizing system validity and ensuring predictability.
• Use of illustrative examples across diverse AI ecosystem configurations to demonstrate how predictability can be achieved in practice.
• Integration of technical and non-technical approaches, including governance, monitoring, and feedback loops, to enhance predictability.
• Differentiation of Predictable AI from related domains like interpretability, robustness, and alignment through distinct hypotheses and research questions.

Experimental results

Research questions

• RQ1 How can we formally define and measure predictability in AI systems, particularly in terms of anticipating performance and safety?
• RQ2 What are the most effective predictors for key validity indicators in complex AI ecosystems?
• RQ3 What trade-offs exist between maximizing AI system validity and ensuring its predictability?
• RQ4 How does Predictable AI differ from existing AI research areas such as interpretability, robustness, and alignment?
• RQ5 What systemic and architectural design choices enable the creation of predictably valid AI systems across diverse deployment contexts?

Key findings

• Predictability is a distinct and essential dimension of AI that must be prioritized over performance to ensure long-term trust and safety.
• Formalizing predictability enables systematic evaluation of AI systems' future behavior, including risks and reliability.
• Predictors such as training data quality, model architecture, and runtime monitoring can significantly improve anticipation of system validity.
• Trade-offs between validity and predictability are inherent and must be managed through deliberate design and governance.
• Predictable AI introduces a new research landscape with unique challenges, techniques, and hypotheses distinct from alignment and interpretability.
• Illustrative examples demonstrate that predictability is achievable across diverse AI ecosystems, including autonomous systems and decision-support tools.