[Paper Review] The Need for Interpretable Features: Motivation and Taxonomy

Alexandra Zytek, Ignacio Arnaldo+3 more|arXiv (Cornell University)|Feb 23, 2022

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

This paper argues that machine learning models are only as interpretable as their features, proposing a formal taxonomy of interpretable feature properties—such as meaningful, understandable, human-worded, and simulatable—to guide the creation of actionable, domain-relevant explanations. It introduces transforms like semantic binning and unstandardized encoding to convert model-ready features into interpretable forms, emphasizing that feature interpretability is as critical as model performance for real-world decision-making.

ABSTRACT

Through extensive experience developing and explaining machine learning (ML) applications for real-world domains, we have learned that ML models are only as interpretable as their features. Even simple, highly interpretable model types such as regression models can be difficult or impossible to understand if they use uninterpretable features. Different users, especially those using ML models for decision-making in their domains, may require different levels and types of feature interpretability. Furthermore, based on our experiences, we claim that the term "interpretable feature" is not specific nor detailed enough to capture the full extent to which features impact the usefulness of ML explanations. In this paper, we motivate and discuss three key lessons: 1) more attention should be given to what we refer to as the interpretable feature space, or the state of features that are useful to domain experts taking real-world actions, 2) a formal taxonomy is needed of the feature properties that may be required by these domain experts (we propose a partial taxonomy in this paper), and 3) transforms that take data from the model-ready state to an interpretable form are just as essential as traditional ML transforms that prepare features for the model.

Motivation & Objective

• To address the gap in understanding what makes features truly interpretable for non-technical domain experts in real-world ML applications.
• To argue that model interpretability is fundamentally limited by the interpretability of its input features, not just the model architecture.
• To develop a formal taxonomy of feature properties (e.g., meaningful, understandable, human-worded) that support usability across diverse domains.
• To highlight the importance of feature transforms—such as semantic binning and unstandardized encoding—that bridge the gap between model-ready and interpretable feature spaces.
• To advocate for a shift in ML development focus from model performance alone to the usability of features in decision-making workflows.

Proposed method

• Proposes a taxonomy of interpretable feature properties based on real-world deployments in domains like child welfare, healthcare, and cybersecurity.
• Identifies key transforms—such as semantic binning, categorical encoding, and unflagged imputation—that yield interpretable features from raw data.
• Maps user roles (e.g., decision makers, ethicists) to required feature properties, showing how different stakeholders need different interpretability levels.
• Introduces the concept of the 'interpretable feature space' as a distinct state of features that are meaningful and actionable for domain experts.
• Recommends integrating interpretable feature transforms into the ML pipeline on par with traditional preprocessing steps like normalization and one-hot encoding.
• Uses case studies from five domains (child welfare, education, cybersecurity, EHRs, satellite monitoring) to ground the taxonomy in practice.

Experimental results

Research questions

• RQ1 What specific properties of features make them interpretable and useful for non-technical domain experts in real-world ML applications?
• RQ2 How do different stakeholder roles (e.g., decision makers, ethicists, developers) require different levels and types of feature interpretability?
• RQ3 What transforms are necessary to convert model-ready features into features that are meaningful, understandable, and human-worded for domain experts?
• RQ4 How does the choice of feature space affect the fidelity, privacy, and performance of ML explanations?
• RQ5 What are the trade-offs between interpretability, model performance, and data privacy when engineering features for explainability?

Key findings

• Even simple models like linear regression become opaque when using uninterpretable features, proving that model interpretability is contingent on feature interpretability.
• Features must be meaningful, understandable, human-worded, and readable to be useful for decision makers without ML expertise, as demonstrated in child welfare and healthcare applications.
• Transforms such as semantic binning and unstandardized encoding are essential to create interpretable features, while model-ready transforms like one-hot encoding or standardized imputation reduce interpretability.
• The interpretable feature space is not a single state but a spectrum of properties tailored to specific user needs, with distinct requirements across domains like cybersecurity and education.
• Intentional feature engineering—such as grouping sensitive variables into abstract concepts—can hide model biases, posing ethical risks in high-stakes applications.
• Prioritizing interpretable features may reduce model performance and compromise data privacy, requiring context-aware trade-offs in feature design.