[Paper Review] EDDI: Efficient Dynamic Discovery of High-Value Information with Partial VAE

Chao Ma, Sebastian Tschiatschek+4 more|arXiv (Cornell University)|Sep 28, 2018

Machine Learning and Data Classification49 citations

TL;DR

EDDI introduces a scalable framework that uses a Partial Variational Autoencoder to handle partially observed data and an information-theoretic acquisition function to sequentially query the most valuable missing variables under cost constraints.

ABSTRACT

Many real-life decision-making situations allow further relevant information to be acquired at a specific cost, for example, in assessing the health status of a patient we may decide to take additional measurements such as diagnostic tests or imaging scans before making a final assessment. Acquiring more relevant information enables better decision making, but may be costly. How can we trade off the desire to make good decisions by acquiring further information with the cost of performing that acquisition? To this end, we propose a principled framework, named EDDI (Efficient Dynamic Discovery of high-value Information), based on the theory of Bayesian experimental design. In EDDI, we propose a novel partial variational autoencoder (Partial VAE) to predict missing data entries problematically given any subset of the observed ones, and combine it with an acquisition function that maximizes expected information gain on a set of target variables. We show cost reduction at the same decision quality and improved decision quality at the same cost in multiple machine learning benchmarks and two real-world health-care applications.

Motivation & Objective

• Motivate automated, personalized dynamic information acquisition in cost-sensitive settings.
• Develop a scalable probabilistic model for partially observed data that supports fast inference.
• Design an acquisition function that selects the most informative missing variables to query next.
• Demonstrate that EDDI reduces information gathering cost without sacrificing decision quality across domains.

Proposed method

• Introduce Partial VAE to perform amortized inference with arbitrary observed subsets of variables.
• Represent xO using a permutation-invariant set encoder (PN/PNP) to model p(z|xO).
• Derive a tractable information reward for variable selection based on mutual information in the z-space (Equation 9).
• Approximate the KL terms via q(z|xO), q(z|xi, xO), and shared samples to enable efficient computation.
• Cast active variable selection as maximizing the expected information gain about target variables xφ (Algorithm 1).

Experimental results

Research questions

• RQ1 How can we perform probabilistic inference when only a subset of variables is observed for each instance?
• RQ2 Can we design a scalable, variable-wise acquisition strategy that maximizes information gain under acquisition cost?
• RQ3 Does the Partial VAE enable effective missing-data imputation and uncertainty estimation across tasks?
• RQ4 Is the EDDI approach computationally efficient enough for real-world health-care and large-scale datasets?

Key findings

• Partial VAE provides scalable amortized inference for partially observed data and supports effective imputation.
• PN/PNP encodings yield better inpainting and uncertainty modeling than ZI-based approaches in MNIST experiments.
• EDDI outperforms RAND and SING baselines on six UCI datasets in terms of information efficiency and RMSE AUIC rankings.
• PNP-based EDDI achieves substantial speedups over non-amortized methods and is about 1000x more efficient than DRAL on Boston Housing.
• In MIMIC-III risk assessment and NHANES public health tasks, EDDI with PNP consistently yields better AUIC rankings than baselines.