[Paper Review] DeepNNK: Explaining deep models and their generalization using polytope interpolation

Sarath Shekkizhar, Antonio Ortega|arXiv (Cornell University)|Jul 20, 2020

Adversarial Robustness in Machine Learning42 references4 citations

TL;DR

This paper introduces DeepNNK, a nonparametric, geometrically intuitive framework for explaining deep neural networks using local polytope interpolation via Non-negative Kernel (NNK) regression. By leveraging k-nearest neighbors with adaptive weighting based on data geometry, DeepNNK enables instance-level interpretability, identifies models with good generalization via leave-one-out estimation, and rationalizes adversarial and generative examples as artifacts of sparse interpolation neighborhoods.

ABSTRACT

Modern machine learning systems based on neural networks have shown great success in learning complex data patterns while being able to make good predictions on unseen data points. However, the limited interpretability of these systems hinders further progress and application to several domains in the real world. This predicament is exemplified by time consuming model selection and the difficulties faced in predictive explainability, especially in the presence of adversarial examples. In this paper, we take a step towards better understanding of neural networks by introducing a local polytope interpolation method. The proposed Deep Non Negative Kernel regression (NNK) interpolation framework is non parametric, theoretically simple and geometrically intuitive. We demonstrate instance based explainability for deep learning models and develop a method to identify models with good generalization properties using leave one out estimation. Finally, we draw a rationalization to adversarial and generative examples which are inevitable from an interpolation view of machine learning.

Motivation & Objective

• To address the limited interpretability of deep neural networks, especially in model selection and predictive explainability.
• To develop a practical interpolation framework that combines geometric intuition with theoretical guarantees, avoiding the complexity of simplicial interpolation in high dimensions.
• To enable instance-based explainability by tracing predictions back to training data via local polytope structures.
• To identify models with good generalization using leave-one-out estimation on the interpolation framework.
• To rationalize adversarial and generative examples through the lens of interpolation sparsity and instability.

Proposed method

• Proposes a local polytope interpolation method based on Non-negative Kernel (NNK) regression to construct interpolation neighborhoods from training data.
• Uses k-nearest neighbors as a starting point but replaces fixed neighborhood selection with data-adaptive, geometrically informed weights that prioritize informative neighbors.
• Employs a leave-one-out estimation strategy to assess model generalization by measuring prediction error when each training point is excluded.
• Applies the framework to deep learning models by integrating NNK interpolation into a Wide-ResNet-28-10 architecture trained with AutoAugment on CIFAR-10.
• Uses interpolation spread (equation 14) to detect model brittleness by measuring variation in neighbor weights for specific test instances.
• Analyzes adversarial and generative images by comparing their average number of neighbors in the interpolation space to real images.

Experimental results

Research questions

• RQ1 How can deep neural network predictions be explained at the instance level using geometric interpolation?
• RQ2 Can interpolation-based methods provide theoretical and practical insights into model generalization without relying on traditional regularization?
• RQ3 Why do adversarial and generative examples often fail to generalize, and can this be explained through interpolation behavior?
• RQ4 To what extent does the number of neighbors in the interpolation neighborhood correlate with model stability and data distribution?
• RQ5 Can leave-one-out estimation on the interpolation framework serve as a reliable proxy for model generalization performance?

Key findings

• DeepNNK achieves 97.3% test accuracy on CIFAR-10 using a Wide-ResNet-28-10 with AutoAugment, matching the original network’s performance.
• Adversarial and generative images exhibit significantly fewer interpolation neighbors on average than real images, indicating sparser coverage in the interpolation space.
• The framework identifies brittle model predictions by detecting high variation in neighbor weights for specific test instances, revealing model limitations.
• Leave-one-out estimation using the NNK interpolation framework successfully identifies models with good generalization properties.
• The method rationalizes adversarial examples as arising from interpolation in unstable, low-density regions where only a few training points influence the prediction.
• The study demonstrates that generative and adversarial examples are not random outliers but emerge from interpolation in regions with minimal training data support.