[Paper Review] ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models

Minsuk Kahng, Pierre Andrews+2 more|arXiv (Cornell University)|Apr 6, 2017

Data Visualization and Analytics25 references15 citations

TL;DR

ActiVis is an interactive visual analytics system that enables deep inspection of industry-scale deep neural networks by unifying instance-level and subset-level exploration through coordinated views of model architecture and neuron activations. It supports scalable, flexible analysis of large datasets and models, helping practitioners identify misclassification causes and optimize model design through visual pattern discovery.

ABSTRACT

While deep learning models have achieved state-of-the-art accuracies for many prediction tasks, understanding these models remains a challenge. Despite the recent interest in developing visual tools to help users interpret deep learning models, the complexity and wide variety of models deployed in industry, and the large-scale datasets that they used, pose unique design challenges that are inadequately addressed by existing work. Through participatory design sessions with over 15 researchers and engineers at Facebook, we have developed, deployed, and iteratively improved ActiVis, an interactive visualization system for interpreting large-scale deep learning models and results. By tightly integrating multiple coordinated views, such as a computation graph overview of the model architecture, and a neuron activation view for pattern discovery and comparison, users can explore complex deep neural network models at both the instance- and subset-level. ActiVis has been deployed on Facebook's machine learning platform. We present case studies with Facebook researchers and engineers, and usage scenarios of how ActiVis may work with different models.

Motivation & Objective

• Address the challenge of interpreting complex, large-scale deep neural networks used in industry, where models are often treated as black boxes.
• Overcome limitations of existing visualization tools that focus narrowly on image data or lack support for diverse data types and scalable datasets.
• Design a system that supports both instance-level and subset-level exploration to enhance model interpretability and debugging efficiency.
• Ensure the tool is flexible and generalizable across diverse models and datasets used in real-world machine learning platforms like Facebook's.
• Integrate high-level model architecture overviews with detailed activation analysis to support both broad understanding and granular inspection.

Proposed method

• Employ a coordinated visualization interface with a computation graph overview to represent model architecture and enable navigation.
• Use a neuron activation matrix view to visualize how neurons respond to specific instances or user-defined subsets of data.
• Support 2D projection of instance activations to enable high-level pattern discovery and comparison across classes.
• Allow users to define subsets using raw data attributes, transformed features, or output results, enabling flexible, multi-angle analysis.
• Integrate real-time interaction for selecting, comparing, and drilling down into individual instances and activation patterns.
• Tighten coupling between the model architecture view and activation views to enable seamless navigation from high-level structure to localized neuron behavior.

Experimental results

Research questions

• RQ1 How can visual analytics tools support effective interpretation of large-scale deep neural networks in industrial settings?
• RQ2 What design patterns enable both instance-level and subset-level exploration of deep learning models in a unified interface?
• RQ3 How can visualization systems scale to handle the complexity and size of industrial datasets and models?
• RQ4 What role do coordinated views play in enhancing user understanding of model behavior and performance?
• RQ5 How can visual tools inform model training and hyperparameter tuning decisions in practice?

Key findings

• ActiVis enables users to identify underutilized neurons by observing sparse activation patterns, suggesting potential for model simplification.
• The system revealed that similar activation patterns across subsets can highlight predictive neurons, aiding in understanding model behavior.
• Case studies showed that users could detect misclassification causes by comparing activation patterns across different instance subsets.
• The integration of model architecture visualization with activation analysis reduced cognitive load and accelerated model debugging.
• Engineers reported that ActiVis helped them make informed decisions about hyperparameter tuning, such as reducing redundant neurons.
• The tool was successfully deployed on Facebook’s machine learning platform and used across multiple teams for model interpretation and optimization.