[Paper Review] Visual Interaction with Deep Learning Models through Collaborative Semantic Inference

Sebastian Gehrmann, Hendrik Strobelt+3 more|arXiv (Cornell University)|Jul 24, 2019

Data Visualization and Analytics4 citations

TL;DR

This paper proposes Collaborative Semantic Inference (CSI), a framework that enables visual interaction with deep learning models by exposing their internal reasoning through model-extended latent variables (hooks). By integrating user feedback into the model's inference process via intuitive visual metaphors—demonstrated in a document summarization system—CSI allows users to control, understand, and co-design model decisions, significantly improving transparency and human-AI collaboration.

ABSTRACT

Automation of tasks can have critical consequences when humans lose agency over decision processes. Deep learning models are particularly susceptible since current black-box approaches lack explainable reasoning. We argue that both the visual interface and model structure of deep learning systems need to take into account interaction design. We propose a framework of collaborative semantic inference (CSI) for the co-design of interactions and models to enable visual collaboration between humans and algorithms. The approach exposes the intermediate reasoning process of models which allows semantic interactions with the visual metaphors of a problem, which means that a user can both understand and control parts of the model reasoning process. We demonstrate the feasibility of CSI with a co-designed case study of a document summarization system.

Motivation & Objective

• To address the lack of human agency in black-box deep learning models by enabling interactive, visual control over model reasoning.
• To co-design visualization and model architectures that expose intermediate inference steps in a way aligned with human mental models.
• To develop a framework where users can influence model decisions through intuitive visual interactions, such as selecting or deselecting content.
• To demonstrate the feasibility of CSI in a real-world NLP task—document summarization—where human post-editing is typically required.
• To establish a systematic co-design process for interactive, explainable AI systems that integrate visualization, interaction design, and machine learning.

Proposed method

• Introduces discrete latent variables (hooks) as 'control points' in the model to represent intermediate reasoning decisions, such as content importance in summarization.
• Employs a hook network that predicts tag probabilities $ p(t|x) $, determining which input words the model can copy into the summary, enabling user intervention via priors.
• Uses a separate backward inference model to compute $ p(t_i|x,y) $, estimating the likelihood that each input word was used in the summary, enabling post-hoc analysis.
• Applies contextualized word representations and attention mechanisms to compute context vectors $ c_i $ for each input word based on its alignment with summary words.
• Models the probability of word usage via a differentiable head: $ p(t_i|x,y) = \sigma(W_2 \tanh(W_1[x_i, c_i] + b_1) + b_2) $, trained independently from the forward model.
• Designs a visual interface where users can visually deselect sentences, which sets $ p(t) = 0 $ for all words in those sentences, thereby preventing the model from copying them.

Experimental results

Research questions

• RQ1 How can deep learning models be extended to expose intermediate reasoning steps in a way that aligns with human cognitive processes?
• RQ2 What visual interaction paradigms enable users to meaningfully influence and understand model decisions during inference?
• RQ3 How can user feedback be semantically mapped to model internals to enable collaborative decision-making?
• RQ4 Can a co-designed interface for summarization improve transparency and reduce post-editing effort by exposing model reasoning?
• RQ5 What are the technical and design challenges in building a system that supports both forward and backward inference for interactive model interpretation?

Key findings

• The CSI framework successfully enables users to visually control model behavior in document summarization by deselecting input sentences, which prevents the model from copying content from those segments.
• The backward inference model accurately identifies which input words were used in a summary, even for user-written summaries, by computing $ p(t_i|x,y) $ with a learned attention-based mechanism.
• The integration of latent hooks into the summarization model allows for what-if analysis and semantic interaction, such as testing how model outputs change under different user priors.
• The system demonstrates that exposing intermediate reasoning through hooks improves model interpretability and supports collaborative human-AI interaction in NLP tasks.
• The co-design process reveals that effective CSI requires tight integration between visualization, interaction design, and model architecture, particularly in aligning user actions with model internals.