[Paper Review] ML-based Visualization Recommendation: Learning to Recommend Visualizations from Data

Xin Qian, Ryan A. Rossi+6 more|arXiv (Cornell University)|Sep 25, 2020

Data Visualization and Analytics15 references11 citations

TL;DR

This paper presents the first end-to-end machine learning-based visualization recommendation system that learns from a large corpus of datasets and visualizations to automatically generate, score, and rank effective visualizations for new, unseen datasets. Using a wide-and-deep neural network architecture, the model outperforms rule-based systems by learning nuanced visual design preferences from data, achieving significantly higher expert preference (5.92/7 vs. 3.45) and resolving tie-breaking issues common in rule-based approaches.

ABSTRACT

Visualization recommendation seeks to generate, score, and recommend to users useful visualizations automatically, and are fundamentally important for exploring and gaining insights into a new or existing dataset quickly. In this work, we propose the first end-to-end ML-based visualization recommendation system that takes as input a large corpus of datasets and visualizations, learns a model based on this data. Then, given a new unseen dataset from an arbitrary user, the model automatically generates visualizations for that new dataset, derive scores for the visualizations, and output a list of recommended visualizations to the user ordered by effectiveness. We also describe an evaluation framework to quantitatively evaluate visualization recommendation models learned from a large corpus of visualizations and datasets. Through quantitative experiments, a user study, and qualitative analysis, we show that our end-to-end ML-based system recommends more effective and useful visualizations compared to existing state-of-the-art rule-based systems. Finally, we observed a strong preference by the human experts in our user study towards the visualizations recommended by our ML-based system as opposed to the rule-based system (5.92 from a 7-point Likert scale compared to only 3.45).

Motivation & Objective

• To address the limitations of rule-based visualization recommendation systems, which suffer from poor differentiation, rigid scoring, and high maintenance costs.
• To develop a fully data-driven, automatic, and scalable system that learns effective visualization recommendations from real user-generated visualizations.
• To overcome the fundamental issue of identical scores in rule-based systems by learning continuous, meaningful scores through deep learning.
• To create a generalizable recommendation model that can be applied to any new dataset without manual reconfiguration.
• To validate the system’s effectiveness through quantitative evaluation, user studies, and qualitative analysis.

Proposed method

• Formalize the ML-based visualization recommendation problem as a learning task over attribute combinations and visualization configurations.
• Design a wide-and-deep neural network that combines wide linear features (e.g., attribute types, layout preferences) with deep representations of data and visualization structure.
• Train the model end-to-end on a large corpus of real datasets and their associated visualizations to predict visualization effectiveness scores.
• Use the learned model to generate, score, and rank visualizations for a new, unseen dataset based on learned patterns from training data.
• Incorporate human design preferences (e.g., horizontal over vertical layouts) into the model via data-driven signal, not hard-coded rules.
• Develop a comprehensive evaluation framework to quantitatively assess ranking quality and model performance on unseen data.

Experimental results

Research questions

• RQ1 Can a data-driven, end-to-end machine learning model learn to recommend more effective visualizations than rule-based systems?
• RQ2 How well does the model generalize to new, unseen datasets without retraining or manual configuration?
• RQ3 To what extent can the model resolve the tie-breaking problem common in rule-based systems, where many visualizations receive identical scores?
• RQ4 How do human experts rate the quality and usefulness of visualizations recommended by the ML-based system versus rule-based systems?
• RQ5 What design and layout preferences can the model learn from data that are not encoded in existing rule-based systems?

Key findings

• The ML-based system achieved a mean expert preference score of 5.92 out of 7, significantly higher than the rule-based system’s 3.45, indicating strong human preference for its recommendations.
• In a user study with 20 human experts, the top-ranked visualization from the ML-based system matched the top choice of experts exactly, demonstrating high alignment with expert judgment.
• The model successfully resolved the tie-breaking issue common in rule-based systems, where many visualizations received identical scores of 0, by learning continuous, discriminative scores.
• The wide-and-deep architecture effectively captured complex visualization preferences, such as favoring horizontal over vertical layouts, which aligns with expert practices.
• Qualitative analysis revealed that the model learned subtle, non-trivial visual design rules from data that were absent in rule-based systems.
• Quantitative evaluation confirmed that the model’s ranking of visualizations was significantly more effective than baseline rule-based approaches.