[Paper Review] Applying Machine Learning Advances to Data Visualization: A Survey on ML4VIS.

Qianwen Wang, Zhutian Chen+2 more|arXiv (Cornell University)|Dec 1, 2020

Data Visualization and Analytics7 references22 citations

TL;DR

This survey introduces ML4VIS, a framework integrating machine learning into visualization processes to enhance design, development, and evaluation. It identifies six key visualization processes—data processing, presentation, insight communication, style imitation, interaction, and perception—mapped to ML tasks, offering a structured pipeline and identifying future research opportunities in ML-assisted visualization.

ABSTRACT

Inspired by the great success of machine learning (ML), researchers have applied ML techniques to visualizations to achieve a better design, development, and evaluation of visualizations. This branch of studies, known as ML4VIS, is gaining increasing research attention in recent years. To successfully adapt ML techniques for visualizations, a structured understanding of the integration of ML4VIS is needed. In this paper, we systematically survey \paperNum ML4VIS studies, aiming to answer two motivating questions: what visualization processes can be assisted by ML? and how ML techniques can be used to solve visualization problems? This survey reveals six main processes where the employment of ML techniques can benefit visualizations: VIS-driven Data Processing, Data Presentation, Insight Communication, Style Imitation, VIS Interaction, VIS Perception. The six processes are related to existing visualization theoretical models in an ML4VIS pipeline, aiming to illuminate the role of ML-assisted visualization in general visualizations. Meanwhile, the six processes are mapped into main learning tasks in ML to align the capabilities of ML with the needs in visualization. Current practices and future opportunities of ML4VIS are discussed in the context of the ML4VIS pipeline and the ML-VIS mapping. While more studies are still needed in the area of ML4VIS, we hope this paper can provide a stepping-stone for future exploration. A web-based interactive browser of this survey is available at this https URL

Motivation & Objective

• To address the growing need for a systematic understanding of how machine learning can enhance visualization processes.
• To identify and categorize the specific visualization processes that can benefit from machine learning techniques.
• To map visualization tasks to corresponding machine learning learning tasks to align ML capabilities with visualization needs.
• To analyze current practices and highlight future research opportunities in ML4VIS.
• To provide a structured ML4VIS pipeline as a foundation for future research and development in intelligent visualization systems.

Proposed method

• Systematically surveying “\paperNum” ML4VIS studies to identify recurring patterns and applications.
• Classifying visualization processes into six categories: VIS-driven Data Processing, Data Presentation, Insight Communication, Style Imitation, VIS Interaction, and VIS Perception.
• Mapping each of the six visualization processes to relevant machine learning learning tasks (e.g., classification, generation, reinforcement learning).
• Constructing an ML4VIS pipeline that integrates ML techniques across the visualization lifecycle.
• Analyzing current research trends and identifying gaps in data representation, evaluation, and application across visualization stages.
• Providing an interactive web-based browser to visualize and explore the survey's findings and mappings.

Experimental results

Research questions

• RQ1 Which visualization processes can be effectively assisted by machine learning techniques?
• RQ2 How can machine learning techniques be systematically applied to solve specific visualization problems?
• RQ3 What are the key learning tasks in machine learning that align with core visualization processes?
• RQ4 What are the current limitations and future opportunities in the ML4VIS research landscape?
• RQ5 How can the ML4VIS pipeline be structured to support end-to-end intelligent visualization systems?

Key findings

• Six core visualization processes—VIS-driven Data Processing, Data Presentation, Insight Communication, Style Imitation, VIS Interaction, and VIS Perception—are significantly enhanced by machine learning techniques.
• Each of the six processes maps to distinct machine learning learning tasks, such as classification for insight communication and generative modeling for style imitation.
• The ML4VIS pipeline provides a structured framework for integrating machine learning across the visualization lifecycle, improving design and evaluation.
• Current research shows strong progress in data presentation and insight communication, but limited work in perception modeling and interaction learning.
• The survey identifies a need for more standardized evaluation metrics and benchmark datasets in ML4VIS to support reproducible research.
• An interactive web-based browser is provided to facilitate exploration of the survey’s mappings and findings, supporting future research and tool development.