[Paper Review] Juxtaposing Controlled Empirical Studies in Visualization with Topic Developments in Psychology

Alfie Abdul‐Rahman, Rita Borgo+3 more|arXiv (Cornell University)|Sep 6, 2019

Data Visualization and Analytics204 references4 citations

TL;DR

This paper conducts a comparative analysis of controlled empirical studies in visualization against topic developments in psychology from 1978 to 2017, revealing significant overlaps in perceptual and cognitive phenomena. It proposes a cross-disciplinary framework to bridge visualization and psychology, identifying shared research themes and opportunities for joint discovery in perception, cognition, and visualization effectiveness.

ABSTRACT

Empirical studies form an integral part of visualization research. Not only can they facilitate the evaluation of various designs, techniques, systems, and practices in visualization, but they can also enable the discovery of the causalities explaining why and how visualization works. This state-of-the-art report focuses on controlled and semi-controlled empirical studies conducted in laboratories and crowd-sourcing environments. In particular, the survey provides a taxonomic analysis of over 129 empirical studies in the visualization literature. It juxtaposes these studies with topic developments between 1978 and 2017 in psychology, where controlled empirical studies have played a predominant role in research. To help appreciate this broad context, the paper provides two case studies in detail, where specific visualization-related topics were examined in the discipline of psychology as well as the field of visualization. Following a brief discussion on some latest developments in psychology, it outlines challenges and opportunities in making new discoveries about visualization through empirical studies.

Motivation & Objective

• To identify and analyze the landscape of controlled empirical studies in visualization from 1978 to 2017.
• To map topic developments in psychology over the same period to identify shared research themes with visualization.
• To explore whether key perceptual and cognitive phenomena in visualization have been previously studied in psychology.
• To highlight opportunities for visualization researchers to leverage psychological research for causal discovery in visualization.
• To advocate for the development of a new interdisciplinary field: visualization psychology, integrating methods and findings from both domains.

Proposed method

• Conducted a taxonomic analysis of 129 controlled and semi-controlled empirical studies from the visualization literature.
• Constructed a high-level taxonomy of psychology topics from 1978 to 2017 using literature review and thematic clustering.
• Juxtaposed visualization studies with psychology topic developments using visual analytics techniques to identify thematic overlaps.
• Performed two in-depth case studies on visualization-related topics (e.g., visual averaging, attention limits) in both psychology and visualization literature.
• Mapped psychological research methodologies (e.g., fMRI, eye tracking, controlled experiments) to visualization research practices.
• Identified gaps and synergies between psychological research on perception and cognition and visualization-specific empirical studies.

Experimental results

Research questions

• RQ1 To what extent do controlled empirical studies in visualization overlap with topic developments in psychology between 1978 and 2017?
• RQ2 Which perceptual and cognitive phenomena in visualization have already been studied in psychology, and how do they align?
• RQ3 What are the key methodological and conceptual synergies between controlled empirical studies in psychology and visualization?
• RQ4 How can visualization research benefit from psychological research in terms of theory development and experimental design?
• RQ5 What opportunities exist for establishing a new interdisciplinary field—visualization psychology—through cross-domain collaboration?

Key findings

• A significant number of perceptual and cognitive phenomena studied in visualization—such as visual averaging, attention limits, and correlation perception—have deep roots in psychological research.
• The study identified 129 controlled empirical studies in visualization, many of which align with long-standing psychological research on perception, attention, and cognition.
• Psychological research on topics like mindfulness and cognitive neuroscience shows potential for enhancing visualization task performance and user engagement.
• Functional neuroimaging techniques such as fMRI and PET have been underutilized in visualization research, despite their potential for studying visualization-related brain functions.
• Visualization researchers can leverage psychological methodologies to design more rigorous, theory-driven empirical studies that go beyond usability evaluation.
• The authors advocate for the creation of a new field—visualization psychology—by integrating psychological theories and visualization technologies to advance causal understanding of visualization effectiveness.