[Paper Review] Image as Data: Automated Visual Content Analysis for Political Science

Jungseock Joo, ZACHARY STEINERT-THRELKELD|arXiv (Cornell University)|Oct 3, 2018

Computational and Text Analysis Methods47 references66 citations

TL;DR

This paper introduces automated visual content analysis using computer vision and deep learning to study political phenomena at scale, and demonstrates its application with protest imagery from South Korea and Hong Kong.

ABSTRACT

Image data provide unique information about political events, actors, and their interactions which are difficult to measure from or not available in text data. This article introduces a new class of automated methods based on computer vision and deep learning which can automatically analyze visual content data. Scholars have already recognized the importance of visual data and a variety of large visual datasets have become available. The lack of scalable analytic methods, however, has prevented from incorporating large scale image data in political analysis. This article aims to offer an in-depth overview of automated methods for visual content analysis and explains their usages and implementations. We further elaborate on how these methods and results can be validated and interpreted. We then discuss how these methods can contribute to the study of political communication, identity and politics, development, and conflict, by enabling a new set of research questions at scale.

Motivation & Objective

• Motivate the shift from manual visual analysis to scalable automated methods for political imagery.
• Summarize core computer vision and deep learning techniques applicable to political science data.
• Explain validation, interpretation, and potential research applications of visual content analysis.

Proposed method

• Present an overview of computer vision and deep learning foundations relevant to image data.
• Describe artificial neural networks and convolutional neural networks (CNNs) and their training dynamics.
• Explain key CNN components (convolutional, nonlinear, pooling, and fully connected layers) and their roles in image tasks.
• Discuss training, validation, and interpretation strategies, including transfer learning and visualization methods.
• Illustrate the approach with a demonstration analyzing protest images from South Korea and Hong Kong.

Experimental results

Research questions

• RQ1 How can automated visual content analysis be applied to political science to analyze imagery at scale?
• RQ2 What are the essential deep learning techniques suitable for political image data, and how can they be trained and validated?
• RQ3 How can results from visual content analysis be interpreted and validated in political research contexts?
• RQ4 What kinds of political science questions can large-scale image data help address (e.g., protests, identity, conflict)?

Key findings

• Automated visual methods enable scalable analysis of political imagery beyond manual annotation.
• CNNs and deep learning offer powerful end-to-end learning from raw images without handcrafted features.
• Visualization and transfer learning techniques help interpret and adapt models for political tasks.
• The paper demonstrates a protest image analysis case study in South Korea and Hong Kong as a proof of concept.