[Paper Review] Large-scale Quantitative Evidence of Media Impact on Public Opinion toward China

Junming Huang, Gavin G. Cook+1 more|arXiv (Cornell University)|Dec 8, 2020

Media Influence and Politics17 references4 citations

TL;DR

This study uses BERT-based NLP to analyze 267,907 China-related New York Times articles (1970–2019), linking media sentiment to public opinion. It finds that coverage on Chinese culture and democracy in one year explains 53.9% of variance in American public opinion toward China the following year.

ABSTRACT

Do mass media influence people's opinion of other countries? Using BERT, a deep neural network-based natural language processing model, we analyze a large corpus of 267,907 China-related articles published by The New York Times since 1970. We then compare our output from The New York Times to a longitudinal data set constructed from 101 cross-sectional surveys of the American public's views on China. We find that the reporting of The New York Times on China in one year explains 54% of the variance in American public opinion on China in the next. Our result confirms hypothesized links between media and public opinion and helps shed light on how mass media can influence public opinion of foreign countries.

Motivation & Objective

• To investigate whether elite media coverage, specifically The New York Times, influences American public opinion toward China.
• To quantify media sentiment toward China across multiple thematic domains using state-of-the-art NLP techniques.
• To establish a temporal link between media sentiment and public opinion using longitudinal survey data.
• To assess the relative explanatory power of different media sentiment topics on public opinion trends.
• To provide large-scale empirical evidence of media impact on foreign policy attitudes, despite limitations in establishing causality.

Proposed method

• Employed BERT, a deep neural network-based NLP model, to extract sentiment from 267,907 China-related New York Times articles published between 1970 and 2019.
• Segmented sentiment into eight thematic topics—culture, democracy, economy, human rights, military, politics, society, and trade—each with positive, negative, and neutral sentiment scores.
• Calculated yearly fractions of positive and negative sentiment for each topic to create time-series predictors.
• Used linear regression to model public opinion as a function of prior-year media sentiment, with the final model including culture (positive) and democracy (negative) sentiment.
• Applied a greedy model selection strategy to identify the most informative sentiment topics, achieving maximum R² at eight topics.
• Validated model predictions against actual survey data using a case example (1993–1994), showing strong predictive accuracy.

Experimental results

Research questions

• RQ1 To what extent does The New York Times’ media sentiment on China predict changes in American public opinion toward China?
• RQ2 Which specific thematic domains of media coverage (e.g., culture, democracy, economy) have the strongest influence on public opinion?
• RQ3 How does the explanatory power of media sentiment vary when segmented by topic, and at what point does adding more topics yield diminishing returns?
• RQ4 Can sentiment from elite media outlets like The New York Times serve as a leading indicator of shifts in public opinion toward foreign nations?
• RQ5 How does the temporal lag between media coverage and public opinion manifest, and what does this imply about media influence?

Key findings

• Media sentiment from The New York Times on China in one year explains 53.9% of the variance in American public opinion toward China in the following year.
• The sentiment on Chinese culture, particularly positive coverage, has the highest explanatory power, accounting for 31.2% of the variance in public opinion.
• The inclusion of negative sentiment on Chinese democracy further increases model explanatory power, with the combined model of culture (positive) and democracy (negative) sentiment achieving the highest R².
• The model’s predictive accuracy was validated using the 1993–1994 data point, where the predicted public opinion score (0.236) closely matched the actual value (0.218).
• Explanatory power increased monotonically with added topics but showed diminishing returns after two topics were included, indicating saturation in predictive gain.
• The final linear predictor model is μₜ = -0.791 + 3.112×F_{culture,t-1,positive} + 1.452×F_{democracy,t-1,negative}, demonstrating a strong, statistically significant relationship between media sentiment and public opinion.