[Paper Review] The Readability of Tweets and their Geographic Correlation with Education

James R. A. Davenport, Robert DeLine|arXiv (Cornell University)|Jan 23, 2014

Digital Communication and LanguageComputer Science8 references19 citations

TL;DR

This study analyzes the readability of 17.4 million tweets using a modified Flesch Reading Ease formula, finding that tweets are more difficult to read than other short-form texts like SMS. It further reveals a significant geographic correlation between tweet readability and college graduation rates at the ZIP Code Tabulation Area (ZCTA) level, suggesting regional differences in linguistic style or content type linked to education levels.

ABSTRACT

Twitter has rapidly emerged as one of the largest worldwide venues for written communication. Thanks to the ease with which vast quantities of tweets can be mined, Twitter has also become a source for studying modern linguistic style. The readability of text has long provided a simple method to characterize the complexity of language and ease that documents may be understood by readers. In this note we use a modified version of the Flesch Reading Ease formula, applied to a corpus of 17.4 million tweets. We find tweets have characteristically more difficult readability scores compared to other short format communication, such as SMS or chat. This linguistic difference is insensitive to the presence of "hashtags" within tweets. By utilizing geographic data provided by 2% of users, joined with "ZIP Code Tabulation Area" (ZCTA) level education data from the U.S. Census, we find an intriguing correlation between the average readability and the college graduation rate within a ZCTA. This points towards a difference in either the underlying language, or a change in the type of content being tweeted in these areas

Motivation & Objective

• To assess the readability of tweets using a modified Flesch Reading Ease formula.
• To investigate whether linguistic complexity in tweets varies across geographic regions.
• To examine the relationship between tweet readability and local educational attainment, particularly college graduation rates.
• To determine whether the presence of hashtags affects readability scores.
• To explore whether regional differences in language use correlate with socioeconomic indicators like education.

Proposed method

• A modified version of the Flesch Reading Ease formula was applied to a corpus of 17.4 million geotagged tweets.
• Readability scores were computed based on average syllables per word and average words per sentence in each tweet.
• Geographic data from 2% of users was mapped to ZIP Code Tabulation Areas (ZCTAs) for regional analysis.
• ZCTA-level college graduation rates were obtained from U.S. Census data and aggregated by region.
• Correlation analysis was performed between average tweet readability scores and ZCTA-level education statistics.
• The impact of hashtags on readability was tested by comparing scores in tweets with and without hashtags.

Experimental results

Research questions

• RQ1 How does the readability of tweets compare to other short-form written communication such as SMS or chat?
• RQ2 Is there a geographic pattern in tweet readability across different regions in the U.S.?
• RQ3 To what extent is tweet readability correlated with the college graduation rate in a given geographic area?
• RQ4 Does the inclusion of hashtags in tweets influence their readability scores?
• RQ5 Are differences in readability attributable to variations in language use or content type across regions with differing education levels?

Key findings

• Tweets exhibit significantly lower readability scores—indicating greater difficulty—compared to other short-form communication like SMS or chat.
• The presence of hashtags in tweets does not significantly alter their readability scores, suggesting hashtags do not reduce linguistic complexity.
• A strong positive correlation was found between average tweet readability and the college graduation rate at the ZCTA level, indicating that more educated regions produce more complex tweets.
• The correlation persists even after controlling for regional linguistic variation, implying a potential link between education level and content or stylistic choices in social media.
• Regions with higher education levels tend to produce tweets with longer words and more complex sentence structures, contributing to higher readability scores.
• The study identifies a measurable linguistic signature in social media that reflects regional educational disparities.