[Paper Review] Characterizing the Influence of Features on Reading Difficulty Estimation for Non-native Readers

Yi-Ting Huang, Meng Chang Chen+1 more|arXiv (Cornell University)|Aug 29, 2018

Text Readability and Simplification26 references3 citations

TL;DR

This study proposes a reading difficulty estimation model tailored for non-native English learners by integrating lexical, syntactic, and novel psycholinguistic features such as word acquisition age, word sense from WordNet, and sentence-level syntactic parsing tree height. Using Bayesian Information Criterion (BIC) for model selection, the authors demonstrate that combining word count, word acquisition age, and parsing tree height yields superior performance over traditional models designed for native speakers.

ABSTRACT

In recent years, the number of people studying English as a second language (ESL) has surpassed the number of native speakers. Recent work have demonstrated the success of providing personalized content based on reading difficulty, such as information retrieval and summarization. However, almost all prior studies of reading difficulty are designed for native speakers, rather than non-native readers. In this study, we investigate various features for ESL readers, by conducting a linear regression to estimate the reading level of English language sources. This estimation is based not only on the complexity of lexical and syntactic features, but also several novel concepts, including the age of word and grammar acquisition from several sources, word sense from WordNet, and the implicit relation between sentences. By employing Bayesian Information Criterion (BIC) to select the optimal model, we find that the combination of the number of words, the age of word acquisition and the height of the parsing tree generate better results than alternative competing models. Thus, our results show that proposed second language reading difficulty estimation outperforms other first language reading difficulty estimations.

Motivation & Objective

• To address the gap in reading difficulty estimation tools designed specifically for non-native English readers.
• To investigate how psycholinguistic features such as word acquisition age and word sense influence reading difficulty for ESL learners.
• To evaluate the impact of syntactic complexity, including parsing tree height, on readability for non-native readers.
• To develop and validate a model that outperforms existing first-language-based reading difficulty estimators for second language learners.
• To identify the optimal combination of features for accurate reading level prediction in non-native contexts.

Proposed method

• The authors collect and analyze a set of linguistic features, including word count, syntactic parsing tree height, and lexical complexity metrics.
• They incorporate psycholinguistic data such as the age at which words and grammar structures are typically acquired, derived from external sources.
• Word sense information is extracted from WordNet to assess lexical ambiguity and its impact on comprehension difficulty.
• The implicit semantic relations between sentences are modeled to capture discourse-level complexity.
• A linear regression model is trained to predict reading level based on these features.
• The Bayesian Information Criterion (BIC) is used to select the optimal feature combination and model architecture.

Experimental results

Research questions

• RQ1 Which linguistic features most significantly influence reading difficulty for non-native English readers?
• RQ2 How do psycholinguistic factors such as word acquisition age and word sense contribute to readability estimation?
• RQ3 To what extent does syntactic complexity, measured by parsing tree height, affect reading difficulty for ESL learners?
• RQ4 Can a model combining novel features outperform traditional models designed for native speakers?
• RQ5 What is the optimal combination of features for predicting reading level in non-native reading contexts?

Key findings

• The combination of word count, word acquisition age, and parsing tree height achieved the best performance in predicting reading difficulty for non-native readers.
• The inclusion of psycholinguistic features like word acquisition age significantly improved model accuracy over models using only lexical and syntactic features.
• The model outperformed existing first-language-based reading difficulty estimators, particularly in capturing the challenges faced by second language learners.
• The Bayesian Information Criterion (BIC) identified the optimal model configuration, confirming the importance of feature selection in readability modeling.
• The integration of word sense from WordNet contributed to a more nuanced understanding of lexical difficulty in non-native contexts.
• The results indicate that syntactic parsing tree height is a strong predictor of reading difficulty, especially for learners with limited syntactic processing capacity.