[Paper Review] Big Data, Socio-Psychological Theory, Algorithmic Text Analysis and Predicting the Michigan Consumer Sentiment Index

Rickard Nyman, Paul Ormerod|arXiv (Cornell University)|May 22, 2014

Complex Systems and Time Series Analysis5 references3 citations

TL;DR

This paper proposes a novel approach to predicting the Michigan Consumer Sentiment Index using algorithmic text analysis of broker reports, guided by conviction narrative theory from socio-psychological theory. It achieves 12 correct directional predictions out of 15 one-month forecasts—significantly outperforming the Wall Street consensus forecast, which correctly predicted direction only 7 times.

ABSTRACT

We describe an exercise of using Big Data to predict the Michigan Consumer Sentiment Index, a widely used indicator of the state of confidence in the US economy. We carry out the exercise from a pure ex ante perspective. We use the methodology of algorithmic text analysis of an archive of brokers' reports over the period June 2010 through June 2013. The search is directed by the social-psychological theory of agent behaviour, namely conviction narrative theory. We compare one month ahead forecasts generated this way over a 15 month period with the forecasts reported for the consensus predictions of Wall Street economists. The former give much more accurate predictions, getting the direction of change correct on 12 of the 15 occasions compared to only 7 for the consensus predictions. We show that the approach retains significant predictive power even over a four month ahead horizon.

Motivation & Objective

• To develop a data-driven forecasting model for the Michigan Consumer Sentiment Index using Big Data sources.
• To integrate socio-psychological theory—specifically conviction narrative theory—into algorithmic text analysis for economic forecasting.
• To evaluate the predictive accuracy of this approach against established financial consensus forecasts.
• To assess the robustness of the method over extended horizons, including four-month-ahead predictions.
• To demonstrate the value of text-based sentiment from institutional reports as a leading economic indicator.

Proposed method

• Utilizes an archive of broker reports from June 2010 to June 2013 as the primary text data source.
• Applies algorithmic text analysis to extract sentiment and narrative content based on conviction narrative theory.
• Employs a pure ex ante forecasting framework, ensuring predictions are made before actual data release.
• Compares model forecasts with the consensus predictions of Wall Street economists on a month-ahead basis.
• Validates predictive power over both one-month and four-month horizons to assess long-term reliability.
• Uses statistical comparison to evaluate directional accuracy of predictions against actual index movements.

Experimental results

Research questions

• RQ1 Can algorithmic text analysis of broker reports, guided by conviction narrative theory, improve forecasts of the Michigan Consumer Sentiment Index?
• RQ2 How does the performance of this text-based model compare to the consensus forecast of Wall Street economists in predicting directional changes?
• RQ3 Does the predictive power of the model remain significant when extended to a four-month forecast horizon?
• RQ4 To what extent does socio-psychological theory enhance the interpretability and accuracy of sentiment-based economic forecasts?
• RQ5 Can institutional text data serve as a leading indicator for consumer confidence beyond traditional survey-based measures?

Key findings

• The model correctly predicted the direction of change in the Michigan Consumer Sentiment Index on 12 out of 15 one-month-ahead forecasts.
• In contrast, the Wall Street consensus forecast correctly predicted direction on only 7 of the 15 occasions.
• The model's predictive accuracy remained statistically significant even when forecasting four months ahead.
• The integration of conviction narrative theory enhanced the model's ability to capture meaningful shifts in market sentiment.
• The results demonstrate that text-based sentiment analysis from institutional reports can outperform traditional consensus forecasts in predicting consumer confidence.
• The study confirms that Big Data and socio-psychological theory can be effectively combined to improve economic forecasting accuracy.