[Paper Review] Mapping the Economic Crisis: Some Preliminary Investigations

Pierre Bourreau, Thierry Poibeau|arXiv (Cornell University)|Jun 17, 2014

Advanced Text Analysis Techniques9 references3 citations

TL;DR

This paper presents a natural language processing pipeline to map the 2007–2008 financial crisis using named entity recognition, normalization, and dynamic visualization. By applying Stanford NER and clustering rules to congressional reports, it constructs time-resolved semantic networks and Sankey diagrams that reveal shifting actor roles and topic associations—especially the transformation of Fannie Mae’s associations from subprime loans to regulators post-2007.

ABSTRACT

In this paper we describe our contribution to the PoliInformatics 2014 Challenge on the 2007-2008 financial crisis. We propose a state of the art technique to extract information from texts and provide different representations, giving first a static overview of the domain and then a dynamic representation of its main evolutions. We show that this strategy provides a practical solution to some recent theories in social sciences that are facing a lack of methods and tools to automatically extract information from natural language texts.

Motivation & Objective

• To address the challenge of extracting and visualizing complex socio-economic dynamics from unstructured textual data on the 2007–2008 financial crisis.
• To overcome methodological gaps in social science research that lack scalable tools for automatic analysis of natural language texts.
• To provide interactive, dynamic visualizations that reveal temporal shifts in key actors, institutions, and topics without requiring full manual reading of documents.
• To evaluate and compare normalization strategies for named entities (organizations and persons) to improve consistency in entity clustering.
• To demonstrate how NLP techniques can support expert-driven corpus exploration by generating interpretable, interactive visual maps of crisis evolution.

Proposed method

• Used Stanford Named Entity Recognizer (CRF-based) with MUC tags (Person, Organization, Date, etc.) to extract entities from three congressional reports: AoC, SMP, and FCIC.
• Applied rule-based normalization: for organizations, matching by initials or hierarchical inclusion; for persons, matching by name substring or last name overlap.
• Implemented two clustering strategies—P_Max (based on most frequent entities) and P_Av (based on average frequency threshold)—with P_Av selected as most effective.
• Constructed static co-occurrence graphs using Gephi to visualize relationships between named entities and topics.
• Built dynamic Sankey diagrams using Cortext and Gephi to track temporal evolution of entity-topic associations across time periods (e.g., 1990–2007 vs. 2008–2010).
• Integrated interactive features into visualizations to allow users to explore term co-occurrences and shifts in associations across time intervals.

Experimental results

Research questions

• RQ1 How can named entities and their temporal associations be systematically extracted from unstructured crisis-related reports?
• RQ2 What normalization rules best resolve variations in entity representation (e.g., 'S&P' vs. 'Standard & Poor') without relying on complex models?
• RQ3 How do the relationships between key actors (e.g., Fannie Mae, regulators) and topics (e.g., subprime loans, bank failures) evolve over time during the crisis?
• RQ4 Can dynamic visualizations such as Sankey diagrams effectively reveal structural shifts in discourse and institutional roles during the crisis?
• RQ5 To what extent can NLP-based tools support expert navigation and interpretation of large textual corpora in social science research?

Key findings

• The P_Av normalization strategy outperformed P_Max in clustering consistency and was selected for final analysis.
• Fannie Mae’s co-occurrence profile shifted dramatically after 2007, transitioning from 'subprime loans' to 'bank regulators' and 'Federal Financial Services Supervisory Authority'.
• The Sankey diagram for FCIC revealed a major structural shift in 2007–2008, with a clear reconfiguration of entity-topic associations across all streams.
• The AoC diagram similarly showed a sharp change in data flow after 2007, with the default of Washington Mutual Bank (WaMu) clearly marked as a pivotal event.
• Dynamic visualizations enabled immediate identification of key events and evolving narratives without full-text reading, demonstrating utility for expert exploration.
• Interactive visualizations allowed users to inspect term co-occurrences between time periods, revealing how entity roles and perceptions changed over time.