[Paper Review] The debt aversion survey module: An experimentally validated tool to measure individual debt aversion

David Albrecht, Thomas Meißner|arXiv (Cornell University)|Nov 4, 2022

Economic and Environmental Valuation4 citations

TL;DR

This paper introduces a two-question survey module that experimentally validates a brief, reliable, and easy-to-implement tool for measuring individual debt aversion. Using structural estimation from incentivized experiments and multivariate regression, the module achieves a 0.3073 correlation with experimentally measured debt aversion and predicts choices with 89.76% accuracy—comparable to full structural estimation—making it ideal for large-scale surveys.

ABSTRACT

We develop an experimentally validated, short and easy-to-use survey module for measuring individual debt aversion. To this end, we first estimate debt aversion on an individual level, using choice data from Meissner and Albrecht (2022). This data also contains responses to a large set of debt aversion survey items, consisting of existing items from the literature and novel items developed for this study. Out of these, we identify a survey module comprising two qualitative survey items to best predict debt aversion in the incentivized experiment.

Motivation & Objective

• To develop a short, practical, and experimentally validated survey tool for measuring individual debt aversion.
• To overcome the limitations of costly, time-intensive incentivized experiments in large-scale studies of debt aversion.
• To identify a minimal set of survey items that best predict experimentally measured debt aversion while balancing brevity and predictive power.
• To enable researchers to control for debt aversion in non-experimental settings, improving causal inference in behavioral economics research.
• To provide a benchmark tool comparable to the GPS survey module but tailored specifically to debt aversion preferences.

Proposed method

• Structural estimation of individual debt aversion using hierarchical maximum likelihood on choice data from an incentivized experiment (Meissner & Albrecht, 2022).
• Collection of 100+ survey items on debt aversion, including existing literature items and new, experimentally developed items.
• Multivariate OLS regression to predict the experimentally estimated debt aversion parameter (γ) from survey responses.
• Model selection using cross-validation (k=5 and k=10), BIC, AIC, and adjusted R² to balance predictive accuracy and model brevity.
• Use of Likert-scale responses (1 = strongly agree, 6 = strongly disagree) with estimated weights to compute a composite debt aversion score.
• Validation via k-fold cross-validation and comparison of prediction accuracy against structural estimation benchmarks.

Experimental results

Research questions

• RQ1 Which short set of survey items best predicts experimentally measured individual debt aversion?
• RQ2 How does the predictive accuracy of the survey module compare to full structural estimation in predicting actual choices?
• RQ3 What is the in-sample and out-of-sample performance of the survey module in terms of correlation and mean absolute error?
• RQ4 How does the survey module perform relative to established preference measurement tools like the GPS module?
• RQ5 Can the survey module reliably control for debt aversion in non-experimental, large-scale survey research?

Key findings

• The survey module, consisting of two Likert-scale items (Q1: 'Debt is an integral part of today’s life.' and Q2: 'There is no excuse for borrowing money.'), achieves a correlation of 0.3073 with experimentally measured debt aversion (γ).
• The module explains approximately 9.45% of the variance in the debt aversion parameter (R² = 0.0945), indicating moderate but meaningful in-sample fit.
• K-fold cross-validation yields a mean absolute prediction error of 0.0272, demonstrating strong out-of-sample predictive performance.
• Survey module-based predictions correctly anticipate actual choices in 89.76% of decision scenarios, compared to 91.48% for the full structural estimation—indicating near-equivalent predictive power.
• The module performs comparably to the GPS survey module in terms of predictive quality, despite being shorter and more focused on debt aversion.
• The module is robust to measurement error concerns, with performance comparable to established preference measures like risk and time discounting in test-retest reliability.