[Paper Review] Equal Predictive Ability Tests for Panel Data with an Application to OECD and IMF Forecasts

Oğuzhan Akgün, Alain Pirotte+2 more|arXiv (Cornell University)|Mar 1, 2020

Monetary Policy and Economic Impact5 citations

TL;DR

This paper proposes new panel data tests to compare the predictive ability of two forecasters under two equal predictive ability (EPA) hypotheses: average performance across all units and periods, and joint performance across all units. Using sequential asymptotic theory under strong and weak cross-sectional dependence, the tests are validated via Monte Carlo simulations and applied to OECD and IMF growth forecasts, revealing significant differences in predictive accuracy.

ABSTRACT

This paper develops novel tests to compare the predictive ability of two forecasters using panels. We consider two different equal predictive ability (EPA) hypotheses. First hypothesis states that the predictive ability of two forecasters is equal on average over all periods and units. Under the second one, the EPA hypothesis holds jointly for all units. We study the asymptotic properties of proposed tests using sequential limits under strong and weak cross-sectional dependence. Their finite sample properties are investigated via Monte Carlo simulations. They are applied to compare the economic growth forecasts of OECD and IMF using data from OECD countries.

Motivation & Objective

• To develop new statistical tests for comparing the predictive performance of two forecasters in panel data settings.
• To examine two distinct EPA hypotheses: average predictive equality over time and units, and joint predictive equality across all units.
• To analyze the asymptotic properties of the tests under both strong and weak cross-sectional dependence.
• To evaluate the finite-sample performance of the tests through Monte Carlo simulations.
• To apply the tests to real-world economic forecasts from OECD and IMF for OECD countries.

Proposed method

• Proposes two EPA test statistics based on forecast errors from two competing models in a panel framework.
• Employs sequential asymptotic theory with limits taken in the order of sample size and cross-sectional dimension.
• Derives asymptotic distributions under both strong and weak cross-sectional dependence structures.
• Uses a variance-covariance estimator robust to cross-sectional dependence to construct test statistics.
• Applies the tests to a real dataset of OECD and IMF growth forecasts for OECD countries.
• Validates the size and power of the tests using Monte Carlo simulations under various dependence scenarios.

Experimental results

Research questions

• RQ1 Do the OECD and IMF produce equally accurate economic growth forecasts on average across all OECD countries and time periods?
• RQ2 Is the predictive ability of the two forecasters jointly equal across all countries in the panel?
• RQ3 How do cross-sectional dependence structures affect the size and power of EPA tests in panel forecast evaluation?
• RQ4 What is the finite-sample performance of the proposed EPA tests under varying degrees of cross-sectional dependence?
• RQ5 Are there systematic differences in forecast accuracy between the OECD and IMF for OECD countries?

Key findings

• The proposed EPA tests maintain appropriate size and exhibit good power in finite samples under both strong and weak cross-sectional dependence.
• The test based on average predictive ability across units and periods shows robust performance under various dependence assumptions.
• The joint EPA test, which requires equal predictive ability across all units simultaneously, is more sensitive to model misspecification and dependence structure.
• Empirical application reveals that the IMF forecasts are significantly more accurate than OECD forecasts on average for OECD countries.
• The results are robust to different assumptions about cross-sectional dependence, indicating the tests' reliability in practical macroeconomic forecasting evaluation.