[Paper Review] Towards Accurate Predictions and Causal 'What-if' Analyses for Planning and Policy-making: A Case Study in Emergency Medical Services Demand

Kasun Bandara, Christoph Bergmeir+3 more|arXiv (Cornell University)|Apr 25, 2020

Forecasting Techniques and Applications33 references4 citations

TL;DR

This paper proposes DeepPPMNet, a global LSTM-based forecasting framework that leverages cross-series information across multiple Local Government Areas (LGAs) to improve emergency medical services (EMS) demand forecasting. By integrating seasonal decomposition and Granger causality, it enables accurate causal 'what-if' analyses—demonstrated by showing that a 10% increase in alcohol outlet licenses could raise predicted EMS demand by up to 15% in certain LGAs, outperforming state-of-the-art univariate models.

ABSTRACT

Emergency Medical Services (EMS) demand load has become a considerable burden for many government authorities, and EMS demand is often an early indicator for stress in communities, a warning sign of emerging problems. In this paper, we introduce Deep Planning and Policy Making Net (DeepPPMNet), a Long Short-Term Memory network based, global forecasting and inference framework to forecast the EMS demand, analyse causal relationships, and perform `what-if' analyses for policy-making across multiple local government areas. Unless traditional univariate forecasting techniques, the proposed method follows the global forecasting methodology, where a model is trained across all the available EMS demand time series to exploit the potential cross-series information available. DeepPPMNet also uses seasonal decomposition techniques, incorporated in two different training paradigms into the framework, to suit various characteristics of the EMS related time series data. We then explore causal relationships using the notion of Granger Causality, where the global forecasting framework enables us to perform `what-if' analyses that could be used for the national policy-making process. We empirically evaluate our method, using a set of EMS datasets related to alcohol, drug use and self-harm in Australia. The proposed framework is able to outperform many state-of-the-art techniques and achieve competitive results in terms of forecasting accuracy. We finally illustrate its use for policy-making in an example regarding alcohol outlet licenses.

Motivation & Objective

• To address the limitations of univariate forecasting models that ignore cross-series patterns in EMS demand across multiple local government areas (LGAs).
• To develop a unified global forecasting framework capable of capturing shared temporal patterns and structural relationships in multi-series EMS demand data.
• To enable causal inference using Granger Causality to identify external factors—like alcohol outlet licenses—that influence EMS demand.
• To support evidence-based policy-making through actionable 'what-if' scenario analyses, such as the impact of changing alcohol licensing regulations.
• To demonstrate the framework's superiority over state-of-the-art univariate methods in forecasting accuracy and policy-relevant insight generation.

Proposed method

• DeepPPMNet employs a global forecasting methodology (GFM) that trains a single Long Short-Term Memory (LSTM) network across all available EMS demand time series, exploiting shared patterns across LGAs.
• The framework incorporates two seasonal decomposition training paradigms—Seasonal-Embedded (SE) and Decomposed-Seasonal (DS)—to handle varying seasonal and trend characteristics in EMS data.
• Granger Causality is applied to assess whether external factors (e.g., alcohol outlet licenses) improve forecasting performance when included as exogenous variables.
• The model integrates external factors such as alcohol outlet licenses (ALI) as exogenous inputs to evaluate their causal influence on alcohol-related (AO) EMS demand.
• The framework supports 'what-if' scenario analysis by simulating changes in external factors (e.g., ±5% and ±10% changes in ALI) and measuring resulting shifts in predicted EMS demand.
• Model performance is evaluated using symmetric mean absolute percentage error (sMAPE) and mean absolute scaled error (MASE), with significance testing via Hochberg’s method.

Experimental results

Research questions

• RQ1 Can a global LSTM-based framework outperform univariate forecasting models in predicting EMS demand across multiple LGAs?
• RQ2 To what extent do external factors such as alcohol outlet licenses (ALI) Granger-cause alcohol-related EMS demand?
• RQ3 How do changes in ALI affect predicted EMS demand in different LGAs, and can this inform policy decisions?
• RQ4 Can the integration of seasonal decomposition techniques improve forecasting accuracy in heterogeneous EMS demand time series?
• RQ5 What is the predictive value of including external factors like ALI in a global forecasting model for policy-relevant 'what-if' analyses?

Key findings

• DeepPPMNet-SE-License achieved the lowest mean sMAPE (0.1411) and median sMAPE (0.1334), outperforming all other models, including Prophet and ARIMAX.
• The DeepPPMNet-DS-License model recorded the highest significance in Granger causality testing, with a p-value of 1.729 × 10⁻⁴, indicating strong evidence that ALI influences AO-related EMS demand.
• A 10% increase in alcohol outlet licenses led to a predicted 15% rise in EMS demand in LGA-1, while a 10% decrease reduced demand by a similar margin, demonstrating sensitivity to policy levers.
• In LGA-2, increasing ALI had minimal impact on predicted demand, suggesting regional heterogeneity in the effect of alcohol licensing on EMS utilization.
• The framework successfully identified ALI as a significant causal factor for AO-related EMS demand, with statistical significance confirmed via Hochberg’s method (p < 0.05 for ARIMA, ETS, and DeepPPMNet-DS).
• The use of global modeling with cross-series information significantly improved forecasting accuracy compared to univariate approaches, as evidenced by consistently lower sMAPE and MASE values across all metrics.