[Paper Review] Overspend? Late? Failure? What the Data Say About IT Project Risk in the Public Sector

Alexander Budzier, Bent Flyvbjerg|arXiv (Cornell University)|Apr 16, 2013

Big Data and Business Intelligence16 citations

TL;DR

This paper analyzes 1,355 public sector IT projects to assess risk in cost and schedule performance, revealing that 18% exceed 25% cost overruns despite normative expectations of near-99% compliance. It identifies duration and project type as key risk drivers and proposes four evidence-based solutions: benchmarking, de-biasing decision-making, reducing complexity, and developing 'Masterbuilders' to improve project outcomes.

ABSTRACT

Implementing large-scale information and communication technology (IT) projects carries large risks and easily might disrupt operations, waste taxpayers' money, and create negative publicity. Because of the high risks it is important that government leaders manage the attendant risks. We analysed a sample of 1,355 public sector IT projects. The sample included large-scale projects, on average the actual expenditure was $130 million and the average duration was 35 months. Our findings showed that the typical project had no cost overruns and took on average 24% longer than initially expected. However, comparing the risk distribution with the normative model of a thin-tailed distribution, projects' actual costs should fall within -30% and +25% of the budget in nearly 99 out of 100 projects. The data showed, however, that a staggering 18% of all projects are outliers with cost overruns >25%. Tests showed that the risk of outliers is even higher for standard software (24%) as well as in certain project types, e.g., data management (41%), office management (23%), eGovernment (21%) and management information systems (20%). Analysis showed also that projects duration adds risk: every additional year of project duration increases the average cost risk by 4.2 percentage points. Lastly, we suggest four solutions that public sector organization can take: (1) benchmark your organization to know where you are, (2) de-bias your IT project decision-making, (3) reduce the complexities of your IT projects, and (4) develop Masterbuilders to learn from the best in the field.

Motivation & Objective

• To assess the actual risk profile of large-scale public sector IT projects in terms of cost overruns and schedule delays.
• To identify factors that significantly increase the likelihood of cost overruns and schedule slippage.
• To compare observed risk distributions with normative thin-tailed statistical models to expose systemic deviations.
• To develop actionable, data-driven recommendations for public sector organizations to reduce IT project failure rates.

Proposed method

• Analysis of a dataset of 1,355 public sector IT projects with average expenditure of $130 million and average duration of 35 months.
• Application of statistical modeling to compare actual cost and schedule outcomes against a normative thin-tailed distribution model.
• Categorization of projects by type (e.g., data management, eGovernment) to assess risk variation across domains.
• Quantitative analysis of duration as a risk factor, measuring incremental cost risk per additional year of project length.
• Benchmarking of organizational performance to identify high-performing entities.
• Development of four practical risk mitigation strategies based on empirical findings and behavioral insights.

Experimental results

Research questions

• RQ1 What is the actual distribution of cost overruns and schedule delays in large public sector IT projects compared to normative expectations?
• RQ2 Which project types exhibit the highest risk of cost overruns and schedule slippage?
• RQ3 How does project duration correlate with increased cost risk?
• RQ4 To what extent do standard software projects differ in risk profile from custom development projects?
• RQ5 What evidence-based organizational practices can reduce the likelihood of IT project failure in the public sector?

Key findings

• 18% of public sector IT projects are outliers with cost overruns exceeding 25%, significantly exceeding the 1% expected under a normative thin-tailed distribution.
• On average, projects take 24% longer than initially planned, though the typical project shows no cost overrun.
• Data management projects have the highest risk, with 41% exceeding 25% cost overruns, followed by office management (23%) and eGovernment (21%).
• Each additional year of project duration increases average cost risk by 4.2 percentage points.
• Standard software projects have a 24% outlier rate for cost overruns, indicating higher-than-expected risk.
• The study identifies four practical solutions: benchmarking, de-biasing decision-making, reducing project complexity, and developing 'Masterbuilders' to institutionalize best practices.