[Paper Review] The Machine Learning Canvas: Empirical Findings on Why Strategy Matters More Than AI Code Generation

Martin Prause|arXiv (Cornell University)|Jan 5, 2026

Ethics and Social Impacts of AI0 citations

TL;DR

The paper introduces the Machine Learning Canvas and shows that strategic factors—Strategy, Process, Ecosystem, and Support—drive ML project success, with organizational support influencing strategy and subsequent workflow and infrastructure; AI code generation speeds coding but does not guarantee success.

ABSTRACT

Despite the growing popularity of AI coding assistants, over 80% of machine learning (ML) projects fail to deliver real business value. This study creates and tests a Machine Learning Canvas, a practical framework that combines business strategy, software engineering, and data science in order to determine the factors that lead to the success of ML projects. We surveyed 150 data scientists and analyzed their responses using statistical modeling. We identified four key success factors: Strategy (clear goals and planning), Process (how work gets done), Ecosystem (tools and infrastructure), and Support (organizational backing and resources). Our results show that these factors are interconnected - each one affects the next. For instance, strong organizational support results in a clearer strategy (β= 0.432, p < 0.001), which improves work processes (β= 0.428, p < 0.001) and builds better infrastructure (β= 0.547, p < 0.001). Together, these elements determine whether a project succeeds. The surprising finding? Although AI assistants make coding faster, they don't guarantee project success. AI assists with the "how" of coding but cannot replace the "why" and "what" of strategic thinking.

Motivation & Objective

• Motivate the need for an integrated framework that connects business strategy with ML delivery.
• Develop and test the Machine Learning Canvas as a practical framework.
• Quantify relationships among strategic factors and project success.
• Show that AI coding assistants speed coding but do not guarantee business value.

Proposed method

• Survey of 150 data scientists.
• Statistical modeling to analyze relationships between factors and project success.
• Estimation of path relationships with standardized coefficients and p-values.

Experimental results

Research questions

• RQ1 Do Strategy, Process, Ecosystem, and Support independently and jointly affect ML project success?
• RQ2 How do these factors interact to influence success outcomes?
• RQ3 Can organizational backing shape strategy and downstream processes and infrastructure?
• RQ4 What is the role of AI coding assistants in achieving project success?

Key findings

• Organizational Support strongly influences Strategy (beta = 0.432, p < 0.001).
• Strategy positively affects Process (beta = 0.428, p < 0.001).
• Strategy also improves Ecosystem/Infrastructure (beta = 0.547, p < 0.001).
• The four factors are interconnected and collectively determine project success.
• AI assistants speed coding but do not replace strategic decision-making for success.