[Paper Review] Evaluating Digital Agriculture Recommendations with Causal Inference

Ilias Tsoumas, Georgios Giannarakis+6 more|arXiv (Cornell University)|Nov 30, 2022

Smart Agriculture and AI4 citations

TL;DR

This paper proposes an observational causal inference framework to empirically evaluate digital agriculture recommendations, using a cotton sowing time recommendation system as a case study. By applying the back-door criterion and multiple estimation methods (linear regression, matching, IPW, meta-learners), it demonstrates a statistically significant 12–17% yield increase from following recommendations, with robust results confirmed by refutation tests.

ABSTRACT

In contrast to the rapid digitalization of several industries, agriculture suffers from low adoption of smart farming tools. While AI-driven digital agriculture tools can offer high-performing predictive functionalities, they lack tangible quantitative evidence on their benefits to the farmers. Field experiments can derive such evidence, but are often costly, time consuming and hence limited in scope and scale of application. To this end, we propose an observational causal inference framework for the empirical evaluation of the impact of digital tools on target farm performance indicators (e.g., yield in this case). This way, we can increase farmers' trust via enhancing the transparency of the digital agriculture market and accelerate the adoption of technologies that aim to secure farmer income resilience and global agricultural sustainability. As a case study, we designed and implemented a recommendation system for the optimal sowing time of cotton based on numerical weather predictions, which was used by a farmers' cooperative during the growing season of 2021. We then leverage agricultural knowledge, collected yield data, and environmental information to develop a causal graph of the farm system. Using the back-door criterion, we identify the impact of sowing recommendations on the yield and subsequently estimate it using linear regression, matching, inverse propensity score weighting and meta-learners. The results reveal that a field sown according to our recommendations exhibited a statistically significant yield increase that ranged from 12% to 17%, depending on the method. The effect estimates were robust, as indicated by the agreement among the estimation methods and four successful refutation tests. We argue that this approach can be implemented for decision support systems of other fields, extending their evaluation beyond a performance assessment of internal functionalities.

Motivation & Objective

• To address the low adoption of digital agriculture tools due to lack of empirical evidence on their real-world benefits.
• To overcome the limitations of costly and impractical field experiments by using observational causal inference to estimate causal effects.
• To enhance transparency and trust in digital agriculture by empirically validating recommendations using domain knowledge and data.
• To develop a scalable framework for evaluating decision support systems beyond internal performance metrics.
• To demonstrate the framework's viability through a real-world case study on optimal cotton sowing time using weather forecasts.

Proposed method

• Construct a causal graph (DAG) based on agricultural domain knowledge, environmental data, and yield observations to model the farm system.
• Apply the back-door criterion to identify and adjust for confounders affecting the relationship between sowing recommendations and yield.
• Estimate the average treatment effect (ATE) using multiple methods: linear regression, propensity score matching, inverse probability weighting (IPW), and meta-learners.
• Perform robustness checks via four refutation tests, including adding unobserved confounding, to validate the stability of the estimated effects.
• Use NDVI as a proxy for crop growth to assess mediation effects in the causal pathway from sowing time to yield.
• Leverage the causal graph for potential future application of the front-door criterion and conditional average treatment effect (CATE) estimation.

Experimental results

Research questions

• RQ1 Can observational causal inference reliably estimate the impact of digital agriculture recommendations on farm performance without randomized experiments?
• RQ2 What is the causal effect of AI-driven sowing time recommendations on cotton yield, as measured by the average treatment effect (ATE)?
• RQ3 How robust are the estimated treatment effects across different causal inference methods and refutation tests?
• RQ4 To what extent does the integration of domain knowledge into the causal graph improve the validity of impact estimation?
• RQ5 Can this framework be generalized and scaled to other crops and digital agriculture tools with established domain knowledge?

Key findings

• The causal inference framework successfully estimated a statistically significant yield increase of 12–17% from following the digital sowing time recommendations.
• The effect estimates were consistent across four distinct estimation methods—linear regression, matching, IPW, and meta-learners—indicating robustness.
• All four refutation tests passed successfully, confirming the reliability of the estimated causal effect and the absence of major unmeasured confounding.
• The results were sensitive to strong unobserved confounding, as expected, validating the model’s responsiveness to bias assumptions.
• The framework demonstrated external scalability, with the system already deployed nationally in 2022 for cotton, maize, and sunflower.
• The causal graph enabled future exploration of mediation effects via the front-door criterion and potential personalization through conditional average treatment effect (CATE) estimation.