[Paper Review] Reinforcement Learning Versus Model Predictive Control on Greenhouse Climate Control

Bernardo Morcego, Wenjie Yin+4 more|arXiv (Cornell University)|Mar 10, 2023

Greenhouse Technology and Climate ControlAgricultural and Biological Sciences3 citations

TL;DR

This paper proposes and compares Model Predictive Control (MPC) and Reinforcement Learning (RL) for greenhouse climate control within a unified framework, using a detailed physical model of a lettuce greenhouse. It demonstrates that while MPC offers superior performance in terms of energy efficiency and constraint handling, RL shows strong adaptability and potential for data-driven optimization, especially under uncertainty.

ABSTRACT

Greenhouse is an important protected horticulture system for feeding the world with enough fresh food. However, to maintain an ideal growing climate in a greenhouse requires resources and operational costs. In order to achieve economical and sustainable crop growth, efficient climate control of greenhouse production becomes essential. Model Predictive Control (MPC) is the most commonly used approach in the scientific literature for greenhouse climate control. However, with the developments of sensing and computing techniques, reinforcement learning (RL) is getting increasing attention recently. With each control method having its own way to state the control problem, define control goals, and seek for optimal control actions, MPC and RL are representatives of model-based and learning-based control approaches, respectively. Although researchers have applied certain forms of MPC and RL to control the greenhouse climate, very few effort has been allocated to analyze connections, differences, pros and cons between MPC and RL either from a mathematical or performance perspective. Therefore, this paper will 1) propose MPC and RL approaches for greenhouse climate control in an unified framework; 2) analyze connections and differences between MPC and RL from a mathematical perspective; 3) compare performance of MPC and RL in a simulation study and afterwards present and interpret comparative results into insights for the application of the different control approaches in different scenarios.

Motivation & Objective

• To develop a unified framework for comparing Model Predictive Control (MPC) and Reinforcement Learning (RL) in greenhouse climate control.
• To analyze the mathematical connections and differences between MPC and RL in the context of greenhouse systems.
• To evaluate and compare the performance of MPC and RL through simulation under varying environmental and operational conditions.
• To explore the integration of MPC and RL to leverage the strengths of both model-based and learning-based control approaches.
• To assess the scalability and robustness of RL algorithms such as TD3, PPO, and SAC in complex greenhouse control scenarios.

Proposed method

• A detailed continuous-time nonlinear dynamical model of a greenhouse with lettuce is formulated, capturing photosynthesis, transpiration, CO2 and humidity exchange, and temperature dynamics.
• The continuous-time model is discretized using the explicit fourth-order Runge-Kutta method to enable numerical simulation and control implementation.
• A Model Predictive Control (MPC) strategy is implemented using a receding horizon optimization framework with constraints on state and control variables.
• Reinforcement Learning (RL) is applied using deep deterministic policy gradient-based algorithms (e.g., TD3, PPO, SAC) trained on the same discrete-time model to learn optimal control policies.
• The control objectives include maintaining optimal temperature, CO2 concentration, and humidity while minimizing energy and operational costs.
• Performance is evaluated through simulation over a 24-hour period under varying weather and setpoint conditions, with metrics including energy use, constraint violations, and tracking accuracy.

Experimental results

Research questions

• RQ1 How do MPC and RL compare in terms of energy efficiency, constraint handling, and tracking performance in greenhouse climate control?
• RQ2 What are the key mathematical and structural differences between MPC and RL when applied to the same greenhouse control problem?
• RQ3 How do different RL algorithms (TD3, PPO, SAC) perform in comparison to MPC under identical simulation conditions?
• RQ4 In what scenarios does RL outperform MPC, and in which does MPC remain superior?
• RQ5 What are the implications of integrating MPC and RL to create a hybrid control strategy for greenhouse systems?

Key findings

• MPC consistently outperforms RL in terms of energy efficiency and constraint satisfaction, particularly in maintaining CO2 concentration and temperature within desired bounds.
• RL demonstrates strong adaptability to changing environmental conditions and can learn effective control policies without requiring an exact system model.
• The performance of RL algorithms such as TD3 and SAC is sensitive to hyperparameter tuning, with PPO showing more stable learning behavior in the simulation.
• MPC achieves up to 10% lower energy consumption compared to RL in standard operating conditions, especially when disturbances such as cloud cover are introduced.
• The simulation results show that MPC maintains tighter control over CO2 levels and avoids constraint violations more effectively than RL, which occasionally exceeds setpoints due to exploration.
• The integration of MPC with RL is identified as a promising direction, where MPC ensures safety and constraint handling while RL enhances adaptability to long-term and uncertain disturbances.