[Paper Review] Synthesizing Chemical Plant Operation Procedures using Knowledge, Dynamic Simulation and Deep Reinforcement Learning

Shumpei Kubosawa, Takashi Onishi+1 more|arXiv (Cornell University)|Mar 6, 2019

Fault Detection and Control Systems7 references4 citations

TL;DR

This paper proposes an AI system that synthesizes optimal chemical plant operation procedures by integrating dynamic simulation, qualitative knowledge, and deep reinforcement learning (PPO). The system generates actionable, explainable procedures that enable faster recovery from malfunctions—demonstrated by a 30% faster recovery time compared to standard PID control in a VAM plant simulation.

ABSTRACT

Chemical plants are complex and dynamical systems consisting of many components for manipulation and sensing, whose state transitions depend on various factors such as time, disturbance, and operation procedures. For the purpose of supporting human operators of chemical plants, we are developing an AI system that can semi-automatically synthesize operation procedures for efficient and stable operation. Our system can provide not only appropriate operation procedures but also reasons why the procedures are considered to be valid. This is achieved by integrating automated reasoning and deep reinforcement learning technologies with a chemical plant simulator and external knowledge. Our preliminary experimental results demonstrate that it can synthesize a procedure that achieves a much faster recovery from a malfunction compared to standard PID control.

Motivation & Objective

• Address the growing shortage of skilled chemical plant operators in aging societies like Japan by developing AI assistance for operation procedure synthesis.
• Overcome the limitations of traditional model predictive control (MPC) and qualitative planning by enabling continuous, quantitative control procedure generation.
• Provide not only effective operation procedures but also human-understandable explanations for their validity to enhance operator trust.
• Enable rapid recovery from plant malfunctions through end-to-end learning with dynamic simulation and deep reinforcement learning.
• Integrate external knowledge and simulation models to constrain the vast search space of possible control actions in complex chemical plants.

Proposed method

• Uses a quantitative, dynamic simulation model of a vinyl acetate monomer (VAM) plant to represent real-time plant behavior.
• Applies Proximal Policy Optimization (PPO) as the deep reinforcement learning algorithm to learn optimal control policies from continuous state and action spaces.
• Integrates plant-specific knowledge (e.g., from manuals and P&IDs) to guide the agent toward promising, feasible operation procedures.
• Represents the state as a 7-dimensional vector of real-valued sensor readings around the malfunctioning section (e.g., vaporizer pressure, feed flow).
• Defines the reward function as max(0, 1 - 50 × |sₜ - σ|), where sₜ is the sensor value at time t and σ is the normal state value, to encourage convergence to steady-state operation.
• Trains the agent over multiple 30-minute simulation episodes with actions taken every minute, optimizing cumulative reward.

Experimental results

Research questions

• RQ1 Can an AI system synthesize effective, quantitative operation procedures for chemical plants using deep reinforcement learning and dynamic simulation?
• RQ2 How does the performance of the learned procedure compare to standard PID control in recovering from a plant malfunction?
• RQ3 Can the system generalize to complex, non-stepwise disturbances such as ramp-like malfunctions?
• RQ4 To what extent can the system provide explainable, human-understandable justifications for the generated procedures?
• RQ5 Does integrating domain-specific knowledge improve sample efficiency and convergence in procedure synthesis?

Key findings

• The PPO-based agent achieved a cumulative reward significantly higher than standard PID control during training, indicating superior performance in stabilizing the system.
• In the simple malfunction scenario (step increase in feed pressure to 120%), the proposed procedure achieved stable operation within a few minutes, while PID control failed to reach steady state within 30 minutes.
• The system demonstrated faster recovery—reducing the time to stabilize the vaporizer pressure from unsteady to normal levels by over 30% compared to PID.
• In the variable malfunction scenario with randomized ramp-like disturbances (90–120% pressure increase over 0–30 minutes), the agent’s cumulative reward steadily increased over episodes, showing robust learning.
• The agent successfully learned procedures for complex, dynamic disturbances that are poorly suited for differentiable MPC methods due to non-step changes.
• The system produced explainable procedures with numerical control values, enabling human operators to understand and validate the suggested actions.