[Paper Review] Reinforcement Learning on Job Shop Scheduling Problems Using Graph Networks.

Mohammed Sharafath Abdul Hameed, Andreas Schwung|arXiv (Cornell University)|Sep 8, 2020

Scheduling and Optimization Algorithms42 references18 citations

TL;DR

This paper proposes a distributed reinforcement learning approach using graph neural networks to solve job shop scheduling problems by modeling resource interactions and enabling flexible, scalable optimization. Evaluated on a multi-robot and a complex production benchmark, the method demonstrates strong performance and adaptability in dynamic environments.

ABSTRACT

This paper presents a novel approach for job shop scheduling problems using deep reinforcement learning. To account for the complexity of production environment, we employ graph neural networks to model the various relations within production environments. Furthermore, we cast the JSSP as a distributed optimization problem in which learning agents are individually assigned to resources which allows for higher flexibility with respect to changing production environments. The proposed distributed RL agents used to optimize production schedules for single resources are running together with a co-simulation framework of the production environment to obtain the required amount of data. The approach is applied to a multi-robot environment and a complex production scheduling benchmark environment. The initial results underline the applicability and performance of the proposed method.

Motivation & Objective

• To address the complexity and dynamism of real-world job shop scheduling by modeling inter-resource dependencies.
• To improve scheduling flexibility in changing production environments through decentralized learning agents.
• To enable scalable and adaptive scheduling by assigning individual RL agents to resources.
• To validate the approach in realistic, complex scheduling environments with dynamic constraints.
• To demonstrate the feasibility and performance of graph-based deep reinforcement learning in industrial scheduling.

Proposed method

• Graph neural networks are used to model relationships between jobs, machines, and resources in the production environment.
• Each resource is assigned a dedicated reinforcement learning agent, enabling distributed optimization.
• The agents learn through interaction with a co-simulation framework that emulates real-time production dynamics.
• The approach treats job shop scheduling as a distributed optimization problem, improving scalability and adaptability.
• Training data is collected via continuous interaction between RL agents and the co-simulation environment.
• The method supports dynamic reconfiguration and real-time response to production changes.

Experimental results

Research questions

• RQ1 Can graph neural networks effectively model complex interdependencies in job shop scheduling environments?
• RQ2 How does a distributed RL architecture with per-resource agents improve scheduling adaptability in dynamic settings?
• RQ3 What performance gains does the proposed method achieve compared to centralized or non-graph-based approaches?
• RQ4 Can the method scale effectively across complex, multi-resource production environments?
• RQ5 How well does the approach generalize in real-time simulation of industrial scheduling scenarios?

Key findings

• The proposed method successfully models complex inter-resource relationships using graph neural networks.
• Distributed RL agents achieve better adaptability in dynamic production environments compared to centralized approaches.
• The method demonstrates strong performance on a complex production scheduling benchmark environment.
• The co-simulation framework enables efficient data collection for training in realistic conditions.
• Initial results confirm the method's applicability and scalability in multi-robot and industrial scheduling scenarios.
• The integration of graph networks and distributed RL enables real-time, flexible scheduling optimization.