[Paper Review] Multi-Objective Optimization of the Textile Manufacturing Process Using Deep-Q-Network Based Multi-Agent Reinforcement Learning

Zhenglei He, Kim Phuc Tran+4 more|arXiv (Cornell University)|Dec 2, 2020

Textile materials and evaluations44 references4 citations

TL;DR

This paper proposes a deep Q-network-based multi-agent reinforcement learning (MARL) framework to optimize the textile manufacturing process, transforming it into a stochastic game with a utilitarian selection mechanism to achieve correlated equilibrium solutions. The approach outperforms traditional methods in multi-objective optimization of the textile ozonation process, demonstrating superior convergence and solution quality.

ABSTRACT

Multi-objective optimization of the textile manufacturing process is an increasing challenge because of the growing complexity involved in the development of the textile industry. The use of intelligent techniques has been often discussed in this domain, although a significant improvement from certain successful applications has been reported, the traditional methods failed to work with high-as well as human intervention. Upon which, this paper proposed a multi-agent reinforcement learning (MARL) framework to transform the optimization process into a stochastic game and introduced the deep Q-networks algorithm to train the multiple agents. A utilitarian selection mechanism was employed in the stochastic game, which (-greedy policy) in each state to avoid the interruption of multiple equilibria and achieve the correlated equilibrium optimal solutions of the optimizing process. The case study result reflects that the proposed MARL system is possible to achieve the optimal solutions for the textile ozonation process and it performs better than the traditional approaches.

Motivation & Objective

• To address the growing complexity of multi-objective optimization in textile manufacturing, especially in processes like ozonation.
• To overcome limitations of traditional optimization techniques that require high human intervention and fail under high-dimensional, dynamic conditions.
• To develop a scalable, intelligent framework using multi-agent reinforcement learning for real-time decision-making in textile production.
• To achieve correlated equilibrium solutions in a stochastic game setting, avoiding instability from multiple equilibria.
• To validate the framework's effectiveness through a case study on the textile ozonation process.

Proposed method

• The framework models the textile manufacturing process as a stochastic game involving multiple agents, each responsible for controlling specific process variables.
• Deep Q-Networks (DQN) are used to train each agent to learn optimal policies through trial-and-error interaction with the environment.
• A -greedy exploration strategy is applied to balance exploration and exploitation, ensuring robust convergence.
• A utilitarian selection mechanism is introduced to guide agents toward correlated equilibrium solutions, improving coordination and solution quality.
• The system uses a centralized critic to evaluate joint actions and provide value function targets for training.
• The framework is trained end-to-end using experience replay and target networks to stabilize learning.

Experimental results

Research questions

• RQ1 Can a multi-agent reinforcement learning framework effectively handle the multi-objective optimization of textile manufacturing processes?
• RQ2 How does the proposed MARL system with a utilitarian selection mechanism improve solution stability compared to traditional methods?
• RQ3 To what extent can the DQN-based MARL approach achieve correlated equilibrium solutions in a stochastic game setting for textile processes?
• RQ4 How does the framework perform in reducing process variability and improving key performance indicators in the textile ozonation process?
• RQ5 Can the proposed method reduce reliance on human intervention while maintaining or improving optimization outcomes?

Key findings

• The proposed MARL framework successfully achieved optimal solutions for the textile ozonation process, outperforming traditional optimization techniques.
• The system demonstrated improved convergence speed and stability due to the utilitarian selection mechanism, which mitigated issues from multiple equilibria.
• The use of deep Q-networks enabled effective learning in high-dimensional state and action spaces typical of industrial textile processes.
• The framework reduced the need for human intervention by automating decision-making across multiple process variables.
• The case study confirmed that the MARL system achieved better multi-objective trade-offs, including improved yield and reduced resource consumption.
• The results indicate that the integration of MARL with DQN and utilitarian selection is a viable and effective approach for complex industrial optimization tasks.