[Paper Review] Designing an LLM-Based Copilot for Manufacturing Equipment Selection

Jonas Werheid, Oleksandr Melnychuk+11 more|arXiv (Cornell University)|Jan 1, 2024

Manufacturing Process and Optimization1 citations

TL;DR

This paper proposes a factual-driven LLM-based copilot using Retrieval-Augmented Generation (RAG) to streamline automation equipment selection in manufacturing ramp-up. It integrates structured and semi-structured knowledge to guide engineers through a traceable, state-machine workflow, achieving 19/22 correct equipment selections and 6/22 fully compliant recommendations in industrial testing.

ABSTRACT

Effective decision-making in automation equipment selection is critical for reducing ramp-up time and maintaining production quality, especially in the face of increasing product variation and market demands. However, limited expertise and resource constraints often result in inefficiencies during the ramp-up phase when new products are integrated into production lines. Existing methods often lack structured and tailored solutions to support automation engineers in reducing ramp-up time, leading to compromises in quality. This research investigates whether large-language models (LLMs), combined with Retrieval-Augmented Generation (RAG), can assist in streamlining equipment selection in ramp-up planning. We propose a factual-driven copilot integrating LLMs with structured and semi-structured knowledge retrieval for three component types (robots, feeders and vision systems), providing a guided and traceable state-machine process for decision-making in automation equipment selection. The system was demonstrated to an industrial partner, who tested it on three internal use-cases. Their feedback affirmed its capability to provide logical and actionable recommendations for automation equipment. More specifically, among 22 equipment prompts analyzed, 19 involved selecting the correct equipment while considering most requirements, and in 6 cases, all requirements were fully met.

Motivation & Objective

• Address the challenge of prolonged ramp-up times in manufacturing due to complex equipment selection under resource and expertise constraints.
• Overcome limitations of static rule-based systems and generic LLM responses in automation engineering.
• Develop a transparent, factual, and traceable decision-support system for selecting robots, feeders, and vision systems during production ramp-up.
• Integrate domain-specific knowledge from academic literature, industrial sources, and supplier databases into a structured LLM workflow.
• Demonstrate the system’s effectiveness through industrial validation with real-world use cases.

Proposed method

• Employ a multi-agent LLM architecture with a primary orchestrator agent managing subcomponents via API calls.
• Integrate a relational knowledge system and a semi-structured knowledge system (e.g., supplier databases) to retrieve domain-specific facts.
• Use Retrieval-Augmented Generation (RAG) to ground LLM responses in factual data, reducing hallucinations and improving reliability.
• Implement a state-machine process to guide users through a structured, traceable decision path for equipment selection.
• Train and deploy the system using curated knowledge from scientific literature, lecture materials, and industrial datasets.
• Validate the system through industrial testing on three real-world ramp-up use cases.

Experimental results

Research questions

• RQ1 Can an LLM-based copilot with RAG improve the accuracy and traceability of equipment selection in manufacturing ramp-up?
• RQ2 To what extent can factual grounding reduce hallucinations and improve recommendation quality in automation engineering?
• RQ3 How well does the system perform in real industrial scenarios compared to traditional methods?
• RQ4 What is the impact of structured knowledge retrieval on selecting correct equipment types and subtypes?
• RQ5 Can the system support complex, multi-requirement selections (e.g., precision, payload, speed) in a transparent and repeatable manner?

Key findings

• Among 22 equipment prompts, 19 resulted in correct equipment selection while considering most requirements.
• In 6 out of 22 cases, all specified requirements—including precision, payload, and speed—were fully met by the recommended equipment.
• Industrial feedback confirmed the system’s ability to generate logical, actionable, and traceable recommendations.
• The system demonstrated strong performance in selecting appropriate equipment subtypes, such as SCARA robots and articulated arm robots, based on detailed technical constraints.
• The copilot successfully integrated academic, industrial, and supplier knowledge to produce reliable, localized, and internationally relevant equipment suggestions.
• Limitations include lack of support for layout design and integration with the ramp-up implementation phase, indicating room for future expansion.