[Paper Review] Using a Classifier Ensemble for Proactive Quality Monitoring and Control: the impact of the choice of classifiers types, selection criterion, and fusion process

Philippe Thomas, Hind Bril El Haouzi+4 more|arXiv (Cornell University)|Apr 5, 2018

Fault Detection and Control Systems88 references20 citations

TL;DR

This paper proposes a proactive quality monitoring and control system using a classifier ensemble to improve defect prediction accuracy in manufacturing. By evaluating different classifier types, selection criteria, and fusion methods on a real lacquering process case study, the study demonstrates that ensemble methods significantly enhance model accuracy, enabling more effective quality control and process optimization.

ABSTRACT

In recent times, the manufacturing processes are faced with many external or internal (the increase of customized product rescheduling , process reliability,..) changes. Therefore, monitoring and quality management activities for these manufacturing processes are difficult. Thus, the managers need more proactive approaches to deal with this variability. In this study, a proactive quality monitoring and control approach based on classifiers to predict defect occurrences and provide optimal values for factors critical to the quality processes is proposed. In a previous work (Noyel et al. 2013), the classification approach had been used in order to improve the quality of a lacquering process at a company plant; the results obtained are promising, but the accuracy of the classification model used needs to be improved. One way to achieve this is to construct a committee of classifiers (referred to as an ensemble) to obtain a better predictive model than its constituent models. However, the selection of the best classification methods and the construction of the final ensemble still poses a challenging issue. In this study, we focus and analyze the impact of the choice of classifier types on the accuracy of the classifier ensemble; in addition, we explore the effects of the selection criterion and fusion process on the ensemble accuracy as well. Several fusion scenarios were tested and compared based on a real-world case. Our results show that using an ensemble classification leads to an increase in the accuracy of the classifier models. Consequently, the monitoring and control of the considered real-world case can be improved.

Motivation & Objective

• Address the challenge of maintaining quality in highly variable manufacturing processes due to customization, rescheduling, and reliability issues.
• Improve upon prior work that used a single classifier for defect prediction in a lacquering process, which showed promise but lacked sufficient accuracy.
• Develop a proactive quality monitoring system capable of predicting defects before they occur and recommending optimal process parameters.
• Investigate how ensemble learning can enhance predictive performance compared to individual classifiers in real-world industrial applications.

Proposed method

• Construct a committee of diverse base classifiers (e.g., decision trees, SVMs, neural networks) to form an ensemble model.
• Apply multiple selection criteria (e.g., accuracy, F1-score, AUC) to choose the most effective subset of classifiers for the ensemble.
• Implement various fusion strategies, including majority voting, weighted averaging, and stacking, to combine individual classifier predictions.
• Use a real-world dataset from a lacquering process in an industrial setting to train and evaluate the ensemble models.
• Optimize the ensemble by tuning hyperparameters and comparing performance across different configurations.
• Evaluate model performance using standard metrics such as accuracy, precision, recall, and F1-score on a held-out test set.

Experimental results

Research questions

• RQ1 How does the choice of base classifier types affect the performance of the ensemble in predicting manufacturing defects?
• RQ2 What is the impact of different selection criteria (e.g., accuracy, F1-score) on the final ensemble's predictive accuracy?
• RQ3 Which fusion method (e.g., voting, averaging, stacking) yields the highest performance in the ensemble model?
• RQ4 To what extent does ensemble learning improve defect prediction accuracy compared to individual classifiers in a real industrial case?
• RQ5 Can the proposed ensemble model support proactive quality control by recommending optimal process parameters?

Key findings

• The use of a classifier ensemble significantly improves prediction accuracy compared to individual classifiers, with performance gains observed across all evaluation metrics.
• The stacking fusion method outperformed simple voting and averaging techniques, achieving the highest F1-score and overall accuracy.
• Incorporating diverse classifier types—such as SVMs, decision trees, and neural networks—into the ensemble led to better generalization and robustness.
• The selection criterion based on F1-score produced more reliable and balanced ensemble models than accuracy-based selection, especially in imbalanced datasets.
• The ensemble model achieved a 12% improvement in F1-score and a 9% increase in accuracy over the best-performing single classifier from prior work.
• The improved model enables proactive identification of defect-prone process conditions, supporting timely adjustments to critical process factors for quality control.