[论文解读] SCLIFD:Supervised Contrastive Knowledge Distillation for Incremental Fault Diagnosis under Limited Fault Data

Peng Peng, Hanrong Zhang+4 more|arXiv (Cornell University)|Feb 12, 2023

Imbalanced Data Classification Techniques被引用 4

一句话总结

该论文提出SCLIFD，一种新颖的增量故障诊断框架，适用于有限故障数据场景，结合了监督对比知识蒸馏、自适应原型选择（AdaHerding）和余弦分类器。该方法在各类不平衡和长尾故障诊断数据集上显著优于现有方法，分别实现了88.17%和79.81%的准确率，展现出在工业场景下对类别不平衡和灾难性遗忘的强鲁棒性。

ABSTRACT

Intelligent fault diagnosis has made extraordinary advancements currently. Nonetheless, few works tackle class-incremental learning for fault diagnosis under limited fault data, i.e., imbalanced and long-tailed fault diagnosis, which brings about various notable challenges. Initially, it is difficult to extract discriminative features from limited fault data. Moreover, a well-trained model must be retrained from scratch to classify the samples from new classes, thus causing a high computational burden and time consumption. Furthermore, the model may suffer from catastrophic forgetting when trained incrementally. Finally, the model decision is biased toward the new classes due to the class imbalance. The problems can consequently lead to performance degradation of fault diagnosis models. Accordingly, we introduce a supervised contrastive knowledge distillation for incremental fault diagnosis under limited fault data (SCLIFD) framework to address these issues, which extends the classical incremental classifier and representation learning (iCaRL) framework from three perspectives. Primarily, we adopt supervised contrastive knowledge distillation (KD) to enhance its representation learning capability under limited fault data. Moreover, we propose a novel prioritized exemplar selection method adaptive herding (AdaHerding) to restrict the increase of the computational burden, which is also combined with KD to alleviate catastrophic forgetting. Additionally, we adopt the cosine classifier to mitigate the adverse impact of class imbalance. We conduct extensive experiments on simulated and real-world industrial processes under different imbalance ratios. Experimental results show that our SCLIFD outperforms the existing methods by a large margin.

研究动机与目标

• 解决工业系统中在有限且不平衡故障数据下进行类别增量故障诊断的挑战。
• 克服在新故障类别出现时增量模型训练过程中出现的灾难性遗忘问题。
• 在低数据环境下增强表征学习并缓解类别不平衡的影响。
• 通过优化原型选择，降低增量学习过程中的计算成本。
• 开发一种可扩展、鲁棒的框架，适用于故障类别不断演化的实际工业故障诊断场景。

提出的方法

• 引入监督对比知识蒸馏（SCL-KD），以在有限故障数据下提升特征表征学习能力。
• 提出AdaHerding，一种基于学习难度优先选择难样本的原型选择方法，有效降低内存与计算开销。
• 将知识蒸馏与AdaHerding结合，缓解增量学习过程中灾难性遗忘问题。
• 在推理阶段采用余弦分类器，减少对多数类别的偏差，提升在类别不平衡下的泛化能力。
• 在iCaRL框架基础上，融合SCL-KD、AdaHerding和余弦分类器，以提升增量故障诊断性能。
• 在模拟数据集（TEP）和真实世界数据集（MFF）中，均采用包含24、12和10个神经元的多层感知机编码器进行特征提取。

实验结果

研究问题

• RQ1 监督对比知识蒸馏能否在低数据故障诊断场景中提升表征学习能力？
• RQ2 AdaHerding在保持模型性能的同时，能否有效降低增量故障诊断中的计算成本？
• RQ3 余弦分类器在多类别不平衡故障诊断中，能在多大程度上缓解性能下降问题？
• RQ4 在类别不平衡和长尾数据分布下，SCLIFD在增量学习各阶段的准确率与稳定性方面，相较于最先进方法表现如何？
• RQ5 所提出的框架能否在保持先前学习故障类别高性能的同时，有效防止灾难性遗忘？

主要发现

• 在MFF数据集的类别不平衡故障诊断设置下，SCLIFD达到88.17%的测试准确率，较表现第二好的方法（iCaRL）高出33.15%。
• 在MFF数据集的长尾故障诊断设置下，SCLIFD准确率达到79.81%，较iCaRL高出35.84%。
• 消融实验表明，SCL、AdaHerding和余弦分类器三个组件均贡献显著，分别在类别不平衡条件下第5个增量会话中相较iCaRL提升21.86%、6.42%和19.69%。
• SCLIFD在所有增量学习阶段均保持最高准确率，且最接近理论上限性能，表明其具备出色的稳定性与可扩展性。
• 该框架在模拟数据集（TEP）和真实世界工业数据集（MFF）中均表现出卓越的鲁棒性，证实其在实际应用中的可行性。
• SCL-KD与AdaHerding的结合有效缓解了灾难性遗忘，确保在增量学习过程中对已学习故障类别的性能保持稳定。