[Paper Review] Automatic Bug Triage using Semi-Supervised Text Classification

Jifeng Xuan, He Jiang+3 more|Software Engineering and Knowledge Engineering|Apr 16, 2017

Software Engineering Research22 references85 citations

TL;DR

A semi-supervised text classification approach combines naive Bayes and expectation-maximization to perform bug triage using both labeled and unlabeled bug reports, with a weighted developer-aware training and iterative labeling to improve accuracy over supervised methods.

ABSTRACT

In this paper, we propose a semi-supervised text classification approach for bug triage to avoid the deficiency of labeled bug reports in existing supervised approaches. This new approach combines naive Bayes classifier and expectation-maximization to take advantage of both labeled and unlabeled bug reports. This approach trains a classifier with a fraction of labeled bug reports. Then the approach iteratively labels numerous unlabeled bug reports and trains a new classifier with labels of all the bug reports. We also employ a weighted recommendation list to boost the performance by imposing the weights of multiple developers in training the classifier. Experimental results on bug reports of Eclipse show that our new approach outperforms existing supervised approaches in terms of classification accuracy.

Motivation & Objective

• Address the shortage of labeled bug reports for effective bug triage
• Develop a semi-supervised learning method that utilizes both labeled and unlabeled bug reports
• Improve triage accuracy over traditional supervised approaches
• Incorporate developer influence through weighted training signals
• Demonstrate effectiveness on real-world bug repositories (Eclipse)

Proposed method

• Combine Naive Bayes classifier with expectation-maximization to leverage unlabeled bug reports
• Train initial classifier with a fraction of labeled reports
• Iteratively label unlabeled bug reports and retrain using labels from all reports
• Incorporate a weighted recommendation list that imposes developer weights during training
• Evaluate on Eclipse bug reports and compare to existing supervised methods

Experimental results

Research questions

• RQ1 Can semi-supervised text classification improve bug triage accuracy with limited labeled data?
• RQ2 How does integrating unlabeled data via EM affect classifier performance in bug triage?
• RQ3 Does incorporating developer-weighted training improve triage results?
• RQ4 How does the proposed method compare to standard supervised approaches on real-world datasets (Eclipse)?

Key findings

• The semi-supervised approach with EM and NB outperforms existing supervised methods in classification accuracy on Eclipse bug reports.