[Paper Review] Active Learning for Regression Using Greedy Sampling

Dongrui Wu, Chin‐Teng Lin+1 more|arXiv (Cornell University)|Aug 8, 2018

Machine Learning and Algorithms24 references4 citations

TL;DR

This paper proposes two novel active learning for regression (ALR) methods—GSx (greedy sampling in input space) and iGS (greedy sampling in both input and output spaces)—that select informative samples based on diversity in feature and output spaces without requiring model retraining. iGS outperformed existing methods, including QBC and EMCM, across 12 UCI/StatLib datasets and 15 EEG-based driver drowsiness estimation subjects, achieving statistically significant improvements in RMSE and CC.

ABSTRACT

Regression problems are pervasive in real-world applications. Generally a substantial amount of labeled samples are needed to build a regression model with good generalization ability. However, many times it is relatively easy to collect a large number of unlabeled samples, but time-consuming or expensive to label them. Active learning for regression (ALR) is a methodology to reduce the number of labeled samples, by selecting the most beneficial ones to label, instead of random selection. This paper proposes two new ALR approaches based on greedy sampling (GS). The first approach (GSy) selects new samples to increase the diversity in the output space, and the second (iGS) selects new samples to increase the diversity in both input and output spaces. Extensive experiments on 12 UCI and CMU StatLib datasets from various domains, and on 15 subjects on EEG-based driver drowsiness estimation, verified their effectiveness and robustness.

Motivation & Objective

• Address the challenge of high labeling cost in regression by reducing the number of required labeled samples through active learning.
• Overcome the limitations of existing ALR methods that rely on model retraining or are less effective in capturing sample diversity.
• Develop a computationally efficient, model-agnostic ALR approach that improves generalization with minimal labeled data.
• Evaluate the proposed methods on diverse regression tasks, including UCI/StatLib datasets and real-world EEG-based driver drowsiness estimation.
• Demonstrate the superiority of combining input and output space diversity in sample selection for improved regression accuracy.

Proposed method

• Proposes GSx, a greedy sampling method that selects the first sample closest to the centroid of the unlabeled pool and then iteratively selects the sample farthest from all previously selected samples in the input (feature) space.
• Introduces GSy, which extends GSx by selecting samples that maximize diversity in the output (target value) space, computed via the distance between predicted outputs of unlabeled samples and those of already selected samples.
• Develops iGS, a hybrid approach that combines both input and output space diversity by computing a composite distance metric that considers both feature and output space separations.
• Uses Euclidean distance to measure diversity: for GSx, $ d_n^x = \min_m ||\mathbf{x}_n - \mathbf{x}_m|| $; for GSy, $ d_n^y = \min_m ||\hat{y}_n - \hat{y}_m|| $; for iGS, a weighted combination of both.
• Employs a sequential pool-based active learning framework where samples are selected iteratively without retraining the regression model between steps, ensuring low computational cost.
• Applies the methods to regression tasks using ridge regression as the base model, with performance evaluated via RMSE and correlation coefficient (CC).

Experimental results

Research questions

• RQ1 Can greedy sampling based on input space diversity (GSx) improve active learning for regression compared to random or uncertainty-based sampling?
• RQ2 Does incorporating output space diversity (GSy) lead to better regression performance than input-only diversity?
• RQ3 Can combining input and output space diversity (iGS) yield superior performance across diverse regression tasks?
• RQ4 How does the proposed iGS method compare to existing ALR baselines (e.g., QBC, EMCM) in terms of robustness and generalization across multiple datasets?
• RQ5 Is the proposed method effective in real-world, high-cost labeling scenarios such as EEG-based driver drowsiness estimation?

Key findings

• iGS achieved the best average performance across all 15 subjects in EEG-based driver drowsiness estimation, outperforming BL, QBC, EMCM, GSx, and GSy in both RMSE and CC metrics.
• iGS had statistically significant improvements in RMSE over all other methods (p < 0.001), and in CC over all except GSx (p = 0.0394), confirming its robustness and superiority.
• GSx significantly outperformed QBC and EMCM in RMSE and CC, validating the effectiveness of input-space diversity in active learning for regression.
• GSy showed statistically significant improvements over QBC and EMCM in RMSE, and over GSx in RMSE (p = 0.0955), indicating that output-space diversity enhances performance.
• On the 12 UCI and CMU StatLib datasets, iGS consistently ranked first in performance, with GSy and GSx also outperforming baseline methods.
• Statistical analysis confirmed that all proposed ALR methods significantly reduced RMSE and improved CC compared to the baseline (BL), demonstrating their effectiveness across diverse regression tasks.