[Paper Review] The Greedy Miser: Learning under Test-time Budgets

Zhixiang Xu, Kilian Q. Weinberger+1 more|arXiv (Cornell University)|Jun 27, 2012

Machine Learning and Data ClassificationComputer Science14 references80 citations

TL;DR

This paper proposes a greedy, cost-aware learning algorithm that explicitly optimizes for test-time computational cost during training by modeling feature extraction cost as part of the loss. By relaxing a non-continuous global objective into a continuous surrogate loss, the method enables stage-wise regression via gradient boosting, achieving accuracy comparable to unconstrained models while reducing test-time cost by an order of magnitude.

ABSTRACT

As machine learning algorithms increasingly enter real-world settings, there is rising interest in controlling the cpu-cost during test-time. In industry, computational resources must be budgeted and costs must be strictly accounted for. At its very core, this problem is inherently a tradeoff between accuracy and test-time computation. Test-time computation consists of two components: 1. the actual running time of the algorithm; 2. the time required for feature extraction. The latter can vary drastically if the feature set is diverse. In this abstract, we propose a novel algorithm that explicitly considers the feature extraction cost during training. We first state the (non-continuous) global objective, which explicitly trades off feature cost and accuracy, and then relax it into a continuous loss function. Subsequently, we derive an update rule that shows the resulting loss lends itself naturally to greedy optimization with stage-wise regression [4]. The resulting learning algorithm is much simpler than any prior work, yet leads to superior test-time performance. Its accuracy matches that of the unconstrained baseline (with unlimited resources) while achieving an order of magnitude reduction of test-time cost. Cost-sensitive learning. We use gradient-boosting [4] to learn a classifier H(x) = ∑T t=1 βtht(x) to minimize some loss ℓ(H). Here, ht ∈ H where H is the set of all possible regression trees [1] of some limited

Motivation & Objective

• To address the growing need for computational cost control in real-world machine learning deployments where CPU resources are strictly budgeted.
• To model and optimize for the combined cost of feature extraction and inference time during test-time, which is often overlooked in standard learning frameworks.
• To develop a training procedure that explicitly trades off feature extraction cost and model accuracy in a differentiable, continuous manner.
• To enable efficient, greedy optimization that matches the performance of unconstrained models while drastically reducing test-time resource usage.
• To simplify cost-sensitive learning by leveraging stage-wise regression in gradient boosting for direct optimization of test-time efficiency.

Proposed method

• The method formulates a non-continuous global objective that jointly optimizes for model accuracy and feature extraction cost.
• This objective is relaxed into a continuous surrogate loss function to enable gradient-based optimization.
• The resulting loss function is naturally amenable to greedy, stage-wise optimization using gradient boosting with regression trees.
• The algorithm learns a sequence of weak estimators (regression trees) that incrementally improve the trade-off between accuracy and cost.
• Feature extraction cost is explicitly modeled as a component of the loss, allowing the model to learn which features to prioritize based on their computational cost.
• The final classifier is formed as a weighted sum of weak learners, with each stage minimizing the relaxed loss under cost constraints.

Experimental results

Research questions

• RQ1 Can a learning algorithm be trained to minimize test-time computational cost without sacrificing model accuracy?
• RQ2 How can feature extraction cost be effectively modeled and optimized during training in a differentiable and continuous manner?
• RQ3 Can a greedy, stage-wise optimization strategy be used to achieve superior test-time efficiency while matching unconstrained performance?
• RQ4 What is the trade-off between feature extraction cost and model accuracy when both are explicitly optimized during training?
• RQ5 How does the proposed method compare to unconstrained baselines in terms of test-time cost and accuracy?

Key findings

• The proposed algorithm achieves test-time accuracy that matches that of an unconstrained baseline model with unlimited computational resources.
• The method reduces test-time computational cost by an order of magnitude compared to the unconstrained baseline.
• The continuous relaxation of the global objective enables effective and stable optimization via gradient boosting.
• The greedy, stage-wise training process leads to efficient inference by learning to prioritize low-cost, high-impact features.
• The model demonstrates strong generalization to unseen test-time budgets by learning cost-aware representations during training.
• The approach outperforms prior cost-sensitive learning methods in both efficiency and accuracy under constrained budgets.