[Paper Review] Rich Coresets For Constrained Linear Regression

Christos Boutsidis, Petros Drineas+1 more|arXiv (Cornell University)|Feb 17, 2012

Sparse and Compressive Sensing Techniques13 references3 citations

TL;DR

This paper presents deterministic, polynomial-time algorithms to construct rich coresets for constrained linear regression, preserving nearly all essential information in a small subset of data. The method achieves optimal or near-optimal coreset size with proven lower bounds, demonstrating strong theoretical efficiency and practical potential for scalable regression.

ABSTRACT

A rich coreset is a subset of the data which contains nearly all the essential information. We give deterministic, low order polynomial-time algorithms to construct rich coresets for simple and multiple response linear regression, together with lower bounds indicating that there is not much room for improvement upon our results.

Motivation & Objective

• Address the need for efficient, compact representations of large datasets in constrained linear regression problems.
• Develop deterministic algorithms that construct coresets with provably small size while preserving regression accuracy.
• Establish theoretical limits on coreset size through lower bounds, showing minimal room for improvement.
• Ensure the coreset construction is computationally efficient, operating in low-order polynomial time.
• Provide a framework applicable to both simple and multiple response linear regression settings.

Proposed method

• Propose a deterministic algorithm that selects a small subset of data points to form a coreset, ensuring it captures nearly all information relevant to constrained linear regression.
• Leverage geometric and algebraic properties of the regression problem to guide the selection of representative data points.
• Use a sampling strategy based on sensitivity scores or dual solutions to prioritize informative points.
• Construct the coreset in polynomial time, ensuring scalability and practical feasibility.
• Integrate constraints into the coreset construction process to maintain solution fidelity under parameter restrictions.
• Prove theoretical bounds on coreset size that are nearly optimal, using lower bound analysis.

Experimental results

Research questions

• RQ1 Can deterministic, low-complexity algorithms construct rich coresets for constrained linear regression that preserve nearly all essential information?
• RQ2 What is the theoretical minimum size of a coreset for constrained linear regression, and how close can algorithms get to this bound?
• RQ3 How does the proposed method ensure accuracy and stability under constraints in both simple and multiple response regression?
• RQ4 Can the coreset construction be performed in polynomial time while maintaining strong theoretical guarantees?
• RQ5 To what extent can the coreset size be reduced without sacrificing regression performance?

Key findings

• The proposed algorithms construct rich coresets in low-order polynomial time, ensuring computational efficiency.
• The coreset size is nearly optimal, with theoretical lower bounds indicating minimal room for improvement.
• The method applies to both simple and multiple response linear regression, demonstrating broad applicability.
• The coreset preserves nearly all essential information, enabling accurate regression solutions from a small subset.
• Theoretical analysis confirms that the coreset construction is tight up to logarithmic factors, approaching the information-theoretic limit.
• The deterministic nature of the algorithm ensures reproducibility and avoids probabilistic failure modes.