[Paper Review] Learning from Multiple Outlooks

Maayan Harel, Shie Mannor|arXiv (Cornell University)|Apr 30, 2010

Domain Adaptation and Few-Shot Learning26 references72 citations

TL;DR

This paper introduces Multiple Outlook Learning (MOMAP), a method to improve classification by aligning data from different feature spaces (outlooks) via optimal affine transformations that match empirical moments. The approach boosts performance in activity recognition by transferring knowledge across diverse sensors or feature representations, achieving up to 5.9% lower error rate than baseline classifiers.

ABSTRACT

We propose a novel problem formulation of learning a single task when the data are provided in different feature spaces. Each such space is called an outlook, and is assumed to contain both labeled and unlabeled data. The objective is to take advantage of the data from all the outlooks to better classify each of the outlooks. We devise an algorithm that computes optimal affine mappings from different outlooks to a target outlook by matching moments of the empirical distributions. We further derive a probabilistic interpretation of the resulting algorithm and a sample complexity bound indicating how many samples are needed to adequately find the mapping. We report the results of extensive experiments on activity recognition tasks that show the value of the proposed approach in boosting performance.

Motivation & Objective

• To address the challenge of learning a single classification task when data are available in multiple, distinct feature spaces (outlooks) with no sample correspondence.
• To develop a method that leverages labeled and unlabeled data from all outlooks to improve classification performance beyond what is possible with any single outlook.
• To enable knowledge transfer between outlooks by learning affine mappings that align their empirical distributions through moment matching.
• To provide theoretical guarantees via a sample complexity bound for the mapping estimation process.
• To demonstrate the effectiveness of the approach in real-world scenarios such as multi-sensor activity recognition and cross-lingual document classification.

Proposed method

• The method computes optimal affine transformations from each source outlook to a target outlook by matching the first and second-order empirical moments (mean and covariance) of their distributions.
• The transformation preserves isometry within classes, ensuring that class structure is maintained during the mapping process.
• The algorithm uses a closed-form solution derived from moment matching, making it computationally efficient and scalable.
• The approach is grounded in a probabilistic interpretation, linking the moment-matching objective to likelihood maximization under certain assumptions.
• The method is extended to handle multiple outlooks by iteratively aligning each outlook to a common target space.
• The framework is adaptable to various setups, including different feature dimensions, sampling rates, and noisy or permuted features.

Experimental results

Research questions

• RQ1 Can we effectively transfer knowledge between multiple outlooks with distinct feature representations and no sample correspondence?
• RQ2 How can we learn a shared representation across multiple outlooks by aligning their empirical distributions without assuming feature correspondence?
• RQ3 What is the optimal way to map multiple outlooks to a common space using only labeled and unlabeled data?
• RQ4 How does the proposed moment-matching approach compare to standard baselines in terms of classification accuracy and robustness?
• RQ5 What theoretical guarantees can be provided for the sample complexity of the mapping estimation process?

Key findings

• For two outlooks with different sensors, MOMAP achieved a mean BER reduction of 4.5% (±2.7%) compared to the TRG classifier when using only 5% labeled data.
• In the case of different sampling rates (32Hz vs. 30Hz), MOMAP reduced BER by 5.9% (±2.4%) on average over the TRG baseline with 5% labeled data.
• The method outperformed the OPT classifier in several user pairs, particularly when noise features were added, indicating robustness to irrelevant or corrupted features.
• The performance boost was most pronounced for the running activity in experiments with permuted and noisy features, highlighting the method’s ability to extract useful patterns from complex representations.
• The MOMAP algorithm consistently improved classification accuracy across diverse experimental setups, including varying feature dimensions, noise, and sampling rates.
• The method demonstrated strong generalization, especially in low-data regimes, suggesting effective regularization through multi-outlook knowledge transfer.