[Paper Review] Making transport more robust and interpretable by moving data through a small number of anchor points

Chi-Heng Lin, Mehdi Azabou+1 more|arXiv (Cornell University)|Dec 21, 2020

Domain Adaptation and Few-Shot Learning46 references4 citations

TL;DR

This paper proposes Latent Optimal Transport (LOT), a structured optimal transport method that improves robustness and interpretability by learning separate source and target anchors, enabling data to flow through these hubs. LOT regularizes transport rank, reduces sensitivity to noise and sampling variation, and enhances structural alignment between datasets with different underlying geometries.

ABSTRACT

Optimal transport (OT) is a widely used technique for distribution alignment, with applications throughout the machine learning, graphics, and vision communities. Without any additional structural assumptions on trans-port, however, OT can be fragile to outliers or noise, especially in high dimensions. Here, we introduce a new form of structured OT that simultaneously learns low-dimensional structure in data while leveraging this structure to solve the alignment task. Compared with OT, the resulting transport plan has better structural interpretability, highlighting the connections between individual data points and local geometry, and is more robust to noise and sampling. We apply the method to synthetic as well as real datasets, where we show that our method can facilitate alignment in noisy settings and can be used to both correct and interpret domain shift.

Motivation & Objective

• Address the fragility of standard optimal transport (OT) to noise and outliers, especially in high-dimensional settings.
• Overcome limitations of prior structured OT methods that rely on shared anchors between source and target, which may not capture domain shifts or differing data structures.
• Develop a method that jointly learns low-dimensional data structure and uses it to regularize transport, improving both robustness and interpretability.
• Enable more interpretable transport plans by highlighting connections between individual data points and local data geometry via latent anchor spaces.
• Provide a theoretically grounded relaxation of OT that mitigates the curse of dimensionality in sampling requirements.

Proposed method

• Factorize the transport plan into three components: (1) source points to source anchors, (2) source anchors to target anchors, and (3) target anchors to target points.
• Learn distinct sets of anchors for source and target distributions, allowing for different structural representations in each domain.
• Use latent space alignment between source and target anchors to model high-level structural similarity, enabling robust cross-domain transport.
• Formulate the objective as a regularized optimal transport problem with low-rank constraints via anchor-based factorization.
• Employ entropy regularization (via Sinkhorn iterations) to ensure computational efficiency and numerical stability in optimization.
• Estimate transport maps using a three-stage transformation: source-to-anchor, anchor-to-anchor, and anchor-to-target, with centroid-based reconstruction.

Experimental results

Research questions

• RQ1 Can learning separate anchors for source and target distributions improve the robustness of optimal transport to noise and sampling variation?
• RQ2 How does allowing distinct anchor sets affect the interpretability and structural fidelity of the resulting transport plan?
• RQ3 To what extent does LOT reduce the sensitivity of OT to high-dimensional noise and outliers compared to standard OT?
• RQ4 Does LOT’s anchor-based factorization provide a statistical advantage in terms of sampling efficiency and reduced curse of dimensionality?
• RQ5 Can LOT effectively align datasets with domain shifts or differing intrinsic structures, such as in computer vision domain adaptation tasks?

Key findings

• LOT significantly improves robustness to noise and sampling variation in high-dimensional data, as demonstrated on synthetic GMMs and MNIST-USPS domain adaptation.
• The method reduces the rank of the transport plan through anchor-based factorization, which acts as a denoising mechanism and stabilizes the solution.
• By using separate anchors for source and target, LOT better captures domain shifts and differing local geometries, leading to more interpretable transport maps.
• LOT achieves state-of-the-art performance in domain adaptation on MNIST-USPS, with accuracy improvements over standard OT and factored coupling (FC) baselines.
• The number of anchors can be selected via model selection (e.g., silhouette score or elbow method), and overestimation has minimal impact on performance, indicating robustness to hyperparameter choice.
• LOT provides a statistical relaxation of OT that overcomes the curse of dimensionality in sampling requirements, as shown in theoretical analysis and empirical validation.