[Paper Review] Manifolds' Projective Approximation Using The Moving Least-Squares (MMLS).

Barak Sober, David Levin|arXiv (Cornell University)|Jun 22, 2016

Face and Expression Recognition19 references3 citations

TL;DR

This paper proposes a non-linear moving least-squares (MMLS) method to approximate a smooth, d-dimensional submanifold embedded in high-dimensional space R^n from noisy, scattered data points. By projecting data onto a locally polynomial approximant using a moving least-squares framework, the method achieves infinite smoothness and high approximation order O(h^{m+1}), with computational cost linear in n, even without prior knowledge of the manifold's analytic form.

ABSTRACT

In order to avoid the curse of dimensionality, frequently encountered in Big Data analysis, there was a vast development in the field of linear and non-linear dimension reduction techniques in recent years. These techniques (sometimes referred to as manifold learning) assume that the scattered input data is lying on a lower dimensional manifold, thus the high dimensionality problem can be overcome by learning the lower dimensionality behavior. However, in real life applications, data is often very noisy. In this work, we propose a method to approximate a $d$-dimensional $C^{m+1}$ smooth submanifold $\mathcal{M}$ residing in $\mathbb{R}^n$ ($d << n$) based upon scattered data points (i.e., a data cloud). We assume that the data points are located near the noisy lower dimensional manifold and perform a non-linear moving least-squares projection on an approximating manifold. Under some mild assumptions, the resulting approximant is shown to be infinitely smooth and of high approximation order (i.e., $O(h^{m+1})$, where $h$ is the fill distance and $m$ is the degree of the local polynomial approximation). Furthermore, the method presented here assumes no analytic knowledge of the approximated manifold and the approximation algorithm is linear in the large dimension $n$.

Motivation & Objective

• To address the curse of dimensionality in Big Data by approximating high-dimensional data lying on a lower-dimensional manifold.
• To handle real-world noisy data in manifold learning, where traditional methods may fail due to lack of smoothness or noise sensitivity.
• To develop a method that is infinitely smooth, of high approximation order, and computationally efficient in high-dimensional input space.
• To provide a non-parametric approximation framework that does not require prior knowledge of the manifold's analytic form.

Proposed method

• The method uses a moving least-squares (MLS) framework to project scattered data points onto a local polynomial approximant on the manifold.
• It constructs a local polynomial approximation of degree m around each data point using weighted least squares, minimizing a weighted residual sum of squares.
• The approximation is performed by solving a local optimization problem that minimizes the error between data points and the polynomial surface, with weights decaying with distance.
• The resulting approximant is shown to be C∞ smooth due to the smooth weighting function and polynomial basis used in the local fitting.
• The method operates in the ambient high-dimensional space R^n, with computational cost linear in n, making it scalable to large dimensions.
• No analytic expression of the manifold is required; the method relies solely on the scattered data cloud and local geometric structure.

Experimental results

Research questions

• RQ1 Can a non-linear, high-order approximation of a smooth manifold be achieved from noisy, scattered data without prior knowledge of the manifold’s analytic form?
• RQ2 What is the approximation order of the proposed MMLS-based method, and how does it depend on the fill distance h and polynomial degree m?
• RQ3 How does the method maintain infinite smoothness despite noise and non-uniform sampling in high-dimensional data?
• RQ4 Can the computational cost remain linear in the ambient dimension n while achieving high approximation accuracy?
• RQ5 How does the method perform in terms of convergence and stability under mild assumptions on data distribution and manifold smoothness?

Key findings

• The proposed MMLS approximant is infinitely smooth (C∞) due to the smoothness of the weighting function and polynomial basis in the local approximation.
• The method achieves high approximation order O(h^{m+1}), where h is the fill distance and m is the degree of the local polynomial approximation.
• The computational cost of the algorithm scales linearly with the ambient dimension n, enabling scalability to high-dimensional data.
• The method does not require any analytic knowledge of the underlying manifold, making it suitable for real-world noisy data applications.
• Under mild assumptions on data distribution and manifold smoothness, the method provides stable and accurate manifold approximation even in the presence of noise.
• The approach effectively mitigates the curse of dimensionality by exploiting the low intrinsic dimensionality of the data manifold.