[Paper Review] Memory Approximate Message Passing

Lei Liu, Shunqi Huang+1 more|arXiv (Cornell University)|Jun 4, 2021

Advanced Adaptive Filtering Techniques33 references4 citations

TL;DR

This paper proposes Memory Approximate Message Passing (MAMP), a low-complexity, Bayes-optimal algorithm for high-dimensional linear systems with right-unitarily-invariant matrices. By incorporating a long-memory matched filter and orthogonalization of estimation errors, MAMP achieves performance matching OAMP/VAMP—previously requiring high-complexity LMMSE—while maintaining AMP-level complexity.

ABSTRACT

Approximate message passing (AMP) is a low-cost iterative parameter-estimation technique for certain high-dimensional linear systems with non-Gaussian distributions. However, AMP only applies to independent identically distributed (IID) transform matrices, but may become unreliable for other matrix ensembles, especially for ill-conditioned ones. To handle this difficulty, orthogonal/vector AMP (OAMP/VAMP) was proposed for general right-unitarily-invariant matrices. However, the Bayes-optimal OAMP/VAMP requires high-complexity linear minimum mean square error estimator. To solve the disadvantages of AMP and OAMP/VAMP, this paper proposes a memory AMP (MAMP), in which a long-memory matched filter is proposed for interference suppression. The complexity of MAMP is comparable to AMP. The asymptotic Gaussianity of estimation errors in MAMP is guaranteed by the orthogonality principle. A state evolution is derived to asymptotically characterize the performance of MAMP. Based on the state evolution, the relaxation parameters and damping vector in MAMP are optimized. For all right-unitarily-invariant matrices, the optimized MAMP converges to OAMP/VAMP, and thus is Bayes-optimal if it has a unique fixed point. Finally, simulations are provided to verify the validity and accuracy of the theoretical results.

Motivation & Objective

• To address the limitations of AMP, which fails for non-i.i.d. or ill-conditioned matrices.
• To overcome the high computational cost of OAMP/VAMP, which relies on costly linear MMSE estimation.
• To develop a low-complexity algorithm that achieves Bayes-optimality for general right-unitarily-invariant matrices.
• To ensure asymptotic Gaussianity of estimation errors through strict orthogonality constraints.
• To optimize relaxation parameters and damping for fast, stable convergence without breaking statistical properties.

Proposed method

• Proposes a memory AMP (MAMP) that replaces the standard matched filter in AMP with a long-memory matched filter to suppress interference.
• Implements a strict orthogonality condition where the current output estimation error is orthogonal to all prior input estimation errors, ensuring asymptotic Gaussianity of errors.
• Derives a covariance-matrix state evolution to analytically characterize MAMP’s performance and guide parameter optimization.
• Introduces relaxation parameters and a damping vector to improve convergence speed while preserving orthogonality and Gaussianity.
• Uses state evolution to optimize parameters such that MAMP converges to OAMP/VAMP performance when a unique fixed point exists.
• Employs a damping strategy applied to orthogonal outputs, preserving the statistical properties required for accurate state evolution.

Experimental results

Research questions

• RQ1 Can a low-complexity message-passing algorithm achieve Bayes-optimality for general right-unitarily-invariant matrices without requiring high-complexity LMMSE estimation?
• RQ2 How can memory and orthogonalization be combined to maintain asymptotic Gaussianity of estimation errors in iterative algorithms?
• RQ3 What parameter tuning strategy (relaxation and damping) ensures stable and fast convergence in memory-based AMP?
• RQ4 Does the state evolution of MAMP accurately predict its empirical performance across different matrix condition numbers?
• RQ5 How does MAMP compare in convergence speed and accuracy to CAMP and OAMP/VAMP, especially for ill-conditioned matrices?

Key findings

• MAMP achieves Bayes-optimality for all right-unitarily-invariant matrices when it converges to a unique fixed point, matching the performance of OAMP/VAMP.
• The state evolution of MAMP accurately predicts its mean squared error (MSE) performance across various matrix condition numbers, including κ=100.
• With optimized relaxation parameters and damping length L=3, MAMP converges faster than CAMP and matches OAMP/VAMP performance.
• Damping applied to orthogonal outputs preserves the asymptotic Gaussianity of estimation errors, unlike CAMP where damping on a-posteriori outputs breaks this property.
• For ill-conditioned matrices (κ=100), MAMP remains stable and converges, while CAMP diverges when κ>15.
• The MSE of MAMP is nearly identical for damping lengths L≥3, indicating that L=3 is sufficient for optimal convergence speed and stability.