[Paper Review] Zero Attracting PNLMS Algorithm and Its Convergence in Mean

Rajib Lochan Das, Mrityunjoy Chakraborty|arXiv (Cornell University)|Jul 10, 2015

Advanced Adaptive Filtering Techniques23 references3 citations

TL;DR

This paper proposes the Zero-Attracting PNLMS (ZA-PNLMS) algorithm, which enhances the PNLMS algorithm by incorporating an l1-norm penalty to promote sparsity, thereby accelerating convergence and reducing steady-state excess mean square error (EMSE). The method introduces zero attractors that shrink inactive tap coefficients, mitigating the convergence slowdown typical of PNLMS in later iterations.

ABSTRACT

The proportionate normalized least mean square (PNLMS) algorithm and its variants are by far the most popular adaptive filters that are used to identify sparse systems. The convergence speed of the PNLMS algorithm, though very high initially, however, slows down at a later stage, even becoming worse than sparsity agnostic adaptive filters like the NLMS. In this paper, we address this problem by introducing a carefully constructed l1 norm (of the coefficients) penalty in the PNLMS cost function which favors sparsity. This results in certain zero attractor terms in the PNLMS weight update equation which help in the shrinkage of the coefficients, especially the inactive taps, thereby arresting the slowing down of convergence and also producing lesser steady state excess mean square error (EMSE). We also carry out the convergence analysis (in mean) of the proposed algorithm.

Motivation & Objective

• To address the slow convergence of PNLMS in later iterations due to diminishing step sizes for inactive taps.
• To improve both transient and steady-state performance of PNLMS for sparse system identification.
• To reduce steady-state excess mean square error (EMSE) by promoting sparsity through coefficient shrinkage.
• To provide a rigorous mean convergence analysis of the proposed ZA-PNLMS algorithm despite the complexity introduced by gain matrices and zero attractors.

Proposed method

• Introduces an l1-norm penalty in the PNLMS cost function to favor sparse solutions.
• Derives a modified weight update equation with zero attractor terms that pull inactive tap coefficients toward zero.
• Uses a transform domain equivalent model to simplify convergence analysis.
• Applies angular discretization of continuous-valued random vectors to handle the statistical complexity of the gain matrix and zero attractors.
• Employs a recursive least mean square (RLMS) framework with a diagonal gain matrix G(n) that adapts step sizes proportionally to tap weight magnitudes.
• Analyzes convergence in mean by assuming statistical independence between G⁻¹/²(n) and sign(w(n)) in steady state, enabling derivation of bias expressions.

Experimental results

Research questions

• RQ1 How does the inclusion of an l1-norm penalty affect the convergence behavior of PNLMS in sparse systems?
• RQ2 To what extent does the zero attractor mechanism reduce steady-state EMSE compared to standard PNLMS?
• RQ3 Can a rigorous mean convergence analysis be performed for ZA-PNLMS despite the presence of a gain matrix and nonlinear zero attractor terms?
• RQ4 How does the bias in estimated tap weights vary with the magnitude of the true optimal weights in the ZA-PNLMS algorithm?
• RQ5 What is the impact of the zero attractor on the convergence rate and stability of the algorithm in later iterations?

Key findings

• The ZA-PNLMS algorithm achieves faster convergence and lower steady-state EMSE than standard PNLMS by actively shrinking inactive tap coefficients.
• Active taps with larger magnitudes experience less bias in estimation, while smaller-magnitude active taps show increased bias proportional to the inverse of their gain factor.
• Inactive taps (with w_opt,i = 0) exhibit negligible bias in steady state due to the zero attractor, which confines their estimates to a small band around zero.
• The mean convergence analysis confirms that the estimated weights converge to values closer to the origin than the true optimal weights, especially for active taps.
• Simulation results validate that E[w_i(n)] converges to the optimal value for large-magnitude active taps (e.g., w_opt,37 = 0.9), while smaller active taps (e.g., w_opt,55 = 0.1) show measurable bias.
• The bias is quantitatively shown to decrease with increasing optimal tap magnitude, as confirmed by the learning curves and bias-vs-magnitude plot in Figure 6.