A study of LMS-based algorithms: exploiting plain and hidden sparsity

Abstract

Over the last decades, adaptive filters have been used in many different applications. This dissertation focuses on one in particular: the identification of systems presenting some kind of sparsity. A system is sparse when we represent it by only a few coefficients in some domain. This work addresses the following kinds of sparsity: the plain and the hidden. The plain one is well-known in the adaptive filtering field and occurs when the sparsity is directly observed in the coefficients space, without any mathematical manipulation. There are two distinct families of algorithms that were intended to tackle this problem. The proportionate family tries to exploit the plain sparsity by assigning particular step sizes to each of the coefficients. The second family is the regularized-type, which relies on sparsity promoting functions. Since the proportionate family of algorithms does not model the sparsity explicitly, this work performs a thorough study of this family, addressing some undocumented properties. Therefore, we present several numerical results, concluding that they exploit something more general than the plain sparsity. The hidden sparsity is a more recent problem in the adaptive filtering field, in which we reveal the system sparsity through some mathematical manipulation. The feature least-mean-square (F-LMS) algorithm addresses this problem introducing a feature matrix in the objective function of the LMS algorithm. However, several applications have both types of sparsity, and the F-LMS can not exploit the plain one. Then, we propose the simple sparsity-aware F-LMS (SSF-LMS), which exploits both of them while requiring less arithmetic operations by using the discard function.