4

The block LMS and conventional LMS have the same convergence rate and the same misadjustment. I am having trouble wrapping my head around this. The block LMS uses a more accurate estimate of the gradient vector at each iteration. Conceptually, why does a better gradient estimate have no benefit on the descent?

Dres
  • 41
  • 1
  • 2
    The reduction of the variance of the gradient estimate is exactly compensated by the fact that the filter coefficients are only updated *once per block*. Note, however, that the restrictions on the step size to avoid filter divergence are stronger for the block version of the algorithm. This in many cases leads to a performance of the block LMS which is lower than the performance of the conventional LMS. – applesoup Mar 11 '18 at 10:59
  • Oh, welcome to DSP.SE, by the way! :-) – applesoup Mar 11 '18 at 17:12
  • (I am OP - forgot I already had an account). I don't have the reputation to comment, but I still wanted to thank applesoup for the reply. Makes perfect sense. Instead of steps every iteration, you make one "big" step every $L$ iterations, where this big step is an aggregate of $L$ gradient estimates. So you benefit on variance of the lump gradient estimate, but lose on the "granularity" of the estimates, so to speak. – Probably Mar 11 '18 at 19:16
  • 1
    This does not provide an answer to the question. Once you have sufficient [reputation](https://dsp.stackexchange.com/help/whats-reputation) you will be able to [comment on any post](https://dsp.stackexchange.com/help/privileges/comment); instead, [provide answers that don't require clarification from the asker](https://meta.stackexchange.com/questions/214173/why-do-i-need-50-reputation-to-comment-what-can-i-do-instead). - [From Review](/review/low-quality-posts/28707) – MBaz Mar 11 '18 at 23:54
  • 2
    @Probably Please go through [account merging](https://dsp.stackexchange.com/help/merging-accounts) to remove one of these accounts. – Peter K. Mar 12 '18 at 00:39

2 Answers2

2

here is a short and little (a.k.a. "consise but terse") derivation of the LMS and normalized LMS adaptive filter.

once the LMS has converged on a reasonably stable equilibrium for $h_n[k]$, they won't move around that much. then it doesn't matter so much how long the block is. and the only difference between block-LMS and the plain-old ordinary LMS is the block size. (the block size for the latter is 1.)

robert bristow-johnson
  • 16,504
  • 3
  • 29
  • 68
0

The reason is in the correlation matrix $\textbf{R}$. In the pointwise LMS, the entry in your $\textbf{R}$ is $E[x(i)\,x(j)]$. However, in block LMS, $\textbf{R}_{\text{bk}}$ is $L\cdot E[x(i)\,x(j)]$, where $L$ is block length. Or you can write $\textbf{R}_{\text{bk}} = L \cdot \textbf{R}$. Recall that $\textbf{R} = \textbf{P} \Lambda \textbf{P}^T$, where $\textbf{P} \textbf{P}^T = I$ and $\Lambda$ is diagonal. Hence, $\textbf{R}_{\text{bk}} = \textbf{P} (L\Lambda) \textbf{P}^T$------(1). Assume $\Lambda$ is 2 by 2 with diagonal $\lambda_1$ and $ \lambda_2$ and $\lambda_1 > \lambda_2$. Recall the condition of convergence is $0< \mu< 1/\lambda_1$ for pointwise LMS, where $\mu$ is step size. Hence the convergent condition for block LMS is $0< \mu< \frac{1}{L\lambda_1}$ from (1). So you did have better estimate of matrix but your stepsize is $L$ times smaller.

Po-Yu Paul Chen
  • 1
  • 1