**4. LMS and NLMS algorithms for multilinear forms**

The limitations of the Wiener filter (e.g., matrix inversion, statistics estimation) can restrict the applicability of the previously presented approach in real-world situations (for example, in nonstationary conditions, or when real-time processing is needed). Therefore, a better approach may be represented by adaptive filters. In this context, the well-known least-mean-square (LMS) algorithm is among the most popular solutions, due to its simplicity. In the following, a family of LMS-based algorithms for multilinear forms identification is presented.

By using the estimated impulse responses **<sup>h</sup>**^*i*ð Þ*<sup>t</sup>* , *<sup>i</sup>* <sup>¼</sup> 1, 2, … , *<sup>N</sup>*, the corresponding a priori error signals can be defined

⋮

$$e\_{\hat{\mathbf{h}}\_{2}\hat{\mathbf{h}}\_{3}\ldots\hat{\mathbf{h}}\_{N}}(t) = d(t) - \hat{\mathbf{h}}\_{1}^{T}(t-1)\mathbf{x}\_{\hat{\mathbf{h}}\_{2}\hat{\mathbf{h}}\_{3}\ldots\hat{\mathbf{h}}\_{N}}(t),\tag{33}$$

$$d\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_3 \dots \hat{\mathbf{h}}\_N}(t) = d(t) - \hat{\mathbf{h}}\_2^T(t-1) \mathbf{x}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_3 \dots \hat{\mathbf{h}}\_N}(t),\tag{34}$$

$$\boldsymbol{\sigma}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_2 \dots \hat{\mathbf{h}}\_{N-1}}(t) = d(t) - \hat{\mathbf{h}}\_N^T(t-1) \mathbf{x}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_2 \dots \hat{\mathbf{h}}\_{N-1}}(t), \tag{35}$$

where

$$\mathbf{x}\_{\hat{\mathbf{h}}\_{2}\hat{\mathbf{h}}\_{3}\ldots\hat{\mathbf{h}}\_{N}}(t) = [\hat{\mathbf{h}}\_{N}(t-1)\otimes\hat{\mathbf{h}}\_{N-1}(t-1)\otimes\cdots\otimes\hat{\mathbf{h}}\_{2}(t-1)\otimes\mathbf{I}\_{L\_{1}}]\mathbf{x}(t),\tag{36}$$

$$\mathbf{x}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_3 \dots \hat{\mathbf{h}}\_N}(t) = [\hat{\mathbf{h}}\_N(t-1) \otimes \hat{\mathbf{h}}\_{N-1}(t-1) \otimes \dots \otimes \hat{\mathbf{h}}\_3(t-1) \otimes \mathbf{I}\_{L4} \otimes \hat{\mathbf{h}}\_1(t-1)] \mathbf{x}(t), \tag{37}$$
 
$$\vdots$$

$$\mathbf{x}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_2 \dots \hat{\mathbf{h}}\_{N-1}}(t) = [\mathbf{I}\_{L\_N} \otimes \hat{\mathbf{h}}\_{N-1}(t-1) \otimes \dots \otimes \hat{\mathbf{h}}\_2(t-1) \hat{\mathbf{h}}\_1(t-1)] \mathbf{x}(t). \tag{38}$$

We can easily check that *<sup>e</sup>***h**^2**h**^<sup>3</sup> … **<sup>h</sup>**^*<sup>N</sup>* ðÞ¼ *<sup>t</sup> <sup>e</sup>***h**^1**h**^<sup>3</sup> … **<sup>h</sup>**^*<sup>N</sup>* ðÞ¼ *<sup>t</sup>* … <sup>¼</sup> *<sup>e</sup>***h**^1**h**^<sup>2</sup> … **<sup>h</sup>**^*N*�<sup>1</sup> ð Þ*t* . Hence, the LMS updates of the individual filters will be

$$\begin{split} \hat{\mathbf{h}}\_{1}(t) &= \hat{\mathbf{h}}\_{1}(t-1) - \frac{\mu\_{\hat{\mathbf{h}}\_{1}}}{2} \times \frac{\partial \mathbf{e}\_{\hat{\mathbf{h}}\_{1}\hat{\mathbf{h}}\_{\hat{\mathbf{h}}\_{\hat{\mathbf{h}}\_{N}}}}^{2}(t)}{\partial \hat{\mathbf{h}}\_{1}(t-1)} \\ &= \hat{\mathbf{h}}\_{1}(t-1) + \mu\_{\hat{\mathbf{h}}\_{1}} \mathbf{x}\_{\hat{\mathbf{h}}\_{2}\hat{\mathbf{h}}\_{3}\ldots\hat{\mathbf{h}}\_{N}}(t) e\_{\hat{\mathbf{h}}\_{1}\hat{\mathbf{h}}\_{3}\ldots\hat{\mathbf{h}}\_{N}}(t), \end{split} \tag{39}$$
 
$$\begin{split} \hat{\mathbf{h}}\_{2}(t) &= \hat{\mathbf{h}}\_{2}(t-1) - \frac{\mu\_{\hat{\mathbf{h}}\_{2}}}{2} \times \frac{\partial \mathbf{e}\_{\hat{\mathbf{h}}\_{1}\hat{\mathbf{h}}\_{3}\ldots\hat{\mathbf{h}}\_{N}}^{2}(t)}{\partial \hat{\mathbf{h}}\_{2}(t-1)} \\ &= \hat{\mathbf{h}}\_{2}(t-1) + \mu\_{\hat{\mathbf{h}}\_{1}} \mathbf{x}\_{\hat{\mathbf{h}}\_{1}\hat{\mathbf{h}}\_{3}\ldots\hat{\mathbf{h}}\_{N}}(t) \mathbf{e}\_{\hat{\mathbf{h}}\_{1}\hat{\mathbf{h}}\_{3}\ldots\hat{\mathbf{h}}\_{N}}(t), \end{split} \tag{40}$$

*Identification of Multilinear Systems: A Brief Overview DOI: http://dx.doi.org/10.5772/intechopen.102765*

$$\begin{split} \hat{\mathbf{h}}\_{N}(t) &= \hat{\mathbf{h}}\_{N}(t-\mathbf{1}) - \frac{\mu\_{\hat{\mathbf{h}}\_{N}}}{2} \times \frac{\partial e^{2}\_{\hat{\mathbf{h}}\_{1}\hat{\mathbf{h}}\_{2}\ldots\hat{\mathbf{h}}\_{N-1}}(t)}{\partial \hat{\mathbf{h}}\_{N}(t-\mathbf{1})} \\ &= \hat{\mathbf{h}}\_{N}(t-\mathbf{1}) + \mu\_{\hat{\mathbf{h}}\_{N}} \mathbf{x}\_{\mathbf{\hat{h}}\_{1}\hat{\mathbf{h}}\_{2}\ldots\hat{\mathbf{h}}\_{N-1}}(t) e\_{\mathbf{\hat{h}}\_{1}\hat{\mathbf{h}}\_{2}\ldots\hat{\mathbf{h}}\_{N-1}}(t), \end{split} \tag{41}$$

where *μ***h**^*<sup>i</sup>* > 0, *i* ¼ 1, 2, … , *N* represent the step-size parameters. Relations (39–41) describe the LMS algorithm for multilinear forms (LMS-MF). The initialization of the estimated impulse responses could be

$$\hat{\mathbf{h}}\_1(\mathbf{0}) = \begin{bmatrix} \mathbf{1} & \mathbf{0} & \cdots & \mathbf{0} \end{bmatrix}^T,\tag{42}$$

$$\hat{\mathbf{h}}\_k(\mathbf{0}) = \frac{1}{L\_k} [\mathbf{1} \quad \mathbf{1} \quad \cdots \quad \mathbf{1}]^T, \ k = 2, 3, \dots, N. \tag{43}$$

The global filter estimate is obtained as

$$
\hat{\mathbf{g}}(t) = \hat{\mathbf{h}}\_N(t) \otimes \hat{\mathbf{h}}\_{N-1}(t) \otimes \dots \otimes \hat{\mathbf{h}}\_1(t). \tag{44}
$$

We may also identify the global impulse response using the classical LMS algorithm:

$$
\hat{\mathbf{g}}(t) = \hat{\mathbf{g}}(t-\mathbf{1}) + \mu\_{\hat{\mathbf{g}}} \mathbf{x}(t)e(t), \tag{45}
$$

$$e(t) = d(t) - \hat{\mathbf{g}}(t-1)\mathbf{x}(t),\tag{46}$$

where *μ***g**^ denotes the global step-size parameter, but this would involve a single longlength adaptive filter.

When choosing the constant values of the step-size parameters from Eqs. (39)– (41), we need to consider the compromise between convergence rate and steady-state misadjustment. In certain cases, it can be more useful to have variable step-size parameters. Hence, the update equations become

⋮

$$
\hat{\mathbf{h}}\_1(t) = \hat{\mathbf{h}}\_1(t-1) + \mu\_{\hat{\mathbf{h}}\_1}(t) \mathbf{x}\_{\hat{\mathbf{h}}\_2 \hat{\mathbf{h}}\_3 \dots \hat{\mathbf{h}}\_N}(t) e\_{\hat{\mathbf{h}}\_2 \hat{\mathbf{h}}\_3 \dots \hat{\mathbf{h}}\_N}(t), \tag{47}
$$

$$
\hat{\mathbf{h}}\_2(t) = \hat{\mathbf{h}}\_2(t-\mathbf{1}) + \mu\_{\hat{\mathbf{h}}\_2}(t) \mathbf{x}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_3 \dots \hat{\mathbf{h}}\_N}(t) e\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_3 \dots \hat{\mathbf{h}}\_N}(t), \tag{48}
$$

$$
\hat{\mathbf{h}}\_N(t) = \hat{\mathbf{h}}\_N(t-\mathbf{1}) + \mu\_{\hat{\mathbf{h}}\_N}(t) \mathbf{x}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_2 \dots \hat{\mathbf{h}}\_{N-1}}(t) e\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_2 \dots \hat{\mathbf{h}}\_{N-1}}(t). \tag{49}
$$

Then, the a posteriori error signals can be defined as

$$\boldsymbol{\varepsilon}\_{\hat{\mathbf{h}}\_{2}\hat{\mathbf{h}}\_{3}\ldots\hat{\mathbf{h}}\_{N}}(t) = \boldsymbol{d}(t) - \hat{\mathbf{h}}\_{1}^{T}(t)\mathbf{x}\_{\hat{\mathbf{h}}\_{2}\hat{\mathbf{h}}\_{3}\ldots\hat{\mathbf{h}}\_{N}}(t),\tag{50}$$

$$\boldsymbol{\varepsilon}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_3 \dots \hat{\mathbf{h}}\_N}(t) = \boldsymbol{d}(t) - \hat{\mathbf{h}}\_2^T(t) \mathbf{x}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_3 \dots \hat{\mathbf{h}}\_N}(t), \tag{51}$$

$$\boldsymbol{\varepsilon}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_2 \dots \hat{\mathbf{h}}\_{N-1}}(t) = d(t) - \hat{\mathbf{h}}\_N^T(t) \mathbf{x}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_2 \dots \hat{\mathbf{h}}\_{N-1}}(t). \tag{52}$$

After replacing Eq. (39) in Eq. (50), Eq. (40) in Eq. (51), and Eq. (41) in Eq. (52), and then canceling the a posteriori error signals, we get

⋮

$$\mathbf{e}\_{\hat{\mathbf{h}}\_{2}\hat{\mathbf{h}}\_{3}\ldots\hat{\mathbf{h}}\_{N}}(t)\Big[\mathbf{1}-\mu\_{\hat{\mathbf{h}}\_{1}}(t)\mathbf{x}\_{\mathbf{\hat{h}}\_{2}\hat{\mathbf{h}}\_{3}\ldots\hat{\mathbf{h}}\_{N}}^{T}(t)\mathbf{x}\_{\hat{\mathbf{h}}\_{2}\hat{\mathbf{h}}\_{3}\ldots\hat{\mathbf{h}}\_{N}}(t)\Big]=\mathbf{0},\tag{53}$$

$$\sigma\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_3 \dots \hat{\mathbf{h}}\_N}(t) \Big[ \mathbf{1} - \mu\_{\hat{\mathbf{h}}\_1}(t) \mathbf{x}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_3 \dots \hat{\mathbf{h}}\_N}^T(t) \mathbf{x}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_3 \dots \hat{\mathbf{h}}\_N}(t) \Big] = \mathbf{0},\tag{54}$$
 
$$\vdots$$

$$\sigma\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_2 \dots \hat{\mathbf{h}}\_{N-1}}(t) \Big[ \mathbf{1} - \mu\_{\hat{\mathbf{h}}\_N}(t) \mathbf{x}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_2 \dots \hat{\mathbf{h}}\_{N-1}}^T(t) \mathbf{x}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_2 \dots \hat{\mathbf{h}}\_{N-1}}(t) \Big] = \mathbf{0}. \tag{55}$$

We assume that *<sup>e</sup>***h**^2**h**^<sup>3</sup> … **<sup>h</sup>**^*<sup>N</sup>* ð Þ*<sup>t</sup>* 6¼ 0, *<sup>e</sup>***h**^1**h**^<sup>3</sup> … **<sup>h</sup>**^*<sup>N</sup>* ð Þ*<sup>t</sup>* 6¼ 0, … , *<sup>e</sup>***h**^1**h**^<sup>2</sup> … **<sup>h</sup>**^*N*�<sup>1</sup> ð Þ*t* 6¼ 0. Hence, the step-sizes become

$$\mu\_{\hat{\mathbf{h}}\_1}(t) = \frac{1}{\mathbf{x}\_{\hat{\mathbf{h}}\_2 \hat{\mathbf{h}}\_3 \dots \hat{\mathbf{h}}\_N}^T(t) \mathbf{x}\_{\hat{\mathbf{h}}\_2 \hat{\mathbf{h}}\_3 \dots \hat{\mathbf{h}}\_N}(t)},\tag{56}$$

$$\mu\_{\hat{\mathbf{h}}\_{\mathcal{I}}}(t) = \frac{1}{\mathbf{x}\_{\hat{\mathbf{h}}\_{1}\hat{\mathbf{h}}\_{3}\ldots\hat{\mathbf{h}}\_{N}}^{T}(t)\mathbf{x}\_{\hat{\mathbf{h}}\_{1}\hat{\mathbf{h}}\_{3}\ldots\hat{\mathbf{h}}\_{N}}(t)},\tag{57}$$

$$\mu\_{\hat{\mathbf{h}}\_N}(t) = \frac{1}{\mathbf{x}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_2 \dots \hat{\mathbf{h}}\_{N-1}}(t) \mathbf{x}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_2 \dots \hat{\mathbf{h}}\_{N-1}}(t)} \,. \tag{58}$$

In the numerators of Eqs. (56)–(58), the normalized step-size parameters 0<*α***h**^*<sup>i</sup>* <1, *i* ¼ 1, 2, … , *N* can be used to achieve a good compromise between convergence rate and misadjustment. Some regularization constants denoted by *δ***h**^*i* > 0, *i* ¼ 1, 2, … , *N* are also introduced in the denominators of the step-sizes in order to ensure robust adaptation. Consequently, we obtain the update equations of the normalized LMS (NLMS) algorithm for multilinear forms (NLMS-MF):

⋮

$$
\hat{\mathbf{h}}\_1(t) = \hat{\mathbf{h}}\_1(t-1) + \frac{\alpha\_{\mathbf{\hat{h}}\_1} \mathbf{x}\_{\mathbf{\hat{h}}\_2 \mathbf{\hat{h}}\_3 \dots \mathbf{\hat{h}}\_N}(t) e\_{\mathbf{\hat{h}}\_2 \mathbf{\hat{h}}\_3 \dots \mathbf{\hat{h}}\_N}(t)}{\delta\_{\mathbf{\hat{h}}\_1} + \mathbf{x}\_{\mathbf{\hat{h}}\_2 \mathbf{\hat{h}}\_3 \dots \mathbf{\hat{h}}\_N}^T(t) \mathbf{x}\_{\mathbf{\hat{h}}\_2 \mathbf{\hat{h}}\_3 \dots \mathbf{\hat{h}}\_N}(t)}, \tag{59}
$$

$$
\hat{\mathbf{h}}\_2(t) = \hat{\mathbf{h}}\_2(t-1) + \frac{\alpha\_{\hat{\mathbf{h}}\_2} \mathbf{x}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_3 \dots \hat{\mathbf{h}}\_N}(t) e\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_3 \dots \hat{\mathbf{h}}\_N}(t)}{\delta\_{\hat{\mathbf{h}}\_2} + \mathbf{x}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_3 \dots \hat{\mathbf{h}}\_N}^T(t) \mathbf{x}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_3 \dots \hat{\mathbf{h}}\_N}(t)}, \tag{60}
$$

$$
\hat{\mathbf{h}}\_N(t) = \hat{\mathbf{h}}\_N(t-1) + \frac{a\_{\hat{\mathbf{h}}\_N} \mathbf{x}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_2 \dots \hat{\mathbf{h}}\_{N-1}}(t) \boldsymbol{\varepsilon}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_2 \dots \hat{\mathbf{h}}\_{N-1}(t)}}{\boldsymbol{\delta}\_{\hat{\mathbf{h}}\_N} + \mathbf{x}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_2 \dots \hat{\mathbf{h}}\_{N-1}}^T(t) \mathbf{x}\_{\hat{\mathbf{h}}\_1 \hat{\mathbf{h}}\_2 \dots \hat{\mathbf{h}}\_{N-1}}(t)}. \tag{61}
$$

The initialization of the individual impulse responses can be done using Eqs. (42, 43). We may also identify the global impulse response using the regular NLMS algorithm:

⋮

$$
\hat{\mathbf{g}}(t) = \hat{\mathbf{g}}(t-1) + \frac{a\_{\hat{\mathbf{g}}} \mathbf{x}(t) \mathbf{e}(t)}{\mathbf{x}^T(t) \mathbf{x}(t) + \delta\_{\hat{\mathbf{g}}}},\tag{62}
$$

where 0 <*α***g**^ ≤1 denotes the normalized step-size parameter, *δ***g**^ > 0 represents the regularization constant, and *e t*ð Þ is defined in Eq. (46). However, this would involve a single (long length) adaptive filter, with *L* ¼ *L*1*L*2⋯*LN*.
