Ubiquitous Filtering for Nonlinear Problems

Valeri Kontorovich and Fernando Ramos-Alarcon

#### Abstract

This chapter develops and extends the general theoretical results, previously published in the chapter "Nonlinear filtering of weak chaotic signals", and presents detailed implementations of a computationally simple, robust (filtering fidelity almost insensitive to changes of the desired input signal properties) and rather precise approach for the filtering of weak signals of different physical nature (biological, seismic, voice, etc.) in presence of white Gaussian noise. The implementations rely on non-linear filtering techniques that in general can be considered as either one-moment or multi-moment, in the sense that they operate with a single sample (instantaneous fashion) or with various adjacent samples (non-instantaneous fashion). Chaotic modeling of the real input signals allows achieving an almost ubiquitous filtering approach with a computationally simple implementation. Application of the linearization strategies (for both one and twomoment filtering) provide, additionally, "invariance" of the processing algorithms to variations on the nature and statistics of the input signals.

Keywords: system identification toolbox, linearized models, weak chaos, quasi-linear filtering algorithms

#### 1. Introduction

The signal filtering plays a fundamental role in the design of signal processing algorithms for many problems, that is, the first step is to remove (to filter) the background noise from the input (incoming) signal, and the second step is to perform the corresponding signal processing [1, 2]. In this sense, the filtering approach based on the theory of dynamic systems [3–5] pops up immediately as one of the possible ways to address this issue. The dynamic filtering approach, such as classic linear Kalman filtering, has been applied for many problems long ago [6] and recently as well [7]. However, in the following the dynamic filtering is proposed adopting a different (nonlinear) angle [8, 9], namely, using signals from nonlinear chaotic attractors as a model for the desired signals arriving at the filtering structure. The modeling of real phenomena using chaos has been used for more than 50 years, and there is a wide range of scientific and practical applications, such as seismology [10–12], statistical theory of communication [13, 14], control, geophysics, biomedical telemetry [15, 16] under water signal processing [17], and many other areas related to applied physics as well [18].

When the signals of interest are significantly weak (smaller than the background additive white Gaussian noise, AWGN), the problem is far from trivial. The

following material will show the effectiveness of using a dynamic nonlinear strategy (introduced in [4, 5]) for filtering signals, belonging to different types of real phenomena, which are modeled through components of chaotic attractors, all this in presence of strong AWGN, which concretely means a signal to noise ratio, SNR, < 1 (<0 dB). Note that even though weak signals are treated in the literature, [10, 12], their processing is not addressed from the dynamic filtering point of view, and therefore (in our opinion) optimum fidelity solutions are still required.

been accomplished so far, at least to the author's knowledge. The reason behind this is the identification of Eq. (1) is an identification of the "inertial vector nonlinear system" which does not have an unique solution and can be formulated only for a previously defined class of nonlinear systems; the complexity of this task has been addressed elsewhere [8, 9, 18, 20] and will not be considered in the following. As examples for F(∙), which will be used in the following, there are the equations for the chaotic attractors corresponding to Rossler, Lorenz, and Chua

xkþ<sup>1</sup> ¼ xk þ Ts �yk � zk

ykþ<sup>1</sup> <sup>¼</sup> yk <sup>þ</sup> Ts xk <sup>þ</sup> <sup>0</sup>:2yk

zkþ<sup>1</sup> <sup>¼</sup> zk <sup>þ</sup> Ts � <sup>8</sup>

ykþ<sup>1</sup> <sup>¼</sup> ykþ<sup>1</sup> <sup>þ</sup> Ts xk � yk <sup>þ</sup> <sup>z</sup><sup>3</sup>

zkþ<sup>1</sup> ¼ zkþ<sup>1</sup> þ Ts �14:3yk

<sup>2</sup> ð Þ <sup>m</sup><sup>0</sup> � <sup>m</sup><sup>1</sup> ½ � j j xk <sup>þ</sup> <sup>1</sup> � j j xk � <sup>1</sup> , <sup>m</sup><sup>0</sup> ¼ � <sup>1</sup>

In order to neglect the uncertainty effects of the initial conditions, at least for real data filtering, the approach used in [21], based on the fundamental statement of statistical dynamics for deterministic systems related to Kolmogorov and Max Born [18], together with the introduction of the so-called additive "process noise" in Eq. (1), can be applied. The latter transforms the ODE Eq. (1) into a stochastic differential equation (SDE) [20]. The transformation of Eq. (1) into SDE is relevant

The equation for strange attractor Eq. (1) can be transformed into the equivalent stochastic form as a stochastic differential equation (SDE), which "generates" the

where f(x(t)) is identical to F(x(t)) from Eq. (1). The influence of a weak external source of white noise is denoted by ξ(t); the noise intensities are given in a

The solutions proposed hereafter might be encountered from the structural analysis of the quasi-optimum filtering algorithms for weak chaos in presence of

When one uses the SDE Eq. (5) as a model for chaos, the first strategy that comes up immediately to mind is the nonlinear filtering of chaotic signals which was rigorously developed in [8, 9]. The kernel presented in [8] is the Stratonovich-Kushner equation (SKE) [4, 5], which allows to describe the dynamic equation for the a posteriori probability density function for the chaos x(t). For the filtering with this generalized approach, some additional information from the received aggregate

AWGN [8, 9] and are synthesized in the following for convenience.

signal has to be incorporated on several sequential time instants, i.e., the

ykþ<sup>1</sup> <sup>¼</sup> yk <sup>þ</sup> Ts <sup>28</sup>xk � yk <sup>þ</sup> xk � zk

3

,

,

x\_ ¼ f xð Þþ ð Þt εξð Þt (5)

zk þ xk � yk 

,

(2)

(3)

(4)

<sup>7</sup> and

<sup>7</sup>, and <sup>m</sup><sup>1</sup> <sup>¼</sup> <sup>2</sup>

zkþ<sup>1</sup> ¼ zk þ Tsð Þ 0:2 � zkð Þ 5:7 � xk

xkþ<sup>1</sup> ¼ xk þ Ts 10 xk � yk

xkþ<sup>1</sup> ¼ xkþ<sup>1</sup> þ Ts 9:205 yk � U xð Þ<sup>k</sup>

Continuous time: Discrete time:

x t \_ðÞ¼�y tðÞ� z tð Þ y t \_ðÞ¼ x tð Þþ 0:2y tð Þ

Ubiquitous Filtering for Nonlinear Problems DOI: http://dx.doi.org/10.5772/intechopen.88409

x t \_ðÞ¼ 10ð Þ x tðÞ� y tð Þ

3

x t \_ðÞ¼ 9:205ð Þ y tð Þ� Uxt ð Þ ð Þ , y t \_ðÞ¼ x tð Þ� y tðÞþ z tð Þ,

z t \_ðÞ¼� <sup>8</sup>

z t \_ðÞ¼�14:3y tð Þ:

where U xð Þ¼ <sup>k</sup> <sup>m</sup>1xk <sup>þ</sup> <sup>1</sup>

TS is the sampling time.

for the following material.

matrix form ε = [εij]

91

z t \_ðÞ¼ 0:2 � z tð Þð Þ 5:7 � x tð Þ

y t \_ðÞ¼ 28x tðÞ� y tðÞþ x tðÞ� z tð Þ

n-dimensional Markov stochastic process [18, 20]:

nxn .

z tðÞþ x tðÞ� y tð Þ

types [8, 9]:

Rossler

Lorenz

Chua

In the following the term, effective filtering is used to indicate high precision which is evaluated considering values of the normalized mean square error, NMSE, <1% (in the following the normalization of the MSE will be considered in relation to the variance of the desired input signal). In the regular practice, there might be several precision measures corresponding to each specific filtering scenario. The use of the NMSE, or RMSE (root mean square error for nonstationary scenarios), as a measure of precision (fidelity) for filtering is well established from the statistical theory [19], and so, it can be considered as "universal" because its formal definition is the same irrespectively to specific filtering scenarios [1, 2]. Also note that the case of small values of the NMSE might adequately correspond to concrete practical criteria of fidelity [3, 19].

The proposed strategy is robust but not in the sense used in control theory, where the term "robust" means that the filter's structure is invariant to a priori unknown features of the input signals. The proposed chaotic filtering is considered as robust in the sense that its fixed structure and fidelity are almost invariant to signals from rather different filtering scenarios, which in the following correspond to seismic signals, heart beat signals, voice-like signals, and radio frequency interference (RFI) signals. Actually one can see that such invariance makes the filtering "ubiquitous."

For conditions when the SNR ≤ 0 dB, the term "weak chaotic signals" will be used, keeping in mind that chaos modeling is applied for the abovementioned filtering scenarios. Chaos modeling might be immensely useful because almost all quasi-optimum filtering algorithms (which formally are nonlinear but are essentially quasi-linear) show rather high precision in the sense of low NMSE (≤1%) and exhibit low computational complexity among other benefits. These properties were broadly discussed in [8, 9], where theoretical proofs can be found. The material presented here contains some experimental applications, for rather different scenarios that apply and extend the ideas presented in [9], and so, both have to be considered together. In [9] it was impossible for the authors to include the present material, not even partially, for the lack of space.

#### 2. Extraction of some theoretical principles

#### 2.1 Chaotic modeling and filtering

Let us assume that a chaos vector process x(t) can be generated from the following ordinary differential equation (ODE) of certain attractor [18]:

$$
\dot{\boldsymbol{x}} = F(\boldsymbol{x}, t) \tag{1}
$$

where the initial condition is x(t0) = x<sup>0</sup> and F(∙) is a vector function, which (for any real application) is a priori unknown (together with the initial conditions) and needs to be somehow identified beforehand and moreover is usually time varying. It is worth mentioning that the identification problem for F(∙) has attracted a lot of interest in the last decades, but being rather complex, it has not

Ubiquitous Filtering for Nonlinear Problems DOI: http://dx.doi.org/10.5772/intechopen.88409

been accomplished so far, at least to the author's knowledge. The reason behind this is the identification of Eq. (1) is an identification of the "inertial vector nonlinear system" which does not have an unique solution and can be formulated only for a previously defined class of nonlinear systems; the complexity of this task has been addressed elsewhere [8, 9, 18, 20] and will not be considered in the following. As examples for F(∙), which will be used in the following, there are the equations for the chaotic attractors corresponding to Rossler, Lorenz, and Chua types [8, 9]:

Continuous time: Discrete time:

following material will show the effectiveness of using a dynamic nonlinear strategy (introduced in [4, 5]) for filtering signals, belonging to different types of real phenomena, which are modeled through components of chaotic attractors, all this in presence of strong AWGN, which concretely means a signal to noise ratio, SNR, < 1 (<0 dB). Note that even though weak signals are treated in the literature, [10, 12], their processing is not addressed from the dynamic filtering point of view,

In the following the term, effective filtering is used to indicate high precision which is evaluated considering values of the normalized mean square error, NMSE, <1% (in the following the normalization of the MSE will be considered in relation to the variance of the desired input signal). In the regular practice, there might be several precision measures corresponding to each specific filtering scenario. The use of the NMSE, or RMSE (root mean square error for nonstationary scenarios), as a measure of precision (fidelity) for filtering is well established from the statistical theory [19], and so, it can be considered as "universal" because its formal definition is the same irrespectively to specific filtering scenarios [1, 2]. Also note that the case of small values of the NMSE might adequately correspond to concrete practical

The proposed strategy is robust but not in the sense used in control theory, where the term "robust" means that the filter's structure is invariant to a priori unknown features of the input signals. The proposed chaotic filtering is considered as robust in the sense that its fixed structure and fidelity are almost invariant to signals from rather different filtering scenarios, which in the following correspond to seismic signals, heart beat signals, voice-like signals, and radio frequency interference (RFI) signals. Actually one can see that such invariance makes the filtering

For conditions when the SNR ≤ 0 dB, the term "weak chaotic signals" will be used, keeping in mind that chaos modeling is applied for the abovementioned filtering scenarios. Chaos modeling might be immensely useful because almost all quasi-optimum filtering algorithms (which formally are nonlinear but are essentially quasi-linear) show rather high precision in the sense of low NMSE (≤1%) and exhibit low computational complexity among other benefits. These properties were broadly discussed in [8, 9], where theoretical proofs can be found. The material presented here contains some experimental applications, for rather different scenarios that apply and extend the ideas presented in [9], and so, both have to be considered together. In [9] it was impossible for the authors to include the present

Let us assume that a chaos vector process x(t) can be generated from the following ordinary differential equation (ODE) of certain attractor [18]:

where the initial condition is x(t0) = x<sup>0</sup> and F(∙) is a vector function, which (for any real application) is a priori unknown (together with the initial conditions) and needs to be somehow identified beforehand and moreover is usually time varying. It is worth mentioning that the identification problem for F(∙) has attracted a lot of interest in the last decades, but being rather complex, it has not

x\_ ¼ F xð Þ ; t (1)

and therefore (in our opinion) optimum fidelity solutions are still required.

criteria of fidelity [3, 19].

Research Advances in Chaos Theory

material, not even partially, for the lack of space.

2. Extraction of some theoretical principles

2.1 Chaotic modeling and filtering

"ubiquitous."

90

Rossler x t \_ðÞ¼�y tðÞ� z tð Þ y t \_ðÞ¼ x tð Þþ 0:2y tð Þ z t \_ðÞ¼ 0:2 � z tð Þð Þ 5:7 � x tð Þ xkþ<sup>1</sup> ¼ xk þ Ts �yk � zk ykþ<sup>1</sup> <sup>¼</sup> yk <sup>þ</sup> Ts xk <sup>þ</sup> <sup>0</sup>:2yk zkþ<sup>1</sup> ¼ zk þ Tsð Þ 0:2 � zkð Þ 5:7 � xk (2) Lorenz x t \_ðÞ¼ 10ð Þ x tðÞ� y tð Þ y t \_ðÞ¼ 28x tðÞ� y tðÞþ x tðÞ� z tð Þ z t \_ðÞ¼� <sup>8</sup> 3 z tðÞþ x tðÞ� y tð Þ xkþ<sup>1</sup> ¼ xk þ Ts 10 xk � yk ykþ<sup>1</sup> <sup>¼</sup> yk <sup>þ</sup> Ts <sup>28</sup>xk � yk <sup>þ</sup> xk � zk zkþ<sup>1</sup> <sup>¼</sup> zk <sup>þ</sup> Ts � <sup>8</sup> 3 zk þ xk � yk (3) Chua x t \_ðÞ¼ 9:205ð Þ y tð Þ� Uxt ð Þ ð Þ , y t \_ðÞ¼ x tð Þ� y tðÞþ z tð Þ, z t \_ðÞ¼�14:3y tð Þ: xkþ<sup>1</sup> ¼ xkþ<sup>1</sup> þ Ts 9:205 yk � U xð Þ<sup>k</sup> , ykþ<sup>1</sup> <sup>¼</sup> ykþ<sup>1</sup> <sup>þ</sup> Ts xk � yk <sup>þ</sup> <sup>z</sup><sup>3</sup> , zkþ<sup>1</sup> ¼ zkþ<sup>1</sup> þ Ts �14:3yk , (4)

where U xð Þ¼ <sup>k</sup> <sup>m</sup>1xk <sup>þ</sup> <sup>1</sup> <sup>2</sup> ð Þ <sup>m</sup><sup>0</sup> � <sup>m</sup><sup>1</sup> ½ � j j xk <sup>þ</sup> <sup>1</sup> � j j xk � <sup>1</sup> , <sup>m</sup><sup>0</sup> ¼ � <sup>1</sup> <sup>7</sup>, and <sup>m</sup><sup>1</sup> <sup>¼</sup> <sup>2</sup> <sup>7</sup> and TS is the sampling time.

In order to neglect the uncertainty effects of the initial conditions, at least for real data filtering, the approach used in [21], based on the fundamental statement of statistical dynamics for deterministic systems related to Kolmogorov and Max Born [18], together with the introduction of the so-called additive "process noise" in Eq. (1), can be applied. The latter transforms the ODE Eq. (1) into a stochastic differential equation (SDE) [20]. The transformation of Eq. (1) into SDE is relevant for the following material.

The equation for strange attractor Eq. (1) can be transformed into the equivalent stochastic form as a stochastic differential equation (SDE), which "generates" the n-dimensional Markov stochastic process [18, 20]:

$$
\dot{\mathfrak{x}} = \mathfrak{f}(\mathfrak{x}(t)) + \mathfrak{e}\mathfrak{f}(t) \tag{5}
$$

where f(x(t)) is identical to F(x(t)) from Eq. (1). The influence of a weak external source of white noise is denoted by ξ(t); the noise intensities are given in a matrix form ε = [εij] nxn .

The solutions proposed hereafter might be encountered from the structural analysis of the quasi-optimum filtering algorithms for weak chaos in presence of AWGN [8, 9] and are synthesized in the following for convenience.

When one uses the SDE Eq. (5) as a model for chaos, the first strategy that comes up immediately to mind is the nonlinear filtering of chaotic signals which was rigorously developed in [8, 9]. The kernel presented in [8] is the Stratonovich-Kushner equation (SKE) [4, 5], which allows to describe the dynamic equation for the a posteriori probability density function for the chaos x(t). For the filtering with this generalized approach, some additional information from the received aggregate signal has to be incorporated on several sequential time instants, i.e., the

information has to be considered in the block manner by aggregating data, in our case, from several time instants (multi-moment processing). Multi-moment algorithms are carried out through the generalization of the Stratonovich-Kushner Equations for the corresponding multi-moment data. In this way the resulting heuristics are not arbitrary; they are actually generalized heuristics from the standard one-moment SKE. All this gives hope that one can achieve rather good MSE for successively lower thresholds of the SNR using an algorithm with rather low complexity.

developed for many real scenarios [20]. Let a chaotic attractor with certain observable component in Eq. (1) together with its stochastic characterization be a model of the input data. One might consider a random process, generated by a stochastic differential equation of the first order (SDE-1), and name it as a stochastic equivalent as long as it has the same probability density function (PDF) and the same covariance function as the observable component. So, if one assumes that the stochastic equivalent (through the solution of the scalar SDE-1) is an adequate approach to substitute the model of the real phenomena (in the form of an observable component of the multidimensional chaotic attractor), then the actual model is

<sup>x</sup>\_ <sup>¼</sup> f xð Þþ ffiffiffiffi

where the local characteristics, here denoted as K2(x) and K1(x) for Eq. (6), are K1(x) = f (x), K2(x) = K, and ξ(t) is AWGN with unitary intensity [5, 20, 23].<sup>1</sup> If the

where n0(t) is AWGN with intensity N0, then applying the standard procedure of local Gaussian approximation approach for the a posteriori PDF (which for this particular case includes Taylor series representation for all nonlinearities and also includes the PDF exponent and is limited to only quadratic terms at the SKE [4, 5]),

<sup>P</sup>\_ <sup>11</sup>ðÞ¼ <sup>t</sup>

K 2 þ

one can get the following quasi-optimum filtering algorithms:

<sup>∂</sup>Sð Þ <sup>x</sup>^; <sup>t</sup> ∂x

where x t ^ð Þ and P11(t) are a posteriori mean (estimated value) and variance (error) of filtering, respectively. Applying then the well-known standard EKF synthesis procedure [4] for Eqs. (6) and (7), one can also easily obtain the algorithm Eq. (8). It is worth mentioning that the difference between the above-listed algorithms for the local Gaussian approximation depends only on the way the localization of the instantaneous estimation of x(t) is chosen (as it will be commented in the

For the case of high filtering accuracy, all other algorithms that apply local Gaussian approximation [8] can be successfully approximated by the EKF, because the true value of the filtered process and the reference point for application of the

The algorithm Eq. (8) is related to the so-called one-moment (1MM) regime which is classical for the EKF. In the 1MM regime during each processing cycle, one sample from one instant of time is processed (instantaneous processing). The 2MM regime was exhaustively presented at [8, 9] as a special case of multi-moment filtering and could be easily reviewed by the interested reader. In the 2MM regime during each cycle, two samples from two instants of time are processed (non-instantaneous processing). The main parameter for 2MM algorithm is "ρ", which is the correlation coefficient between two adjacent samples of the processing

One can see that for the SDE Eq. (5) the Stratonovich and Ito forms for K1(x) and K2(x) are identical.

½ � y tðÞ� Sð Þ x^; t

Gaussian approximation are obviously very "close."

K

<sup>p</sup> <sup>ξ</sup>ð Þ<sup>t</sup> (6)

y tðÞ¼ S xð Þþ ; t n0ð Þt (7)

<sup>2</sup>∂fð Þ <sup>x</sup>^ ∂x

<sup>P</sup><sup>11</sup> � <sup>2</sup> N<sup>0</sup>

<sup>∂</sup>Sð Þ <sup>x</sup>^ ∂x � �<sup>2</sup>

P2 11

(8)

[5, 20]:

\_

<sup>x</sup>^ <sup>¼</sup> <sup>f</sup>ð Þþ <sup>x</sup>^ <sup>2</sup>P11ð Þ<sup>t</sup>

following).

algorithm.

1

93

N<sup>0</sup>

input signal for filtering is:

Ubiquitous Filtering for Nonlinear Problems DOI: http://dx.doi.org/10.5772/intechopen.88409

Note that the time evolution for the a posteriori PDF for x(t) is completely described by the SKE, but, unfortunately, it does not provide exact analytical solutions. There are very few exceptions: linear SDE Eq. (4) which yields the wellknown Kalman filtering algorithm [4, 22] and some others [4, 5]. Due to this the nonlinear filtering algorithms are practically always simplified, as quasi-optimum or even quasi-linear [4, 5]. In practical applications quasi-linear filters are broadly applied [4, 5].

One might wonder, what is the reason behind the application of chaotic modeling for weak signal filtering? The kernel for this lies in the "singular" properties of the solution of the SKE (see Eqs. (9) and (10) in reference [9]) for the dynamic ODE for chaos Eq. (1), when the solution of the SKE is almost "tuned" to the deterministic chaos from Eq. (1) without any dependence to the SNR [8]. Sure, this statement has to be interpreted as a qualitative explanation for the solution properties of the SKE, and it is almost true for the behavior of the quasi-linear algorithms as well [8, 9].

The following is a list of several quasi-linear filtering algorithms for chaotic signals, based on so-called "Local Gaussian Approximation Approach for the a posteriori PDF" [4, 9], which was found as rather opportunistic for real-time implementations:


All these algorithms certainly show different filtering precision for a fixed SNR and completely different computational complexity for a fixed filtering fidelity. So, in the selection of a concrete filtering algorithm for a concrete scenario, one has to consider as possible selecting criteria the MSE (NMSE and RMSE) together with the computational complexity.

Theoretically, the simplest way to get a comparative analysis of the abovementioned algorithms for the case of weak chaos filtering is in the framework of the so-called stochastic equivalent approximation of the observable component of the chaotic attractor, considered as an adequate model of the real process for filtering.

#### 2.2 Stochastic differential equation of the first order (SDE-1)

The idea of the stochastically equivalent dynamic system (or SDE) was presented for the first time by Stratonovich and Kushner in [5] and extensively information has to be considered in the block manner by aggregating data, in our case, from several time instants (multi-moment processing). Multi-moment algorithms are carried out through the generalization of the Stratonovich-Kushner Equations for the corresponding multi-moment data. In this way the resulting heuristics are not arbitrary; they are actually generalized heuristics from the standard one-moment SKE. All this gives hope that one can achieve rather good MSE for successively lower thresholds of the SNR using an algorithm with rather low

Note that the time evolution for the a posteriori PDF for x(t) is completely described by the SKE, but, unfortunately, it does not provide exact analytical solutions. There are very few exceptions: linear SDE Eq. (4) which yields the wellknown Kalman filtering algorithm [4, 22] and some others [4, 5]. Due to this the nonlinear filtering algorithms are practically always simplified, as quasi-optimum or even quasi-linear [4, 5]. In practical applications quasi-linear filters are broadly

One might wonder, what is the reason behind the application of chaotic modeling for weak signal filtering? The kernel for this lies in the "singular" properties of the solution of the SKE (see Eqs. (9) and (10) in reference [9]) for the dynamic ODE for chaos Eq. (1), when the solution of the SKE is almost "tuned" to the deterministic chaos from Eq. (1) without any dependence to the SNR [8]. Sure, this statement has to be interpreted as a qualitative explanation for the solution properties of the SKE, and it is almost true for the behavior of the quasi-linear algorithms

The following is a list of several quasi-linear filtering algorithms for chaotic signals, based on so-called "Local Gaussian Approximation Approach for the a posteriori PDF" [4, 9], which was found as rather opportunistic for real-time

All these algorithms certainly show different filtering precision for a fixed SNR and completely different computational complexity for a fixed filtering fidelity. So, in the selection of a concrete filtering algorithm for a concrete scenario, one has to consider as possible selecting criteria the MSE (NMSE and RMSE) together with the

abovementioned algorithms for the case of weak chaos filtering is in the framework of the so-called stochastic equivalent approximation of the observable component of the chaotic attractor, considered as an adequate model of the real process for

Theoretically, the simplest way to get a comparative analysis of the

The idea of the stochastically equivalent dynamic system (or SDE) was presented for the first time by Stratonovich and Kushner in [5] and extensively

2.2 Stochastic differential equation of the first order (SDE-1)

complexity.

Research Advances in Chaos Theory

applied [4, 5].

as well [8, 9].

implementations:

1.Extended Kalman filter (EKF)

2.Unscented Kalman filter (UKF)

3.Kalman quadrature filter (KQF)

5.Conditionally optimum filter

computational complexity.

filtering.

92

4.Gauss-Hermite quadrature filter (GQF)

developed for many real scenarios [20]. Let a chaotic attractor with certain observable component in Eq. (1) together with its stochastic characterization be a model of the input data. One might consider a random process, generated by a stochastic differential equation of the first order (SDE-1), and name it as a stochastic equivalent as long as it has the same probability density function (PDF) and the same covariance function as the observable component. So, if one assumes that the stochastic equivalent (through the solution of the scalar SDE-1) is an adequate approach to substitute the model of the real phenomena (in the form of an observable component of the multidimensional chaotic attractor), then the actual model is [5, 20]:

$$
\dot{\mathfrak{x}} = f(\mathfrak{x}) + \sqrt{K}\xi(t) \tag{6}
$$

where the local characteristics, here denoted as K2(x) and K1(x) for Eq. (6), are K1(x) = f (x), K2(x) = K, and ξ(t) is AWGN with unitary intensity [5, 20, 23].<sup>1</sup> If the input signal for filtering is:

$$\mathbf{y}(t) = \mathbf{S}(\mathbf{x}, t) + \mathbf{n}\_0(t) \tag{7}$$

where n0(t) is AWGN with intensity N0, then applying the standard procedure of local Gaussian approximation approach for the a posteriori PDF (which for this particular case includes Taylor series representation for all nonlinearities and also includes the PDF exponent and is limited to only quadratic terms at the SKE [4, 5]), one can get the following quasi-optimum filtering algorithms:

$$\dot{\hat{\mathbf{x}}} = f(\hat{\mathbf{x}}) + \frac{2\mathbf{P}\_{11}(t)}{N\_0} [\mathbf{y}(t) - \mathbf{S}(\hat{\mathbf{x}}, t)] \frac{\partial \mathbf{S}(\hat{\mathbf{x}}, t)}{\partial \mathbf{x}} \ \dot{P}\_{11}(t) = \frac{K}{2} + \frac{2\delta f(\hat{\mathbf{x}})}{\partial \mathbf{x}} P\_{11} - \frac{2}{N\_0} \left[ \frac{\partial \mathbf{S}(\hat{\mathbf{x}})}{\partial \mathbf{x}} \right]^2 P\_{11}^2 \tag{8}$$

where x t ^ð Þ and P11(t) are a posteriori mean (estimated value) and variance (error) of filtering, respectively. Applying then the well-known standard EKF synthesis procedure [4] for Eqs. (6) and (7), one can also easily obtain the algorithm Eq. (8). It is worth mentioning that the difference between the above-listed algorithms for the local Gaussian approximation depends only on the way the localization of the instantaneous estimation of x(t) is chosen (as it will be commented in the following).

For the case of high filtering accuracy, all other algorithms that apply local Gaussian approximation [8] can be successfully approximated by the EKF, because the true value of the filtered process and the reference point for application of the Gaussian approximation are obviously very "close."

The algorithm Eq. (8) is related to the so-called one-moment (1MM) regime which is classical for the EKF. In the 1MM regime during each processing cycle, one sample from one instant of time is processed (instantaneous processing). The 2MM regime was exhaustively presented at [8, 9] as a special case of multi-moment filtering and could be easily reviewed by the interested reader. In the 2MM regime during each cycle, two samples from two instants of time are processed (non-instantaneous processing). The main parameter for 2MM algorithm is "ρ", which is the correlation coefficient between two adjacent samples of the processing algorithm.

<sup>1</sup> One can see that for the SDE Eq. (5) the Stratonovich and Ito forms for K1(x) and K2(x) are identical.

Let us stress here that the concept of stochastic equivalence of the observable component together with the SDE-1 was used only to make our statements in a simple and "friendly" way and to provide computationally simple algorithms. For the general case of the vector SDE (vector ODE) Eq. (1), when the stochastic equivalence in the above presented form cannot be applied, because the high-order statistics (HOS) play a significant role [4], all of the above qualitative comments are true as well; the term <sup>∂</sup>Sð Þ <sup>x</sup>^;<sup>t</sup> <sup>∂</sup><sup>x</sup> at Eq. (8) has to be substituted by the Jacobian matrix, which is usually considered as a "linearization coefficient" at the point x ¼ x^ [4]. It follows that the synthesis approach for the filtering algorithms (in the framework of the local Gaussian approximations for the a posteriori PDF) can be considered as an instantaneous (miscellaneous) linearization approach.

has to be taken into account that the local Gaussian approximation of the a

therefore the accuracy losses might be rather moderate.

Ubiquitous Filtering for Nonlinear Problems DOI: http://dx.doi.org/10.5772/intechopen.88409

background, and so the desired signals are rather weak.

3. Results and discusion

imental settings.

95

posteriori PDF assumes that actually all the model components are almost linear and

interference mitigation, seismology, biomedical telemetry, etc. For weak chaotic signals, in this condition it is possible to consider the EKF with "linear" Jacobian matrix or even SKF instead of the EKF, which additionally simplifies the problem. To obtain the linearization procedure, i.e., operate with a linear matrix A(t) at Eq. (1), that comes from the linear approximation of the attractor's model for chaos, one can use the broadly applied "system identification toolbox" (SIT), [24, 25], which actually provides a solution for A(t) with the spectral properties of the real data. It is worth mentioning that the way how the SIT identifies the linear matrix A(t) follows

from four "canonical representations" for the linear systems stated at [3].

These filtering assumptions seem to be valid for several practical problems such as

Once more, it is only an approximation of an instantaneous linearization procedure required by the quasi-optimum filtering using local Gaussian approximation, but it gives a hope that for a high filtering precision NMSE of about 1% or less (see comments above) the filtering precision losses (by use of the mentioned identification approach) might be moderate and rather acceptable for practice (see also results of the experimental setup). As a final comment, let us note that the "linearization ideology," as an approach, is rather common (see the references already cited above) for quasi-optimum filtering algorithms with varying input data.

The aim of this section is twofold and it will be considered separately. On one side the aim is to show that the stochastic equivalent approach (SDE-1) is efficient and has good accuracy for filtering purposes, taking the sufficiently nonlinear Chua attractor Eq. (4) as the most attractive example. On the other side, the aim is to illustrate the efficiency of the proposed methodology when it is applied to several real-world signals, of absolutely different physical nature, namely, seismic signals, electrocardiogram (ECG) signals, voice-like signals, and RFI signals. These experimental settings have been associated to nonlinear chaotic signals [10–12, 15, 26, 27], and very often, the scenario of such kind of signals includes a strong AWGN

An experimental real-time test bed was developed, containing block generators for the AWGN, the EKF estimation (with their SDE-1 equivalents), the SKF estimation (with the linearization matrix coefficients evaluated from the SIT block), and the real input signals. The chaos EKF segment is a discrete implementation of the EKF which internally contains the discrete version of the equations for the strange attractors of Rossler, Lorenz, and Chua. It also performs a linearization by calculating the Jacobian in each processing cycle. For each signal setting, one of the attractor components (x, y or z) has to be adapted as a possible signal model. For this purpose, first, the sampling time of the chaotic discrete equations is varied so as to achieve a "match" between the temporal variations of the selected attractor component and the desired signal (make the time scales as close as possible). Second, the desired signal is normalized in relation to the mean and variance of the attractor component. The material of [8, 9] shows that the x-component of the three strange attractors might be suitable for modeling the signals from the exper-

The SKF segment is a discrete implementation of the standard Kalman filter which in this case is tridimensional in order to make a fair comparison with the tridimensional EKF. In this segment the linearization matrix is obtained from the

#### 2.3 Computational complexity

The next issue, which has to be analyzed here, is the computational complexity of the quasi-linear algorithms. This subject is crucial for the applications addressed in the next section.

For the general case, when EKF, UKF, GHF, and QKF algorithms are applied considering Chua, Lorenz, and Rössler attractor signals as desired input signals, the computational complexity for the processing is presented in the following table, where all operations, additions (subtractions), multiplications (divisions), Cholesky decompositions, Jacobian calculation (linearization), and nonlinear propagation are included.

From Table 1, it can be easily seen that UKF involves the bigger complexity, while EKF seems to be the simplest algorithm. However, the linearization process performed by the Jacobian calculation involves partial derivatives. For that reason, and depending on the mathematical model of the attractor, the EKF may not always be the fastest algorithm. It follows from Table 1 that the EKF algorithm provides the simplest implementation. Moreover, as it will be shown in the following section, the EKF fidelity for weak signal detection is acceptable in all practical cases. Together with the simple theoretical analysis, the EKF can be considered as an opportunistic approach for applications (see the next section as well).

But one has to notice that for the robust (ubiquitous) solution and applications (see above) the EKF has to be additionally modified by the following heuristics. One can assume, as an alternative to the quasi-linear EKF algorithms, where the linearization is instantaneously updated, that the robust solution for the EKF applications might be found if a "fixed linearization" (with predefined linearization matrix) is used instead of an "instantaneous" one. It actually means that instead of the EKF, the standard Kalman filtering (SKF) approach is applied [3–7], and obviously one has to admit some "losses" in the filtering accuracy for this case. At the same time, it


Table 1. Computational complexity.

#### Ubiquitous Filtering for Nonlinear Problems DOI: http://dx.doi.org/10.5772/intechopen.88409

Let us stress here that the concept of stochastic equivalence of the observable component together with the SDE-1 was used only to make our statements in a simple and "friendly" way and to provide computationally simple algorithms. For the general case of the vector SDE (vector ODE) Eq. (1), when the stochastic equivalence in the above presented form cannot be applied, because the high-order statistics (HOS) play a significant role [4], all of the above qualitative comments are

which is usually considered as a "linearization coefficient" at the point x ¼ x^ [4]. It follows that the synthesis approach for the filtering algorithms (in the framework of the local Gaussian approximations for the a posteriori PDF) can be considered as an

The next issue, which has to be analyzed here, is the computational complexity of the quasi-linear algorithms. This subject is crucial for the applications addressed

For the general case, when EKF, UKF, GHF, and QKF algorithms are applied considering Chua, Lorenz, and Rössler attractor signals as desired input signals, the computational complexity for the processing is presented in the following table, where all operations, additions (subtractions), multiplications (divisions), Cholesky decompositions, Jacobian calculation (linearization), and nonlinear propagation are

From Table 1, it can be easily seen that UKF involves the bigger complexity, while EKF seems to be the simplest algorithm. However, the linearization process performed by the Jacobian calculation involves partial derivatives. For that reason, and depending on the mathematical model of the attractor, the EKF may not always be the fastest algorithm. It follows from Table 1 that the EKF algorithm provides the simplest implementation. Moreover, as it will be shown in the following section,

But one has to notice that for the robust (ubiquitous) solution and applications (see above) the EKF has to be additionally modified by the following heuristics. One can assume, as an alternative to the quasi-linear EKF algorithms, where the linearization is instantaneously updated, that the robust solution for the EKF applications might be found if a "fixed linearization" (with predefined linearization matrix) is used instead of an "instantaneous" one. It actually means that instead of the EKF, the standard Kalman filtering (SKF) approach is applied [3–7], and obviously one has to admit some "losses" in the filtering accuracy for this case. At the same time, it

Additions 8 50 25 25 Multiplications 15 77 33 40 Cholesky decomposition 1 2 2 2 Nonlinear propagation 0 15 21 6 Jacobian calculation 1 0 0 0

EKF UKF GHF QKF

the EKF fidelity for weak signal detection is acceptable in all practical cases. Together with the simple theoretical analysis, the EKF can be considered as an

opportunistic approach for applications (see the next section as well).

<sup>∂</sup><sup>x</sup> at Eq. (8) has to be substituted by the Jacobian matrix,

true as well; the term <sup>∂</sup>Sð Þ <sup>x</sup>^;<sup>t</sup>

Research Advances in Chaos Theory

2.3 Computational complexity

in the next section.

included.

Table 1.

94

Computational complexity.

instantaneous (miscellaneous) linearization approach.

has to be taken into account that the local Gaussian approximation of the a posteriori PDF assumes that actually all the model components are almost linear and therefore the accuracy losses might be rather moderate.

These filtering assumptions seem to be valid for several practical problems such as interference mitigation, seismology, biomedical telemetry, etc. For weak chaotic signals, in this condition it is possible to consider the EKF with "linear" Jacobian matrix or even SKF instead of the EKF, which additionally simplifies the problem. To obtain the linearization procedure, i.e., operate with a linear matrix A(t) at Eq. (1), that comes from the linear approximation of the attractor's model for chaos, one can use the broadly applied "system identification toolbox" (SIT), [24, 25], which actually provides a solution for A(t) with the spectral properties of the real data. It is worth mentioning that the way how the SIT identifies the linear matrix A(t) follows from four "canonical representations" for the linear systems stated at [3].

Once more, it is only an approximation of an instantaneous linearization procedure required by the quasi-optimum filtering using local Gaussian approximation, but it gives a hope that for a high filtering precision NMSE of about 1% or less (see comments above) the filtering precision losses (by use of the mentioned identification approach) might be moderate and rather acceptable for practice (see also results of the experimental setup). As a final comment, let us note that the "linearization ideology," as an approach, is rather common (see the references already cited above) for quasi-optimum filtering algorithms with varying input data.

#### 3. Results and discusion

The aim of this section is twofold and it will be considered separately. On one side the aim is to show that the stochastic equivalent approach (SDE-1) is efficient and has good accuracy for filtering purposes, taking the sufficiently nonlinear Chua attractor Eq. (4) as the most attractive example. On the other side, the aim is to illustrate the efficiency of the proposed methodology when it is applied to several real-world signals, of absolutely different physical nature, namely, seismic signals, electrocardiogram (ECG) signals, voice-like signals, and RFI signals. These experimental settings have been associated to nonlinear chaotic signals [10–12, 15, 26, 27], and very often, the scenario of such kind of signals includes a strong AWGN background, and so the desired signals are rather weak.

An experimental real-time test bed was developed, containing block generators for the AWGN, the EKF estimation (with their SDE-1 equivalents), the SKF estimation (with the linearization matrix coefficients evaluated from the SIT block), and the real input signals. The chaos EKF segment is a discrete implementation of the EKF which internally contains the discrete version of the equations for the strange attractors of Rossler, Lorenz, and Chua. It also performs a linearization by calculating the Jacobian in each processing cycle. For each signal setting, one of the attractor components (x, y or z) has to be adapted as a possible signal model.

For this purpose, first, the sampling time of the chaotic discrete equations is varied so as to achieve a "match" between the temporal variations of the selected attractor component and the desired signal (make the time scales as close as possible). Second, the desired signal is normalized in relation to the mean and variance of the attractor component. The material of [8, 9] shows that the x-component of the three strange attractors might be suitable for modeling the signals from the experimental settings.

The SKF segment is a discrete implementation of the standard Kalman filter which in this case is tridimensional in order to make a fair comparison with the tridimensional EKF. In this segment the linearization matrix is obtained from the

Figure 2 shows the result for the NMSE. The dotted line corresponds to the SDE-1

The experimental data were obtained from a database ATM at PhysioNet [28]. The signal for this experiment corresponds to a baby's heart in fetal stage at the 36th week of the pregnancy cycle. For an SNR = 3 dB, Figure 3 shows the original signal and the filtered signal using 1MM EKF with Rossler x-component as a model.

filtering according to Eq. (9) and the continuous line to the 1MM 3D EKF using Eq. (4) in (A3), and in both cases the input signal is the Chua's x-component. The reason for doing this is that when one filters the input signal (Chua's x-component) using the Chua's Eq. (4), it is almost the best one can do (quasi-optimum solution), and that is why it gives the most adequate benchmark. From Figure 2 it is possible to see that there are some very moderate losses due to the use of the SDE-1 methodology, as it is logically expected; however the NMSE for the SDE-1 does not differ too much from the 1MM 3D EKF, and so the SDE-1 approach offers almost the best accuracy. The following examples are devoted to the filtering of real data, which obviously differ from the theoretical chaos. The NMSE will increase because there is a mismatch between the input signal and the "chaotic signal component" from the filtering algorithm. This "mismatch" as it was mentioned above can be "compensated" by introducing a process noise with intensity Q, in the filtering structure (A3).

3.2 Experiment two: fetal electrocardiogram signals (FECG)

Full results for the NMSE are shown in Table 2.

Ubiquitous Filtering for Nonlinear Problems DOI: http://dx.doi.org/10.5772/intechopen.88409

Comparison between SDE method and 3D EKF.

Snapshot for signals in experiment two.

Figure 2.

Figure 3.

97

Figure 1. Block diagram of the experimental test bed.

experimental signal (seismic, EGG, voice-like) using MATLAB's SIT. The matrix evaluation is made offline, calling the MATLAB's command "ident." Once the signal is loaded in the workspace, the identification is made selecting the option "state space models" [3, 22] for the tridimensional case. The program offers three estimation options, and at the end it yields the confidence percentage for the selected estimation option. It was found experimentally that the PEM option (prediction error method) gives the best confidence for the estimated matrix. Note that for a fixed scenario from the real life, the matrix should be evaluated for each incoming signal offline before the signal processing is done (to obtain information both a priori and (or) from experimental data) as it is illustrated in Figure 1 by introducing a "virtual" delay "ρ," which means the separation in time of the matrix identification and filtering procedure; as the signals are stationary, the identification made for a large vector signal will suffice for any short vector signal. This experimental strategy is shown at Figure 1.

In the following, the experimental results apply the 1MM and 2MM filtering strategies. The 2MM strategy requires for its processing the correlation between two samples which in our case was set to ρ = 0.85. The 2MM shows a bit better NMSE values as it is intuitively expected. For the scenario of seismic signals, it was not possible to calculate the linearization matrix from the SIT, as the signals are not tractable (limited signal durations for the spectral analysis). For all filtering scenarios, a weak process noise value (Q) has been introduced (EKF and SKF) in order to exclude the uncertainty of the initial conditions and is indicated in the corresponding tables.

#### 3.1 Experiment one

This experiment shows the efficiency of the stochastic equivalent SDE-1 for filtering. For illustration purposes, the intended signal here is the x-component of the chaotic attractor from Chua. It is worth noticing that upon taking x(t) in Eq. (4) as the observable component, the correspondent PDF is bimodal due to the function U(�) [20]. The statistically equivalent SDE-1 for the case of Chua's x-component can be obtained straightforward from Eq. (8) [8, 9]:

$$\dot{\hat{\chi}}\_1 = -2\text{K}\hat{\chi}\_1(p\_1 + q\_1) + \frac{2(y(t) - \hat{\chi}\_1)}{N\_0} \quad \dot{\hat{P}}\_{11} = -\frac{K}{2} + \frac{\hat{P}\_{11}^2}{N\_0} + 4K(p\_1 + q\_1)\hat{P}\_{11} \tag{9}$$

where p<sup>1</sup> = 3.5 and q<sup>1</sup> = 1.5.

#### Ubiquitous Filtering for Nonlinear Problems DOI: http://dx.doi.org/10.5772/intechopen.88409

Figure 2 shows the result for the NMSE. The dotted line corresponds to the SDE-1 filtering according to Eq. (9) and the continuous line to the 1MM 3D EKF using Eq. (4) in (A3), and in both cases the input signal is the Chua's x-component. The reason for doing this is that when one filters the input signal (Chua's x-component) using the Chua's Eq. (4), it is almost the best one can do (quasi-optimum solution), and that is why it gives the most adequate benchmark. From Figure 2 it is possible to see that there are some very moderate losses due to the use of the SDE-1 methodology, as it is logically expected; however the NMSE for the SDE-1 does not differ too much from the 1MM 3D EKF, and so the SDE-1 approach offers almost the best accuracy.

The following examples are devoted to the filtering of real data, which obviously differ from the theoretical chaos. The NMSE will increase because there is a mismatch between the input signal and the "chaotic signal component" from the filtering algorithm. This "mismatch" as it was mentioned above can be "compensated" by introducing a process noise with intensity Q, in the filtering structure (A3).

#### 3.2 Experiment two: fetal electrocardiogram signals (FECG)

The experimental data were obtained from a database ATM at PhysioNet [28]. The signal for this experiment corresponds to a baby's heart in fetal stage at the 36th week of the pregnancy cycle. For an SNR = 3 dB, Figure 3 shows the original signal and the filtered signal using 1MM EKF with Rossler x-component as a model. Full results for the NMSE are shown in Table 2.

Figure 2. Comparison between SDE method and 3D EKF.

Figure 3. Snapshot for signals in experiment two.

experimental signal (seismic, EGG, voice-like) using MATLAB's SIT. The matrix evaluation is made offline, calling the MATLAB's command "ident." Once the signal is loaded in the workspace, the identification is made selecting the option "state space models" [3, 22] for the tridimensional case. The program offers three estimation options, and at the end it yields the confidence percentage for the selected estimation option. It was found experimentally that the PEM option (prediction error method) gives the best confidence for the estimated matrix. Note that for a fixed scenario from the real life, the matrix should be evaluated for each incoming signal offline before the signal processing is done (to obtain information both a priori and (or) from experimental data) as it is illustrated in Figure 1 by introducing a "virtual" delay "ρ," which means the separation in time of the matrix identification and filtering procedure; as the signals are stationary, the identification made for a large vector signal will suffice for any short vector signal. This experimental

In the following, the experimental results apply the 1MM and 2MM filtering strategies. The 2MM strategy requires for its processing the correlation between two samples which in our case was set to ρ = 0.85. The 2MM shows a bit better NMSE values as it is intuitively expected. For the scenario of seismic signals, it was not possible to calculate the linearization matrix from the SIT, as the signals are not tractable (limited signal durations for the spectral analysis). For all filtering scenarios, a weak process noise value (Q) has been introduced (EKF and SKF) in order to

This experiment shows the efficiency of the stochastic equivalent SDE-1 for filtering. For illustration purposes, the intended signal here is the x-component of the chaotic attractor from Chua. It is worth noticing that upon taking x(t) in Eq. (4) as the observable component, the correspondent PDF is bimodal due to the function U(�) [20]. The statistically equivalent SDE-1 for the case of Chua's x-component can

> \_ <sup>P</sup>^<sup>11</sup> ¼ � <sup>K</sup>

2 þ P^2 11 N<sup>0</sup>

þ 4K p<sup>1</sup> þ q<sup>1</sup>

P^<sup>11</sup> (9)

exclude the uncertainty of the initial conditions and is indicated in the

be obtained straightforward from Eq. (8) [8, 9]:

2ð Þ y tðÞ� x^<sup>1</sup> N<sup>0</sup>

<sup>þ</sup>

where p<sup>1</sup> = 3.5 and q<sup>1</sup> = 1.5.

strategy is shown at Figure 1.

Block diagram of the experimental test bed.

Research Advances in Chaos Theory

Figure 1.

corresponding tables.

3.1 Experiment one

x^<sup>1</sup> ¼ �2Kx^<sup>1</sup> p<sup>1</sup> þ q<sup>1</sup>

\_

96


#### Table 2.

NMSE results for experiment two.

#### 3.3 Experiment three: voice sounds

For this experiment sustained vowel sounds were used. These kinds of signals are used for voice synthesis procedures [26]. Figure 4 shows the snapshot (continuous line) of the vowel sound "O" (recorded in a sustained fashion for 5 s at 22050 Hz) and also the filtered signal (broken line) using 2MM SKF with its matrix evaluated with the SIT. Almost identical results as in the previous experiments are shown in Table 3. For this experiment none of the components from the Lorenz attractor were suitable as a model for the voice-like signals.

#### 3.4 Experiment four: seismic signals

For this experiment a MATLAB simulator based on the seismic models of [29] was used. For an SNR = 3 dB, Figure 5 shows the seismic signal and its filtered version using 2MM EKF with Rossler x-component as a model. Full results are presented in Table 4. For the seismic signals, it was not possible to obtain an adequate linearization matrix, and so the SKF was not applied for this scenario.

#### 3.5 Experiment five: radio frequency interference (RFI) signals

This experiment considers the RFI generated by computing equipment [27, 30] that affects the transmission of the desired information signals. For an SNR = 3 dB,

Figure 6 shows the RFI signal and its filtered version using 1MM SKF with its

The simulation results obtained from the linearization approach, applying SIT, are presented at Tables 2, 3, and 5. Comparative analysis of the data in the tables allows the following conclusions. All the filtering approaches presented above are rather effective, as all of them show low values of NMSE. One can notice that for the worst-case scenario (10 dB), signals are visually impossible to be distinguished

SNR 0 dB 3 dB 10 dB Processing time

1MM 0.0048 0.0074 0.0178 1.79 s 2MM 0.0047 0.0073 0.0135 3.53 s

1MM 0.0058 0.0093 0.0245 1.807 s 2MM 0.0054 0.0081 0.0187 3.62 s

1MM 0.0057 0.0095 0.029 1.816 s 2MM 0.0051 0.0084 0.023 3.65 s

SNR 0 dB 3 dB 10 dB Processing time

1MM 0.0025 0.0037 0.0079 0.47 s 2MM 0.0015 0.0024 0.0053 0.95 s

1MM 0.0029 0.0044 0.0124 1.792 s 2MM 0.0027 0.0039 0.011 3.611 s

1MM 0.0031 0.0048 0.0137 1.81 s 2MM 0.0025 0.0043 0.0130 3.58 s

SKF Q = 0.0081 (with matrix from SIT)

Ubiquitous Filtering for Nonlinear Problems DOI: http://dx.doi.org/10.5772/intechopen.88409

EKF Rossler x Q = 0.23

EKF Chua x Q = 0.76

NMSE results for experiment three.

Snapshot for signals in experiment four.

EKF Rossler x Q = 0.35

EKF Lorenz x Q = 0.135

EKF Chua x Q = 0.135

NMSE results for experiment four.

Table 3.

Figure 5.

Table 4.

99

matrix evaluated with the SIT. Full results are presented in Table 5.

Figure 4. Snapshot for signals in experiment three.

Ubiquitous Filtering for Nonlinear Problems DOI: http://dx.doi.org/10.5772/intechopen.88409


#### Table 3.

3.3 Experiment three: voice sounds

SKF Q = 0.04 (with matrix from SIT)

Research Advances in Chaos Theory

EKF Rossler x Q = 0.21

EKF Lorenz x Q = 0.42

EKF Chua x Q = 0.075

NMSE results for experiment two.

Table 2.

Figure 4.

98

Snapshot for signals in experiment three.

3.4 Experiment four: seismic signals

attractor were suitable as a model for the voice-like signals.

3.5 Experiment five: radio frequency interference (RFI) signals

For this experiment sustained vowel sounds were used. These kinds of signals are used for voice synthesis procedures [26]. Figure 4 shows the snapshot (continuous line) of the vowel sound "O" (recorded in a sustained fashion for 5 s at 22050 Hz) and also the filtered signal (broken line) using 2MM SKF with its matrix evaluated with the SIT. Almost identical results as in the previous experiments are shown in Table 3. For this experiment none of the components from the Lorenz

SNR 0 dB 3 dB 10 dB Processing time

1MM 0.0025 0.0037 0.0078 0.43 s 2MM 0.0021 0.0032 0.0065 0.89 s

1MM 0.0026 0.0040 0.0098 1.825 s 2MM 0.0023 0.0036 0.0079 3.503 s

1MM 0.0029 0.0042 0.010 1.782 s 2MM 0.0023 0.0034 0.0083 3.59 s

1MM 0.0034 0.0053 0.015 1.812 s 2MM 0.0026 0.0042 0.012 3.61 s

For this experiment a MATLAB simulator based on the seismic models of [29] was used. For an SNR = 3 dB, Figure 5 shows the seismic signal and its filtered version using 2MM EKF with Rossler x-component as a model. Full results are presented in Table 4. For the seismic signals, it was not possible to obtain an adequate linearization matrix, and so the SKF was not applied for this scenario.

This experiment considers the RFI generated by computing equipment [27, 30] that affects the transmission of the desired information signals. For an SNR = 3 dB, NMSE results for experiment three.

#### Figure 5. Snapshot for signals in experiment four.


#### Table 4.

NMSE results for experiment four.

Figure 6 shows the RFI signal and its filtered version using 1MM SKF with its matrix evaluated with the SIT. Full results are presented in Table 5.

The simulation results obtained from the linearization approach, applying SIT, are presented at Tables 2, 3, and 5. Comparative analysis of the data in the tables allows the following conclusions. All the filtering approaches presented above are rather effective, as all of them show low values of NMSE. One can notice that for the worst-case scenario (10 dB), signals are visually impossible to be distinguished

practice, it is not so important what particular model of chaotic attractor or linear-

Why it happened? This spectacular issue was briefly mentioned above, but in the following the feasible explanation is presented once more. It is worth to stress that all the chaotic attractors mentioned and applied for modeling of the real data are "generating" chaos as quasi-deterministic stochastic process Eqs. (1) and (5). Therefore all quasi-optimum filtering algorithms listed before (including EKF and its modifications) that apply chaotic modeling are working in almost "singular" regime, i.e., the shape of the a posteriori PDF is "concentrated" along the a priori PDF of the desired signal "irrespective" to the value of the SNR [8, 9]. That is why it is possible to obtain so low values of NMSE for weak signals (SNR <0 dB and down to 10 dB). Thus, for high filtering fidelity, the linear term of the Taylor expansion for the quasi-linear algorithm [4, 5, 22] significantly prevails over the terms related to the "nonlinearities" (Jacobian matrix, etc.), i.e., the linear approximation is "enough." So the influence of the nonlinear character of the ODE of the attractors

on the value of the NMSE will be relatively small, which follows from the experimental data in tables. Sure, the explanation above is "qualitative" but well corresponding to the theoretic development of the quasi-optimum

linearized filtering approach is considered as well.

on certain fidelity requirements and computational complexity.

In this material a rather simple and robust structure for weak signal filtering is proposed, based on the EKF algorithm and its 2MM modification. In addition, the

Based on this it is possible to suggest, for chaotic modeling of input of non-Gaussian data, a "high degree of freedom" for the filtering block design depending

Taking advantage of the quasi-linear character of the effective real-time filtering algorithms for stochastic non-Gaussian real signals, an approach using the wellknown "system identification toolbox" was proposed as well and might be selected as a reasonable compromise between computational complexity and

The experimental results show that the filtering accuracy losses for the lineari-

For the interested reader, it is highly recommended to consider together the material of the previous chapter "nonlinear filtering of weak chaotic signals" and

All the results presented in the plots and in the tables clearly show that the implementation of the proposed strategy for solving filtering problems might be

zation case and even for the application of the simplified SDE-1 equivalent approach are very moderate and almost negligible for practical implementations. This issue might significantly simplify the theoretical study applied for comparative

the material presented above as it gives a complete "panorama" of the

recommended algorithms and their real-life implementations.

ization matrix from SIT is applied!

Ubiquitous Filtering for Nonlinear Problems DOI: http://dx.doi.org/10.5772/intechopen.88409

algorithms [4, 22].

4. Conclusions

filtering accuracy.

101

selection of the filtering algorithms.

recommended for the practical scenarios.

Figure 6.

Snapshot for signals in experiment five.


#### Table 5.

NMSE results for experiment five.

from noise; however, the NMSE is around 1% for both strategies (SKF and EKF) with either 1MM or 2MM.

The tables also show the average time (in seconds) required to process 5000 samples applied for statistical processing, for each filtering scenario. One has to notice that the 2MM approach consumes more time than the 1MM algorithm but (roughly speaking) no more than the double of the time required for the 1MM processing, as an upper bound. Second, the use of SKF is faster (almost 3 times) because there is no time consumed for the linearization process. The processing time together with the filtering complexity and fidelity might be considered as "criteria" while choosing the appropriate filtering algorithm for concrete implementations.

The SKF with the linearization approach yielded the best results; this once more confirms what was pointed above that for the processing of the quasi-linear algorithms of filtering, the influence of the spectral properties of the input data for these algorithms prevails over the influence of the "non-Gaussian" statistics of the data. The values of the NMSE, obtained by simulations, can be regarded as below or equal to the requirements for many cases of the practice, at least from the study of the corresponding references [1, 2] and the author's knowledge as well. Moreover, one can see that the NMSE values are rather close for all filtering scenarios, and for the

#### Ubiquitous Filtering for Nonlinear Problems DOI: http://dx.doi.org/10.5772/intechopen.88409

practice, it is not so important what particular model of chaotic attractor or linearization matrix from SIT is applied!

Why it happened? This spectacular issue was briefly mentioned above, but in the following the feasible explanation is presented once more. It is worth to stress that all the chaotic attractors mentioned and applied for modeling of the real data are "generating" chaos as quasi-deterministic stochastic process Eqs. (1) and (5). Therefore all quasi-optimum filtering algorithms listed before (including EKF and its modifications) that apply chaotic modeling are working in almost "singular" regime, i.e., the shape of the a posteriori PDF is "concentrated" along the a priori PDF of the desired signal "irrespective" to the value of the SNR [8, 9]. That is why it is possible to obtain so low values of NMSE for weak signals (SNR <0 dB and down to 10 dB). Thus, for high filtering fidelity, the linear term of the Taylor expansion for the quasi-linear algorithm [4, 5, 22] significantly prevails over the terms related to the "nonlinearities" (Jacobian matrix, etc.), i.e., the linear approximation is "enough." So the influence of the nonlinear character of the ODE of the attractors on the value of the NMSE will be relatively small, which follows from the experimental data in tables. Sure, the explanation above is "qualitative" but well corresponding to the theoretic development of the quasi-optimum algorithms [4, 22].

#### 4. Conclusions

In this material a rather simple and robust structure for weak signal filtering is proposed, based on the EKF algorithm and its 2MM modification. In addition, the linearized filtering approach is considered as well.

Based on this it is possible to suggest, for chaotic modeling of input of non-Gaussian data, a "high degree of freedom" for the filtering block design depending on certain fidelity requirements and computational complexity.

Taking advantage of the quasi-linear character of the effective real-time filtering algorithms for stochastic non-Gaussian real signals, an approach using the wellknown "system identification toolbox" was proposed as well and might be selected as a reasonable compromise between computational complexity and filtering accuracy.

The experimental results show that the filtering accuracy losses for the linearization case and even for the application of the simplified SDE-1 equivalent approach are very moderate and almost negligible for practical implementations. This issue might significantly simplify the theoretical study applied for comparative selection of the filtering algorithms.

For the interested reader, it is highly recommended to consider together the material of the previous chapter "nonlinear filtering of weak chaotic signals" and the material presented above as it gives a complete "panorama" of the recommended algorithms and their real-life implementations.

All the results presented in the plots and in the tables clearly show that the implementation of the proposed strategy for solving filtering problems might be recommended for the practical scenarios.

from noise; however, the NMSE is around 1% for both strategies (SKF and EKF)

The tables also show the average time (in seconds) required to process 5000 samples applied for statistical processing, for each filtering scenario. One has to notice that the 2MM approach consumes more time than the 1MM algorithm but (roughly speaking) no more than the double of the time required for the 1MM processing, as an upper bound. Second, the use of SKF is faster (almost 3 times) because there is no time consumed for the linearization process. The processing time together with the filtering complexity and fidelity might be considered as "criteria" while choosing the appropriate filtering algorithm for concrete

SNR 0 dB 3 dB 10 dB Processing time

1MM 0.0018 0.003 0.0098 0.51 s 2MM 0.0015 0.0025 0.0085 0.92 s

1MM 0.0026 0.005 0.019 1.872 s 2MM 0.0023 0.0036 0.011 3.9 s

1MM 0.0023 0.0032 0.04 1.76 s 2MM 0.0016 0.0027 0.0083 3.81 s

1MM 0.0034 0.0053 0.035 1.86 s 2MM 0.0026 0.0042 0.029 3.77 s

The SKF with the linearization approach yielded the best results; this once more confirms what was pointed above that for the processing of the quasi-linear algorithms of filtering, the influence of the spectral properties of the input data for these algorithms prevails over the influence of the "non-Gaussian" statistics of the data. The values of the NMSE, obtained by simulations, can be regarded as below or equal to the requirements for many cases of the practice, at least from the study of the corresponding references [1, 2] and the author's knowledge as well. Moreover, one can see that the NMSE values are rather close for all filtering scenarios, and for the

with either 1MM or 2MM.

NMSE results for experiment five.

Figure 6.

Snapshot for signals in experiment five.

Research Advances in Chaos Theory

EKF Rossler x Q = 0.2

EKF Lorenz x Q = 0.6

EKF Chua x Q = 0.4

Table 5.

SKF Q = 0.02 (with matrix from SIT)

implementations.

100

Research Advances in Chaos Theory

### Author details

Valeri Kontorovich\* and Fernando Ramos-Alarcon Department of Electrical Engineering, Communications Section, CINVESTAV-IPN, Mexico

References

Francis; 2005

Academic; 1970

Breach; 1967

915-926

[1] Morris AS, Langari R. Measurement and Instrumentation: Theory and Application. 2nd ed. London, UK: Academic Press Elsevier; 2016

Ubiquitous Filtering for Nonlinear Problems DOI: http://dx.doi.org/10.5772/intechopen.88409

Chinese Science Bulletin. 2006;51(24):

[11] Vieira M. Chaos and synchronized chaos in an earthquake model. Physical Review Letters. 1998;82(1):201-204

[12] Li Y, Yang Bj, Badal J, et al. Chaotic system detection of weak seismic signals. Geophysics Journal

International. 2009;178(3):1493-1522

Chua LO. The role of synchronization in digital communications using chaos. I.

communications. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications. 1997;44(10):

[13] Kolumban G, Kennedy MP,

Fundamentals of digital

[14] Kaddoum G. Design and performance analysis of a multiuser OFDM based differential chaos shift keying communication system. IEEE Transactions on Communications. 2016;

[15] Thakor NV. Chaos in the heart: Signals and models. In: Proceedings of the 2nd International Conference Biomedical Engineering; May 1998;

[16] Rodriguez-Bermudez G, Garcia-Laencina PJ. Analysis of EEG signals using nonlinear dynamics and chaos: A review. Applied Mathematics and Information Sciences. 2015;9(5):

[17] Shu-Yao J, Fei Y, et al., Application of stochastic resonance technology in underwater acoustic weak signal detection. In: Conference OCEANS;

[18] Anischenko VS et al. Non-Linear Dynamics of Chaotic and Stochastic Systems. Berlin Heidelberg: Springer-

927-936

64(1):249-260

pp. 11-18

2309-2321

Verlag; 2007

April 2016; pp. 978-982

3010-3017

[2] Northtrop RB. Introduction to Instrumentation and Measurements. 2nd ed. Florida, USA: CRC Press Taylor

[3] Van Trees H. Detection, Estimation and Modulation Theory. New York, USA: John Wiley & Sons; 2001

[4] Jazwinski A. Stochastic Processing and Filtering Theory. NewYork, USA:

[5] Stratonovich R. Topics of the Theory of Random Noise. London, UK: Gordon

estimation using fast modified reduced update Kalman filter. IEEE Transactions on Signal Processing. 1992;40(4):

[7] Parseh R, Kansanen K, Slock D. Distortion outage analysis for joint space-time coding and Kalman filter. IEEE Transactions on Signal Processing.

[8] Kontorovich V, Lovtchikova Z. Nonlinear filtering of chaos for real time

[9] Kontorovich V, Lovtchikova Z, Ramos F. Nonlinear filtering of weak chaotic signals. In: Kais A, Al Naimee M, editors. Chaos Theory, Chapter 5.

[10] Yue L, Baojun Y, et al. Ability to detect weak effective seismic signals by utilizing chaotic vibrator system.

London, UK: Intech; 2018

103

HalangWA, et al., editors. Selected Topics in Nonlinear Dynamics and Theoretical Electrical Engineering, Chapter 3. Berlin Heidelberg: Springer-Verlag; 2013

applications. In: Kyamakya K,

2017;65(9):2291-2305

[6] Wu W-R, Kundu A. Image

\*Address all correspondence to: valeri@cinvestav.mx

© 2019 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Ubiquitous Filtering for Nonlinear Problems DOI: http://dx.doi.org/10.5772/intechopen.88409

#### References

[1] Morris AS, Langari R. Measurement and Instrumentation: Theory and Application. 2nd ed. London, UK: Academic Press Elsevier; 2016

[2] Northtrop RB. Introduction to Instrumentation and Measurements. 2nd ed. Florida, USA: CRC Press Taylor Francis; 2005

[3] Van Trees H. Detection, Estimation and Modulation Theory. New York, USA: John Wiley & Sons; 2001

[4] Jazwinski A. Stochastic Processing and Filtering Theory. NewYork, USA: Academic; 1970

[5] Stratonovich R. Topics of the Theory of Random Noise. London, UK: Gordon Breach; 1967

[6] Wu W-R, Kundu A. Image estimation using fast modified reduced update Kalman filter. IEEE Transactions on Signal Processing. 1992;40(4): 915-926

[7] Parseh R, Kansanen K, Slock D. Distortion outage analysis for joint space-time coding and Kalman filter. IEEE Transactions on Signal Processing. 2017;65(9):2291-2305

[8] Kontorovich V, Lovtchikova Z. Nonlinear filtering of chaos for real time applications. In: Kyamakya K, HalangWA, et al., editors. Selected Topics in Nonlinear Dynamics and Theoretical Electrical Engineering, Chapter 3. Berlin Heidelberg: Springer-Verlag; 2013

[9] Kontorovich V, Lovtchikova Z, Ramos F. Nonlinear filtering of weak chaotic signals. In: Kais A, Al Naimee M, editors. Chaos Theory, Chapter 5. London, UK: Intech; 2018

[10] Yue L, Baojun Y, et al. Ability to detect weak effective seismic signals by utilizing chaotic vibrator system.

Chinese Science Bulletin. 2006;51(24): 3010-3017

[11] Vieira M. Chaos and synchronized chaos in an earthquake model. Physical Review Letters. 1998;82(1):201-204

[12] Li Y, Yang Bj, Badal J, et al. Chaotic system detection of weak seismic signals. Geophysics Journal International. 2009;178(3):1493-1522

[13] Kolumban G, Kennedy MP, Chua LO. The role of synchronization in digital communications using chaos. I. Fundamentals of digital communications. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications. 1997;44(10): 927-936

[14] Kaddoum G. Design and performance analysis of a multiuser OFDM based differential chaos shift keying communication system. IEEE Transactions on Communications. 2016; 64(1):249-260

[15] Thakor NV. Chaos in the heart: Signals and models. In: Proceedings of the 2nd International Conference Biomedical Engineering; May 1998; pp. 11-18

[16] Rodriguez-Bermudez G, Garcia-Laencina PJ. Analysis of EEG signals using nonlinear dynamics and chaos: A review. Applied Mathematics and Information Sciences. 2015;9(5): 2309-2321

[17] Shu-Yao J, Fei Y, et al., Application of stochastic resonance technology in underwater acoustic weak signal detection. In: Conference OCEANS; April 2016; pp. 978-982

[18] Anischenko VS et al. Non-Linear Dynamics of Chaotic and Stochastic Systems. Berlin Heidelberg: Springer-Verlag; 2007

Author details

Research Advances in Chaos Theory

Mexico

102

Valeri Kontorovich\* and Fernando Ramos-Alarcon

\*Address all correspondence to: valeri@cinvestav.mx

provided the original work is properly cited.

Department of Electrical Engineering, Communications Section, CINVESTAV-IPN,

© 2019 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/3.0), which permits unrestricted use, distribution, and reproduction in any medium,

[19] Middleton D. Introduction to the Statistical Theory of Communications. Piscataway, New Jersey: IEEE Press; 1999

[20] Primak S, Kontorovich V, Lyandres V. Stochastic Methods and their Applications to Communications: Stochastic Differential Equations Approach. Chichester West Sussex, UK: John Wiley & Sons; 2004

[21] Pecora LM, Carroll TL. Synchronization in chaotic systems. Physical Review Letters. 1990;64(8): 821-824

[22] Pugachev V, Sinitsyn I. Stochastic Differential Systems Analysis and Filtering. Chichester West Sussex, UK: John Wiley & Sons; 1987

[23] Kontorovich V, Lovtchikova Z. Cumulant analysis of strange attractors: Theory and applications. In: Kyamakya K, Halang WA, et al., editors. Recent Advances in Non-Linear Dynamics and Synchronization. Berlin Heidelberg: Springer-Verlag; 2009. pp. 77-115

[24] Nikulchev EV. Simulation of robust chaotic signal with given properties. Advanced Studies in Theoretical Physics. 2014;8(21):939-944

[25] Ljung L. System Identification: Theory for the User. New Jersey, USA: Prentice Hall; 1999

[26] Banbrook M, McLaughlin S, Mann I. Speech characterization and synthesis by nonlinear methods. IEEE Transactions on Speech and Audio Processing. 1999;7(1):1-17

[27] Arditti D. Alcocer A, Ramos F, Kontorovich V. Adaptive mitigation of platform-generated radio-frequency interference. Patents EP2838205A3, 2014 and CN104378128B. 2018

[28] PhysioNet ATM. Available from: h ttps://physionet.org/cgi-bin/atm/ATM

[29] Rofooel FR, Mobarake A, Ahmadi G. Generation of artificial earthquake records with a nonstationary Kanai-Tajimi model. Engineering Structures. 2001;23(7): 827-837

[30] Alban EX, Magaña ME, Skinner HG, Slattery KP. Statistical modelling of the interference noise generated by computing platforms. IEEE Transactions on Electromagnetic Compatibility. 2012;54(3):574-584

Section 3

Biology and Economics

105

### Section 3
