**4. Convergence analysis**

In this section, the convergence of the algorithm is discussed. The following assumptions are needed:

The derivatives of , and *h* exist.

The solution (\*, \*, \*) is the optimal solution to Problem (P). That is, the optimal smoothing solution.

The convergence result is presented in Theorem 4.1, while the accuracy of the smoothed state in term of state error covariance is proven in Corollary 4.1.

Theorem 4.1: The converged solution of Problem (M) is the correct optimal smoothing solution of Problem (P).

Proof: Consider the real plant and the output measurement of Problem (P) with the exact optimal smoothing solution (\*, \*, \*) as given below:

$$\text{tr}^{\bullet}(k+l) = f\left(\text{x}^{\bullet}(k), \text{u}^{\bullet}(k), k\right) \text{ and } \text{y}^{\bullet}(k) = h(\text{x}^{\bullet}(k), k) \tag{29}$$

In Problem (M), the model used consists of

$$
\hat{\boldsymbol{\chi}}^{\mathcal{C}}(k) = \overline{\boldsymbol{\chi}}^{\mathcal{C}}(k) + K\_f(k)(\mathbf{y}(k) - \overline{\mathbf{y}}^{\mathcal{C}}(k)) \tag{30a}
$$

$$
\overline{\boldsymbol{X}}^{\mathcal{C}}(k+\mathsf{I}) = A\hat{\boldsymbol{X}}^{\mathcal{C}}(k) + B\boldsymbol{u}^{\mathcal{C}}(k) + \alpha\_{\mathsf{I}}(k) \tag{30b}
$$

$$
\overline{\boldsymbol{y}}^{\mathbf{c}}(k) = \boldsymbol{C}\overline{\boldsymbol{x}}^{\mathbf{c}}(k) + \boldsymbol{a}\_2(k) \tag{30c}
$$

$$
\hat{\boldsymbol{\chi}}\_{\mathcal{S}}^{\mathcal{C}}(k) = \hat{\boldsymbol{\chi}}^{\mathcal{C}}(k) + K\_{\mathcal{S}}(k)(\hat{\boldsymbol{\chi}}\_{\mathcal{S}}^{\mathcal{C}}(k+l) - \overline{\boldsymbol{\chi}}^{\mathcal{C}}(k+l))\tag{30d}
$$

Smoothing Solution for Discrete-Time Nonlinear Stochastic Optimal Control Problem with Model-Reality Differences http://dx.doi.org/10.5772/64564 77

$$
\hat{\mathbf{y}}\_{\mathcal{S}}^{\mathcal{C}}(k) = \mathbf{C} \hat{\mathbf{x}}\_{\mathcal{S}}^{\mathcal{C}}(k) \tag{30e}
$$

where (), (), (), (), (), and () are, respectively, the converged sequences for control law, smoothed state estimate, filtered state estimate, expected state estimate, smoothed output, and expected output. Here, () is the output measured from the real plant.

Applying the adjusted parameters 1() and 2(), which are given by

$$\begin{aligned} a\_1(k) &= f(\mathbf{z}(k), v(k), k) - Az(k) - Bv(k) \text{and} \\\\ a\_2(k) &= h(\mathbf{z}(k), k) - Cz(k), \end{aligned}$$

into the model used given by Eq. (30b) and (30c), the differences between the real plant and the model used can be measured at each iteration. Moreover, at the end of iteration, from Eqs. (29) and (30a) – (30e) yields

$$
\hat{\mathfrak{X}}\_S^{\mathcal{C}}(k+\mathsf{l}) = f\left(\boldsymbol{\varepsilon}(k), \boldsymbol{\nu}(k), k\right) \\
\text{and} \\
\hat{\mathfrak{Y}}\_S^{\mathcal{C}}(k) = h(\boldsymbol{\varepsilon}(k), k).
$$

which () = () and () = () = () are satisfied. Hence, this implies that

$$u^{\mathcal{C}}(k) = u^\*(k), \,\hat{\boldsymbol{\chi}}^{\mathcal{C}}\_{\mathcal{S}}(k) = \boldsymbol{\varkappa}^\*(k), \,\hat{\boldsymbol{\chi}}^{\mathcal{C}}\_{\mathcal{S}}(k) = \boldsymbol{\jmath}^\*(k)$$

This completes the proof.

As a rule of this case, the algorithm (from Step 1 to Step 4) is required to run few times. Initially, for first run of the algorithm (from Step 1 to Step 4), these scalars are set at *kv* = *kz* = *kp* = 1, and then, with different values chosen from 0.1 to 0.9, the algorithm is run again. The value with the optimal number of iterations can be determined after that. Applying the parameters *r*1 and *r*2 is to enhance the convexity such that the convergence of the

In this section, the convergence of the algorithm is discussed. The following assumptions are

The solution (\*, \*, \*) is the optimal solution to Problem (P). That is, the optimal smoothing

The convergence result is presented in Theorem 4.1, while the accuracy of the smoothed state

Theorem 4.1: The converged solution of Problem (M) is the correct optimal smoothing solution

Proof: Consider the real plant and the output measurement of Problem (P) with the exact

*x k f x k u k k y k hx k k* ( 1) ( ( ), ( ), ) and ( ) ( ( ), ) + = = (29)

<sup>ˆ</sup> ( ) ( ) ( )( ( ) ( )) *c c <sup>c</sup> <sup>f</sup> x k x k K k yk y k* =+ - (30a)

(30b)

(30c)

a

*<sup>s</sup> s s xk xk Kkxk x k* = + +- + (30d)

\* \*\* \* \*

<sup>1</sup> ( 1) ( ) ( ) ( ) <sup>ˆ</sup> *c cc x k Ax k Bu k k* += + +

<sup>2</sup> () () () *c c y k Cx k k* = +

ˆˆ ˆ ( ) ( ) ( )( ( 1) ( 1)) *cc c c*

a

algorithm can be improved.

76 Nonlinear Systems - Design, Analysis, Estimation and Control

**4. Convergence analysis**

The derivatives of , and *h* exist.

in term of state error covariance is proven in Corollary 4.1.

optimal smoothing solution (\*, \*, \*) as given below:

In Problem (M), the model used consists of

needed:

solution.

of Problem (P).

Corollary 4.1: The smoothed state error covariance is the smallest among the values of state error covariance.

Proof: From Eq. (6), it is clear that the filtered state error covariance () is less than the predicted state error covariance () . That is, () < () . Now, to prove () < (),, we shall show that ( + 1) < ( + 1). Consider the boundary condition () = () and taking = − 1, we have

$$P\_s(N-\mathbf{l}) = P(N-\mathbf{l}) < M\_\chi(N-\mathbf{l}).$$

For = − 2, it shows that

$$P\_s(N-2) < P(N-2) < M\_\chi(N-2).$$

This statement can be deduced that

$$P\_s(k+1) - M\_\chi(k+1) < 0 \text{ for } k = k+1\dots$$

Thus, we conclude that

$$P\_s(k) < P(k) < M\_\chi(k), \quad k = 0, 1, \dots, N - 2,$$

which shows the accuracy of the smoothed state estimate. This completes the proof.

### **5. Illustrative example**

Consider a continuous stirred‐tank reactor problem [19], which consists of the state difference equations

$$\begin{split} \mathbf{x}\_{\mathrm{l}}(k+1) &= \mathbf{x}\_{\mathrm{l}}(k) - 0.02(\mathbf{x}\_{\mathrm{l}}(k) + 0.25) + 0.01(\mathbf{x}\_{\mathrm{2}}(k) + 0.5) \exp\left[\frac{25\mathbf{x}\_{\mathrm{l}}(k)}{\mathbf{x}\_{\mathrm{l}}(k) + 2}\right] \\ &- 0.01(\mathbf{x}\_{\mathrm{l}}(k) + 0.25)\boldsymbol{\mu}(k) + o\_{\mathrm{l}}(k) \end{split}$$

$$\ln \mathbf{x}\_2(k+l) = 0.99\mathbf{x}\_2(k) - 0.005 - 0.01(\mathbf{x}\_2(k) + 0.5)\exp\left[\frac{25\mathbf{x}\_1(k)}{\mathbf{x}\_1(k) + 2}\right] + o\_2(k)$$

for = 0, ..., 77, and the output measurement () = 1() + (). The initial state (0) = 0 is a random vector with mean and covariance given, respectively, by 1(0) = 0.05, 2(0) = 0, and 0 = 10−2 2 .

Here, ()=[1() 2()] <sup>T</sup> and () are Gaussian white noise sequences with their respective covariance given by = 10−3 <sup>2</sup> and = 10−3. The expected cost function

$$J\_0(\boldsymbol{\mu}) = 0.5 \sum\_{k=0}^{N-1} E[\left(\mathbf{x}\_1(k)\right)^2 + \left(\mathbf{x}\_2(k)\right)^2 + 0.1(\boldsymbol{\mu}(k))^2]^{\frac{1}{2}}$$

is to be minimized over the state difference equations and the output measurement. This problem is referred to as Problem (P).

To obtain the optimal smoothing solution of Problem (P), we simplify the plant dynamics of Problem (P) and refer it as Problem (M), given by

$$\min\_{\mu(k)} J\_m(\mu) = \frac{1}{2} \sum\_{k=0}^{N-1} \left[ \left( \hat{\mathfrak{x}}\_s(k) \right)^2 + 0.1 \left( \mu(k) \right)^2 + 2\gamma(k) \right],$$

subject to

This statement can be deduced that

78 Nonlinear Systems - Design, Analysis, Estimation and Control

Thus, we conclude that

**5. Illustrative example**

equations

0 = 10−2

2 .

Here, ()=[1() 2()]

covariance given by = 10−3

This problem is referred to as Problem (P).

( 1) ( 1) 0for 1. *Pk M k k k s x* +- + < =+

( ) ( ) ( ), 0,1,..., 2, *P k Pk M k k N s x* << = -

Consider a continuous stirred‐tank reactor problem [19], which consists of the state difference

25 ( ) ( 1) ( ) 0.02( ( ) 0.25) 0.01( ( ) 0.5)exp () 2

w

+= - + + + ê ú

*x k xk xk xk x k*

22 2 2

+= - - + ê ú +

for = 0, ..., 77, and the output measurement () = 1() + (). The initial state (0) = 0 is a random vector with mean and covariance given, respectively, by 1(0) = 0.05, 2(0) = 0, and

( ) 0.5 [( ( )) ( ( )) 0.1( ( )) ]

*J u E x k x k uk*

= ++ å

is to be minimized over the state difference equations and the output measurement.

25 ( ) ( 1) 0.99 ( ) 0.005 0.01( ( ) 0.5)exp ( ) () 2 *x k xk xk x k <sup>k</sup>*

1

+ ë û

w

é ù

1

1

+ ë û

é ù

1

<sup>T</sup> and () are Gaussian white noise sequences with their respective

= 10−3. The expected cost function

22 2

*x k*

*x k*

which shows the accuracy of the smoothed state estimate. This completes the proof.

11 1 2


<sup>2</sup> and

1


*N*

*k*

=

0 12 0

1 1

*x k uk k*

0.01( ( ) 0.25) ( ) ( )

$$
\hat{\boldsymbol{\alpha}}\_s(k) = \hat{\boldsymbol{\alpha}}(k) + K\_s(k)(\hat{\boldsymbol{\alpha}}\_s(k+1) - \overline{\boldsymbol{\alpha}}(k+1)),
$$

$$
\hat{\boldsymbol{\nu}}\_s(k) = C \hat{\boldsymbol{\chi}}\_s(k)
$$

with

$$
\hat{\mathfrak{x}}(k) = \overline{\mathfrak{x}}(k) + K\_f(k)(\mathcal{y}(k) - \overline{\mathfrak{y}}(k)),
$$

$$
\begin{bmatrix}
\overline{\mathbf{x}}\_1(k+1) \\
\overline{\mathbf{x}}\_2(k+1)
\end{bmatrix} = \begin{bmatrix}
1.0895 & 0.0184 \\
\end{bmatrix} \begin{bmatrix}
\hat{\mathbf{x}}\_1(k) \\
\hat{\mathbf{x}}\_2(k)
\end{bmatrix} + \begin{bmatrix}
0.000
\end{bmatrix} \boldsymbol{\mu}(k) + \begin{bmatrix}
\boldsymbol{a}\_{11}(k) \\
\boldsymbol{a}\_{12}(k)
\end{bmatrix}
$$

$$
\overline{\mathbf{y}}(k) = \overline{\mathbf{x}}\_{\mathbf{l}}(k) + \alpha\_2(k)
$$

with the initial condition (0) = 0 and the boundary value () = () . Here, (), 2() and 1()=[11() 12()] T are the adjusted parameters.


**Table 1.** Iteration result.

The iteration results, both for filtering and smoothing models, are shown in **Table 1**. The final cost of the smoothing model is the least compared to the final cost of the filtering model. When the trace matrix terms are considered in the cost function, the total final cost of the smoothing model is 0.019188 unit, while the total final cost of the filtering model is 0.039725 unit. The value of the trace matrix terms is 0.0185 unit. It is noticed that the output residual could be dropped to almost 52% from the filtering output residual by using the approach proposed in

this chapter. This statement is valid since the output residual of smoothing model is least than the output residual of filtering model.

**Figure 2.** Filtering trajectory for final control.

**Figure 3.** Filtering trajectory for final state.

To identify the accuracy of the resulting algorithm, the norms of the differences between the real plant and the model used at the end of iteration, which are 0.0128 unit for filtering model and 0.0099 unit for smoothing model, are calculated. These values show that the smoothing model can approximate closely to the correct optimal solution of the original optimal control problem rather than the filtering model. Hence, the accuracy of the smoothing model is proven.

**Figure 4.** Filtering trajectory for final output and real output.

this chapter. This statement is valid since the output residual of smoothing model is least than

To identify the accuracy of the resulting algorithm, the norms of the differences between the real plant and the model used at the end of iteration, which are 0.0128 unit for filtering model

the output residual of filtering model.

80 Nonlinear Systems - Design, Analysis, Estimation and Control

**Figure 2.** Filtering trajectory for final control.

**Figure 3.** Filtering trajectory for final state.

**Figure 5.** Smoothing trajectory for final control.

**Figure 6.** Smoothing trajectory for final state.

**Figure 7.** Smoothing trajectory for final output and real output.

The trajectories of final control, final state and final output for filtering, and smoothing mod‐ els are shown in **Figures 2**–**7**. With the smallest output residual, the output, which is associ‐ ated with the smoothed state estimate, is definitely applicable to measure the real output trajectory.
