**5. Numerical Examples**

In this section, we present two examples to illustrate the design methods displayed in Sec‐ tion 3 and 4 respectively.

*Example 5.1* Consider the following linear discrete-time system (15) under the influence of state feedback of the form*u*(*k*)= *Kx*(*k*), its parameter matrices are

$$\begin{aligned} A &= \begin{bmatrix} 0 & 2 \\ 4 & 0.2 \end{bmatrix} \cdot B\_0 = \begin{bmatrix} 0.1 \\ 0.2 \end{bmatrix} \cdot B\_1 = \begin{bmatrix} 0.5 \\ 0.3 \end{bmatrix} \cdot B\_2 = \begin{bmatrix} 1 \\ 0.2 \end{bmatrix} \\\ C\_1 &= \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix} \cdot C\_2 = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \cdot D\_{12} = \begin{bmatrix} 0 \\ 1 \end{bmatrix} \end{aligned}$$

The above system satisfies Assumption 3.1-3.3, and the open-loop poles of this system are *p*<sup>1</sup> = −2.7302,*p*<sup>2</sup> =2.9302; thus it is open-loop unstable.

Let*<sup>R</sup>* =1,*<sup>Q</sup>* <sup>=</sup> <sup>1</sup> <sup>0</sup> <sup>0</sup> <sup>1</sup> , *<sup>γ</sup>* =9.5, *<sup>δ</sup>* =0.01,*<sup>M</sup>* <sup>=</sup> <sup>−</sup>0.04 <sup>−</sup>1.2 ; by using algorithm 3.1, we solve the discrete-time Riccati equation (20) to get*Xi* ,*U*1(*i*) , *Ki* (*i* =0,1,2,⋯,10)and the corresponding closed-loop poles. The calculating results of algorithm 3.1 are listed in Table 1.

It is shown in Table 1 that when the iteration index*i* =10,*X*100 and *U*1(10)= −0.2927 0, thus the discrete-time state feedback stochastic mixed LQR/*H∞* controller does not exist in this case. Of course, Table 1 does not list all discrete-time state feedback stochastic mixed LQR/*H<sup>∞</sup>* controllers because we do not calculating all these controllers by using Algorithm in this ex‐ ample. In order to illustrate further the results, we give the trajectories of state of the system (15) with the state feedback of the form *u*(*k*)= *Kx*(*k*) for the resulting discrete-time state feed‐ back stochastic mixed LQR/*H∞* controller *K* = −0.3071 −2.0901 . The resulting closed-loop system is

$$\begin{aligned} \mathbf{x}(k+1) &= (A + B\_2K)\mathbf{x}(k) + B\_0\mathbf{w}\_0(k) + B\_1\mathbf{w}(k) \\ \mathbf{z}(k) &= (C\_1 + D\_{12}K)\mathbf{x}(k) \\ \text{where, } A + B\_2K &= \begin{bmatrix} -0.3071 & -0.0901 \\ 4.0000 & 0.2000 \end{bmatrix} \mathbf{C}\_1 + D\_{12}K = \begin{bmatrix} 1.0000 & 0 \\ -0.3071 & -2.0901 \end{bmatrix} .\end{aligned}$$


**Table 1.** The calculating results of algorithm 3.1.

*F<sup>∞</sup>* = −*U*<sup>2</sup>

er subject.

*<sup>A</sup>*<sup>=</sup> <sup>0</sup> <sup>2</sup>

*<sup>C</sup>*<sup>1</sup> <sup>=</sup> <sup>1</sup> <sup>0</sup>

system is

*z*(*k*)=(*C*<sup>1</sup> + *D*12*K*)*x*(*k*)

−1 *B*2

22 Advances in Discrete Time Systems

*<sup>T</sup> <sup>U</sup>*3*AC*<sup>2</sup>

**5. Numerical Examples**

tion 3 and 4 respectively.

<sup>4</sup> 0.2 , *<sup>B</sup>*<sup>0</sup> <sup>=</sup> 0.1

<sup>0</sup> <sup>0</sup> , *<sup>C</sup>*<sup>2</sup> <sup>=</sup> <sup>1</sup> <sup>0</sup>

Let*<sup>R</sup>* =1,*<sup>Q</sup>* <sup>=</sup> <sup>1</sup> <sup>0</sup>

*<sup>T</sup>* (*C*2*C*<sup>2</sup> *T* ) −1

0.2 , *<sup>B</sup>*<sup>1</sup> <sup>=</sup> 0.5

<sup>0</sup> <sup>1</sup> , *<sup>D</sup>*<sup>12</sup> <sup>=</sup> <sup>0</sup>

discrete-time Riccati equation (20) to get*Xi*

*x*(*k* + 1)=(*A* + *B*2*K*)*x*(*k*) + *B*0*w*0(*k*) + *B*1*w*(*k*)

where, *<sup>A</sup>* <sup>+</sup> *<sup>B</sup>*2*<sup>K</sup>* <sup>=</sup> <sup>−</sup>0.3071 <sup>−</sup>0.0901

In this chapter, we will not prove the convergence of the above algorithm. This will is anoth‐

In this section, we present two examples to illustrate the design methods displayed in Sec‐

*Example 5.1* Consider the following linear discrete-time system (15) under the influence of

The above system satisfies Assumption 3.1-3.3, and the open-loop poles of this system are

,*U*1(*i*)

It is shown in Table 1 that when the iteration index*i* =10,*X*100 and *U*1(10)= −0.2927 0, thus the discrete-time state feedback stochastic mixed LQR/*H∞* controller does not exist in this case. Of course, Table 1 does not list all discrete-time state feedback stochastic mixed LQR/*H<sup>∞</sup>* controllers because we do not calculating all these controllers by using Algorithm in this ex‐ ample. In order to illustrate further the results, we give the trajectories of state of the system (15) with the state feedback of the form *u*(*k*)= *Kx*(*k*) for the resulting discrete-time state feed‐ back stochastic mixed LQR/*H∞* controller *K* = −0.3071 −2.0901 . The resulting closed-loop

<sup>0</sup> <sup>1</sup> , *<sup>γ</sup>* =9.5, *<sup>δ</sup>* =0.01,*<sup>M</sup>* <sup>=</sup> <sup>−</sup>0.04 <sup>−</sup>1.2 ; by using algorithm 3.1, we solve the

, *Ki*

<sup>−</sup>0.3071 <sup>−</sup>2.0901 .

(*i* =0,1,2,⋯,10)and the corresponding

state feedback of the form*u*(*k*)= *Kx*(*k*), its parameter matrices are

0.3 , *<sup>B</sup>*<sup>2</sup> <sup>=</sup> <sup>1</sup>

1

*p*<sup>1</sup> = −2.7302,*p*<sup>2</sup> =2.9302; thus it is open-loop unstable.

0

closed-loop poles. The calculating results of algorithm 3.1 are listed in Table 1.

4.0000 0.2000 ,*C*<sup>1</sup> <sup>+</sup> *<sup>D</sup>*12*<sup>K</sup>* <sup>=</sup> 1.0000 <sup>0</sup>

To determine the mean value function, we take mathematical expectation of the both hand of the above two equations to get

$$\begin{aligned} \bar{\boldsymbol{x}}(k+1) &= (\boldsymbol{A} + \boldsymbol{B}\_2 \boldsymbol{K}) \bar{\boldsymbol{x}}(k) + \boldsymbol{B}\_1 \boldsymbol{w}(k) \\ \bar{\boldsymbol{z}}(k) &= (\boldsymbol{C}\_1 + \boldsymbol{D}\_{12} \boldsymbol{K}) \bar{\boldsymbol{x}}(k) \end{aligned}$$

where, *E*{*x*(*k*)}= *<sup>x</sup>*¯(*k*), *E*{*z*(*k*)}= *<sup>z</sup>*¯(*k*),*E*{*x*(0)} <sup>=</sup> *<sup>x</sup>*¯ <sup>0</sup>.

Let*w*(*k*)=*γ* <sup>−</sup><sup>2</sup> *U*1 −1 *B*1 *<sup>T</sup> <sup>X</sup>∞*(*<sup>A</sup>* <sup>+</sup> *<sup>B</sup>*2*K*)*x*¯(*k*), then the trajectories of mean values of states of result‐ ing closed-loop system with *x*¯ <sup>0</sup> <sup>=</sup> <sup>3</sup> <sup>2</sup> *<sup>T</sup>* are given in Fig. 1.

where, *<sup>A</sup>* <sup>+</sup> *<sup>B</sup>*2*F∞C*<sup>2</sup> <sup>=</sup> <sup>−</sup>0.3727 <sup>−</sup>0.0174

*<sup>x</sup>*¯(*<sup>k</sup>* <sup>+</sup> 1)=(*<sup>A</sup>* <sup>+</sup> *<sup>B</sup>*2*F∞C*2)*x*¯(*k*) <sup>+</sup> *<sup>B</sup>*1*w*(*k*)

where, *E*{*x*(*k*)}= *<sup>x</sup>*¯(*k*), *E*{*z*(*k*)}= *<sup>z</sup>*¯(*k*),*E*{*x*(0)} <sup>=</sup> *<sup>x</sup>*¯ <sup>0</sup>.

sulting closed-loop system with *x*¯ <sup>0</sup> <sup>=</sup> <sup>1</sup> <sup>2</sup> *<sup>T</sup>* are given in Fig. 2.

**Figure 2.** The trajectories of mean values of states of resulting system in Example 5.2.

In this chapter, we provide a characterization of all state feedback controllers for solving the discrete-time stochastic mixed LQR/*H∞* control problem for linear discrete-time systems by the technique of Xu (2008 and 2011) with the well known LQG theory. Sufficient conditions for the existence of all state feedback controllers solving the discrete-time stochastic mixed LQR/*H<sup>∞</sup>* control problem are given in terms of a single algebraic Riccati equation with a free parameter matrix, plus two constrained conditions: One is a free parameter matrix con‐ strained condition on the form of the gain matrix, another is an assumption that the free pa‐ rameter matrix is a free admissible controller error. Also, a numerical algorithm for calculating a kind of discrete-time state feedback stochastic mixed LQR/*H∞* controllers are proposed. As one special case, the central discrete-time state feedback stochastic mixed LQR./*H<sup>∞</sup>* controller is given in terms of an algebraic Riccati equation. This provides an inter‐ pretation of discrete-time state feedback mixed LQR/*H<sup>∞</sup>* control problem. As another special

*<sup>z</sup>*(*k*)=(*C*<sup>1</sup> <sup>+</sup> *<sup>D</sup>*12*F∞C*2)*x*¯(*k*)

*U*1 −1 *B*1

Let*w*(*k*)=*γ* <sup>−</sup><sup>2</sup>

**6. Conclusion**

4.0000 0.2000 ,*C*<sup>1</sup> <sup>+</sup> *<sup>D</sup>*12*F∞C*<sup>2</sup> <sup>=</sup> 1.0000 <sup>0</sup>

Taking mathematical expectation of the both hand of the above two equations to get

<sup>−</sup>0.3727 <sup>−</sup>2.0174 .

Stochastic Mixed LQR/H∞ Control for Linear Discrete-Time Systems

http://dx.doi.org/10.5772/51019

25

*<sup>T</sup> <sup>X</sup>∞*(*<sup>A</sup>* <sup>+</sup> *<sup>B</sup>*2*F∞C*2)*x*¯(*k*), then the trajectories of mean values of states of re‐

**Figure 1.** The trajectories of mean values of states of resulting system in Example 5.1.

*Example 5.2* Consider the following linear discrete-time system (15) with static output feed‐ back of the form*u*(*k*)= *F<sup>∞</sup> y*(*k*), its parameter matrices are as same as Example 5.1.

When *C*2 is quare and invertible, that is, all state variable are measurable, we may assume without loss of generality that *C*<sup>2</sup> <sup>=</sup> *<sup>I</sup>* ; let*<sup>γ</sup>* =6.5, *<sup>R</sup>* =1and*<sup>Q</sup>* <sup>=</sup> <sup>1</sup> <sup>0</sup> <sup>0</sup> <sup>1</sup> , by solving the discretetime Riccati equation (24), we get that the central discrete-time state feedback stochastic mixed LQR/*H∞* controller displayed in Theorem 3.2 is

$$K \triangleq \begin{bmatrix} -0.3719 & -2.0176 \end{bmatrix}$$

and the poles of resulting closed-loop system are*p*<sup>1</sup> = −0.1923,*p*<sup>2</sup> =0.0205.

When*C*<sup>2</sup> <sup>=</sup> <sup>1</sup> 5.4125 , let*<sup>γ</sup>* =6.5,*<sup>R</sup>* =1 , *<sup>Q</sup>* <sup>=</sup> <sup>1</sup> <sup>0</sup> <sup>0</sup> <sup>1</sup> , by using Algorithm 4.1, we solve the dis‐ crete-time Riccati equation (32) to get

$$X\_{\sim} = \begin{bmatrix} 148.9006 & 8.8316 \\ 8.8316 & 9.5122 \end{bmatrix} > 0,\\ U\_1 = 0.03607$$

Thus the discrete-time static output feedback stochastic mixed LQR/*H∞* controller displayed in Theorem 4.1 is*F<sup>∞</sup>* = −0.3727. The resulting closed-loop system is

$$\begin{aligned} \mathbf{x}(k+1) &= (A + B\_2 F\_\circ \mathbf{C\_2}) \mathbf{x}(k) + B\_0 \mathbf{w}\_0(k) + B\_1 \mathbf{w}(k) \\ \mathbf{z}(k) &= (\mathbf{C\_1} + D\_{12} F\_\circ \mathbf{C\_2}) \mathbf{x}(k) \end{aligned}$$

$$\text{where, } A + B\_2 F\_\text{\\_C} \mathbf{C}\_2 = \begin{bmatrix} -0.3727 & -0.0174 \\ 4.0000 & 0.2000 \end{bmatrix} \mathbf{C}\_1 + D\_{12} F\_\text{\\_C} \mathbf{C}\_2 = \begin{bmatrix} 1.0000 & 0 \\ -0.3727 & -2.0174 \end{bmatrix} \mathbf{J}$$

Taking mathematical expectation of the both hand of the above two equations to get

$$\begin{aligned} \bar{\boldsymbol{x}}(k+1) &= (\boldsymbol{A} + \boldsymbol{B}\_2 \boldsymbol{F}\_\circ \mathbf{C}\_2) \bar{\boldsymbol{x}}(k) + \boldsymbol{B}\_1 \boldsymbol{w}(k) \\ \boldsymbol{z}(k) &= (\mathbf{C}\_1 + \mathbf{D}\_{12} \boldsymbol{F}\_\circ \mathbf{C}\_2) \bar{\boldsymbol{x}}(k) \end{aligned}$$

where, *E*{*x*(*k*)}= *<sup>x</sup>*¯(*k*), *E*{*z*(*k*)}= *<sup>z</sup>*¯(*k*),*E*{*x*(0)} <sup>=</sup> *<sup>x</sup>*¯ <sup>0</sup>.

ing closed-loop system with *x*¯ <sup>0</sup> <sup>=</sup> <sup>3</sup> <sup>2</sup> *<sup>T</sup>* are given in Fig. 1.

**Figure 1.** The trajectories of mean values of states of resulting system in Example 5.1.

without loss of generality that *C*<sup>2</sup> <sup>=</sup> *<sup>I</sup>* ; let*<sup>γ</sup>* =6.5, *<sup>R</sup>* =1and*<sup>Q</sup>* <sup>=</sup> <sup>1</sup> <sup>0</sup>

and the poles of resulting closed-loop system are*p*<sup>1</sup> = −0.1923,*p*<sup>2</sup> =0.0205.

mixed LQR/*H∞* controller displayed in Theorem 3.2 is

>0,*U*<sup>1</sup> =0.0360

in Theorem 4.1 is*F<sup>∞</sup>* = −0.3727. The resulting closed-loop system is

When*C*<sup>2</sup> <sup>=</sup> <sup>1</sup> 5.4125 , let*<sup>γ</sup>* =6.5,*<sup>R</sup>* =1 , *<sup>Q</sup>* <sup>=</sup> <sup>1</sup> <sup>0</sup>

*x*(*k* + 1)=(*A* + *B*2*F∞C*2)*x*(*k*) + *B*0*w*0(*k*) + *B*1*w*(*k*)

crete-time Riccati equation (32) to get

*K* <sup>∗</sup> = −0.3719 −2.0176

*<sup>X</sup><sup>∞</sup>* <sup>=</sup> 148.9006 8.8316 8.8316 9.5122

*z*(*k*)=(*C*<sup>1</sup> + *D*12*F∞C*2)*x*(*k*)

*Example 5.2* Consider the following linear discrete-time system (15) with static output feed‐

When *C*2 is quare and invertible, that is, all state variable are measurable, we may assume

time Riccati equation (24), we get that the central discrete-time state feedback stochastic

Thus the discrete-time static output feedback stochastic mixed LQR/*H∞* controller displayed

<sup>0</sup> <sup>1</sup> , by solving the discrete-

<sup>0</sup> <sup>1</sup> , by using Algorithm 4.1, we solve the dis‐

back of the form*u*(*k*)= *F<sup>∞</sup> y*(*k*), its parameter matrices are as same as Example 5.1.

*<sup>T</sup> <sup>X</sup>∞*(*<sup>A</sup>* <sup>+</sup> *<sup>B</sup>*2*K*)*x*¯(*k*), then the trajectories of mean values of states of result‐

Let*w*(*k*)=*γ* <sup>−</sup><sup>2</sup>

*U*1 −1 *B*1

24 Advances in Discrete Time Systems

where, *E*{*x*(*k*)}= *<sup>x</sup>*¯(*k*), *E*{*z*(*k*)}= *<sup>z</sup>*¯(*k*),*E*{*x*(0)} <sup>=</sup> *<sup>x</sup>*¯ <sup>0</sup>.

Let*w*(*k*)=*γ* <sup>−</sup><sup>2</sup> *U*1 −1 *B*1 *<sup>T</sup> <sup>X</sup>∞*(*<sup>A</sup>* <sup>+</sup> *<sup>B</sup>*2*F∞C*2)*x*¯(*k*), then the trajectories of mean values of states of re‐ sulting closed-loop system with *x*¯ <sup>0</sup> <sup>=</sup> <sup>1</sup> <sup>2</sup> *<sup>T</sup>* are given in Fig. 2.

**Figure 2.** The trajectories of mean values of states of resulting system in Example 5.2.

#### **6. Conclusion**

In this chapter, we provide a characterization of all state feedback controllers for solving the discrete-time stochastic mixed LQR/*H∞* control problem for linear discrete-time systems by the technique of Xu (2008 and 2011) with the well known LQG theory. Sufficient conditions for the existence of all state feedback controllers solving the discrete-time stochastic mixed LQR/*H<sup>∞</sup>* control problem are given in terms of a single algebraic Riccati equation with a free parameter matrix, plus two constrained conditions: One is a free parameter matrix con‐ strained condition on the form of the gain matrix, another is an assumption that the free pa‐ rameter matrix is a free admissible controller error. Also, a numerical algorithm for calculating a kind of discrete-time state feedback stochastic mixed LQR/*H∞* controllers are proposed. As one special case, the central discrete-time state feedback stochastic mixed LQR./*H<sup>∞</sup>* controller is given in terms of an algebraic Riccati equation. This provides an inter‐ pretation of discrete-time state feedback mixed LQR/*H<sup>∞</sup>* control problem. As another special case, sufficient conditions for the existence of all static output feedback controllers solving the discrete-time stochastic mixed LQR/*H∞* control problem are given. A numerical algo‐ rithm for calculating a static output feedback stochastic mixed LQR/*H∞* controller is also presented.

[10] Furata, K., & Phoojaruenchanachai, S. (1990). An algebraic approach to discretetime H∞ control problems. *Proceedings of 1990 American Control Conference*, San

Stochastic Mixed LQR/H∞ Control for Linear Discrete-Time Systems

http://dx.doi.org/10.5772/51019

27

[11] Gadewadikar, J., Lewis, F. L., Xie, L., Kucera, V., & Abu-Khalaf, M. (2007). Parameterization of all stabilizing H∞ static state-feedback gains: application to

[12] Geromel, J. C., & Peres, P. L. D. (1985). Decentrailised load-frequency control. *IEE*

[13] Geromel, J. C., Yamakami, A., & Armentano, V. A. (1989). Structrual constrained controllers for discrete-time linear systems. *Journal of Optimization and Applications*,

[14] Haddad, W. M., Bernstein, D. S., & Mustafa, D. (1991). Mixed-norm H2/H<sup>∞</sup> regulation and estimation: the discrete-time case. *Systems & Control Letters*, 16,

[15] Iglesias, P. A., & Glover, K. (1991). State-space approach to discrete-time H<sup>∞</sup>

[16] Khargonekar, P. P., & Rotea, M. A. (1991). Mixed H2/H∞ control: A convex

[17] Kucera, V., & de Souza, C. E. (1995). A necessary and sufficient condition for

[18] Kwakernaak, H. (2002). H2-optimization-theory and application to robust control

[19] Limebeer, D. J. N., Anderson, B. D. O., Khargonekar, P. P., & Green, M. (1992). A game theoretic approach to H∞ control for time-varying systems. *SIAM J. Control*

[20] Limebeer, D. J. N., Anderson, B. D. O., & Hendel, B. (1994). A Nash game approach to mixed H2/H∞ control. *IEEE Trans. Aut. Control*, 39(1), 69-82.

[21] Tse, E. (1971). On the optimal control of stochastic linear systems. *IEEE Trans. Aut.*

[22] Xu, X. (1996). A study on robust control for discrete-time systems with uncertainty. *A Master Thesis of 1995*, Kobe university, Kobe, Japan, January, 1996.

[23] Xu, X. (2008). Characterization of all static state feedback mixed LQR/H<sup>∞</sup> controllers for linear continuous-time systems. *Proceedings of the 27th Chinese*

*Control Conference*, Kunming, Yunnan, China, 678-682, July 16-18, 2008.

optimization approach. *IEEE Trans. Aut. Control*, 36(7), 824-837.

output feedback stabilizability. *Automatica*, 31(9), 1357-1359.

output-feedback design. *Automatica*, 43, 1597-1604.

Diego, 2067-3072.

61(1), 73-94.

235-247.

*Proceedings*, 132(5), 225-230.

control. *INT. J. Control*, 54(5), 1031-1073.

design. *Annual Reviews in Control*, 26, 45-56.

*and Optimization*, 30(2), 262-283.

*Control*, 16(6), 776-785.
