**4. Static Output Feedback**

This section consider discrete-time static output feedback stochastic mixed LQR/*H<sup>∞</sup>* control problem. This problem is defined as follows:

In this case, the discrete-time static output feedback stochastic mixed LQR/*H<sup>∞</sup>* controller

*<sup>Δ</sup><sup>K</sup>* <sup>=</sup> *<sup>F</sup>∞C*<sup>2</sup> <sup>−</sup> *<sup>K</sup>* ∗. As is discussed in Remark 3.1, suppose that there exists a suboptimal con‐ troller *<sup>K</sup>* <sup>∗</sup> such that *AK* <sup>∗</sup> <sup>=</sup> *<sup>A</sup>* <sup>+</sup> *<sup>B</sup>*2*<sup>K</sup>* <sup>∗</sup> is stable, then *ΔK* is an admissible controller error if it

It should be noted that Theorem 4.1 does not tell us how to calculate a discrete-time static output feedback stochastic mixed LQR/*H<sup>∞</sup>* controller*F∞*. In order to do this, we present, based on the algorithms proposed by Geromel & Peres (1985) and Kucera & de Souza (1995), a numerical algorithm for computing a discrete-time static output feedback stochastic mixed LQR/*H∞* controller *F<sup>∞</sup>* and a solution *X<sup>∞</sup>* to discrete-time Riccati equation (32). This numeri‐

−1 *B*2

http://dx.doi.org/10.5772/51019

Stochastic Mixed LQR/H∞ Control for Linear Discrete-Time Systems

*<sup>T</sup> <sup>U</sup>*3*A*, then

21

*<sup>T</sup> <sup>X</sup>∞B*0*R*1(*k*)) subject to *Tzw <sup>∞</sup>* <sup>&</sup>lt;*<sup>γ</sup>*

*Remark 4.1* In Theorem 4.1, define a suboptimal controller as*<sup>K</sup>* <sup>∗</sup> <sup>=</sup> <sup>−</sup>*U*<sup>2</sup>

Step 1: Fix the two weighting matrices *Q* and*R*, set*i* =0, *ΔKi* =0, and*U*2(*i*)=0.

^ *<sup>T</sup> XiA* is stable and*U*1(*i*)<sup>=</sup> *<sup>I</sup>* <sup>−</sup>*<sup>γ</sup>* <sup>−</sup><sup>2</sup>

*B*1

, ⋯, *U*1(*i*)

, ⋯ converges, say to*X∞*,*U*<sup>1</sup> ,*U*<sup>2</sup> and*U*3, respectively; then the both

, ⋯,*U*2(1)

, *U*2(2)

, ⋯, *U*2(*i*)

, ⋯ ,

and *ΔKi*+1 by using the following formulas

*<sup>T</sup> XiB*<sup>1</sup> >0.

will achieve

{*JE*} = lim *T* →*∞* 1 *<sup>T</sup>* ∑ *k*=0 *T tr*(*B*<sup>0</sup>

belongs to the set:

*Algorithm 4.1*

+*C*<sup>1</sup>

*A* ^

for *Xi*

*ci* = *A*− *B* ^ (*B* ^ *<sup>T</sup> XiB* ^ <sup>+</sup> *<sup>R</sup>* ^ ) −1 *B*

*<sup>A</sup><sup>T</sup> XiA*<sup>−</sup> *Xi* <sup>−</sup> *<sup>A</sup><sup>T</sup> XiB*

*TC*<sup>1</sup> <sup>+</sup> *<sup>Q</sup>* <sup>+</sup> *<sup>Δ</sup>Ki*

Step 3: Calculate*U*3(*i*+1)

*U*2(*i*+1)=*R* + *I* + *B*<sup>2</sup>

*<sup>Δ</sup>Ki*+1 <sup>=</sup> <sup>−</sup>*U*2(*i*+1) <sup>−</sup><sup>1</sup> *<sup>B</sup>*<sup>2</sup>

wise stop.

and *U*3(1)

*<sup>U</sup>*3(*i*+1)<sup>=</sup> *Xi* <sup>+</sup> *<sup>γ</sup>* <sup>−</sup><sup>2</sup>*XiB*1*U*1(*i*)

, *U*3(2)

*Ω* : ={*ΔK* : *A* + *B*2*F∞C*2 is stable}

cal algorithm is given as follows:

Step 2: Solve the discrete-time Riccati equation

*ΔKi* =0

, *U*2(*i*+1)

<sup>−</sup><sup>1</sup> *<sup>B</sup>*<sup>1</sup> *<sup>T</sup> Xi*

*B*2

*A*(*C*<sup>2</sup>

*<sup>T</sup>* (*C*2*C*<sup>2</sup>

*<sup>T</sup>* )−<sup>1</sup>

feedback stochastic mixed LQR/*H∞* controllers is parameterized as follows:

*C*<sup>2</sup> − *I*)

, ⋯, *U*1(1)

Step 4: If *ΔKi*+1 is an admissible controller error, then increase *i* by1, and goto Step 2; other‐

two conditions displayed in Theorem 4.1 are met. In this case, a discrete-time static output

, *U*1(2)

*<sup>T</sup> <sup>U</sup>*3(*i*+1)

*<sup>T</sup> <sup>U</sup>*3(*i*+1)

If the four sequences*X*0, *X*1, ⋯, *Xi*

, ⋯, *U*3(*i*)

symmetric non-negative definite such that

^(*B* ^ *<sup>T</sup> XiB* ^ <sup>+</sup> *<sup>R</sup>* ^ )−<sup>1</sup> *B* ^ *<sup>T</sup> XiA*

*<sup>T</sup> <sup>U</sup>*2(*i*)

sup *w*∈*L* 2+

Discrete-time static ouput feedback stochastic mixed LQR/*H∞* control problem: Consider the system (15) under the influence of static output feedback of the form

*u*(*k*)= *F<sup>∞</sup> y*(*k*)

with*w* ∈ *L* <sup>2</sup> 0,*∞*), for a given number*γ* >0, determine an admissible static output feedback controller *F∞* such that

$$\sup\_{w \in \mathcal{L}\_{z\_\ast}} \left\| I\_E \right\| \text{ subject to } \left\| \begin{array}{c} T\_{zw} \end{array} \right\|\_\ast \le \gamma$$

If this admissible controller exists, it is said to be a discrete-time static output feedback sto‐ chastic mixed LQR/*H<sup>∞</sup>* controller. As is well known, the discrete-time static output feedback stochastic mixed LQR/*H∞* control problem is equivalent to the discrete-time state feedback stochastic mixed LQR/*H∞* control problem for the systems (15) (16), where, *K*is constrained to have the form of*K* = *F∞C*2. This problem is also said to be a structural constrained state feedback stochastic mixed LQR/*H<sup>∞</sup>* control problem.Based the above, we can obtain all solu‐ tion to discrete-time static output feedback stochastic mixed LQR/*H∞* control problem by us‐ ing the result of Theorem 3.1 as follows:

*Theorem 4.1* There exists a discrete-time static output feedback stochastic mixed LQR/ *H<sup>∞</sup>* controller if the following two conditions hold:

i.There exists a matrix *ΔK* such that

$$
\Delta K = F \, \text{C}\_2 + \, \text{U}\_2^{-1} \text{B}\_2^{T} \, \text{U}\_3 \text{A} \tag{31}
$$

and *X∞* is a symmetric non-negative definite solution of the following discrete-time Riccati equation

$$\begin{aligned} & \mathbf{A}^T \mathbf{X}\_{\circ \circ} \mathbf{A} - \mathbf{X}\_{\circ \circ} - \mathbf{A}^T \mathbf{X}\_{\circ \circ} \overset{\wedge}{\mathbf{B}} \left( \overset{\wedge}{\mathbf{B}}^T \mathbf{X}\_{\circ \circ} \overset{\wedge}{\mathbf{B}} + \overset{\wedge}{\mathbf{R}} \right)^{-1} \overset{\wedge}{\mathbf{B}}^T \mathbf{X}\_{\circ \circ} \mathbf{A} \\ & + \mathbf{C}\_1^T \mathbf{C}\_1 + \mathbf{Q} + \Delta \mathbf{K}^T \mathbf{U}\_2 \Delta \mathbf{K} = \mathbf{0} \end{aligned} \tag{32}$$

and *A* ^ *<sup>c</sup>* = *A*− *B* ^ (*B* ^ *<sup>T</sup> <sup>X</sup>∞<sup>B</sup>* ^ <sup>+</sup> *<sup>R</sup>* ^ ) −1 *B* ^ *<sup>T</sup> <sup>X</sup>∞A* is stable and*U*<sup>1</sup> <sup>=</sup> *<sup>I</sup>* <sup>−</sup>*<sup>γ</sup>* <sup>−</sup><sup>2</sup> *B*1 *<sup>T</sup> <sup>X</sup>∞B*<sup>1</sup> >0.

Where, *B* ^ <sup>=</sup> *<sup>γ</sup>* <sup>−</sup><sup>1</sup> *B*<sup>1</sup> *B*<sup>2</sup> , *R* ^ <sup>=</sup> <sup>−</sup> *<sup>I</sup>* <sup>0</sup> <sup>0</sup> *<sup>R</sup>* <sup>+</sup> *<sup>I</sup>* , *U*<sup>3</sup> <sup>=</sup> *<sup>X</sup><sup>∞</sup>* <sup>+</sup> *<sup>γ</sup>* <sup>−</sup>2*X∞B*1*U*<sup>1</sup> −1 *B*1 *<sup>T</sup> <sup>X</sup>∞*,*U*<sup>2</sup> <sup>=</sup>*<sup>R</sup>* <sup>+</sup> *<sup>I</sup>* <sup>+</sup> *<sup>B</sup>*<sup>2</sup> *<sup>T</sup> <sup>U</sup>*3*B*2.

ii. *ΔK*is an admissible controller error.

In this case, the discrete-time static output feedback stochastic mixed LQR/*H<sup>∞</sup>* controller will achieve

$$\sup\_{\mathcal{I}\_{w \in \mathcal{I}\_{z\star}}} \left\{ I\_E \right\} = \lim\_{T \to \infty} \frac{1}{T} \sum\_{k=0}^T \operatorname{tr} \left( B\_0^{\operatorname{T}} X\_{\leadsto} B\_0 R\_1(k) \right) \operatorname{subject } \operatorname{to } \left\| \begin{array}{c} T\_{zw} \end{array} \right\|\_{\infty} \leqslant \gamma \lambda$$

*Remark 4.1* In Theorem 4.1, define a suboptimal controller as*<sup>K</sup>* <sup>∗</sup> <sup>=</sup> <sup>−</sup>*U*<sup>2</sup> −1 *B*2 *<sup>T</sup> <sup>U</sup>*3*A*, then *<sup>Δ</sup><sup>K</sup>* <sup>=</sup> *<sup>F</sup>∞C*<sup>2</sup> <sup>−</sup> *<sup>K</sup>* ∗. As is discussed in Remark 3.1, suppose that there exists a suboptimal con‐ troller *<sup>K</sup>* <sup>∗</sup> such that *AK* <sup>∗</sup> <sup>=</sup> *<sup>A</sup>* <sup>+</sup> *<sup>B</sup>*2*<sup>K</sup>* <sup>∗</sup> is stable, then *ΔK* is an admissible controller error if it belongs to the set:

$$\mathcal{Q} := \{ \Delta K : A + B\_2 F\_{\bowtie} \mathcal{C}\_2 \text{ is stable} \}$$

It should be noted that Theorem 4.1 does not tell us how to calculate a discrete-time static output feedback stochastic mixed LQR/*H<sup>∞</sup>* controller*F∞*. In order to do this, we present, based on the algorithms proposed by Geromel & Peres (1985) and Kucera & de Souza (1995), a numerical algorithm for computing a discrete-time static output feedback stochastic mixed LQR/*H∞* controller *F<sup>∞</sup>* and a solution *X<sup>∞</sup>* to discrete-time Riccati equation (32). This numeri‐ cal algorithm is given as follows:

*Algorithm 4.1*

**4. Static Output Feedback**

20 Advances in Discrete Time Systems

*u*(*k*)= *F<sup>∞</sup> y*(*k*)

sup *w*∈*L* 2+

equation

and *A* ^

Where, *B*

*<sup>c</sup>* = *A*− *B* ^ (*B* ^ *<sup>T</sup> <sup>X</sup>∞<sup>B</sup>*

^ <sup>=</sup> *<sup>γ</sup>* <sup>−</sup><sup>1</sup>

controller *F∞* such that

{*JE*} subject to *Tzw <sup>∞</sup>* <*γ*

ing the result of Theorem 3.1 as follows:

i.There exists a matrix *ΔK* such that

controller if the following two conditions hold:

+*C*<sup>1</sup>

*B*<sup>1</sup> *B*<sup>2</sup> , *R*

ii. *ΔK*is an admissible controller error.

^ <sup>+</sup> *<sup>R</sup>* ^ ) −1 *B*

problem. This problem is defined as follows:

This section consider discrete-time static output feedback stochastic mixed LQR/*H<sup>∞</sup>* control

Discrete-time static ouput feedback stochastic mixed LQR/*H∞* control problem: Consider the

with*w* ∈ *L* <sup>2</sup> 0,*∞*), for a given number*γ* >0, determine an admissible static output feedback

If this admissible controller exists, it is said to be a discrete-time static output feedback sto‐ chastic mixed LQR/*H<sup>∞</sup>* controller. As is well known, the discrete-time static output feedback stochastic mixed LQR/*H∞* control problem is equivalent to the discrete-time state feedback stochastic mixed LQR/*H∞* control problem for the systems (15) (16), where, *K*is constrained to have the form of*K* = *F∞C*2. This problem is also said to be a structural constrained state feedback stochastic mixed LQR/*H<sup>∞</sup>* control problem.Based the above, we can obtain all solu‐ tion to discrete-time static output feedback stochastic mixed LQR/*H∞* control problem by us‐

*Theorem 4.1* There exists a discrete-time static output feedback stochastic mixed LQR/ *H<sup>∞</sup>*

−1 *B*2

and *X∞* is a symmetric non-negative definite solution of the following discrete-time Riccati

^ <sup>+</sup> *<sup>R</sup>* ^ )−<sup>1</sup> *B* ^ *<sup>T</sup> <sup>X</sup>∞<sup>A</sup>*

*B*1

−1 *B*1

*<sup>T</sup> <sup>X</sup>∞B*<sup>1</sup> >0.

*<sup>T</sup> <sup>X</sup>∞*,*U*<sup>2</sup> <sup>=</sup>*<sup>R</sup>* <sup>+</sup> *<sup>I</sup>* <sup>+</sup> *<sup>B</sup>*<sup>2</sup>

^(*B* ^ *<sup>T</sup> <sup>X</sup>∞<sup>B</sup>*

^ *<sup>T</sup> <sup>X</sup>∞A* is stable and*U*<sup>1</sup> <sup>=</sup> *<sup>I</sup>* <sup>−</sup>*<sup>γ</sup>* <sup>−</sup><sup>2</sup>

<sup>0</sup> *<sup>R</sup>* <sup>+</sup> *<sup>I</sup>* , *U*<sup>3</sup> <sup>=</sup> *<sup>X</sup><sup>∞</sup>* <sup>+</sup> *<sup>γ</sup>* <sup>−</sup>2*X∞B*1*U*<sup>1</sup>

*<sup>T</sup> <sup>U</sup>*3*<sup>A</sup>* (31)

(32)

*<sup>T</sup> <sup>U</sup>*3*B*2.

*ΔK* = *FC*<sup>2</sup> + *U*<sup>2</sup>

*<sup>A</sup><sup>T</sup> <sup>X</sup>∞A*<sup>−</sup> *<sup>X</sup><sup>∞</sup>* <sup>−</sup> *<sup>A</sup><sup>T</sup> <sup>X</sup>∞<sup>B</sup>*

^ <sup>=</sup> <sup>−</sup> *<sup>I</sup>* <sup>0</sup>

*TC*<sup>1</sup> <sup>+</sup> *<sup>Q</sup>* <sup>+</sup> *<sup>Δ</sup><sup>K</sup> <sup>T</sup> <sup>U</sup>*2*Δ<sup>K</sup>* =0

system (15) under the influence of static output feedback of the form

Step 1: Fix the two weighting matrices *Q* and*R*, set*i* =0, *ΔKi* =0, and*U*2(*i*)=0.

Step 2: Solve the discrete-time Riccati equation

$$\begin{aligned} &\mathbf{A}^T \mathbf{X}\_i \mathbf{A} - \mathbf{X}\_i - \mathbf{A}^T \mathbf{X}\_i \overset{\wedge}{\mathbf{B}} \overset{\wedge}{\mathbf{B}}^T \mathbf{X}\_i \overset{\wedge}{\mathbf{B}} + \overset{\wedge}{\mathbf{R}} \overset{\wedge}{\mathbf{B}}^T \mathbf{X}\_i \mathbf{A} \\ &+ \mathbf{C}\_1^T \mathbf{C}\_1 + \mathbf{Q} + \boldsymbol{\Delta} \, \mathbf{K}\_i^T \mathbf{U} \mathbf{2}\_{\{i\}} \boldsymbol{\Delta} \, \mathbf{K}\_i = \mathbf{0} \end{aligned}$$

for *Xi* symmetric non-negative definite such that

$$\stackrel{\wedge}{A}\_{ci} = A - \stackrel{\wedge}{B} (\stackrel{\wedge}{B}^T X\_i \stackrel{\wedge}{B} + \stackrel{\wedge}{R})^{-1} \stackrel{\wedge}{B}^T X\_i A \text{ is stable and} \\ U\_{1(i)} = I - \gamma^{-2} B\_1^T X\_i B\_1 > 0.$$

Step 3: Calculate*U*3(*i*+1) , *U*2(*i*+1) and *ΔKi*+1 by using the following formulas

$$\begin{aligned} \mathcal{U}\mathcal{U}\_{3(i+1)} &= \mathcal{X}\_i + \gamma^{-2} \mathcal{X}\_i \mathcal{B}\_1 \mathcal{U}\_{1(i)}^{-1} \mathcal{B}\_1^T \mathcal{X}\_i \\ \mathcal{U}\mathcal{U}\_{2(i+1)} &= \mathcal{R} + I + \mathcal{B}\_2^T \mathcal{U}\_{3(i+1)} \mathcal{B}\_2 \\ \mathcal{A}\mathcal{K}\_{i+1} &= -\mathcal{U}\_{2(i+1)}^{-1} \mathcal{B}\_2^T \mathcal{U}\_{3(i+1)} \mathcal{A} \{\mathcal{C}\_2^T \{\mathcal{C}\_2 \mathcal{C}\_2^T\}^{-1} \mathcal{C}\_2 - I\} \end{aligned}$$

Step 4: If *ΔKi*+1 is an admissible controller error, then increase *i* by1, and goto Step 2; other‐ wise stop.

If the four sequences*X*0, *X*1, ⋯, *Xi* , ⋯, *U*1(1) , *U*1(2) , ⋯, *U*1(*i*) , ⋯,*U*2(1) , *U*2(2) , ⋯, *U*2(*i*) , ⋯ , and *U*3(1) , *U*3(2) , ⋯, *U*3(*i*) , ⋯ converges, say to*X∞*,*U*<sup>1</sup> ,*U*<sup>2</sup> and*U*3, respectively; then the both two conditions displayed in Theorem 4.1 are met. In this case, a discrete-time static output feedback stochastic mixed LQR/*H∞* controllers is parameterized as follows:

*F<sup>∞</sup>* = −*U*<sup>2</sup> −1 *B*2 *<sup>T</sup> <sup>U</sup>*3*AC*<sup>2</sup> *<sup>T</sup>* (*C*2*C*<sup>2</sup> *T* ) −1

In this chapter, we will not prove the convergence of the above algorithm. This will is anoth‐ er subject.

**Iteration Index** *i*

**Solution of DARE**

10 *X*<sup>10</sup> >0 −0.2927

**Table 1.** The calculating results of algorithm 3.1.

of the above two equations to get

*<sup>x</sup>*¯(*<sup>k</sup>* <sup>+</sup> 1)=(*<sup>A</sup>* <sup>+</sup> *<sup>B</sup>*2*K*)*x*¯(*k*) <sup>+</sup> *<sup>B</sup>*1*w*(*k*)

*<sup>z</sup>*¯(*k*)=(*C*<sup>1</sup> <sup>+</sup> *<sup>D</sup>*12*K*)*x*¯(*k*)

**Additional Condition** *U*1(*i*)

0 *X*<sup>0</sup> >0 0.5571 −0.2886 −1.9992 *p*<sup>1</sup> = −0.2951,

1 *X*<sup>1</sup> >0 0.5553 −0.2892 −2.0113 *p*<sup>1</sup> = −0.1653,

2 *X*<sup>2</sup> >0 0.5496 −0.2903 −2.0237 *p*1,2 = −0.0451

3 *X*<sup>3</sup> >0 0.5398 −0.2918 −2.0363 *p*1,2 = −0.0459

4 *X*<sup>4</sup> >0 0.5250 −0.2940 −2.0492 *p*1,2 = −0.0470

5 *X*<sup>5</sup> >0 0.5040 −0.2969 −2.0624 *p*1,2 = −0.0485

6 *X*<sup>6</sup> >0 0.4739 −0.3010 −2.0760 *p*1,2 = −0.0505

7 *X*<sup>7</sup> >0 0.4295 −0.3071 −2.0901 *p*1,2 = −0.0535

8 *X*<sup>8</sup> >0 0.3578 −0.3167 −2.1049 *p*1,2 = −0.0584

9 *X*<sup>9</sup> >0 0.2166 −0.3360 −2.1212 *p*1,2 = −0.0680

To determine the mean value function, we take mathematical expectation of the both hand

**State Feedback Controller** *Ki*

**The Closed-Loop Poles**

http://dx.doi.org/10.5772/51019

23

Stochastic Mixed LQR/H∞ Control for Linear Discrete-Time Systems

*p*<sup>2</sup> =0.2065

*p*<sup>2</sup> =0.0761

± *j*0.1862

± *j*0.2912

± *j*0.3686

± *j*0.4335

± *j*0.4911

± *j*0.5440

± *j*0.5939

± *j*0.6427

*Xi*
