Game Theory Applications

*Systems-of-Systems Perspectives and Applications - Design, Modeling, Simulation…*

[82] Shin D-H, Kim H-K. A study on user experience based Storydoing operating principles. The Journal of Digital Contents Society. 2015;**16**(3):425-436

[83] Leigh A. GDC 2012: Sid Meier on how to See Games as Sets of Interesting Decisions. 2012. Available from: https://www.gamasutra.com/view/ news/164869/GDC\_2012\_Sid\_Meier\_ on\_how\_to\_see\_games\_as\_sets\_of\_

interesting\_decisions.php

editore; 2017

pp. 171-179

2018

[84] Morrison B. Gamasutra Blog. 2013. [Online]. Available from: www. gamasutra.com/blog/bricemorrison

[85] Viola F, Cassone VI. L'arte del coinvolgimento: emozioni e stimoli per cambiare il mondo. Milano: Hoepli

[86] Marczewski A. Gamification–A simple introduction. Tips, advice and thoughts on gamification [Internet]. 2012. Available from: https://cutt.ly/ ltLAJTS [Accessed: 13 January 2020]

[87] Freeman B, Hawkins R. Evokedeveloping Skills in Youth to Solve the World's Most Complex Problems: The Social Innovators' Framework. 11th ed. Washington, DC: World Bank Education, Technology and Innovation: SABER-ICT Technical Paper Series; 2016

[88] Moccozet L, Tardy C, Opprecht W, Léonard M. Gamification-based assessment of group work. In: 2013 International Conference on Interactive Collaborative Learning (ICL). 2013.

[89] Getplayoff.com [Internet]. 2020. Available from: https://www.getplayoff. com/ [Accessed: 12 February 2020]

[90] Federculture. Impresa Cultura. Comunità, territori, sviluppo: 14° Federculture. Roma: Gangemi Editore;

[71] Dochy F, Segers M, Sluijsmans D. The use of self-, peer and co-assessment in higher education: A review. Studies in Higher Education. 1999;**24**(3):331-350

[72] Tessaro F. La valutazione dei processi formativi. Roma: Armando

[73] Chan KMA et al. Opinion: Why protect nature? Rethinking values and the environment. Proceedings of the National Academy of Sciences.

[74] Kim AJ. Game Thinking: Innovate smarter & Drive Deep Engagement with Design Techniques from Hit Games. 2nd edition. Burlingame: Gamethinking.io;

[75] Coppock P, Ferri G. Serious Urban Games. From play in the city to play for the city. In: Media and the City. Urbanism, Technology and Communication. Newcastle: Cambridge Scholars Publishing; 2013. p. 120-134

[76] Schell J. Jesse Schell: When Games Invade Real Life. In: *DICE Summit*. 2010

[77] Deterding S, Khaled R, Nacke LE, Dixon D. Gamification: Toward a definition. In: CHI 2011 Gamification Workshop Proceedings. Vol. 12. Vancouver BC, Canada; 2011

[78] Viola F. Gamification/Pointsfication/ Serious Games. 2011 [Online]. Available from: http://www.gameifications.com/

gamification-pointsification/

Penguin; 2011

tuomuseo.it/

[79] McGonigal J. Reality Is Broken: Why Games Make us Better and How They Can Change the World. New York:

[80] Viola F. Fabio Viola speaker con TuoMuseo a FareTurismo. 2015[Online]. Available from: http:// www.gameifications.com/?s=tuomuseo

[81] TuoMuseo. tuomuseo.it. 2018. [Online]. Available from: https://www.

Editore; 1997

2018

2016;**113**(6):1462-1465

**182**

**Chapter 11**

**Abstract**

**1. Introduction**

**185**

Existence of Open Loop Equilibria

for Disturbed Stackelberg Games

*T.-P. Azevedo Perdicoúlis, G. Jank and P. Lopes dos Santos*

In this work, we derive necessary and sufficient conditions for the existence of an hierarchic equilibrium of a disturbed two player linear quadratic game with open loop information structure. A convexity condition guarantees the existence of a unique Stackelberg equilibria; this solution is first obtained in terms of a pair of symmetric Riccati equations and also in terms of a coupled of system of Riccati equations. In this latter case, the obtained equilibrium controls are of feedback type.

**Keywords:** differential games, linear quadratic, Riccati differential equations,

The study of linear quadratic (LQ) games has been addressed by many authors [1–4]. This type of games is often used as a benchmark to assess the game equilibrium strategies and its respective outcomes. In a disturbed differential game, each player calculates its strategy taking into account a worst-case unknown disturbance. In non-cooperative game theory, the concept of hierarchical or Stackelberg games is very important, since different applications in economics and engineering exist [1, 5]. This is also the case of gas networks where a hierarchy may be assigned to its controllable elements—compressors, sources, reductors, etc… Also, for this application, the modelling as a disturbed game makes a lot of sense, since the unknown offtakes of the network can be modelled as unknown disturbances. Further research on Stackelberg games can be found for instance in AbouKandil and Bertrand [6];

No assumptions/constraints are made of the disturbance. To be easier to understand the hierarchical concept, we consider only two players. Therefore, we study a LQ game of two players with Open Loop (OL) information structure where the players choose its strategy according to a modified Stackelberg equilibrium. Player-1 is the follower and chooses its strategy after the nomination of the strategy of the leader. Player-2, the leader, chooses its strategy assuming rationality of the follower.

In this work, we consider a finite time horizon, where for applications this is chosen according to the periodicity of the operation of the problem being studied. The disturbed case of the representation of optimal equilibria for noncooperative games has been studied [10, 11] considering a Nash equilibrium. It is the aim of this paper to generalise the work of Jank and Kun to Stackelberg games and extend the results presented in Freiling and Jank [12]; Freiling et al. [13] to the disturbed

Both players find their strategies assuming a worst-case disturbance.

Stackelberg equilibrium, worst-case disturbance

Medanic [7]; Yong [8]; Tolwinski [9].

### **Chapter 11**

## Existence of Open Loop Equilibria for Disturbed Stackelberg Games

*T.-P. Azevedo Perdicoúlis, G. Jank and P. Lopes dos Santos*

#### **Abstract**

In this work, we derive necessary and sufficient conditions for the existence of an hierarchic equilibrium of a disturbed two player linear quadratic game with open loop information structure. A convexity condition guarantees the existence of a unique Stackelberg equilibria; this solution is first obtained in terms of a pair of symmetric Riccati equations and also in terms of a coupled of system of Riccati equations. In this latter case, the obtained equilibrium controls are of feedback type.

**Keywords:** differential games, linear quadratic, Riccati differential equations, Stackelberg equilibrium, worst-case disturbance

#### **1. Introduction**

The study of linear quadratic (LQ) games has been addressed by many authors [1–4]. This type of games is often used as a benchmark to assess the game equilibrium strategies and its respective outcomes. In a disturbed differential game, each player calculates its strategy taking into account a worst-case unknown disturbance. In non-cooperative game theory, the concept of hierarchical or Stackelberg games is very important, since different applications in economics and engineering exist [1, 5]. This is also the case of gas networks where a hierarchy may be assigned to its controllable elements—compressors, sources, reductors, etc… Also, for this application, the modelling as a disturbed game makes a lot of sense, since the unknown offtakes of the network can be modelled as unknown disturbances. Further research on Stackelberg games can be found for instance in AbouKandil and Bertrand [6]; Medanic [7]; Yong [8]; Tolwinski [9].

No assumptions/constraints are made of the disturbance. To be easier to understand the hierarchical concept, we consider only two players. Therefore, we study a LQ game of two players with Open Loop (OL) information structure where the players choose its strategy according to a modified Stackelberg equilibrium. Player-1 is the follower and chooses its strategy after the nomination of the strategy of the leader. Player-2, the leader, chooses its strategy assuming rationality of the follower. Both players find their strategies assuming a worst-case disturbance.

In this work, we consider a finite time horizon, where for applications this is chosen according to the periodicity of the operation of the problem being studied.

The disturbed case of the representation of optimal equilibria for noncooperative games has been studied [10, 11] considering a Nash equilibrium. It is the aim of this paper to generalise the work of Jank and Kun to Stackelberg games and extend the results presented in Freiling and Jank [12]; Freiling et al. [13] to the disturbed

case. To calculate the controls, we use a value function approach, appropriately guessed. Thence, we obtain sufficient conditions of existence of these controls and its representation in terms of the solution of certain Riccati equations. Furthermore, a feedback form of the worst-case Stackelberg equilibrium is obtained.

where

*i* ¼ 1, 2, … , *N:*

by the leader.

*J* ∗ *<sup>i</sup> ui*, *u*<sup>∗</sup>

*γ* ∗

Player-1 is the follower.

ð Þ �*<sup>i</sup>* , *<sup>w</sup>*

The leader seeks a strategy *u*<sup>∗</sup>

**Problem 2.1.** Find the control *u*<sup>∗</sup>

controls and disturbance, respectively.

*J* ∗ *<sup>i</sup> ui*, *u*<sup>∗</sup>

2.We say that the controls *u*<sup>∗</sup>

i. The leader chooses *u*<sup>∗</sup>

for all *u*<sup>2</sup> ∈ U2*:*

**187**

max *<sup>γ</sup>*<sup>1</sup> <sup>∈</sup> <sup>R</sup><sup>1</sup> *<sup>u</sup>*<sup>∗</sup> ð Þ <sup>2</sup>

ii. The follower then chooses *u*<sup>∗</sup>

K *x tf*

*Existence of Open Loop Equilibria for Disturbed Stackelberg Games*

*DOI: http://dx.doi.org/10.5772/intechopen.92202*

� � � � <sup>¼</sup> *xT tf*

*j*¼1 *uT*

þ X *N*

<sup>Ψ</sup> *ui*, *<sup>u</sup>*ð Þ �*<sup>i</sup>* , *<sup>w</sup>* � � <sup>¼</sup> *xT*ð Þ*<sup>t</sup> Qi*ð Þ*<sup>t</sup> x t*ðÞþ *<sup>w</sup>T*ð Þ*<sup>t</sup> Pi*ð Þ*<sup>t</sup> w t*ð Þ

with symmetric matrices *Kif* ∈ *<sup>n</sup>*�*<sup>n</sup>* and symmetric, piecewise continuous and bounded matrix valued functions *Qi*ð Þ*<sup>t</sup>* <sup>∈</sup> *<sup>n</sup>*�*n*, *Rij*ð Þ*<sup>t</sup>* <sup>∈</sup> *mi*�*<sup>m</sup> <sup>j</sup>* and *Pi*ð Þ*<sup>t</sup>* <sup>∈</sup> *<sup>m</sup>*�*m*,

We observe that no cost functional is assigned to the disturbance term because no constraints can be applied on an "unpredictable" parameter. In what follows, we consider *N* ¼ 2*:* To extend the theory to *N* > 2, since this is an hierarchical solution, we need to define the structure of the leaders and followers in the game. We can even have more than two hierachy levels. We assume that Player-2 is the leader and

before the game starts. This strategy is found knowing how the follower reacts to his choices. The follower calculates its strategy as a best reply to the strategy announced

Consider U*i*, *i* ¼ 1, 2, the sets of functions such that (1) is solvable and *Ji* exists, with *ui*, *i* ¼ 1, 2, in these conditions U*i*, *i* ¼ 1, 2, W are said the sets of admissible

**Definition 2.3.** (Stackelberg equilibrium) Let Γ<sup>2</sup> be a 2-person differential game,

1.A function *w*^*i*ð Þ *u* ∈ W is called the worst-case disturbance, from the point of view of the *i*th player belonging to the set of admissible controls, if

*<sup>i</sup> ui*, *u*<sup>∗</sup>

holds for each *w* ∈ W. There exists exactly one worst-case disturbance from

the point of view of the *i*th player according to every set of controls.

<sup>1</sup> , *u*<sup>∗</sup> 2

<sup>2</sup> such that

<sup>2</sup> , *w*^<sup>2</sup> � �<sup>≤</sup> max *<sup>γ</sup>*<sup>1</sup> <sup>∈</sup> <sup>R</sup>1ð Þ *<sup>u</sup>*<sup>2</sup>

<sup>1</sup> such that

*J*<sup>2</sup> *γ*1, *u*<sup>∗</sup>

ð Þ �*<sup>i</sup>* , *w*

� �, *<sup>i</sup>* <sup>¼</sup> 1, 2, (5)

� � form a worst-case Stackelberg equilibrium if

*J*<sup>2</sup> *γ*<sup>1</sup> ð Þ , *u*2, *w*^<sup>2</sup>

� �, *<sup>i</sup>* <sup>¼</sup> 1, 2, is minimal when subject to constraints *<sup>u</sup>*<sup>∗</sup>

we define the Stackelberg/worst-case equilibrium in two stages.

ð Þ �*<sup>i</sup>* , *<sup>w</sup>*^*<sup>i</sup>* � � <sup>≥</sup> *<sup>J</sup>* <sup>∗</sup>

*<sup>i</sup> t*, *η<sup>i</sup>* ð Þ ð Þ*t* , *i* ¼ 1, 2, and (1) and considering a worst-case disturbance.

� �*Kif x tf*

� � (3)

*<sup>j</sup>* ð Þ*<sup>t</sup> Rij*ð Þ*<sup>t</sup> <sup>u</sup> <sup>j</sup>*ð Þ*<sup>t</sup>* , (4)

<sup>2</sup> ð Þ*t* in OL information structure and announces it

*<sup>i</sup>* ðÞ¼ *t*

*<sup>i</sup>* ∈U*i*, *i* ¼ 1, 2, in T for which

In a future paper, we expect to present analogous conditions using an operator approach.

In Section 2, we define the disturbed LQ game and define Stackelberg worst-case equilibrium. In Section 3, we derive sufficient conditions for the existence of a worst-case Stackelberg equilibrium under OL information structure and investigate how are these solutions related to certain Riccati differential equations. Section 4 concludes the paper and outlines some directions for future work.

#### **2. Fundamental notions**

We start with the concept of best reply:

**Definition 2.1.** (Best reply) Let Γ*<sup>N</sup>* be a *N*-player differential game. For *i* ∈f g 1, … , *N* ,

$$\gamma\_{(-i)} \coloneqq (\gamma\_1, \dots, \gamma\_{i-1}, \gamma\_{i+1}, \dots, \gamma\_N) \in \otimes\_{\, j \neq i} \mathcal{U}\_j \cdot \mathbf{1}$$

We say that ~*γ<sup>i</sup>* is the best reply against *γ*ð Þ �*<sup>i</sup>* if

$$J\_i(\boldsymbol{\gamma}\_1, \dots, \boldsymbol{\gamma}\_{i-1}, \tilde{\boldsymbol{\gamma}}\_i, \boldsymbol{\gamma}\_{i+1}, \dots, \boldsymbol{\gamma}\_N) \leq J\_i(\boldsymbol{\gamma}\_1, \dots, \boldsymbol{\gamma}\_N)^T$$

holds for any strategy *γ<sup>i</sup>* ∈U*i*. We denote the set of all best replies by R*<sup>i</sup> γ*ð Þ �*<sup>i</sup>* � �.

We study games of quadratic criteria, defined in a finite time horizon *t*0, *tf* � �⊂ and subject to a linear dynamics, controlled players and also an unknown disturbance. Hereby also consider *u <sup>j</sup>* ¼ *γ <sup>j</sup> t*, *η <sup>j</sup>* � �, where *<sup>η</sup> <sup>j</sup>* is the information structure of Player-j. In this case, *η <sup>j</sup>*, *j* ¼ 1, … , *N*, is of OL type.

**Definition 2.2.** (Linear Quadratic (LQ) differential game) Let Γ*<sup>N</sup>* be an *N*� player differential game finite time horizon *T* ¼ *t*0, *tf* � �*:* Suppose further that:

i. the dynamics of the game are assumed to obey a linear differential equation

$$\begin{aligned} \dot{\mathbf{x}}(t) &= \mathbf{A}(t)\mathbf{x}(t) + \sum\_{j=1}^{N} B\_j(t)\boldsymbol{u}\_j(t) + \mathbf{C}(t)\boldsymbol{w}(t), \\ \mathbf{x}(t\_0) &= \mathbf{x}\_0. \end{aligned} \tag{1}$$

In this equation, *t* ∈T , where the initial *t*<sup>0</sup> and the final *tf* are finite and fixed, the state *x t*ð Þ is an *n*� dimension vector of continuous functions defined in T and with *x tf* � � <sup>¼</sup> *<sup>x</sup> <sup>f</sup>* . The controls *ui*, *<sup>i</sup>* <sup>¼</sup> 1, … , *<sup>N</sup>*, are square (Lebesgue) integrable and the *mi*� dimension vector of continuous functions is also defined in T . Also, the disturbance *w t*ð Þ∈L*<sup>m</sup>*ð Þ <sup>T</sup> *:* The different matrices are of adequate dimension and with elements continuous in T .

i. the performance criteria are of the form

$$J\_i(u\_i, u\_{(-i)}, w) = \mathcal{K}(\mathfrak{x}(t\_f)) + \int\_{t\_0}^{t\_f} \Psi(u\_i, u\_{(-i)}, w)dt. \tag{2}$$

*Existence of Open Loop Equilibria for Disturbed Stackelberg Games DOI: http://dx.doi.org/10.5772/intechopen.92202*

where

case. To calculate the controls, we use a value function approach, appropriately guessed. Thence, we obtain sufficient conditions of existence of these controls and its representation in terms of the solution of certain Riccati equations. Furthermore,

equilibrium. In Section 3, we derive sufficient conditions for the existence of a worst-case Stackelberg equilibrium under OL information structure and investigate how are these solutions related to certain Riccati differential equations. Section 4

**Definition 2.1.** (Best reply) Let Γ*<sup>N</sup>* be a *N*-player differential game. For

� �∈ ⊗ *<sup>j</sup>*6¼*<sup>i</sup>*<sup>U</sup> *<sup>j</sup>:*

� � .

� �⊂

(1)

, where *η <sup>j</sup>* is the information structure of

� �*:* Suppose further that:

<sup>Ψ</sup> *ui*, *<sup>u</sup>*ð Þ �*<sup>i</sup>* , *<sup>w</sup>* � �*dt:* (2)

� �≤*Ji <sup>γ</sup>*<sup>1</sup> ð Þ , … , *<sup>γ</sup><sup>N</sup>*

holds for any strategy *γ<sup>i</sup>* ∈U*i*. We denote the set of all best replies by R*<sup>i</sup> γ*ð Þ �*<sup>i</sup>*

We study games of quadratic criteria, defined in a finite time horizon *t*0, *tf*

and subject to a linear dynamics, controlled players and also an unknown distur-

**Definition 2.2.** (Linear Quadratic (LQ) differential game) Let Γ*<sup>N</sup>* be an *N*�

*N*

*j*¼1

the *mi*� dimension vector of continuous functions is also defined in T . Also, the disturbance *w t*ð Þ∈L*<sup>m</sup>*ð Þ <sup>T</sup> *:* The different matrices are of adequate dimension and

� � � � <sup>þ</sup>

In this equation, *t* ∈T , where the initial *t*<sup>0</sup> and the final *tf* are finite and fixed, the state *x t*ð Þ is an *n*� dimension vector of continuous functions defined in T and

� � <sup>¼</sup> *<sup>x</sup> <sup>f</sup>* . The controls *ui*, *<sup>i</sup>* <sup>¼</sup> 1, … , *<sup>N</sup>*, are square (Lebesgue) integrable and

ð*t f t*0

i. the dynamics of the game are assumed to obey a linear differential equation

*B <sup>j</sup>*ð Þ*t u <sup>j</sup>*ðÞþ*t C t*ð Þ*w t*ð Þ,

� �

*γ*ð Þ �*<sup>i</sup>* ≔ *γ*1, … , *γ<sup>i</sup>*�1, *γ<sup>i</sup>*þ1, … , *γ<sup>N</sup>*

*Ji γ*1, … , *γ<sup>i</sup>*�1, ~*γi*, *γ<sup>i</sup>*þ1, … , *γ<sup>N</sup>*

In a future paper, we expect to present analogous conditions using an operator

In Section 2, we define the disturbed LQ game and define Stackelberg worst-case

a feedback form of the worst-case Stackelberg equilibrium is obtained.

*Systems-of-Systems Perspectives and Applications - Design, Modeling, Simulation…*

concludes the paper and outlines some directions for future work.

approach.

*i* ∈f g 1, … , *N* ,

with *x tf*

**186**

with elements continuous in T .

**2. Fundamental notions**

We start with the concept of best reply:

We say that ~*γ<sup>i</sup>* is the best reply against *γ*ð Þ �*<sup>i</sup>* if

Player-j. In this case, *η <sup>j</sup>*, *j* ¼ 1, … , *N*, is of OL type.

*x t*ð Þ¼ <sup>0</sup> *x*0*:*

i. the performance criteria are of the form

*Ji ui*, *<sup>u</sup>*ð Þ �*<sup>i</sup>* , *<sup>w</sup>* � � ¼ K *x tf*

player differential game finite time horizon *T* ¼ *t*0, *tf*

*x t* \_ðÞ¼ *A t*ð Þ*x t*ðÞþ<sup>X</sup>

bance. Hereby also consider *u <sup>j</sup>* ¼ *γ <sup>j</sup> t*, *η <sup>j</sup>*

$$\mathcal{K}(\mathfrak{x}(t\_f)) = \mathfrak{x}^T(t\_f) K\_{\mathfrak{y}} \mathfrak{x}(t\_f) \tag{3}$$

$$\begin{aligned} \Psi(u\_i, u\_{(-i)}, w) &= \mathbf{x}^T(t) Q\_i(t) \mathbf{x}(t) + w^T(t) P\_i(t) w(t) \\ &+ \sum\_{j=1}^N u\_j^T(t) R\_{ij}(t) u\_j(t), \end{aligned} \tag{4}$$

with symmetric matrices *Kif* ∈ *<sup>n</sup>*�*<sup>n</sup>* and symmetric, piecewise continuous and bounded matrix valued functions *Qi*ð Þ*<sup>t</sup>* <sup>∈</sup> *<sup>n</sup>*�*n*, *Rij*ð Þ*<sup>t</sup>* <sup>∈</sup> *mi*�*<sup>m</sup> <sup>j</sup>* and *Pi*ð Þ*<sup>t</sup>* <sup>∈</sup> *<sup>m</sup>*�*m*, *i* ¼ 1, 2, … , *N:*

We observe that no cost functional is assigned to the disturbance term because no constraints can be applied on an "unpredictable" parameter. In what follows, we consider *N* ¼ 2*:* To extend the theory to *N* > 2, since this is an hierarchical solution, we need to define the structure of the leaders and followers in the game. We can even have more than two hierachy levels. We assume that Player-2 is the leader and Player-1 is the follower.

The leader seeks a strategy *u*<sup>∗</sup> <sup>2</sup> ð Þ*t* in OL information structure and announces it before the game starts. This strategy is found knowing how the follower reacts to his choices. The follower calculates its strategy as a best reply to the strategy announced by the leader.

**Problem 2.1.** Find the control *u*<sup>∗</sup> *<sup>i</sup>* ∈U*i*, *i* ¼ 1, 2, in T for which *J* ∗ *<sup>i</sup> ui*, *u*<sup>∗</sup> ð Þ �*<sup>i</sup>* , *<sup>w</sup>* � �, *<sup>i</sup>* <sup>¼</sup> 1, 2, is minimal when subject to constraints *<sup>u</sup>*<sup>∗</sup> *<sup>i</sup>* ðÞ¼ *t γ* ∗ *<sup>i</sup> t*, *η<sup>i</sup>* ð Þ ð Þ*t* , *i* ¼ 1, 2, and (1) and considering a worst-case disturbance.

Consider U*i*, *i* ¼ 1, 2, the sets of functions such that (1) is solvable and *Ji* exists, with *ui*, *i* ¼ 1, 2, in these conditions U*i*, *i* ¼ 1, 2, W are said the sets of admissible controls and disturbance, respectively.

**Definition 2.3.** (Stackelberg equilibrium) Let Γ<sup>2</sup> be a 2-person differential game, we define the Stackelberg/worst-case equilibrium in two stages.

1.A function *w*^*i*ð Þ *u* ∈ W is called the worst-case disturbance, from the point of view of the *i*th player belonging to the set of admissible controls, if

$$J\_i^\*\left(u\_i, u\_{(-i)}^\*, \hat{w}\_i\right) \ge J\_i^\*\left(u\_i, u\_{(-i)}^\*, w\right), \quad i = 1, 2,\tag{5}$$

holds for each *w* ∈ W. There exists exactly one worst-case disturbance from the point of view of the *i*th player according to every set of controls.

2.We say that the controls *u*<sup>∗</sup> <sup>1</sup> , *u*<sup>∗</sup> 2 � � form a worst-case Stackelberg equilibrium if

i. The leader chooses *u*<sup>∗</sup> <sup>2</sup> such that

$$\max\_{\gamma\_1 \in \mathcal{R}\_1(u\_2^\*)} J\_2(\gamma\_1, u\_2^\*, \hat{w}\_2) \le \max\_{\gamma\_1 \in \mathcal{R}\_1(u\_2)} J\_2(\gamma\_1, u\_2, \hat{w}\_2)$$

for all *u*<sup>2</sup> ∈ U2*:*

ii. The follower then chooses *u*<sup>∗</sup> <sup>1</sup> such that

$$\mathcal{R}\_1(\boldsymbol{\mu}\_2) = \{\boldsymbol{\mu}\_1 | J\_1(\boldsymbol{\mu}\_1, \boldsymbol{\mu}\_2, \hat{\boldsymbol{w}}\_1) \le \boldsymbol{I}\_1(\boldsymbol{\gamma}\_1, \boldsymbol{\mu}\_2, \hat{\boldsymbol{w}}\_1)\}\,.$$

*u*∗

*Existence of Open Loop Equilibria for Disturbed Stackelberg Games*

*w*<sup>∗</sup>

where the maximum disturbance,

*DOI: http://dx.doi.org/10.5772/intechopen.92202*

solution of

<sup>1</sup> ¼ �*R*�<sup>1</sup>

<sup>1</sup> ¼ �*P*�<sup>1</sup>

*<sup>x</sup>*\_ <sup>¼</sup> ½ � *<sup>A</sup>* � ð Þ *<sup>S</sup>*<sup>1</sup> <sup>þ</sup> *<sup>T</sup>*<sup>1</sup> *<sup>E</sup>*<sup>1</sup> *<sup>x</sup>* � <sup>1</sup>

*<sup>x</sup>*\_ <sup>¼</sup> ½ � *<sup>A</sup>* � ð Þ *<sup>S</sup>*<sup>1</sup> <sup>þ</sup> *<sup>T</sup>*<sup>1</sup> *<sup>E</sup>*<sup>1</sup> *<sup>x</sup>* � <sup>1</sup>

*K*2*<sup>f</sup>* 0

1 A,

� � exist in <sup>T</sup> , where *<sup>E</sup>*<sup>2</sup> <sup>∈</sup> <sup>2</sup>*n*�2*<sup>n</sup>:* Also *<sup>B</sup>* <sup>≔</sup> *<sup>B</sup>*<sup>2</sup>

*<sup>e</sup>*\_<sup>2</sup> ¼ �*H<sup>T</sup>* <sup>þ</sup> *<sup>E</sup>*2ð Þ *<sup>S</sup>* <sup>þ</sup> *<sup>T</sup>* � �*e*2, *<sup>e</sup>*<sup>2</sup> *tf*

0 @

0 0

<sup>1</sup> , *S*<sup>2</sup> ≔ *B*2*R*�<sup>1</sup>

and *d*<sup>2</sup> ∈ in T by the following initial and terminal value problems:

The corresponding minimal costs then are

minimal control and themaximal disturbance.

*E*<sup>2</sup> *tf* � � <sup>¼</sup>

<sup>11</sup> *R*21*R*�<sup>1</sup> <sup>11</sup> *BT*

� �, *<sup>Q</sup>* <sup>≔</sup> *<sup>Q</sup>*<sup>2</sup> <sup>0</sup>

the trajectory (1) to obtain:

defined by (10).

with *S*<sup>21</sup> ≔ *B*1*R*�<sup>1</sup>

�*Q*<sup>1</sup> *<sup>E</sup>*1*T*<sup>1</sup> � *<sup>A</sup><sup>T</sup>*

*<sup>H</sup>* <sup>≔</sup> *<sup>A</sup>* �*S*<sup>1</sup>

*<sup>T</sup>* <sup>≔</sup> *<sup>T</sup>*<sup>2</sup> *<sup>T</sup>*2*E*<sup>1</sup> *E*1*T*<sup>2</sup> *E*1*T*2*E*<sup>1</sup>

**189**

solution of

*<sup>J</sup>*<sup>10</sup> <sup>¼</sup> <sup>2</sup>*J*1ð Þ¼ *<sup>u</sup>*1, *<sup>u</sup>*<sup>2</sup> *<sup>x</sup><sup>T</sup>*

<sup>11</sup> *B<sup>T</sup>*

was considered. *E*<sup>1</sup> and *e*<sup>1</sup> are the solutions of (6)–(7) and *x* is then the

<sup>1</sup> *E*1*x* þ

<sup>1</sup> *<sup>C</sup><sup>T</sup> <sup>E</sup>*1*<sup>x</sup>* <sup>þ</sup>

2

<sup>0</sup>*E*1ð Þ *<sup>t</sup>*<sup>0</sup> *<sup>x</sup>*<sup>0</sup> <sup>þ</sup> *<sup>x</sup><sup>T</sup>*

2

The cost functional minimal value is obtained when we substitute in (9) the

**Theorem 3.3.** Let the solution of the Riccati differential Eq. (6) and the

*<sup>E</sup>*\_ <sup>2</sup> ¼ �*E*2*<sup>H</sup>* � *<sup>H</sup>TE*<sup>2</sup> � *<sup>Q</sup>* <sup>þ</sup> *<sup>E</sup>*2ð Þ *<sup>S</sup>* <sup>þ</sup> *<sup>T</sup> <sup>E</sup>*2,

<sup>22</sup> *BT*

For any given control *u*<sup>2</sup> of the leader, define functions *e*<sup>2</sup> ∈ <sup>3</sup>*<sup>n</sup>*, *v*1, *vw*, *x*∈ *<sup>n</sup>*

<sup>0</sup> *<sup>S</sup>*<sup>21</sup> � �, *<sup>S</sup>* <sup>≔</sup> *<sup>S</sup>*<sup>2</sup> <sup>0</sup>

<sup>2</sup> and *T*<sup>2</sup> ≔ *CP*�<sup>1</sup>

<sup>2</sup> *CT:* Also

0*<sup>m</sup>*1�*<sup>n</sup>* � �*:*

� � <sup>¼</sup> <sup>0</sup> (16)

0 0 � � and

Notice that *J*10ð Þ *u*<sup>2</sup> is not depending on *u*1*:* This, as a matter of fact, is only true if we consider OL information structure, since otherwise *u*<sup>2</sup> would depend on the trajectory *x* and hence, via (1), also on *u*1*:* In OL Stackelberg games, the leader tries next to find an optimal OL control *u*<sup>2</sup> that minimises *J*2ð Þ *u*1ð Þ *u*<sup>2</sup> , *u*<sup>2</sup> while *u*1ð Þ *u*<sup>2</sup> is

**Proof:** We have that the unique OL response of the follower to the leader's announced strategy *u*<sup>2</sup> (10) under worst-case disturbance (11), that we substitute in

1 2 *e*1

1 2 *e*1

� �, (10)

� �, (11)

ð Þ *S*<sup>1</sup> þ *T*<sup>1</sup> *e*<sup>1</sup> þ *B*2*u*2, (12)

<sup>0</sup> *e*1ð Þþ *t*<sup>0</sup> *d*1ð Þ *t*<sup>0</sup> *:* (14)

(15)

*x t*ð Þ¼ <sup>0</sup> *x*0*:* (13)

ð Þ *S*<sup>1</sup> þ *T*<sup>1</sup> *e*<sup>1</sup> þ *B*2*u*2*:*

To guarantee the uniqueness of OL Stackelberg solutions, matrices are assumed to satisfy *Kif* ≥ 0, *Qi* ≥ 0, *Rij* >0, *i* 6¼ *j* and *Rii* ≥ 0, *i*, *j* ¼ 1, … , *N* in T Simaan and Cruz [14].

In what follows, we drop the dependence of the parameters in *t* to reduce the length of the formulas.

#### **3. Sufficient conditions for the existence of OL Stackelberg equilibria**

In this section, we withdraw sufficient conditions for the existence of the worstcase Stackelberg equilibrium, using a value function approach.

A disturbed differential LQ game as defined in Definition 2.2 is said *playable* if there exists a unique Stackelberg worst-case equilibrium.

**Theorem 3.1.** Let the solution of the Riccati differential equation

$$\begin{aligned} \dot{E}\_1 &= -E\_1 A - A^T E\_1 - Q\_1 + E\_1 (S\_1 + T\_1) E\_1, \\ E\_1(t\_f) &= K\_{\mathcal{Y}}, \end{aligned} \tag{6}$$

with *<sup>S</sup>*<sup>1</sup> <sup>¼</sup> *<sup>B</sup>*1*R*�<sup>1</sup> <sup>11</sup> *BT* <sup>1</sup> and *<sup>T</sup>*<sup>1</sup> <sup>¼</sup> *CP*�<sup>1</sup> <sup>1</sup> *<sup>C</sup><sup>T</sup>* exist on <sup>T</sup> .

For any given admissible OL control of the leader, *u*2, define *e*<sup>1</sup> ∈ *<sup>n</sup>*, *d*<sup>1</sup> ∈ by

$$\begin{aligned} \dot{e}\_1 &= E\_1(\mathbb{S}\_1 + T\_1)e\_1 - 2E\_1B\_2u\_2 - A^T e\_1^T, \\ e\_1(t\_f) &= 0 \\ &\mathbf{1} \end{aligned} \tag{7}$$

$$\begin{aligned} \dot{d}\_1 &= -(u\_2^T R\_{12} + e\_1^T B\_2) u\_2 + \frac{1}{4} e\_1^T (\mathbb{S}\_1 + T\_1) e\_1, \\ d\_1(t\_f) &= 0. \end{aligned} \tag{8}$$

Then, the following identity holds:

$$\begin{split} \mathcal{Q}I\_1(u\_1, u\_2) &= \boldsymbol{\varkappa}\_0^T \boldsymbol{E}\_1(t\_0) \boldsymbol{\varkappa}\_0 + \boldsymbol{\varkappa}\_0^T \boldsymbol{e}\_1(t\_0) + \boldsymbol{d}\_1(t\_0) \\ &+ \int\_{t\_0}^{t\_f} \|\boldsymbol{\varkappa}\_1(t)\|\_{R\_{11}}^2 dt + \int\_{t\_0}^{t\_f} \|\boldsymbol{z}(t)\|\_{P\_1}^2 dt, \end{split} \tag{9}$$

where ∥*z*1∥<sup>2</sup> *<sup>R</sup>*<sup>11</sup> ¼ *z*1*R*11*z*<sup>1</sup> with

$$z\_1 = u\_1 + R\_{11}^{-1} B\_1^T \left( E\_1 \varkappa + \frac{1}{2} e\_1 \right),$$

and ∥*z*∥<sup>2</sup> *<sup>P</sup>*<sup>1</sup> ¼ *zP*1*z* with

$$z = w + P\_1^{-1} \mathcal{C}^T \left( E\_1 \mathfrak{x} + \frac{1}{2} e\_1 \right),$$

and *x* a solution of (1).

**Proof:** The proof is similar to the analogous result for the non-disturbed case Freiling et al. [13].

**Theorem 3.2.** Let the solution *E*<sup>1</sup> of (6) exist on T . Then the unique response of the follower to the leader's OL strategy *u*2ð Þ*t* is given by:

*Existence of Open Loop Equilibria for Disturbed Stackelberg Games DOI: http://dx.doi.org/10.5772/intechopen.92202*

$$
\mu\_1^\* = -R\_{11}^{-1} B\_1^T \left( E\_1 \mathfrak{x} + \frac{1}{2} \mathfrak{e}\_1 \right),
\tag{10}
$$

where the maximum disturbance,

R1ð Þ¼ *u*<sup>2</sup> *u*1j*J*1ð Þ *u*1, *u*2, *w*^<sup>1</sup> ≤*J*<sup>1</sup> *γ*<sup>1</sup> f g ð Þ , *u*2, *w*^<sup>1</sup> *:*

*Systems-of-Systems Perspectives and Applications - Design, Modeling, Simulation…*

Cruz [14].

length of the formulas.

with *<sup>S</sup>*<sup>1</sup> <sup>¼</sup> *<sup>B</sup>*1*R*�<sup>1</sup>

where ∥*z*1∥<sup>2</sup>

and ∥*z*∥<sup>2</sup>

Freiling et al. [13].

**188**

To guarantee the uniqueness of OL Stackelberg solutions, matrices are assumed to satisfy *Kif* ≥ 0, *Qi* ≥ 0, *Rij* >0, *i* 6¼ *j* and *Rii* ≥ 0, *i*, *j* ¼ 1, … , *N* in T Simaan and

In what follows, we drop the dependence of the parameters in *t* to reduce the

**3. Sufficient conditions for the existence of OL Stackelberg equilibria**

case Stackelberg equilibrium, using a value function approach.

**Theorem 3.1.** Let the solution of the Riccati differential equation

<sup>2</sup> *R*<sup>12</sup> þ *e*

there exists a unique Stackelberg worst-case equilibrium.

<sup>1</sup> and *<sup>T</sup>*<sup>1</sup> <sup>¼</sup> *CP*�<sup>1</sup>

*E*<sup>1</sup> *tf*

*e*<sup>1</sup> *tf* � � <sup>¼</sup> <sup>0</sup>

> \_ *<sup>d</sup>*<sup>1</sup> ¼ � *uT*

*<sup>R</sup>*<sup>11</sup> ¼ *z*1*R*11*z*<sup>1</sup> with

*<sup>P</sup>*<sup>1</sup> ¼ *zP*1*z* with

and *x* a solution of (1).

<sup>2</sup>*J*1ð Þ¼ *<sup>u</sup>*1, *<sup>u</sup>*<sup>2</sup> *<sup>x</sup><sup>T</sup>*

þ ð*t f t*0

*<sup>z</sup>*<sup>1</sup> <sup>¼</sup> *<sup>u</sup>*<sup>1</sup> <sup>þ</sup> *<sup>R</sup>*�<sup>1</sup>

*<sup>z</sup>* <sup>¼</sup> *<sup>w</sup>* <sup>þ</sup> *<sup>P</sup>*�<sup>1</sup>

the follower to the leader's OL strategy *u*2ð Þ*t* is given by:

*d*<sup>1</sup> *tf* � � <sup>¼</sup> <sup>0</sup>*:*

Then, the following identity holds:

<sup>11</sup> *BT*

In this section, we withdraw sufficient conditions for the existence of the worst-

A disturbed differential LQ game as defined in Definition 2.2 is said *playable* if

*<sup>E</sup>*\_ <sup>1</sup> ¼ �*E*1*<sup>A</sup>* � *<sup>A</sup>TE*<sup>1</sup> � *<sup>Q</sup>*<sup>1</sup> <sup>þ</sup> *<sup>E</sup>*1ð Þ *<sup>S</sup>*<sup>1</sup> <sup>þ</sup> *<sup>T</sup>*<sup>1</sup> *<sup>E</sup>*1,

<sup>1</sup> *<sup>C</sup><sup>T</sup>* exist on <sup>T</sup> . For any given admissible OL control of the leader, *u*2, define *e*<sup>1</sup> ∈ *<sup>n</sup>*, *d*<sup>1</sup> ∈ by

*<sup>e</sup>*\_<sup>1</sup> <sup>¼</sup> *<sup>E</sup>*1ð Þ *<sup>S</sup>*<sup>1</sup> <sup>þ</sup> *<sup>T</sup>*<sup>1</sup> *<sup>e</sup>*<sup>1</sup> � <sup>2</sup>*E*1*B*2*u*<sup>2</sup> � *ATe*

*T* <sup>1</sup> *B*<sup>2</sup> � �*u*<sup>2</sup> <sup>þ</sup>

<sup>0</sup>*E*1ð Þ *<sup>t</sup>*<sup>0</sup> *<sup>x</sup>*<sup>0</sup> <sup>þ</sup> *<sup>x</sup><sup>T</sup>*

*<sup>R</sup>*11*dt* þ

<sup>1</sup> *E*1*x* þ

<sup>1</sup> *CT <sup>E</sup>*1*<sup>x</sup>* <sup>þ</sup>

**Proof:** The proof is similar to the analogous result for the non-disturbed case

**Theorem 3.2.** Let the solution *E*<sup>1</sup> of (6) exist on T . Then the unique response of

<sup>∥</sup>*z*1ð Þ*<sup>t</sup>* <sup>∥</sup><sup>2</sup>

<sup>11</sup> *BT*

� � <sup>¼</sup> *<sup>K</sup>*1*<sup>f</sup>* , (6)

1 4 *e T* *T* 1 ,

<sup>1</sup> ð Þ *S*<sup>1</sup> þ *T*<sup>1</sup> *e*1,

<sup>0</sup> *e*1ð Þþ *t*<sup>0</sup> *d*1ð Þ *t*<sup>0</sup>

<sup>∥</sup>*z t*ð Þ∥<sup>2</sup> *P*1

ð*t f t*0

1 2 *e*1 � �

1 2 *e*1 � � (7)

(8)

*dt*, (9)

$$w\_1^\* = -P\_1^{-1}C^T \left(E\_1 \mathfrak{x} + \frac{1}{2}e\_1\right),\tag{11}$$

was considered. *E*<sup>1</sup> and *e*<sup>1</sup> are the solutions of (6)–(7) and *x* is then the solution of

$$
\dot{\mathbf{x}} = [A - (\mathbf{S}\_1 + T\_1)E\_1]\mathbf{x} - \frac{\mathbf{1}}{2}(\mathbf{S}\_1 + T\_1)e\_1 + B\_2u\_2,\tag{12}
$$

$$
\mathfrak{x}(t\_0) = \mathfrak{x}\_0. \tag{13}
$$

The corresponding minimal costs then are

$$J\_{10} = \mathcal{Q} I\_1(u\_1, u\_2) = \varkappa\_0^T E\_1(t\_0) \varkappa\_0 + \varkappa\_0^T e\_1(t\_0) + d\_1(t\_0). \tag{14}$$

**Proof:** We have that the unique OL response of the follower to the leader's announced strategy *u*<sup>2</sup> (10) under worst-case disturbance (11), that we substitute in the trajectory (1) to obtain:

$$\dot{\mathbf{x}} = [\mathbf{A} - (\mathbf{S}\_1 + T\_1)E\_1]\mathbf{x} - \frac{\mathbf{1}}{2}(\mathbf{S}\_1 + T\_1)\mathbf{e}\_1 + B\_2\boldsymbol{\mu}\_2\mathbf{I}$$

The cost functional minimal value is obtained when we substitute in (9) the minimal control and themaximal disturbance.

Notice that *J*10ð Þ *u*<sup>2</sup> is not depending on *u*1*:* This, as a matter of fact, is only true if we consider OL information structure, since otherwise *u*<sup>2</sup> would depend on the trajectory *x* and hence, via (1), also on *u*1*:* In OL Stackelberg games, the leader tries next to find an optimal OL control *u*<sup>2</sup> that minimises *J*2ð Þ *u*1ð Þ *u*<sup>2</sup> , *u*<sup>2</sup> while *u*1ð Þ *u*<sup>2</sup> is defined by (10).

**Theorem 3.3.** Let the solution of the Riccati differential Eq. (6) and the solution of

$$\begin{aligned} \dot{E}\_2 &= -E\_2H - H^T E\_2 - Q + E\_2(S+T)E\_2, \\ E\_2(t\_f) &= \begin{pmatrix} K\_{2'} & 0 \\ & 0 \\ 0 & 0 \end{pmatrix}, \end{aligned} \tag{15}$$

$$\begin{aligned} \text{with } \mathbf{S}\_{21} &:= B\_1 R\_{11}^{-1} R\_{21} R\_{11}^{-1} B\_1^T, \mathbf{S}\_2 := B\_2 R\_{22}^{-1} B\_2^T \text{ and } \mathbf{T}\_2 := C P\_2^{-1} C^T. \text{ Also} \\ H &:= \begin{pmatrix} A & -\mathbf{S}\_1 & \\ -\mathbf{Q}\_1 & E\_1 T\_1 - A^T \end{pmatrix}, \; Q := \begin{pmatrix} \mathbf{Q}\_2 & \mathbf{0} \\ \mathbf{0} & S\_{21} \end{pmatrix}, \; S := \begin{pmatrix} \mathbf{S}\_2 & \mathbf{0} \\ \mathbf{0} & \mathbf{0} \end{pmatrix} \text{ and } \\ T &:= \begin{pmatrix} T\_2 & T\_2 E\_1 \\ E\_1 T\_2 & E\_1 T\_2 E\_1 \end{pmatrix} \text{ exist in } \mathcal{T}, \; \text{where } E\_2 \in \mathbb{R}^{2n \times 2n}. \text{ Also } B := \begin{pmatrix} B\_2 \\ \mathbf{0}\_{m\_1 \times n} \end{pmatrix}. \\ \text{Then } \mathcal{T}\_2 &:= \begin{pmatrix} \mathbf{0}\_{m\_1 \times n} & \mathbf{0}\_{m\_2 \times n} \end{pmatrix} \text{ and } \; \mathcal{T}\_2 &:= \begin{pmatrix} \mathbf{0}\_{m\_2 \times n} & \mathbf{0}\_{m\_1 \times n} \end{pmatrix}. \end{aligned}$$

For any given control *u*<sup>2</sup> of the leader, define functions *e*<sup>2</sup> ∈ <sup>3</sup>*<sup>n</sup>*, *v*1, *vw*, *x*∈ *<sup>n</sup>* and *d*<sup>2</sup> ∈ in T by the following initial and terminal value problems:

$$
\dot{e}\_2 = \left(-H^T + E\_2(\mathbb{S} + T)\right)e\_2, \quad e\_2(t\_f) = \mathbf{0} \tag{16}
$$

*Systems-of-Systems Perspectives and Applications - Design, Modeling, Simulation…*

$$\dot{d}\_2 = \frac{1}{4} e\_2^T (\mathbb{S} + T) e\_2, \quad d\_2(t\_f) = \mathbf{0} \tag{17}$$

for some mappings *<sup>E</sup>*<sup>2</sup> : T ! <sup>2</sup>*n*�2*n*,*e*<sup>2</sup> : T ! <sup>2</sup>*n*, and *<sup>d</sup>*<sup>2</sup> : T ! <sup>2</sup>

*T* <sup>2</sup> *y* þ *d*<sup>2</sup>

*TE*2*<sup>y</sup>* <sup>þ</sup> *yTE*\_ <sup>2</sup>*<sup>y</sup>* <sup>þ</sup> *<sup>y</sup>TE*2*y*\_ <sup>þ</sup> *<sup>e</sup>*\_

� �*E*2*y*

<sup>1</sup> *<sup>R</sup>*21*u*<sup>1</sup> <sup>þ</sup> *<sup>u</sup><sup>T</sup>*

<sup>2</sup> *BTwTCT* 1

<sup>2</sup> <sup>ð</sup>*Hy* <sup>þ</sup> *Bu*<sup>2</sup> <sup>þ</sup> *<sup>C</sup>*1*w*Þ þ \_

*<sup>y</sup>* <sup>þ</sup> *uT*

<sup>þ</sup>*yTE*\_ <sup>2</sup>*<sup>y</sup>* <sup>þ</sup> *<sup>y</sup>TE*2ð Þ *Hy* <sup>þ</sup> *Bu*<sup>2</sup> <sup>þ</sup> *<sup>C</sup>*1*<sup>w</sup>*

*T* <sup>2</sup> *y* þ *e T* <sup>2</sup> *<sup>y</sup>*\_ <sup>þ</sup> \_ *d*2

<sup>2</sup> *<sup>R</sup>*22*u*<sup>2</sup> <sup>þ</sup> *<sup>w</sup>TP*2*<sup>w</sup>* � *<sup>ψ</sup>*<sup>2</sup>

*d*2

<sup>2</sup> *<sup>R</sup>*22*u*<sup>2</sup> <sup>þ</sup> *<sup>w</sup>TP*2*<sup>w</sup>* � *<sup>ψ</sup>*<sup>2</sup>

1 2 *e T* 2 *B*

> 1 2 *e T* <sup>2</sup> *C*<sup>1</sup>

<sup>2</sup> *R*22*y*<sup>2</sup>

� �*<sup>w</sup>*

� �*u*<sup>2</sup>

� �

We consider (21), where we substitute (20):

<sup>þ</sup>*xTQ*2*<sup>x</sup>* <sup>þ</sup> *uT*

<sup>¼</sup> *<sup>y</sup>THT* <sup>þ</sup> *uT*

<sup>þ</sup>*y<sup>T</sup> <sup>Q</sup>*<sup>2</sup> <sup>0</sup> <sup>0</sup> *<sup>S</sup>*21 !

þ *u*<sup>2</sup> � *y*<sup>2</sup> � �*<sup>T</sup>*

<sup>2</sup> *<sup>R</sup>*22*u*<sup>2</sup> <sup>þ</sup> *uT*

þð Þ *<sup>w</sup>* � *<sup>α</sup> TP*2ð Þ *<sup>w</sup>* � *<sup>α</sup>*

<sup>2</sup> *<sup>B</sup>TE*2*<sup>y</sup>* <sup>þ</sup> *BTe*<sup>2</sup>

þ *u*<sup>2</sup> � *y*<sup>2</sup> � �*<sup>T</sup>*

<sup>þ</sup> *<sup>y</sup>TE*2*<sup>B</sup>* <sup>þ</sup>

<sup>þ</sup>*w<sup>T</sup> CT*

<sup>þ</sup> *<sup>y</sup>TE*2*C*<sup>1</sup> <sup>þ</sup>

<sup>þ</sup>*uT*

*e*\_ *T* <sup>2</sup> þ *e T* <sup>2</sup> *<sup>H</sup>* � �*<sup>y</sup>* <sup>þ</sup> \_

<sup>1</sup> *<sup>E</sup>*2*<sup>y</sup>* <sup>þ</sup> *CT*

<sup>þ</sup>*wTP*2*<sup>α</sup>* <sup>þ</sup> *<sup>α</sup>TP*2*<sup>w</sup>* � *<sup>α</sup>TP*2*<sup>α</sup>*

<sup>þ</sup>*y<sup>T</sup>*

<sup>þ</sup>*u<sup>T</sup>*

*e*\_ *T* <sup>2</sup> þ *e T* <sup>2</sup> *<sup>H</sup>* � �*<sup>y</sup>* <sup>þ</sup> \_

and furthermore

**191**

<sup>þ</sup>*w<sup>T</sup> <sup>C</sup><sup>T</sup>*


<sup>¼</sup> *<sup>y</sup><sup>T</sup> <sup>H</sup>TE*<sup>2</sup> <sup>þ</sup> *<sup>E</sup>*\_ <sup>2</sup> <sup>þ</sup> *<sup>E</sup>*2*<sup>H</sup>* <sup>þ</sup> *<sup>Q</sup>* � �*<sup>y</sup>*

*R*<sup>22</sup> *u*<sup>2</sup> � *y*<sup>2</sup> � �

<sup>2</sup> *<sup>R</sup>*22*y*<sup>2</sup> � *<sup>y</sup><sup>T</sup>*

� � <sup>þ</sup> *yTE*2*<sup>B</sup>* <sup>þ</sup>

<sup>1</sup> *e*<sup>2</sup> � � <sup>þ</sup> *yTE*2*C*<sup>1</sup> <sup>þ</sup>

*d*<sup>2</sup> � *ψ*<sup>2</sup>

<sup>¼</sup> *<sup>y</sup><sup>T</sup> <sup>H</sup>TE*<sup>2</sup> <sup>þ</sup> *<sup>E</sup>*\_ <sup>2</sup> <sup>þ</sup> *<sup>E</sup>*2*<sup>H</sup>* <sup>þ</sup> *<sup>Q</sup>* � �*<sup>y</sup>* � *<sup>ψ</sup>*<sup>2</sup>

*R*<sup>22</sup> *u*<sup>2</sup> � *y*<sup>2</sup> � � � *yT*

� �

2

� �

*e*<sup>2</sup> þ *R*22*y*<sup>2</sup>

*e*<sup>2</sup> þ *P*2*α*

<sup>2</sup> *<sup>C</sup>*<sup>1</sup> � *<sup>α</sup>TP*<sup>2</sup>

þð Þ *<sup>w</sup>* � *<sup>α</sup> TP*2ð Þ� *<sup>w</sup>* � *<sup>α</sup> <sup>α</sup>TP*2*<sup>α</sup>*

<sup>2</sup> *BTE*2*<sup>y</sup>* <sup>þ</sup> *BT* <sup>1</sup>

1 2 *e T* <sup>2</sup> *By<sup>T</sup>* <sup>2</sup> *<sup>R</sup>*<sup>22</sup> � �*u*<sup>2</sup>

> <sup>1</sup> *<sup>E</sup>*2*<sup>y</sup>* <sup>þ</sup> *<sup>C</sup><sup>T</sup>* 1 1 2

> > 1 2 *e T*

� �*<sup>w</sup>*

*d*<sup>2</sup> � *ψ*<sup>2</sup>

<sup>2</sup> *R*22*y*<sup>2</sup>

*dt <sup>y</sup>TE*2*<sup>y</sup>* <sup>þ</sup> *<sup>e</sup>*

*Existence of Open Loop Equilibria for Disturbed Stackelberg Games*

symmetric for each *t* ∈*T:*

*dV*~ <sup>2</sup> *dt* <sup>¼</sup> *<sup>d</sup>*

¼ *y*\_

*DOI: http://dx.doi.org/10.5772/intechopen.92202*

þ*e*\_ *T* <sup>2</sup> *y* þ *e T*

Now we associate certain terms

, where *E*<sup>2</sup> is

$$\begin{aligned} \dot{\upsilon}\_1 = -Q\_1 \mathfrak{x} + \left( E\_1 T\_1 - A^T \right) \upsilon\_1 + E\_1 \text{G} \upsilon, \\ \upsilon\_1(t\_0) = \upsilon\_{10} \end{aligned} \tag{18}$$

$$
\dot{\mathbf{x}} = A\mathbf{x} - \mathbf{S}\_1\mathbf{v}\_1 + B\_2\mathbf{u}\_2 + \mathbf{C}\mathbf{w}, \quad \mathbf{x}(t\_0) = \mathbf{x}\_0,\tag{19}
$$

with *<sup>v</sup>*<sup>1</sup> <sup>≔</sup> *<sup>E</sup>*<sup>1</sup> <sup>þ</sup> <sup>1</sup> 2 *e*1 � �*:* Then, we obtain

$$\begin{aligned} \boldsymbol{u}\_1^\* &= -\boldsymbol{R}\_{11}^{-1} \boldsymbol{B}\_1^T \boldsymbol{v}\_1, \\ \boldsymbol{w}\_1^\* &= -\boldsymbol{P}\_1^{-1} \boldsymbol{C}^T \boldsymbol{v}\_1, \end{aligned}$$

and the following identity

$$\begin{aligned} 2J\_2(\boldsymbol{u}\_1^\*, \boldsymbol{u}\_2^\*, \boldsymbol{w}\_2^\*) &= \begin{pmatrix} \boldsymbol{\kappa}\_0^T \ \boldsymbol{v}\_{10} \end{pmatrix} E\_2(t\_0) \begin{pmatrix} \boldsymbol{\kappa}\_0 \\ \boldsymbol{v}\_{10} \end{pmatrix} \\ &+ \begin{pmatrix} \boldsymbol{\kappa}\_0^T & \boldsymbol{v}\_{10} \end{pmatrix} \boldsymbol{e}\_2(t\_0) + \boldsymbol{d}\_2(t\_0) \\ &+ \int\_{t\_0}^{t\_f} \|\boldsymbol{\kappa}\_2\|\_{R\_{22}}^2 dt + \int\_{t\_0}^{t\_f} \|\boldsymbol{z}\|\_{P\_2}^2 dt, \end{aligned}$$

where *<sup>y</sup>* <sup>¼</sup> *<sup>x</sup> v*1 � �, <sup>∥</sup>*z*2∥<sup>2</sup> *<sup>R</sup>*<sup>22</sup> ¼ *z*2*R*22*z*<sup>2</sup> and

$$z\_2 = \mathfrak{u}\_2 + \left(R\_{22}^{-1}B\_2^T \quad \mathbf{0}\_{m\_1 \times n}\right)\left(E\_2\mathfrak{y} + \frac{1}{2}e\_2\right),$$

and 0*mi*�*<sup>n</sup>*, *<sup>i</sup>* <sup>¼</sup> 1, 2 the *mi* � *<sup>n</sup>* dimensional zero matrix and <sup>∥</sup>*z*∥<sup>2</sup> *<sup>P</sup>*<sup>2</sup> ¼ *zP*2*z* and

$$z = w\_2 + P\_2^{-1} \mathcal{C}\_1^T \left( E\_2 y + \frac{1}{2} e\_2 \right).$$

**Proof:** Consider (10): *u*<sup>∗</sup> <sup>1</sup> ¼ �*R*�<sup>1</sup> <sup>11</sup> *BT* <sup>1</sup> *E*1*x* þ 1 2 *e*1 � � |fflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflffl} <sup>≔</sup> *<sup>v</sup>*<sup>1</sup> *:* Then, differentiate *v*<sup>1</sup> and

substitute the derivatives into the obtained expression using (6), (7) and (8). Also, the optimal control *u*<sup>∗</sup> <sup>1</sup> and disturbance *w*<sup>∗</sup> <sup>1</sup> in (11). Hence:

$$\begin{aligned} \dot{v}\_1 &= -Q\_1 \mathfrak{x} - A^T v\_1, \\ \dot{\mathfrak{x}} &= A \mathfrak{x} - \mathfrak{S}\_1 v\_1 + B\_2 \mathfrak{u}\_2 + \mathcal{C} \mathfrak{w}. \end{aligned}$$

Hence defining *<sup>H</sup>* <sup>≔</sup> *<sup>A</sup>* �*S*<sup>1</sup> �*Q*<sup>1</sup> *<sup>E</sup>*1*T*<sup>1</sup> � *<sup>A</sup><sup>T</sup>* � �, *<sup>B</sup>* <sup>≔</sup> *<sup>B</sup>*<sup>2</sup> *On*�*m*<sup>2</sup> � � and *<sup>C</sup>*<sup>1</sup> <sup>≔</sup> *<sup>I</sup> E*1 � �*C:* We define *<sup>y</sup>* <sup>≔</sup> *<sup>x</sup> v*1 � � to write these two equations as: (??) as:

$$
\dot{y} = H\mathbf{y} + Bu\_2 + C\_1 w \tag{20}
$$

Next, we consider the following value function

$$
\tilde{V}\_2(t) = V\_2(t, y(t)) = y^T E\_2 y + e\_2^T y + d\_2 \tag{21}
$$

*Existence of Open Loop Equilibria for Disturbed Stackelberg Games DOI: http://dx.doi.org/10.5772/intechopen.92202*

for some mappings *<sup>E</sup>*<sup>2</sup> : T ! <sup>2</sup>*n*�2*n*,*e*<sup>2</sup> : T ! <sup>2</sup>*n*, and *<sup>d</sup>*<sup>2</sup> : T ! <sup>2</sup> , where *E*<sup>2</sup> is symmetric for each *t* ∈*T:*

We consider (21), where we substitute (20):

\_ *<sup>d</sup>*<sup>2</sup> <sup>¼</sup> <sup>1</sup> 4 *e T*

with *<sup>v</sup>*<sup>1</sup> <sup>≔</sup> *<sup>E</sup>*<sup>1</sup> <sup>þ</sup> <sup>1</sup>

Then, we obtain

where *<sup>y</sup>* <sup>¼</sup> *<sup>x</sup>*

2 *e*1 � �*:*

> 2*J*<sup>2</sup> *u*<sup>∗</sup> <sup>1</sup> *; u*<sup>∗</sup> <sup>2</sup> *; w*<sup>∗</sup> 2 � � <sup>¼</sup> *xT*

, ∥*z*2∥<sup>2</sup>

and the following identity

*v*1 � �

**Proof:** Consider (10): *u*<sup>∗</sup>

Hence defining *<sup>H</sup>* <sup>≔</sup> *<sup>A</sup>* �*S*<sup>1</sup>

Next, we consider the following value function

the optimal control *u*<sup>∗</sup>

define *<sup>y</sup>* <sup>≔</sup> *<sup>x</sup>*

**190**

*v*1 � � <sup>2</sup> ð Þ *S* þ *T e*2, *d*<sup>2</sup> *tf*

*<sup>v</sup>*\_ <sup>1</sup> ¼ �*Q*1*<sup>x</sup>* <sup>þ</sup> *<sup>E</sup>*1*T*<sup>1</sup> � *<sup>A</sup><sup>T</sup>* � �*v*<sup>1</sup> <sup>þ</sup> *<sup>E</sup>*1*Cw*,

*u*∗

*Systems-of-Systems Perspectives and Applications - Design, Modeling, Simulation…*

*w*<sup>∗</sup>

<sup>1</sup> ¼ �*R*�<sup>1</sup>

<sup>1</sup> ¼ �*P*�<sup>1</sup>

þ *xT*

<sup>þ</sup> <sup>Ð</sup>*tf <sup>t</sup>*<sup>0</sup> <sup>∥</sup>*z*2∥<sup>2</sup>

*<sup>R</sup>*<sup>22</sup> ¼ *z*2*R*22*z*<sup>2</sup> and

and 0*mi*�*<sup>n</sup>*, *<sup>i</sup>* <sup>¼</sup> 1, 2 the *mi* � *<sup>n</sup>* dimensional zero matrix and <sup>∥</sup>*z*∥<sup>2</sup>

*<sup>z</sup>* <sup>¼</sup> *<sup>w</sup>*<sup>2</sup> <sup>þ</sup> *<sup>P</sup>*�<sup>1</sup>

<sup>1</sup> ¼ �*R*�<sup>1</sup>

<sup>1</sup> and disturbance *w*<sup>∗</sup>

<sup>22</sup> *BT*

<sup>11</sup> *BT*

*<sup>v</sup>*\_ <sup>1</sup> ¼ �*Q*1*<sup>x</sup>* � *ATv*1,

�*Q*<sup>1</sup> *<sup>E</sup>*1*T*<sup>1</sup> � *<sup>A</sup><sup>T</sup>* � �

to write these two equations as: (??) as:

*<sup>V</sup>*<sup>~</sup> <sup>2</sup>ðÞ¼ *<sup>t</sup> <sup>V</sup>*2ð Þ¼ *<sup>t</sup>*, *y t*ð Þ *<sup>y</sup>TE*2*<sup>y</sup>* <sup>þ</sup> *<sup>e</sup>*

*<sup>z</sup>*<sup>2</sup> <sup>¼</sup> *<sup>u</sup>*<sup>2</sup> <sup>þ</sup> *<sup>R</sup>*�<sup>1</sup>

<sup>11</sup> *BT* <sup>1</sup> *v*1,

<sup>0</sup> *v*<sup>10</sup> � �*E*2ð Þ *<sup>t</sup>*<sup>0</sup>

<sup>2</sup> 0*<sup>m</sup>*1�*<sup>n</sup>* � � *<sup>E</sup>*2*<sup>y</sup>* <sup>þ</sup>

<sup>2</sup> *C<sup>T</sup>*

<sup>1</sup> *E*1*x* þ

substitute the derivatives into the obtained expression using (6), (7) and (8). Also,

*x*\_ ¼ *Ax* � *S*1*v*<sup>1</sup> þ *B*2*u*<sup>2</sup> þ *Cw:*

<sup>1</sup> *E*2*y* þ


1 2 *e*1 � �

<sup>1</sup> in (11). Hence:

, *<sup>B</sup>* <sup>≔</sup> *<sup>B</sup>*<sup>2</sup>

*On*�*m*<sup>2</sup> � �

*y*\_ ¼ *Hy* þ *Bu*<sup>2</sup> þ *C*1*w* (20)

*T*

1 2 *e*2 � �

<sup>0</sup> *v*<sup>10</sup>

� �*e*2ð Þþ *<sup>t</sup>*<sup>0</sup> *<sup>d</sup>*2ð Þ *<sup>t</sup>*<sup>0</sup>

*<sup>R</sup>*22*dt* <sup>þ</sup> <sup>Ð</sup>*tf*

<sup>1</sup> *CTv*1,

� � <sup>¼</sup> <sup>0</sup> (17)

(18)

*v*1ð Þ¼ *t*<sup>0</sup> *v*<sup>10</sup>

*x*0 *v*<sup>10</sup>

*<sup>t</sup>*<sup>0</sup> <sup>∥</sup>*z*∥<sup>2</sup> *P*2 *dt*,

1 2 *e*2 � �

*:*

*<sup>P</sup>*<sup>2</sup> ¼ *zP*2*z* and

*:* Then, differentiate *v*<sup>1</sup> and

and *<sup>C</sup>*<sup>1</sup> <sup>≔</sup> *<sup>I</sup>*

<sup>2</sup> *y* þ *d*<sup>2</sup> (21)

*E*1 � �

*C:* We

!

*x*\_ ¼ *Ax* � *S*1*v*<sup>1</sup> þ *B*2*u*<sup>2</sup> þ *Cw*, *x t*ð Þ¼ <sup>0</sup> *x*0, (19)

*dV*~ <sup>2</sup> *dt* <sup>¼</sup> *<sup>d</sup> dt <sup>y</sup>TE*2*<sup>y</sup>* <sup>þ</sup> *<sup>e</sup> T* <sup>2</sup> *y* þ *d*<sup>2</sup> � � ¼ *y*\_ *TE*2*<sup>y</sup>* <sup>þ</sup> *yTE*\_ <sup>2</sup>*<sup>y</sup>* <sup>þ</sup> *<sup>y</sup>TE*2*y*\_ <sup>þ</sup> *<sup>e</sup>*\_ *T* <sup>2</sup> *y* þ *e T* <sup>2</sup> *<sup>y</sup>*\_ <sup>þ</sup> \_ *d*2 <sup>þ</sup>*xTQ*2*<sup>x</sup>* <sup>þ</sup> *uT* <sup>1</sup> *<sup>R</sup>*21*u*<sup>1</sup> <sup>þ</sup> *<sup>u</sup><sup>T</sup>* <sup>2</sup> *<sup>R</sup>*22*u*<sup>2</sup> <sup>þ</sup> *<sup>w</sup>TP*2*<sup>w</sup>* � *<sup>ψ</sup>*<sup>2</sup> <sup>¼</sup> *<sup>y</sup>THT* <sup>þ</sup> *uT* <sup>2</sup> *BTwTCT* 1 � �*E*2*y* <sup>þ</sup>*yTE*\_ <sup>2</sup>*<sup>y</sup>* <sup>þ</sup> *<sup>y</sup>TE*2ð Þ *Hy* <sup>þ</sup> *Bu*<sup>2</sup> <sup>þ</sup> *<sup>C</sup>*1*<sup>w</sup>* þ*e*\_ *T* <sup>2</sup> *y* þ *e T* <sup>2</sup> <sup>ð</sup>*Hy* <sup>þ</sup> *Bu*<sup>2</sup> <sup>þ</sup> *<sup>C</sup>*1*w*Þ þ \_ *d*2 <sup>þ</sup>*y<sup>T</sup> <sup>Q</sup>*<sup>2</sup> <sup>0</sup> <sup>0</sup> *<sup>S</sup>*21 ! |fflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflffl} <sup>≔</sup> *<sup>Q</sup> <sup>y</sup>* <sup>þ</sup> *uT* <sup>2</sup> *<sup>R</sup>*22*u*<sup>2</sup> <sup>þ</sup> *<sup>w</sup>TP*2*<sup>w</sup>* � *<sup>ψ</sup>*<sup>2</sup>

Now we associate certain terms

$$\begin{aligned} &=y^T \left(H^T E\_2 + \dot{E}\_2 + E\_2 H + Q\right)y \\ &+ \left(\mu\_2 - \wp\_2\right)^T R\_{22} \left(\mu\_2 - \wp\_2\right) \\ &+ \wp\_2^T R\_{22} \mu\_2 + \nu\_2^T R\_{22} \wp\_2 - \wp\_2^T R\_{22} \wp\_2 \\ &+ (w - \alpha)^T P\_2 (w - \alpha) \\ &+ w^T P\_2 \alpha + a^T P\_2 w - \alpha^T P\_2 \alpha \\ &+ \mu\_2^T \left(B^T E\_2 \mathcal{y} + B^T \epsilon\_2\right) + \left(\boldsymbol{\chi}^T E\_2 \mathcal{B} + \frac{1}{2} \boldsymbol{\epsilon}\_2^T \boldsymbol{B}\right) \mu\_2 \\ &+ w^T \left(C\_1^T E\_2 \mathcal{y} + C\_1^T \epsilon\_2\right) + \left(\boldsymbol{\chi}^T E\_2 G\_1 + \frac{1}{2} \boldsymbol{\epsilon}\_2^T G\_1\right) w \\ &+ \left(\dot{\epsilon}\_2^T + \boldsymbol{\epsilon}\_2^T H\right) \boldsymbol{y} + \dot{d}\_2 - \mu\_2 \end{aligned}$$

and furthermore

$$\begin{aligned} &=y^T(H^TE\_2+\dot{E}\_2+E\_2H+Q)y-\mu\_2\\ &+(\mu\_2-\nu\_2)^TR\_{22}(\mu\_2-\nu\_2)-y\_2^TR\_{22}\nu\_2\\ &+(w-a)^TP\_2(w-a)-a^TP\_2a\\ &+u\_2^T\left(B^TE\_2y+B^TE\_2+R\_{22}\nu\_2\right)\\ &+\left(y^TE\_2B+\frac{1}{2}e\_2^TB\_{22}^TB\_{22}\right)u\_2\\ &+w^T\left(C\_1^TE\_2y+C\_1^TA\_{22}^TB\_{22}\right)u\_2\\ &+\left(y^TE\_1C\_2+\frac{1}{2}e\_2^TB\_1-a^TP\_2\right)w\\ &+\left(\dot{e}\_2^T+e\_2^TB\_2^TB\_{22}-\mu\_2\right)\end{aligned}$$

**191**

*Systems-of-Systems Perspectives and Applications - Design, Modeling, Simulation…*

Consider

$$(R\_{22}\mathbf{y}\_2 + \left(B\_2^T \quad O\_{m\_2 \times n}\right)\left(E\_2 \mathbf{y} + \frac{1}{2}\mathbf{e}\_2\right) = \mathbf{0})$$

Further, we assume the mappings *E*2,*e*2, *d*<sup>2</sup> to be chosen in such a way that the

� � <sup>¼</sup> *<sup>K</sup>*<sup>2</sup> *<sup>f</sup>*

*E*<sup>2</sup> *tf*

*e*<sup>2</sup> *tf* � � <sup>¼</sup> <sup>0</sup>

*d*<sup>2</sup> *tf* � � <sup>¼</sup> <sup>0</sup>

*y tf*

ð*<sup>t</sup> <sup>f</sup> t*

> i *dτ* þ ð*t f t ψ*2*dτ*

Observe that the rhs of (23) does not depend of *u*j½ � *<sup>t</sup>*0,*<sup>t</sup>* and the rls of (23) does not

Now, we substitute this into (23) and consider the infimal values over all possi-

� �*K*<sup>2</sup> *<sup>f</sup> y tf*


> *u*<sup>2</sup> � *y*<sup>2</sup> � �*<sup>T</sup>*

þð Þ *<sup>w</sup>* � *<sup>α</sup> TP*2ð Þ *<sup>w</sup>* � *<sup>α</sup>*

ð*<sup>t</sup> <sup>f</sup> t*

*<sup>u</sup>*j*t*,*<sup>t</sup> <sup>f</sup>*

*u*j *t* 0,*t f*

þð Þ *<sup>w</sup>* � *<sup>α</sup> TP*2ð Þ *<sup>w</sup>* � *<sup>α</sup>*

follower should take also the worst-case disturbance into account.

þð Þ *<sup>w</sup>* � *<sup>α</sup> TP*2ð Þ *<sup>w</sup>* � *<sup>α</sup>*

*<sup>V</sup>*2ð Þ *<sup>t</sup>*, *<sup>y</sup>* equals *<sup>V</sup>*<sup>~</sup> <sup>2</sup>ð Þ*<sup>t</sup>* if *<sup>u</sup>*<sup>2</sup> � *<sup>y</sup>*<sup>2</sup> � <sup>0</sup>∀*<sup>t</sup>* <sup>∈</sup><sup>T</sup> and *<sup>w</sup>* � *<sup>α</sup>* <sup>¼</sup> <sup>0</sup>*:* As the leader chooses his strategy assuming rationality of the follower and worst-case disturbance, the

> ð*t f t*0

*<sup>ψ</sup>*2ð Þ *<sup>τ</sup>*, ^*y*ð Þ*<sup>τ</sup>* , *<sup>u</sup>*ð Þ*<sup>τ</sup> <sup>d</sup><sup>τ</sup>* <sup>þ</sup> *<sup>y</sup><sup>T</sup> tf*

� � <sup>þ</sup>

� � h

i *dτ*

*u*<sup>2</sup> � *y*<sup>2</sup> � �*<sup>T</sup>*

> i *dτ*

*u*<sup>2</sup> � *y*<sup>2</sup> � �*<sup>T</sup>*

> i *dτ*

ð*<sup>t</sup> <sup>f</sup> t ψ*2*dτ*

*R*<sup>22</sup> *u*<sup>2</sup> � *y*<sup>2</sup>

� � h

*R*<sup>22</sup> *u*<sup>2</sup> � *y*<sup>2</sup>

*R*<sup>22</sup> *u*<sup>2</sup> � *y*<sup>2</sup>

� � h

� � and substituting:

*u*<sup>2</sup> � *y*<sup>2</sup> � �*<sup>T</sup>*

*R*<sup>22</sup> *u*<sup>2</sup> � *y*<sup>2</sup>

� �*K*<sup>2</sup> *<sup>f</sup> y tf*

� �

(23)

� � h

� �*Ky <sup>f</sup>*

*y tf* � � �

depend of *<sup>u</sup>*2<sup>j</sup> *<sup>t</sup>*,*<sup>t</sup>* ½ �*<sup>f</sup> :* Then considering now the infimal value, we recall that*:*

þð Þ *<sup>w</sup>* � *<sup>α</sup> TP*2ð Þ *<sup>w</sup>* � *<sup>α</sup>*

ð*t f t*

� � <sup>¼</sup> *<sup>y</sup><sup>T</sup> tf*

*Existence of Open Loop Equilibria for Disturbed Stackelberg Games*

� �*Ky <sup>f</sup>*

*<sup>V</sup>*<sup>~</sup> <sup>2</sup>ðÞ¼ *<sup>t</sup> <sup>y</sup><sup>T</sup> tf*

*V*2ð Þ¼ *t*, *y* inf

*<sup>u</sup>*2<sup>j</sup> *<sup>t</sup>*,*<sup>t</sup>* ½ �*<sup>f</sup>*

� � :

*<sup>V</sup>*<sup>~</sup> <sup>2</sup>ðÞ¼ *<sup>t</sup>* inf *<sup>y</sup><sup>T</sup> tf*

*<sup>V</sup>*2ð Þ¼ *<sup>t</sup>*, *<sup>y</sup> V t* <sup>~</sup> ðÞþ inf

*<sup>V</sup>*2ð Þ¼ *<sup>t</sup>*0, *<sup>y</sup> <sup>V</sup>*<sup>~</sup> <sup>2</sup>ð Þþ *<sup>t</sup>*<sup>0</sup> inf

To conclude, consider *t* ¼ *t*<sup>0</sup> and hence*:*

� inf <sup>ð</sup>*<sup>t</sup> <sup>f</sup> t*

following terminal values hold*:*

*DOI: http://dx.doi.org/10.5772/intechopen.92202*

Then, we obtain *V*~ <sup>2</sup> *tf*

ble control functions in *t*, *t <sup>f</sup>*

then we have:

**193**

and also

$$\mathbf{C}\_1^T \left( E\_2 \mathbf{y} + \frac{1}{2} e\_2 \right) + P\_2 a = \mathbf{0}$$

If *<sup>R</sup>*<sup>22</sup> <sup>&</sup>gt;0 then *<sup>y</sup>*<sup>2</sup> ¼ �*R*�<sup>1</sup> <sup>22</sup> *BT* <sup>2</sup> *Om*2�*<sup>n</sup>* � � *<sup>E</sup>*2*<sup>y</sup>* <sup>þ</sup> <sup>1</sup> 2 *e*2 � �*:* If *P*<sup>2</sup> >0 then *<sup>α</sup>* ¼ �*P*�<sup>1</sup> <sup>2</sup> *<sup>C</sup>*<sup>1</sup> *<sup>E</sup>*2*<sup>y</sup>* <sup>þ</sup> <sup>1</sup> 2 *e*2 � �*:*

Define *<sup>S</sup>* <sup>≔</sup> *<sup>S</sup>*<sup>2</sup> <sup>0</sup> 0 0 � � and *<sup>T</sup>* <sup>≔</sup> *<sup>T</sup>*<sup>2</sup> *<sup>T</sup>*2*E*<sup>1</sup> *E*1*T*<sup>2</sup> *E*1*T*2*E*<sup>1</sup> � �. Substitute this *<sup>y</sup>*<sup>2</sup> and *<sup>α</sup>* in the calculations:

$$\begin{aligned} \dot{\rho} &= \rho^T \left( H^T E\_2 + \dot{E}\_2 + E\_2 H + Q - E\_2 (S + T) E\_2 \right) \mathbf{y} \\ &+ \left( \mu\_2 - \wp\_2 \right)^T R\_{22} \left( \mu\_2 - \wp\_2 \right) + (w - a)^T P\_2 (w - a) \\ &+ \left( \dot{e}\_2^T + e\_2^T H - E\_2 (S + T) \right) \mathbf{y} \\ &+ \dot{d}\_2 - \frac{1}{4} e\_2^T (S + T) \mathbf{e}\_2 - \wp\_2 \end{aligned}$$

Considering:

$$H^T E\_2 + \dot{E}\_2 + E\_2 H + Q - E\_2 (\mathbf{S} + T) E\_2 = \mathbf{0}$$

$$\dot{e}\_2^T + e\_2^T H - e\_2^T (\mathbf{S} + T) E\_2 = \mathbf{0}$$

$$\dot{d}\_2 - \frac{1}{4} e\_2^T (\mathbf{S} + T) e\_2 = \mathbf{0}$$

that is

$$\begin{aligned} \dot{E}\_2 &= -H^T E\_2 - E\_2 H - Q + E\_2 (\mathbf{S} + T) E\_2 \\ \dot{\mathbf{e}}\_2 &= \left( -H^T + E\_2 (\mathbf{S} + T) \right) \mathbf{e}\_2 \\ \dot{d}\_2 &= \frac{1}{4} \mathbf{e}\_2^T (\mathbf{S} + T) \mathbf{e}\_2 \end{aligned}$$

We end up with

$$\begin{split} \frac{d\bar{V}\_2(t)}{dt} &= \left(\mu\_2 - \wp\_2\right)^T R\_{22} \left(\mu\_2 - \wp\_2\right) \\ &+ (w - a)^T P\_2 (w - a) - \wp\_2 \end{split} \tag{22}$$

Integrating yields:

$$\begin{aligned} \tilde{\mathbf{V}}\_2(\mathbf{t}\_f) - \tilde{\mathbf{V}}(\mathbf{t}) &= \int\_{\mathbf{t}}^{\mathbf{t}\_f} \left[ \left( \mathbf{u}\_2 - \mathbf{y}\_2 \right)^T \mathbf{R}\_{22} (\mathbf{u}\_2 - \mathbf{y}\_2) \right] \\ &+ (w - a)^T P\_2 (w - a) \mathbf{s} \Big| d\mathbf{\tau} - \int\_{\mathbf{t}}^{\mathbf{t}\_f} \mathbf{y}\_2 d\mathbf{\tau} . \end{aligned}$$

*Existence of Open Loop Equilibria for Disturbed Stackelberg Games DOI: http://dx.doi.org/10.5772/intechopen.92202*

Consider

and also

*<sup>α</sup>* ¼ �*P*�<sup>1</sup>

calculations:

Considering:

that is

We end up with

Integrating yields:

**192**

If *<sup>R</sup>*<sup>22</sup> <sup>&</sup>gt;0 then *<sup>y</sup>*<sup>2</sup> ¼ �*R*�<sup>1</sup>

2 *e*2 � �*:*

0 0 � �

> þ *u*<sup>2</sup> � *y*<sup>2</sup> � �*<sup>T</sup>*

> > *e*\_ *T* <sup>2</sup> þ *e T* <sup>2</sup> *H* � *e T*

> > > \_ *<sup>d</sup>*<sup>2</sup> � <sup>1</sup> 4 *e T*

> > > > \_ *<sup>d</sup>*<sup>2</sup> <sup>¼</sup> <sup>1</sup> 4 *e T*

*dt* <sup>¼</sup> *<sup>u</sup>*<sup>2</sup> � *<sup>y</sup>*<sup>2</sup>

<sup>~</sup> ðÞ¼

� �*<sup>T</sup>*

ð*<sup>t</sup> <sup>f</sup> t*

þð Þ *<sup>w</sup>* � *<sup>α</sup> TP*2ð Þ *<sup>w</sup>* � *<sup>α</sup> <sup>s</sup>*

*dV*<sup>~</sup> <sup>2</sup>ð Þ*<sup>t</sup>*

� � � *V t*

*V*~ <sup>2</sup> *tf*

þ *e*\_ *T* <sup>2</sup> þ *e T*

þ\_ *<sup>d</sup>*<sup>2</sup> � <sup>1</sup> 4 *e T*

<sup>2</sup> *<sup>C</sup>*<sup>1</sup> *<sup>E</sup>*2*<sup>y</sup>* <sup>þ</sup> <sup>1</sup>

Define *<sup>S</sup>* <sup>≔</sup> *<sup>S</sup>*<sup>2</sup> <sup>0</sup>

*<sup>R</sup>*22*y*<sup>2</sup> <sup>þ</sup> *BT*

*CT* <sup>1</sup> *E*2*y* þ

<sup>22</sup> *BT*

<sup>2</sup> *Om*2�*<sup>n</sup>* � � *<sup>E</sup>*2*<sup>y</sup>* <sup>þ</sup>

*Systems-of-Systems Perspectives and Applications - Design, Modeling, Simulation…*

1 2 *e*2 � �

<sup>2</sup> *Om*2�*<sup>n</sup>* � � *<sup>E</sup>*2*<sup>y</sup>* <sup>þ</sup> <sup>1</sup>

and *<sup>T</sup>* <sup>≔</sup> *<sup>T</sup>*<sup>2</sup> *<sup>T</sup>*2*E*<sup>1</sup>

<sup>¼</sup> *<sup>y</sup><sup>T</sup> <sup>H</sup>TE*<sup>2</sup> <sup>þ</sup> *<sup>E</sup>*\_ <sup>2</sup> <sup>þ</sup> *<sup>E</sup>*2*<sup>H</sup>* <sup>þ</sup> *<sup>Q</sup>* � *<sup>E</sup>*2ð Þ *<sup>S</sup>* <sup>þ</sup> *<sup>T</sup> <sup>E</sup>*<sup>2</sup> � �*y*

*R*<sup>22</sup> *u*<sup>2</sup> � *y*<sup>2</sup>

<sup>2</sup> ð Þ *S* þ *T e*<sup>2</sup> � *ψ*<sup>2</sup>

*<sup>H</sup>TE*<sup>2</sup> <sup>þ</sup> *<sup>E</sup>*\_ <sup>2</sup> <sup>þ</sup> *<sup>E</sup>*2*<sup>H</sup>* <sup>þ</sup> *<sup>Q</sup>* � *<sup>E</sup>*2ð Þ *<sup>S</sup>* <sup>þ</sup> *<sup>T</sup> <sup>E</sup>*<sup>2</sup> <sup>¼</sup> <sup>0</sup>

*<sup>E</sup>*\_ <sup>2</sup> ¼ �*HTE*<sup>2</sup> � *<sup>E</sup>*2*<sup>H</sup>* � *<sup>Q</sup>* <sup>þ</sup> *<sup>E</sup>*2ð Þ *<sup>S</sup>* <sup>þ</sup> *<sup>T</sup> <sup>E</sup>*<sup>2</sup> *<sup>e</sup>*\_<sup>2</sup> ¼ �*H<sup>T</sup>* <sup>þ</sup> *<sup>E</sup>*2ð Þ *<sup>S</sup>* <sup>þ</sup> *<sup>T</sup>* � �*e*<sup>2</sup>

<sup>2</sup> ð Þ *S* þ *T E*<sup>2</sup> ¼ 0

<sup>2</sup> ð Þ *S* þ *T e*<sup>2</sup> ¼ 0

<sup>2</sup> ð Þ *S* þ *T e*<sup>2</sup>

*R*<sup>22</sup> *u*<sup>2</sup> � *y*<sup>2</sup> � �

� � h

i *dτ* � ð*<sup>t</sup> <sup>f</sup> t ψ*2*dτ:*

*R*<sup>22</sup> *u*<sup>2</sup> � *y*<sup>2</sup>

þð Þ *<sup>w</sup>* � *<sup>α</sup> TP*2ð Þ� *<sup>w</sup>* � *<sup>α</sup> <sup>ψ</sup>*<sup>2</sup>

*u*<sup>2</sup> � *y*<sup>2</sup> � �*<sup>T</sup>*

<sup>2</sup> *<sup>H</sup>* � *<sup>E</sup>*2ð Þ *<sup>S</sup>* <sup>þ</sup> *<sup>T</sup>* � �*<sup>y</sup>*

*E*1*T*<sup>2</sup> *E*1*T*2*E*<sup>1</sup> � �

1 2 *e*2 � �

þ *P*2*α* ¼ 0

2 *e*2 � �*:* If *P*<sup>2</sup> >0 then

� � <sup>þ</sup> ð Þ *<sup>w</sup>* � *<sup>α</sup> TP*2ð Þ *<sup>w</sup>* � *<sup>α</sup>*

¼ 0

. Substitute this *y*<sup>2</sup> and *α* in the

(22)

Further, we assume the mappings *E*2,*e*2, *d*<sup>2</sup> to be chosen in such a way that the following terminal values hold*:*

$$E\_2(t\_f) = K\_{2\_f}$$

$$e\_2(t\_f) = \mathbf{0}$$

$$d\_2(t\_f) = \mathbf{0}$$

Then, we obtain *V*~ <sup>2</sup> *tf* � � <sup>¼</sup> *<sup>y</sup><sup>T</sup> tf* � �*Ky <sup>f</sup> y tf* � � and substituting:

$$\begin{aligned} \tilde{\mathbf{V}}\_{2}(t) &= \mathbf{y}^{T}(\mathbf{t}\_{f}) \mathbf{K}\_{\mathbf{y}\_{f}} \mathbf{y}(\mathbf{t}\_{f}) - \int\_{t}^{t\_{f}} \left[ \left( \mathbf{u}\_{2} - \mathbf{y}\_{2} \right)^{T} \mathbf{R}\_{22} \left( \mathbf{u}\_{2} - \mathbf{y}\_{2} \right) \right. \\\\ &+ \left( \mathbf{w} - \mathbf{a} \right)^{T} \mathbf{P}\_{2} (\mathbf{w} - \mathbf{a}) \Big] d\mathbf{\tau} + \int\_{t}^{t\_{f}} \mathbf{y}\_{2} d\mathbf{\tau} \end{aligned} \tag{23}$$

Observe that the rhs of (23) does not depend of *u*j½ � *<sup>t</sup>*0,*<sup>t</sup>* and the rls of (23) does not depend of *<sup>u</sup>*2<sup>j</sup> *<sup>t</sup>*,*<sup>t</sup>* ½ �*<sup>f</sup> :* Then considering now the infimal value, we recall that*:*

$$W\_2(t, \boldsymbol{y}) = \inf\_{\boldsymbol{u}\_2|\_{\boldsymbol{\xi}\_{\mathcal{I}}} \atop \boldsymbol{t}\_{\mathcal{I}}} \int\_{\boldsymbol{t}}^{t\_f} \boldsymbol{\nu}\_2(\boldsymbol{\tau}, \boldsymbol{\hat{y}}(\boldsymbol{\tau}), \boldsymbol{u}(\boldsymbol{\tau})) d\boldsymbol{\tau} + \boldsymbol{y}^T(\boldsymbol{t}\_f) K\_{2\_f} \boldsymbol{y}(\boldsymbol{t}\_f)$$

Now, we substitute this into (23) and consider the infimal values over all possible control functions in *t*, *t <sup>f</sup>* � � :

$$\bar{V}\_{2}(t) = \underbrace{\inf \mathbf{y}^{T}(t\_{f})K\_{2}\mathbf{y}(t\_{f}) + \int\_{t}^{t\_{f}} \boldsymbol{\mu}\_{2} d\tau}\_{V\_{2}(t,\boldsymbol{y})}$$

$$- \inf \int\_{t}^{t\_{f}} \left[ \left(\boldsymbol{\mu}\_{2} - \boldsymbol{\upmu}\_{2}\right)^{T} R\_{22} \left(\boldsymbol{\upmu}\_{2} - \boldsymbol{\upmu}\_{2}\right) \right] d\tau$$

$$+ (\boldsymbol{\upmu} - \boldsymbol{a})^{T} P\_{2}(\boldsymbol{w} - \boldsymbol{a}) \Big] d\tau$$

then we have:

$$V\_2(t, \boldsymbol{y}) = \boldsymbol{\tilde{V}}(t) + \inf\_{\boldsymbol{w}|\_{\boldsymbol{\mu}\_{t\_f}}} \int\_{t}^{t\_f} \left[ \left( \boldsymbol{u}\_2 - \boldsymbol{y}\_2 \right)^T R\_{22} \left( \boldsymbol{u}\_2 - \boldsymbol{y}\_2 \right) \right]$$

$$+ (\boldsymbol{w} - \boldsymbol{a})^T P\_2 (\boldsymbol{w} - \boldsymbol{a}) \Big] d\boldsymbol{\tau}$$

*<sup>V</sup>*2ð Þ *<sup>t</sup>*, *<sup>y</sup>* equals *<sup>V</sup>*<sup>~</sup> <sup>2</sup>ð Þ*<sup>t</sup>* if *<sup>u</sup>*<sup>2</sup> � *<sup>y</sup>*<sup>2</sup> � <sup>0</sup>∀*<sup>t</sup>* <sup>∈</sup><sup>T</sup> and *<sup>w</sup>* � *<sup>α</sup>* <sup>¼</sup> <sup>0</sup>*:* As the leader chooses his strategy assuming rationality of the follower and worst-case disturbance, the follower should take also the worst-case disturbance into account.

To conclude, consider *t* ¼ *t*<sup>0</sup> and hence*:*

$$\begin{aligned} V\_2(t\_0, \boldsymbol{y}) &= \boldsymbol{\tilde{V}}\_2(t\_0) + \inf\_{\boldsymbol{u}|\_{t\_0, t\_f}} \int\_{t\_0}^{t\_f} \left[ \left( \boldsymbol{u}\_2 - \boldsymbol{y}\_2 \right)^T \boldsymbol{R}\_{22} \left( \boldsymbol{u}\_2 - \boldsymbol{y}\_2 \right) \right] \\ &+ (\boldsymbol{w} - \boldsymbol{a})^T \boldsymbol{P}\_2(\boldsymbol{w} - \boldsymbol{a}) \Big] d\tau \end{aligned}$$

*Systems-of-Systems Perspectives and Applications - Design, Modeling, Simulation…*

Then from (21):

$$\begin{aligned} V\_2(t\_0, \boldsymbol{y}) &= \boldsymbol{y}\_0^T \boldsymbol{E}\_2(t\_0) \boldsymbol{y}\_0 + \boldsymbol{e}\_2^T(t\_0) \boldsymbol{y}\_0 + \boldsymbol{d}\_2(t\_0) \\ &+ \inf\_{\boldsymbol{w}|\_{t\_0, t\_f}} \Big[ \Big( \boldsymbol{u}\_2 - \boldsymbol{y}\_2 \Big)^T \boldsymbol{R}\_{22} \big( \boldsymbol{u}\_2 - \boldsymbol{y}\_2 \Big) \\ &+ (\boldsymbol{w} - \boldsymbol{a})^T \boldsymbol{P}\_2(\boldsymbol{w} - \boldsymbol{a}) \Big] d\boldsymbol{\tau} \end{aligned}$$

Defining *<sup>z</sup>*<sup>2</sup> <sup>≔</sup> *<sup>u</sup>*<sup>2</sup> � *<sup>y</sup>*<sup>2</sup> <sup>¼</sup> *<sup>u</sup>*<sup>2</sup> <sup>þ</sup> *<sup>R</sup>*�<sup>1</sup> <sup>22</sup> *BT* <sup>2</sup> <sup>0</sup> � � *<sup>E</sup>*2*<sup>y</sup>* <sup>þ</sup> <sup>1</sup> 2 *e*2 � � and *<sup>z</sup>* <sup>≔</sup> *<sup>w</sup>* � *<sup>α</sup>* <sup>¼</sup> *<sup>w</sup>* <sup>þ</sup> *<sup>P</sup>*�<sup>1</sup> <sup>2</sup> *<sup>C</sup>*<sup>1</sup> *<sup>E</sup>*2*<sup>y</sup>* <sup>þ</sup> <sup>1</sup> 2 *e*2 � �, we have:

$$\begin{aligned} V\_2(t\_0, y) &= \mathcal{y}\_0^T E\_2(t\_0) \mathcal{y}\_0 + \mathcal{e}\_2^T(t\_0) \mathcal{y}\_0 + d\_2(t\_0) \\ &+ \int\_{t\_0}^{t\_f} \left\| z\_2 \right\|\_{R\_{22}}^2 + \left\| z \right\|\_{P\_2}^2 dt \end{aligned}$$

Now, we substitute *<sup>y</sup>*<sup>0</sup> <sup>¼</sup> *<sup>x</sup>*<sup>0</sup> *<sup>v</sup>*<sup>10</sup> � �*:*

The leader may choose its best answer either by accounting directly for its worst-case disturbance or by considering that the follower knows that there is a worst-case disturbance. In this work, the leader takes the worst-case disturbance directly into account.

Notice that in the term

$$J\_{20} = \begin{pmatrix} \varkappa\_0^T & \nu\_{10} \end{pmatrix} E\_2(\mathfrak{t}\_0) \begin{pmatrix} \varkappa\_0 \\ \nu\_{10} \end{pmatrix},\tag{24}$$

considering worst-case disturbamces *w*<sup>∗</sup>

*DOI: http://dx.doi.org/10.5772/intechopen.92202*

*Existence of Open Loop Equilibria for Disturbed Stackelberg Games*

The minimal cost for the follower, *J*<sup>10</sup> *u*<sup>∗</sup>

2 *xT* <sup>0</sup> *In*, *K<sup>T</sup>*

þ *e T* <sup>2</sup> ð Þ *t*<sup>0</sup>

Riccati matrix Eqs. (6) and (15) are globally solvable in �∞, *tf*

*Q*1 *Q*2 0

1

CCA �

0

BB@

þ

þ

0

BB@

0

BB@

*K*1 *K*2 *p*

0

BB@

1

CCA

*K*1*<sup>f</sup> K*2*<sup>f</sup>* 0

1

CCA*:*

As it can be easily observed, all these Riccati equations are of nonsymmetric

*<sup>W</sup>*\_ <sup>¼</sup> *<sup>B</sup>*<sup>21</sup> � *WB*<sup>11</sup> <sup>þ</sup> *<sup>B</sup>*22*<sup>W</sup>* <sup>þ</sup> *WB*12*W*,

where *W* is a matrix of order *k* � *n* whose coefficients are of adequate size. See AbouKandil et al. [16] for results on the existence of solution of Riccati equations.

*W tf*

(ii) solvability of the coupled system of Eqs. (25)–(27).

Riccati matrix differential equation. Hence:

*K*\_ 1 *K*\_ 2 *p*\_

*K*1 *K*2 *p*

1

CCA *tf* � � <sup>¼</sup>

0

BB@

type:

**195**

1

CCA ¼

0

BB@

where *e*2ð Þ *t*<sup>0</sup> , *d*2ð Þ *t*<sup>0</sup> are determined by (16) and (17), respectively.

From the convexity assumptions, it follows that *S*1, *S*, *Q*1, *Q* and *E*<sup>1</sup> *tf*

"

*J*<sup>20</sup> *u*<sup>∗</sup> <sup>1</sup> , *u*<sup>∗</sup> 2 � � <sup>¼</sup> <sup>1</sup>

closed loop equation

*<sup>i</sup>* and where *x t*ð Þ is a solution of the

� �, is as in (14), and for the leader is

*In K*1ð Þ *t*<sup>0</sup>

� � [15].

1

0

BB@

0

BB@

� � <sup>¼</sup> *<sup>W</sup> <sup>f</sup>* , (32)

*K*1 *K*2 *p*

*K*1 *K*2 *p*

1

CCA

1

CCA

CCA

#

*x*0

� �, *E*<sup>2</sup> *tf* � �

(31)

!

*x*\_ ¼ ½ � *A* � *S*1*K*<sup>1</sup> � ð Þ *S*<sup>2</sup> þ *T*<sup>2</sup> *K*<sup>2</sup> � *T*2*E*1*p x*,

2

<sup>1</sup> ð Þ *t*<sup>0</sup> � �*E*2ð Þ *<sup>t</sup>*<sup>0</sup> ð Þ *<sup>t</sup>*<sup>0</sup>

> *In K*1ð Þ *t*<sup>0</sup>

**Proof:** The proof is similar to the analogous result for the non-disturbed case [13].

are all semidefinite. Therefore, as far as the convexity conditions hold, the standard

It still remains the following questions to be answered (i) direct criteria for solvability of these equations if the convexity assumption is guaranteed as well as

Actually, this system of equations can also be written as a single, nonsymmetric

*K*1 *K*2 *p*

�*A<sup>T</sup>* 0 0 <sup>0</sup> �*AT <sup>Q</sup>*<sup>1</sup>

�*S*<sup>21</sup> *S*<sup>1</sup> *A* � *T*1*E*<sup>1</sup>

ð Þ *S*<sup>1</sup> þ *T*1, *S*2, 0

1

CCA *A*

0

BB@

!

*x t*ð Þ¼ <sup>0</sup> *<sup>x</sup>*0*:* (30)

*x*<sup>0</sup> þ *d*2ð Þ *t*<sup>0</sup>

*x*0, *E*2ð Þ *t*<sup>0</sup> , do not depend on the choice of *u*1, *u*2*:* Since we shall study the situation for Player-2 when Player-1 applies his optimal response control defined in (10), we have to set *<sup>v</sup>*<sup>1</sup> <sup>¼</sup> *<sup>E</sup>*1*<sup>x</sup>* <sup>þ</sup> <sup>1</sup> <sup>2</sup> *e*1*:* From (7), we can see that *v*1ð Þ¼ *t*<sup>0</sup> *v*<sup>10</sup> depends on *e*1ð Þ *t*<sup>0</sup> and hence also on *u*2*:*

In order to derive from Theorems (3.1) and (3.3) sufficient conditions for the existence of a unique worst-case Stackelberg equilibrium, we must get rid of the *u*2 dependence on *v*10*:* Therefore, we propose to restrict the set of admissible controls to functions representable in linear feedback form. This is what we do next.

**Theorem 3.4.** Let the solutions *<sup>E</sup>*1ð Þ*<sup>t</sup>* <sup>∈</sup> *<sup>n</sup>*�*<sup>n</sup>*, *<sup>E</sup>*<sup>2</sup> <sup>∈</sup> <sup>2</sup>*n*�2*<sup>n</sup>* of (6) and (15) exist in T , respectively. Let further the coupled system of equations

$$\begin{aligned} \dot{K}\_1 &= -Q\_1 - K\_1 A - A^T K\_1 + K\_1 (S\_1 + T\_1) K\_1 \\ &+ K\_1 S\_2 K\_2, \end{aligned} \tag{25}$$

$$\begin{aligned} \dot{K}\_2 &= -Q\_2 - K\_2 A - A^T K\_2 + Q\_1 p + K\_2 S\_1 K\_1 \\ &+ K\_2 (S\_2 + T\_2) K\_2 + K\_2 T\_2 E\_1 p, \end{aligned} \tag{26}$$

$$\begin{aligned} \dot{p} &= -pA - \mathbf{S\_{21}K\_1 + \mathbf{S\_1}K\_2 + (A - T\_1E\_1)p} \\ &+ p\mathbf{S\_1}K\_1 + p(\mathbf{S\_2} + T\_2)K\_2 + pT\_2E\_1p, \end{aligned} \tag{27}$$

admits a solution in T .

Then, there exists a unique open loop disturbed Stackelberg equilibrium in feedback synthesis which is given by

$$
\mu\_1^\*\left(t\right) = -R\_{11}^{-1}(t)B\_1^T(t)K\_1(t)\mathbf{x}(t),\tag{28}
$$

$$
\mu\_2^\*\left(t\right) = -R\_{22}^{-1}(t)B\_2^T(t)K\_2(t)\varkappa(t),\tag{29}
$$

*Existence of Open Loop Equilibria for Disturbed Stackelberg Games DOI: http://dx.doi.org/10.5772/intechopen.92202*

Then from (21):

<sup>2</sup> *<sup>C</sup>*<sup>1</sup> *<sup>E</sup>*2*<sup>y</sup>* <sup>þ</sup> <sup>1</sup>

directly into account.

Notice that in the term

(10), we have to set *<sup>v</sup>*<sup>1</sup> <sup>¼</sup> *<sup>E</sup>*1*<sup>x</sup>* <sup>þ</sup> <sup>1</sup>

on *e*1ð Þ *t*<sup>0</sup> and hence also on *u*2*:*

admits a solution in T .

**194**

feedback synthesis which is given by

*u*∗

*u*∗

<sup>1</sup> ðÞ¼� *<sup>t</sup> <sup>R</sup>*�<sup>1</sup>

<sup>2</sup> ðÞ¼� *<sup>t</sup> <sup>R</sup>*�<sup>1</sup>

*<sup>w</sup>* <sup>þ</sup> *<sup>P</sup>*�<sup>1</sup>

*<sup>V</sup>*2ð Þ¼ *<sup>t</sup>*0, *<sup>y</sup> <sup>y</sup><sup>T</sup>*

*<sup>V</sup>*2ð Þ¼ *<sup>t</sup>*0, *<sup>y</sup> <sup>y</sup><sup>T</sup>*

Defining *<sup>z</sup>*<sup>2</sup> <sup>≔</sup> *<sup>u</sup>*<sup>2</sup> � *<sup>y</sup>*<sup>2</sup> <sup>¼</sup> *<sup>u</sup>*<sup>2</sup> <sup>þ</sup> *<sup>R</sup>*�<sup>1</sup>

2 *e*2 � �, we have:

Now, we substitute *<sup>y</sup>*<sup>0</sup> <sup>¼</sup> *<sup>x</sup>*<sup>0</sup>

<sup>0</sup>*E*2ð Þ *t*<sup>0</sup> *y*<sup>0</sup> þ *e*

þð Þ *<sup>w</sup>* � *<sup>α</sup> TP*2ð Þ *<sup>w</sup>* � *<sup>α</sup>*

<sup>0</sup>*E*2ð Þ *t*<sup>0</sup> *y*<sup>0</sup> þ *e*

The leader may choose its best answer either by accounting directly for its worst-case disturbance or by considering that the follower knows that there is a worst-case disturbance. In this work, the leader takes the worst-case disturbance

> <sup>0</sup> *v*<sup>10</sup> � �*E*2ð Þ *<sup>t</sup>*<sup>0</sup>

*x*0, *E*2ð Þ *t*<sup>0</sup> , do not depend on the choice of *u*1, *u*2*:* Since we shall study the situation for Player-2 when Player-1 applies his optimal response control defined in

In order to derive from Theorems (3.1) and (3.3) sufficient conditions for the existence of a unique worst-case Stackelberg equilibrium, we must get rid of the *u*2 dependence on *v*10*:* Therefore, we propose to restrict the set of admissible controls to functions representable in linear feedback form. This is what we do next.

**Theorem 3.4.** Let the solutions *<sup>E</sup>*1ð Þ*<sup>t</sup>* <sup>∈</sup> *<sup>n</sup>*�*<sup>n</sup>*, *<sup>E</sup>*<sup>2</sup> <sup>∈</sup> <sup>2</sup>*n*�2*<sup>n</sup>* of (6) and (15) exist in

*<sup>K</sup>*\_ <sup>1</sup> ¼ �*Q*<sup>1</sup> � *<sup>K</sup>*1*<sup>A</sup>* � *ATK*<sup>1</sup> <sup>þ</sup> *<sup>K</sup>*1ð Þ *<sup>S</sup>*<sup>1</sup> <sup>þ</sup> *<sup>T</sup>*<sup>1</sup> *<sup>K</sup>*<sup>1</sup>

*<sup>K</sup>*\_ <sup>2</sup> ¼ �*Q*<sup>2</sup> � *<sup>K</sup>*2*<sup>A</sup>* � *<sup>A</sup>TK*<sup>2</sup> <sup>þ</sup> *<sup>Q</sup>*1*<sup>p</sup>* <sup>þ</sup> *<sup>K</sup>*2*S*1*K*<sup>1</sup>

*p*\_ ¼ �*pA* � *S*21*K*<sup>1</sup> þ *S*1*K*<sup>2</sup> þ ð Þ *A* � *T*1*E*<sup>1</sup> *p*

Then, there exists a unique open loop disturbed Stackelberg equilibrium in

<sup>11</sup> ð Þ*<sup>t</sup> BT*

<sup>22</sup> ð Þ*<sup>t</sup> BT*

<sup>2</sup> <sup>0</sup> � � *<sup>E</sup>*2*<sup>y</sup>* <sup>þ</sup> <sup>1</sup>

ð*<sup>t</sup> <sup>f</sup> t*0

<sup>22</sup> *BT*

þ inf *u*j*t* 0,*t f*

*Systems-of-Systems Perspectives and Applications - Design, Modeling, Simulation…*

þ ð*<sup>t</sup> <sup>f</sup> t*0 k k *z*<sup>2</sup> 2 *<sup>R</sup>*<sup>22</sup> þ k k*z*

*v*<sup>10</sup> � � *:*

*J*<sup>20</sup> ¼ *x<sup>T</sup>*

T , respectively. Let further the coupled system of equations

*T*

*u*<sup>2</sup> � *y*<sup>2</sup> � �*<sup>T</sup>*

*T*

2 *P*2 *dt*

*x*0 *v*<sup>10</sup> � �

<sup>þ</sup>*K*1*S*2*K*2, (25)

<sup>þ</sup>*K*2ð Þ *<sup>S</sup>*<sup>2</sup> <sup>þ</sup> *<sup>T</sup>*<sup>2</sup> *<sup>K</sup>*<sup>2</sup> <sup>þ</sup> *<sup>K</sup>*2*T*2*E*1*p*, (26)

<sup>þ</sup>*pS*1*K*<sup>1</sup> <sup>þ</sup> *p S*ð Þ <sup>2</sup> <sup>þ</sup> *<sup>T</sup>*<sup>2</sup> *<sup>K</sup>*<sup>2</sup> <sup>þ</sup> *pT*2*E*1*p*, (27)

<sup>1</sup> ð Þ*t K*1ð Þ*t x t*ð Þ, (28)

<sup>2</sup> ð Þ*t K*2ð Þ*t x t*ð Þ, (29)

<sup>2</sup> *e*1*:* From (7), we can see that *v*1ð Þ¼ *t*<sup>0</sup> *v*<sup>10</sup> depends

, (24)

<sup>2</sup> ð Þ *t*<sup>0</sup> *y*<sup>0</sup> þ *d*2ð Þ *t*<sup>0</sup>

� � h

i *dτ*

2 *e*2

<sup>2</sup> ð Þ *t*<sup>0</sup> *y*<sup>0</sup> þ *d*2ð Þ *t*<sup>0</sup>

*R*<sup>22</sup> *u*<sup>2</sup> � *y*<sup>2</sup>

� � and *<sup>z</sup>* <sup>≔</sup> *<sup>w</sup>* � *<sup>α</sup>* <sup>¼</sup>

considering worst-case disturbamces *w*<sup>∗</sup> *<sup>i</sup>* and where *x t*ð Þ is a solution of the closed loop equation

$$\begin{aligned} \dot{\mathbf{x}} &= [A - \mathbf{S}\_1 K\_1 - (\mathbf{S}\_2 + T\_2)K\_2 - T\_2 E\_1 p] \mathbf{x}, \\ \mathbf{x}(t\_0) &= \mathbf{x}\_0. \end{aligned} \tag{30}$$

The minimal cost for the follower, *J*<sup>10</sup> *u*<sup>∗</sup> 2 � �, is as in (14), and for the leader is

$$\begin{aligned} J\_{20}(\boldsymbol{u}\_1^\*, \boldsymbol{u}\_2^\*) &= \frac{1}{2} \begin{bmatrix} \boldsymbol{x}\_0^T(\boldsymbol{I}\_n, \boldsymbol{K}\_1^T(t\_0)) \boldsymbol{E}\_2(t\_0)(t\_0) \begin{pmatrix} \boldsymbol{I}\_n\\ \boldsymbol{K}\_1(t\_0) \end{pmatrix} \boldsymbol{\varkappa}\_0\\ &+ \boldsymbol{e}\_2^T(t\_0) \begin{pmatrix} \boldsymbol{I}\_n\\ \boldsymbol{K}\_1(t\_0) \end{pmatrix} \boldsymbol{\varkappa}\_0 + \boldsymbol{d}\_2(t\_0) \end{aligned}$$

where *e*2ð Þ *t*<sup>0</sup> , *d*2ð Þ *t*<sup>0</sup> are determined by (16) and (17), respectively.

**Proof:** The proof is similar to the analogous result for the non-disturbed case [13]. � �

From the convexity assumptions, it follows that *S*1, *S*, *Q*1, *Q* and *E*<sup>1</sup> *tf* � �, *E*<sup>2</sup> *tf* are all semidefinite. Therefore, as far as the convexity conditions hold, the standard Riccati matrix Eqs. (6) and (15) are globally solvable in �∞, *tf* � � [15].

It still remains the following questions to be answered (i) direct criteria for solvability of these equations if the convexity assumption is guaranteed as well as (ii) solvability of the coupled system of Eqs. (25)–(27).

Actually, this system of equations can also be written as a single, nonsymmetric Riccati matrix differential equation. Hence:

$$
\begin{aligned}
\begin{pmatrix}
\dot{K}\_1\\ \dot{K}\_2\\ \dot{p}
\end{pmatrix} &= \begin{pmatrix}
Q\_1\\ Q\_2\\ 0
\end{pmatrix} - \begin{pmatrix}
K\_1\\ K\_2\\ p
\end{pmatrix} A\\ &+ \begin{pmatrix}
\end{pmatrix} \begin{pmatrix} K\_1\\ K\_2\\ p\\ p
\end{pmatrix} \\ &+ \begin{pmatrix} K\_1\\ K\_2\\ p\\ p\\ p \end{pmatrix} (S\_1 + T\_1, \quad S\_2, \quad 0) \begin{pmatrix} K\_1\\ K\_2\\ p\\ p \end{pmatrix} \\
\begin{pmatrix} K\_1\\ K\_2\\ p\\ p \end{pmatrix} (t\_f) &= \begin{pmatrix} K\_{\mathcal{Y}}\\ K\_{\mathcal{Z}}\\ 0 \end{pmatrix}.
\end{aligned} \tag{31}
$$

As it can be easily observed, all these Riccati equations are of nonsymmetric type:

$$\begin{aligned} \dot{W} &= B\_{21} - WB\_{11} + B\_{22}W + WB\_{12}W, \\ W(t\_f) &= W\_f, \end{aligned} \tag{32}$$

where *W* is a matrix of order *k* � *n* whose coefficients are of adequate size. See AbouKandil et al. [16] for results on the existence of solution of Riccati equations.

## **4. Discussion and conclusions**

High dimension problems appeal to the use of hierarchic and decentralised models as differential games. One example of these problems is large networks, as for instance the management and control of high pressure gas networks. Since this is a large dimension and geographically dispersed problem, a decentralised formulation captures the non-cooperative nature, and sometimes even antagonistic, of the different stake-holders in the network.

The network controllable elements can be seen as players that seek their best settings and then interact among themselves to check for network feasibility. The equilibrium sought by the players depends on the way the players are organised among themselves. It makes some sense to have some autonomous elements that run the network and others follow, as is the case of a main inlet point of a country, as it happens with the inlet of Sines in the portuguese network. The ultimate goal of the network is to meet customers' demand at the lowest cost. As the main variation of the problem is due to the off-takes, these may be seen as perturbations to nominal consumption levels of a deterministic model.

Therefore, it makes some sense to view the gas transportation and distribution system as a disturbed Stackelberg game where the players play against a worstcase disturbance, that means a sudden change in weather conditions from one period of operation to the other. Neverthless, the theory is not ready, and also having in mind the development of algorithms, direct solution methods, and explicit solution representations need to be further investigated. In this work, we have obtained sufficient conditions for the existence of the solution of a 2-player game. However, direct criteria for solvability of this problem needs more work. Also, the solvability of the coupled system of Eqs. (25)–(27) has to be further investigated. Also, we would like to solve the same problem using an operator approach.

Similarly to what we have done in the past for Nash games, we would like to study this problem considering the underlying dynamics as a repetitive process, that seems to be adequate to capture the behaviour seemingly periodic of the network. Also, the boundary control of the network depends on the type of strategy sought by the players. The structure of these versions of the problems need to be examined.

The obtained results, in every stage of the work, should be applied to a single pipe and ideally using some operational data. Furthermore, we expect to apply the work to a simple network, which is not exactly a straightforward extension.

**Author details**

**197**

T.-P. Azevedo Perdicoúlis<sup>1</sup>

2 RWTH-Aachen, Germany

\*, G. Jank2 and P. Lopes dos Santos<sup>3</sup>

1 ISR, Engineering Department at ECT UTAD, University of Coimbra, Portugal

© 2020 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/3.0), which permits unrestricted use, distribution, and reproduction in any medium,

3 INESC TEC and EE Department at FEUP, University of Porto, Portugal

\*Address all correspondence to: tazevedo@utad.pt

*Existence of Open Loop Equilibria for Disturbed Stackelberg Games*

*DOI: http://dx.doi.org/10.5772/intechopen.92202*

provided the original work is properly cited.

## **Acknowledgements**

I would like to thank the reviewer for his valuable suggestions.

This work has been financed by National Funds through the Portuguese funding agency, FCT – Fundao para a Cincia e a Tecnologia under project: (i) UID/EEA/ 00048/2019 for the first author and (ii) UID/EEA/50014/2019 for the third author. *Existence of Open Loop Equilibria for Disturbed Stackelberg Games DOI: http://dx.doi.org/10.5772/intechopen.92202*

## **Author details**

**4. Discussion and conclusions**

different stake-holders in the network.

approach.

need to be examined.

**Acknowledgements**

**196**

nominal consumption levels of a deterministic model.

High dimension problems appeal to the use of hierarchic and decentralised models as differential games. One example of these problems is large networks, as for instance the management and control of high pressure gas networks. Since this is a large dimension and geographically dispersed problem, a decentralised formulation captures the non-cooperative nature, and sometimes even antagonistic, of the

*Systems-of-Systems Perspectives and Applications - Design, Modeling, Simulation…*

The network controllable elements can be seen as players that seek their best settings and then interact among themselves to check for network feasibility. The equilibrium sought by the players depends on the way the players are organised among themselves. It makes some sense to have some autonomous elements that run the network and others follow, as is the case of a main inlet point of a country, as it happens with the inlet of Sines in the portuguese network. The ultimate goal of the network is to meet customers' demand at the lowest cost. As the main variation of the problem is due to the off-takes, these may be seen as perturbations to

Therefore, it makes some sense to view the gas transportation and distribution system as a disturbed Stackelberg game where the players play against a worstcase disturbance, that means a sudden change in weather conditions from one period of operation to the other. Neverthless, the theory is not ready, and also having in mind the development of algorithms, direct solution methods, and explicit solution representations need to be further investigated. In this work, we have obtained sufficient conditions for the existence of the solution of a 2-player game. However, direct criteria for solvability of this problem needs more work. Also, the solvability of the coupled system of Eqs. (25)–(27) has to be further investigated. Also, we would like to solve the same problem using an operator

Similarly to what we have done in the past for Nash games, we would like to study this problem considering the underlying dynamics as a repetitive process, that seems to be adequate to capture the behaviour seemingly periodic of the network. Also, the boundary control of the network depends on the type of strategy sought by the players. The structure of these versions of the problems

The obtained results, in every stage of the work, should be applied to a single pipe and ideally using some operational data. Furthermore, we expect to apply the work to a simple network, which is not exactly a straightforward extension.

This work has been financed by National Funds through the Portuguese funding agency, FCT – Fundao para a Cincia e a Tecnologia under project: (i) UID/EEA/ 00048/2019 for the first author and (ii) UID/EEA/50014/2019 for the third author.

I would like to thank the reviewer for his valuable suggestions.


\*Address all correspondence to: tazevedo@utad.pt

© 2020 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/ by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

## **References**

[1] Bagchi A. Differential games in economic models. In: Lecture Notes in Control and Information Sciences. Vol. 64. Springer Verlag; 1984

[2] Eisele T. Nonexistence and nonuniqueness of open-loop equilibria in linear quadratic differential games. Journal of Optimization Theory and Applications. 1982;**37**:443-468

[3] Lukes DL, Russel DL. A global theory for linear quadratic differential games. Journal of Mathematical Analysis and Applications. 1971;**33**:96-123

[4] Mehlmann A. Applied Differential Games. New York: Springer Verlag; 1988

[5] Basar T, Olsder G. Dynamic noncooperative game theory. 2nd ed. London: SIAM; 1999

[6] Abou-Kandil H, Bertrand P. Analytical solution for an open-loop Stackelberg game. IEEE Transactions on Automatic Control. 1985;**AC-30**: 1222-1224

[7] Medanic J, Radojevic D. Multilevel Stackelberg strategies in linear quadratic systems. Journal of Optimization Theory and Applications. 1987;**24**: 485-479

[8] Pan L, Yong J. A differential game with multi-level of hierarchy. Journal of Mathematical Analysis and Applications. 1991;**161**:522-544

[9] Tolwinski B. A Stackelberg solution of dynamic games. IEEE Transactions on Automatic Control. 1973;**AC-28**: 85-93

[10] Jank G, Kun G. Optimal control of disturbed linear quadratic differential games. European Journal of Control. 2002;**8**(2):152-162

[11] Kun G. Stabilizability, controllability and optimal strategies of linear and nonlinear dynamical games [Ph.D. Thesis]. RWTH Aachen; 2000

**Chapter 12**

**Abstract**

Nash Equilibrium Study for

Power Control in D2D

*Sameh Najeh and Ammar Bouallegue*

Communications

conventional centralized algorithms.

**1. Introduction**

**199**

Distributed Mode Selection and

One of the main challenges of LTE-advanced (LTE-A) is to recover the localarea services and improve spectrum effciency. In order to reach those goals technical capabilities are required. D2D is a promising techniques for the 5G wireless communications system using several applications, as: network traffic offoading, public safety, social services and applications such as gaming and military applications. In this chapter, we investigate both mode selection and distributed power control in D2D system. Indeed, the mode selection is provided while respecting a predetermined SINR threshold relative to cellular and D2D users. The amount of minimum and maximum power are then derived to fulffill the predetermined requirements, by limiting the interference created by underlaid D2D users. In order to realize our proposed power control step, a new distributed control approach is proposed using game theory tools for several cellular and D2D users. This distributed approach is based on the mode selection strategy already proposed in the previous step. Finally, simulations were established in order to compare the proposed distributed algorithm in terms of coverage probability which is based on game theory, with other

**Keywords:** mode selection, power control, distributed, Nash equilibrium

munication between varied gadgets without human intercession [2].

applications, in order to make the network with faster speeds and greater

The Internet of Things (IoT) is a developing and promising innovation, which were able to revolutionize the world [1]. IoT manages low-powered gadgets, using the internet by interacting with one another. IoT interconnect "Things" and also helps in machine-to-machine (M2M) communication, which is a way of data com-

IoT applications can be classified into six main categories, such as [1]: smart cities, smart business, smart homes, healthcare, security and surveillance. Regarding these different applications, several requirements should be maintained, like [2]: (1) high scalability, (2) security and privacy, (3) high capacity, (4) security and privacy, (5) energy saving, (6) reduced latency, (7) quality of service (QoS), (8) built-in redundancy, (9) heterogeneity and (10) efficient network and spectrum. The 5G enabled IoT contains a number of key communication techniques for IoT

[12] Freiling G, Jank G. Existence of open-loop Stackelberg equilibria. In: Proceedings of the Mathematical Theory of Networks and Systems, MTNS 98. Symposium held at Padova, Italy; 1998

[13] Freiling G, Jank G, Lee S-R. Openloop Stackelberg equilibria in linear quadratic differential games. Journal of Optimization Theory and Applications. 2001;**110**(3):515-544

[14] Simaan M, Cruz JB. On the Stackelberg strategy in nonzero-sum games. Journal of Optimization Theory and Applications. 1973;**11**(5):533-555

[15] Knobloch HW, Kwakernaak H. Lineare KontrollTheorie. London: Springer Verlag; 1985

[16] Abou-Kandil H, Freiling G, Ionescu V, Jank G. Matrix Riccati Equations in Control and Systems Theory. Basel: Birkhäuser Verlag; 2003

#### **Chapter 12**

**References**

[1] Bagchi A. Differential games in economic models. In: Lecture Notes in Control and Information Sciences. Vol.

*Systems-of-Systems Perspectives and Applications - Design, Modeling, Simulation…*

[11] Kun G. Stabilizability,

2001;**110**(3):515-544

Springer Verlag; 1985

[14] Simaan M, Cruz JB. On the Stackelberg strategy in nonzero-sum games. Journal of Optimization Theory and Applications. 1973;**11**(5):533-555

[15] Knobloch HW, Kwakernaak H. Lineare KontrollTheorie. London:

[16] Abou-Kandil H, Freiling G, Ionescu V, Jank G. Matrix Riccati Equations in Control and Systems Theory. Basel: Birkhäuser Verlag; 2003

controllability and optimal strategies of linear and nonlinear dynamical games [Ph.D. Thesis]. RWTH Aachen; 2000

[12] Freiling G, Jank G. Existence of open-loop Stackelberg equilibria. In: Proceedings of the Mathematical Theory of Networks and Systems, MTNS 98. Symposium held at Padova, Italy; 1998

[13] Freiling G, Jank G, Lee S-R. Openloop Stackelberg equilibria in linear quadratic differential games. Journal of Optimization Theory and Applications.

64. Springer Verlag; 1984

[2] Eisele T. Nonexistence and

Applications. 1971;**33**:96-123

[5] Basar T, Olsder G. Dynamic noncooperative game theory. 2nd ed.

[6] Abou-Kandil H, Bertrand P. Analytical solution for an open-loop Stackelberg game. IEEE Transactions on

Automatic Control. 1985;**AC-30**:

[7] Medanic J, Radojevic D. Multilevel Stackelberg strategies in linear quadratic systems. Journal of Optimization Theory and Applications. 1987;**24**:

[8] Pan L, Yong J. A differential game with multi-level of hierarchy. Journal of

[9] Tolwinski B. A Stackelberg solution of dynamic games. IEEE Transactions on Automatic Control. 1973;**AC-28**:

[10] Jank G, Kun G. Optimal control of disturbed linear quadratic differential games. European Journal of Control.

Mathematical Analysis and Applications. 1991;**161**:522-544

London: SIAM; 1999

1988

1222-1224

485-479

85-93

**198**

2002;**8**(2):152-162

nonuniqueness of open-loop equilibria in linear quadratic differential games. Journal of Optimization Theory and Applications. 1982;**37**:443-468

[3] Lukes DL, Russel DL. A global theory for linear quadratic differential games. Journal of Mathematical Analysis and

[4] Mehlmann A. Applied Differential Games. New York: Springer Verlag;
