**4. Illustrative examples**

In this section, two examples are studied. The first example is for optimizing and controlling a damped harmonic oscillator [7], and the second example is related to optimal control of a continuous stirred-tank chemical reactor [8]. The mathematical models of these examples are discussed, and their optimal solution is obtained by using the algorithm discussed in Section 3. Here, the algorithm is implemented in the Octave 5.1.0 environment.

#### **4.1 Example 1: a damped harmonic oscillator**

Consider a damped harmonic oscillator [7] given by

*Conjugate Gradient Approach for Discrete Time Optimal Control Problems with Model-Reality… DOI: http://dx.doi.org/10.5772/intechopen.89711*

$$
\dot{\mathbf{x}} = \begin{pmatrix} \mathbf{0} & \mathbf{1} \\ -\alpha^2 & -2\delta\alpha \end{pmatrix} \mathbf{x} + \begin{pmatrix} \mathbf{0} \\ \mathbf{1} \end{pmatrix} \mathbf{u} \tag{30}
$$

with the natural frequency *ω* = 0.8, the damping ratio *δ* = 0.1, and the initial state *<sup>x</sup>*<sup>0</sup> <sup>¼</sup> ð Þ 10 10 <sup>T</sup>*:* Define the state *<sup>x</sup>* <sup>¼</sup> ð Þ *<sup>x</sup>*<sup>1</sup> *<sup>x</sup>*<sup>2</sup> T, where *x*<sup>1</sup> is the displacement and *x*<sup>2</sup> is the velocity. For the purpose of controlling this oscillator, the following objective function

$$J(\mathbf{0}) = \frac{1}{2} \int\_{0.0}^{9.4} \left( (\varkappa\_1(t))^2 + (\varkappa\_2(t))^2 + (u(t))^2 \right) dt \tag{31}$$

is minimized. This problem is a continuous-time linear optimal control problem, and the equivalence discrete time optimal control problem, which is regarded as Problem (P), is given by:

$$\min\_{u} J(u) = \sum\_{k=0}^{10} \frac{1}{2} \Delta t \left( \varkappa\_1(k)^2 + \varkappa\_2(k)^2 + u(k)^2 \right) \tag{32}$$

subject to

*<sup>k</sup>* <sup>¼</sup> 0, 1, <sup>⋯</sup>, *<sup>N</sup>* � 1, and *x k*ð Þ<sup>0</sup>

*Control Theory in Engineering*

*:* Step 3: With *<sup>γ</sup>*ð Þ*<sup>k</sup> <sup>i</sup>*

*x k*ð Þ<sup>0</sup> and *p k* ^ð Þ<sup>0</sup> <sup>¼</sup> *p k*ð Þ<sup>0</sup>

*v k*ð Þ*<sup>i</sup>* and *z k*ð Þ*<sup>i</sup>*

method is employed:

and *z k*ð Þ*<sup>i</sup>*þ<sup>1</sup> <sup>¼</sup> *z k*ð Þ*<sup>i</sup>*

**Remark 2:**

satisfied.

established.

**4. Illustrative examples**

the Octave 5.1.0 environment.

**88**

**4.1 Example 1: a damped harmonic oscillator**

Consider a damped harmonic oscillator [7] given by

, *p k*ð Þ<sup>0</sup>

*:* Step 1: Compute the parameters *<sup>γ</sup>*ð Þ*<sup>k</sup> <sup>i</sup>*

, *<sup>α</sup>*ð Þ*<sup>k</sup> <sup>i</sup>*

and repeat the procedure starting from Step 1.

, Γ*<sup>i</sup>* , *<sup>λ</sup>*ð Þ*<sup>k</sup> <sup>i</sup>*

*z k*ð Þ*<sup>i</sup>*þ<sup>1</sup> <sup>¼</sup> *z k*ð Þ*<sup>i</sup>*

where *kv*, *kz*, *kp* <sup>∈</sup>ð � 0, 1 are scalar gains. If *v k*ð Þ*<sup>i</sup>*þ<sup>1</sup> <sup>¼</sup> *v k*ð Þ*<sup>i</sup>*

gain and the Riccati equation are calculated offline.

Step 2: Compute the modifiers Γ*<sup>i</sup>*

0, 1, <sup>⋯</sup>, *<sup>N</sup>* � 1, and using *<sup>r</sup>*1, *<sup>r</sup>*<sup>2</sup> from the data. Set *<sup>i</sup>* <sup>¼</sup> 0, *v k*ð Þ<sup>0</sup> <sup>¼</sup> *u k*ð Þ<sup>0</sup>

0, 1, ⋯, *N* � 1, from (10)–(12). This is called the parameter estimation step.

, *<sup>λ</sup>*ð Þ*<sup>k</sup> <sup>i</sup>*

, *<sup>β</sup>*ð Þ*<sup>k</sup> <sup>i</sup>*

order to provide a mechanism for regulating convergence, a simple relaxation

(15). Notice that this step requires taking the derivatives of *f* and *L* with respect to

using the conjugate gradient algorithm. This is called the system optimization step. Step 4: Test the convergence and update the optimal solution of Problem (P). In

*v k*ð Þ*<sup>i</sup>*þ<sup>1</sup> <sup>¼</sup> *v k*ð Þ*<sup>i</sup>* <sup>þ</sup> *kv u k*ð Þ*<sup>i</sup>* � *v k*ð Þ*<sup>i</sup>*

*p k* ^ð Þ*<sup>i</sup>*þ<sup>1</sup> <sup>¼</sup> *p k* ^ð Þ*<sup>i</sup>* <sup>þ</sup> *kp p k*ð Þ*<sup>i</sup>* � *p k* ^ð Þ*<sup>i</sup>*

a. In Step 0, the nominal solution could be obtained by using the standard procedure of the linear quadratic regulator approach, where the feedback

c. In Step 4, the simple relaxation method in (27)–(29) is used, so that the matching scheme for the parameters and the optimal solution can be

b. In Step 3, applying the conjugate gradient algorithm to obtain the new control sequence will give a good effect if the conjugacy of the search direction is

In this section, two examples are studied. The first example is for optimizing and controlling a damped harmonic oscillator [7], and the second example is related to optimal control of a continuous stirred-tank chemical reactor [8]. The mathematical models of these examples are discussed, and their optimal solution is obtained by using the algorithm discussed in Section 3. Here, the algorithm is implemented in

<sup>þ</sup> *kz x k*ð Þ*<sup>i</sup>*

� *z k*ð Þ*<sup>i</sup>*

, *k* ¼ 0, 1, ⋯, *N*, within a given tolerance, stop; else set *i* ¼ *i* þ 1,

, *k* ¼ 0, 1, ⋯, *N:* Then, with *α*ð Þ¼ *k* 0, *k* ¼

, *<sup>k</sup>* <sup>¼</sup> 0, 1, <sup>⋯</sup>, *<sup>N</sup>*, and *<sup>α</sup>*ð Þ*<sup>k</sup> <sup>i</sup>*

, and *z k*ð Þ*<sup>i</sup>*

and *<sup>β</sup>*ð Þ*<sup>k</sup> <sup>i</sup>*

, *v k*ð Þ*<sup>i</sup>*

, *z k*ð Þ<sup>0</sup> <sup>¼</sup>

(27)

(28)

(29)

, *k* ¼

, solve Problem (Q )

, *k* ¼ 0, 1, ⋯, *N* � 1,

, *k* ¼ 0, 1, ⋯, *N* � 1, from (13)–

$$\boldsymbol{\alpha}(k+1) = \begin{pmatrix} 1.00 & 0.94 \\ -0.60 & 0.85 \end{pmatrix} \boldsymbol{\omega}(k) + \begin{pmatrix} 0.00 \\ 0.94 \end{pmatrix} \boldsymbol{\omega}(k)$$

with the initial state *<sup>x</sup>*<sup>0</sup> <sup>¼</sup> ð Þ 10 10 <sup>T</sup>*:* and the sampling time <sup>Δ</sup>*<sup>t</sup>* <sup>¼</sup> <sup>0</sup>*:*94 s is taken for the discretization transform.

Consider the model-based optimal control problem, which is regarded as Problem (M), given by:

$$\min\_{u} J(u) = \sum\_{k=0}^{10} \left( \frac{1}{2} \left( \varkappa\_1(k)^2 + \varkappa\_2(k)^2 + u(k)^2 \right) + \gamma(k) \right) \Delta t \tag{33}$$

subject to

$$\mathbf{x}(k+1) = \begin{pmatrix} \mathbf{1} & \mathbf{0} \\ \mathbf{0} & \mathbf{1} \end{pmatrix} \mathbf{x}(k) + \begin{pmatrix} \mathbf{1} \\ \mathbf{0} \end{pmatrix} u(k) + a(k)$$

with the initial state *<sup>x</sup>*<sup>0</sup> <sup>¼</sup> ð Þ 10 10 T, and the adjusted parameters *<sup>γ</sup>*ð Þ*<sup>k</sup>* , *k* ¼ 0, 1, ⋯, *N*, and *α*ð Þ*k* , *k* ¼ 0, 1, ⋯, *N* � 1, are supplied to the model used.

By using the algorithm proposed, the simulation result is shown in **Table 1**. Notice that the minimum cost for Problem (M) is 546.05 units without adding the adjusted parameters. Once the adjusted parameters are taken into consideration, the iterative solution approximates to the true optimal solution of the original optimal control problem, in spite of model-reality differences. It is highlighted that there is a 99% of the cost reduction to obtain the final cost of 128.50 units.

**Figures 1** and **2** show the trajectories of control and state, respectively. With this control effort, the state reaches at the steady state after 4 units of time, which presents the oscillator stopped from moving. **Figure 3** shows the changes of the


**Table 1.** *Simulation result, Example 1.*

**Figure 1.** *Final control u(k), Example 1.*

**Figure 4.**

**Figure 5.**

**Figure 6.**

**91**

*Adjusted parameter α(k), Example 1.*

*Adjusted parameter γ(k), Example 1.*

*Stationary Hu(k), Example 1.*

*DOI: http://dx.doi.org/10.5772/intechopen.89711*

*Conjugate Gradient Approach for Discrete Time Optimal Control Problems with Model-Reality…*

**Figure 2.** *Final state x(k), Example 1.*

costate at the first 2 units of time. The optimal solution obtained is verified by satisfying the stationary condition as shown in **Figure 4**. **Figures 5** and **6** show the adjusted parameters after the convergence is achieved, where the model-reality differences are measured during the iterative procedure.

*Conjugate Gradient Approach for Discrete Time Optimal Control Problems with Model-Reality… DOI: http://dx.doi.org/10.5772/intechopen.89711*

**Figure 4.** *Stationary Hu(k), Example 1.*

**Figure 5.** *Adjusted parameter α(k), Example 1.*

**Figure 6.** *Adjusted parameter γ(k), Example 1.*

costate at the first 2 units of time. The optimal solution obtained is verified by satisfying the stationary condition as shown in **Figure 4**. **Figures 5** and **6** show the adjusted parameters after the convergence is achieved, where the model-reality

differences are measured during the iterative procedure.

**Figure 1.**

**Figure 2.**

**Figure 3.**

**90**

*Final costate p(k), Example 1.*

*Final state x(k), Example 1.*

*Final control u(k), Example 1.*

*Control Theory in Engineering*

Therefore, this damped harmonic oscillator is controlled, and the cost function is minimized as desired.

with the initial state *<sup>x</sup>*<sup>0</sup> <sup>¼</sup> ð Þ <sup>0</sup>*:*05 0*:*<sup>00</sup> T, and the adjusted parameters *<sup>γ</sup>*ð Þ*<sup>k</sup>* , *<sup>k</sup>* <sup>¼</sup>

*Conjugate Gradient Approach for Discrete Time Optimal Control Problems with Model-Reality…*

**Table 2** shows the simulation result obtained by using the algorithm proposed. It is mentioned that the minimum cost for the linear model-based optimal control problem is 5.9589 units. At the beginning of the iteration calculation procedure, the initial cost is 0.147463 unit, and a 90% of cost reduction is addressed to give the

The trajectories of the final control and the final state are, respectively, shown in **Figures 7** and **8**. It is noted that the state reaches to the steady state after 40 units of time by associating the control effort taken. This situation indicates that the temperature and the concentration are maintained at their steady state. Thus, the desired objective is confirmed. **Figure 9** shows the costate behavior, which is

**Number of iteration Initial cost Final cost Elapsed time (s)** 9 0.147463 0.014167 4.60934

0, 1, ⋯, *N*, and *α*ð Þ*k* , *k* ¼ 0, 1, ⋯, *N* � 1, are added into the model.

final cost of 0.014167 unit.

*DOI: http://dx.doi.org/10.5772/intechopen.89711*

*Simulation result, Example 2.*

**Table 2.**

**Figure 7.**

**Figure 8.**

**93**

*Final state x(k), Example 2.*

*Final control u(k), Example 2.*

#### **4.2 Example 2: a continuous stirred-tank chemical reactor**

Consider a continuous stirred-tank chemical reactor, which consists of two state equations [8]. The flow of a coolant through a coil inserted in the reactor is to control the first order, irreversible exothermic reaction taking place in the reactor. Assume that *x*1ð Þ*t* is the deviation from the steady-state temperature, *x*2ð Þ*t* is the deviation from the steady-state concentration, and *u t*ð Þ is the normalized control variable that represents the effect of coolant flow on the chemical reaction. The corresponding state equations are given by

$$\dot{\mathbf{x}}\_1(t) = -2(\mathbf{x}\_1(t) + \mathbf{0.25}) + (\mathbf{x}\_2(t) + \mathbf{0.5})\exp\left(\frac{2\mathbf{5x}\_1(t)}{\mathbf{x}\_1(t) + 2}\right) - (\mathbf{x}\_1(t) + \mathbf{0.25})u(t) \tag{34}$$

$$\dot{\varkappa}\_2(t) = 0.5 - \varkappa\_2(t) - (\varkappa\_2(t) + 0.5) \exp\left(\frac{25\varkappa\_1(t)}{\varkappa\_1(t) + 2}\right) \tag{35}$$

with the initial state *<sup>x</sup>*<sup>0</sup> <sup>¼</sup> ð Þ <sup>0</sup>*:*05 0*:*<sup>00</sup> T. The cost function to be minimized is given by

$$J(\mathbf{0}) = \int\_{0.0}^{0.8} \left( (\varkappa\_1(t))^2 + (\varkappa\_2(t))^2 + \mathbf{0}.\mathbf{1}(u(t))^2 \right) dt. \tag{36}$$

Here, the desired objective is to maintain the temperature and the concentration close to their respective steady-state values without expending large amounts of the control effort.

This problem is a continuous time nonlinear optimal control problem. For doing the discretization transform, the sampling time Δ*t* ¼ 0*:*0057 s is used to formulate the equivalence discrete-time optimal control problem, which is referred to as Problem (P), given by:

$$\begin{aligned} \min\_{u} J(u) &= \sum\_{k=0}^{80} \frac{1}{2} \left( 2\mathbf{x}\_{1}(k)^{2} + 2\mathbf{x}\_{2}(k)^{2} + 0.2\mathbf{x}(k)^{2} \right) \Delta t \\ \text{subject to} \\ \mathbf{x}\_{1}(k+1) &= \mathbf{x}\_{1}(k) - 2(\mathbf{x}\_{1}(k) + 0.25)\Delta t + (\mathbf{x}\_{2}(k) + 0.5)\Delta t \exp\left(\frac{25\mathbf{x}\_{1}(k)}{\mathbf{x}\_{1}(k) + 2}\right) \\ &- (\mathbf{x}\_{1}(k) + 0.25)u(k)\Delta t \end{aligned} \tag{37}$$
 
$$\begin{aligned} \mathbf{x}\_{2}(k+1) &= \mathbf{x}\_{2}(k) + (0.5 - \mathbf{x}\_{2}(k))\Delta t - (\mathbf{x}\_{2}(k) + 0.5)\Delta t \exp\left(\frac{25\mathbf{x}\_{1}(k)}{\mathbf{x}\_{1}(k) + 2}\right) \end{aligned}$$

with the initial state *<sup>x</sup>*<sup>0</sup> <sup>¼</sup> ð Þ <sup>0</sup>*:*05 0*:*<sup>00</sup> T.

By applying the algorithm proposed to obtain the optimal solution for Problem (P), the following model, which is referred to as Problem (M), is introduced,

$$\min\_{u} J(u) = \sum\_{k=0}^{80} \frac{1}{2} \left( 2\mathbf{x}\_1(k)^2 + 2\mathbf{x}\_2(k)^2 + \mathbf{0}.2\boldsymbol{\mu}(k)^2 \right) \Delta t \tag{38}$$

subject to

$$\mathbf{x}(k+1) = \begin{pmatrix} 1.048 & 0.010 \\ -0.062 & 0.984 \end{pmatrix} \mathbf{x}(k) + \begin{pmatrix} -0.002 \\ 0.000 \end{pmatrix} \mathbf{u}(k) + \mathbf{a}(k)$$

*Conjugate Gradient Approach for Discrete Time Optimal Control Problems with Model-Reality… DOI: http://dx.doi.org/10.5772/intechopen.89711*

with the initial state *<sup>x</sup>*<sup>0</sup> <sup>¼</sup> ð Þ <sup>0</sup>*:*05 0*:*<sup>00</sup> T, and the adjusted parameters *<sup>γ</sup>*ð Þ*<sup>k</sup>* , *<sup>k</sup>* <sup>¼</sup> 0, 1, ⋯, *N*, and *α*ð Þ*k* , *k* ¼ 0, 1, ⋯, *N* � 1, are added into the model.

**Table 2** shows the simulation result obtained by using the algorithm proposed. It is mentioned that the minimum cost for the linear model-based optimal control problem is 5.9589 units. At the beginning of the iteration calculation procedure, the initial cost is 0.147463 unit, and a 90% of cost reduction is addressed to give the final cost of 0.014167 unit.

The trajectories of the final control and the final state are, respectively, shown in **Figures 7** and **8**. It is noted that the state reaches to the steady state after 40 units of time by associating the control effort taken. This situation indicates that the temperature and the concentration are maintained at their steady state. Thus, the desired objective is confirmed. **Figure 9** shows the costate behavior, which is


**Table 2.**

Therefore, this damped harmonic oscillator is controlled, and the cost function is

Consider a continuous stirred-tank chemical reactor, which consists of two state

25*x*1ð Þ*t x*1ðÞþ*t* 2 � �

> 25*x*1ð Þ*t x*1ð Þþ*t* 2 � �

� ð Þ *x*1ðÞþ*t* 0*:*25 *u t*ð Þ

*dt:* (36)

25*x*1ð Þ*k x*1ð Þþ *k* 2 � �

(37)

(38)

25*x*1ð Þ*k x*1ð Þþ *k* 2 � �

Δ*t*

*u k*ð Þþ *α*ð Þ*k*

(34)

(35)

equations [8]. The flow of a coolant through a coil inserted in the reactor is to control the first order, irreversible exothermic reaction taking place in the reactor. Assume that *x*1ð Þ*t* is the deviation from the steady-state temperature, *x*2ð Þ*t* is the deviation from the steady-state concentration, and *u t*ð Þ is the normalized control variable that represents the effect of coolant flow on the chemical reaction. The

*x*\_2ðÞ¼ *t* 0*:*5 � *x*2ðÞ�*t* ð Þ *x*2ðÞþ*t* 0*:*5 exp

<sup>2</sup>*x*1ð Þ*<sup>k</sup>* <sup>2</sup> <sup>þ</sup> <sup>2</sup>*x*2ð Þ*<sup>k</sup>* <sup>2</sup> <sup>þ</sup> <sup>0</sup>*:*2*u k*ð Þ<sup>2</sup> � �

*x*1ð Þ¼ *k* þ 1 *x*1ð Þ� *k* 2ð Þ *x*1ð Þþ *k* 0*:*25 Δ*t* þ ð Þ *x*2ð Þþ *k* 0*:*5 Δ*t*exp

*x*2ð Þ¼ *k* þ 1 *x*2ð Þþ *k* ð Þ 0*:*5 � *x*2ð Þ*k* Δ*t* � ð Þ *x*2ð Þþ *k* 0*:*5 Δ*t*exp

�ð Þ *x*1ð Þþ *k* 0*:*25 *u k*ð ÞΔ*t*Þ

with the initial state *<sup>x</sup>*<sup>0</sup> <sup>¼</sup> ð Þ <sup>0</sup>*:*05 0*:*<sup>00</sup> T.

80

1 2

1*:*048 0*:*010 �0*:*062 0*:*984

!

*k*¼0

min*<sup>u</sup> J u*ð Þ¼ <sup>X</sup>

with the initial state *<sup>x</sup>*<sup>0</sup> <sup>¼</sup> ð Þ <sup>0</sup>*:*05 0*:*<sup>00</sup> T. The cost function to be minimized is

ð Þ *<sup>x</sup>*1ð Þ*<sup>t</sup>* <sup>2</sup> <sup>þ</sup> ð Þ *<sup>x</sup>*2ð Þ*<sup>t</sup>* <sup>2</sup> <sup>þ</sup> <sup>0</sup>*:*1ð Þ *u t*ð Þ <sup>2</sup> � �

Here, the desired objective is to maintain the temperature and the concentration close to their respective steady-state values without expending large amounts of the

This problem is a continuous time nonlinear optimal control problem. For doing the discretization transform, the sampling time Δ*t* ¼ 0*:*0057 s is used to formulate the equivalence discrete-time optimal control problem, which is referred to as

By applying the algorithm proposed to obtain the optimal solution for Problem

<sup>2</sup>*x*1ð Þ*<sup>k</sup>* <sup>2</sup> <sup>þ</sup> <sup>2</sup>*x*2ð Þ*<sup>k</sup>* <sup>2</sup> <sup>þ</sup> <sup>0</sup>*:*2*u k*ð Þ<sup>2</sup> � �

*x k*ð Þþ �0*:*<sup>002</sup>

0*:*000

!

(P), the following model, which is referred to as Problem (M), is introduced,

Δ*t*

**4.2 Example 2: a continuous stirred-tank chemical reactor**

corresponding state equations are given by

*J*ð Þ¼ 0

ð<sup>0</sup>*:*<sup>8</sup> 0*:*0

*x*\_1ðÞ¼� *t* 2ð Þþ *x*1ð Þþ*t* 0*:*25 ð Þ *x*2ðÞþ*t* 0*:*5 exp

minimized as desired.

*Control Theory in Engineering*

given by

control effort.

Problem (P), given by:

80

1 2

*k*¼0

min*<sup>u</sup> J u*ð Þ¼ <sup>X</sup>

subject to

subject to

**92**

*x k*ð Þ¼ þ 1

*Simulation result, Example 2.*

**Figure 7.** *Final control u(k), Example 2.*

**Figure 8.** *Final state x(k), Example 2.*

**Figure 9.** *Final costate p(k), Example 2.*

reduced gradually to zero at the terminal time, and **Figure 10** shows the stationary condition, which examines the existing of the optimal solution. The adjusted parameters, which are shown in **Figures 11** and **12**, respectively, measure the

*Conjugate Gradient Approach for Discrete Time Optimal Control Problems with Model-Reality…*

Hence, the correct optimal solution of Problem (P) is approximated successfully by solving the model in Problem (M), and the efficiency of the algorithm proposed

The approach, which integrates system optimization and parameter estimation, was discussed in this chapter. The use of the conjugate gradient method in solving the model-based optimal control problem has been examined, and the applicability of the conjugate gradient approach in associating the principle of model-reality differences was identified. Definitely, many computational approaches could be used to solve the model-based optimal control; however, the algorithm proposed in this chapter gives a tractable solution procedure for handling the optimal control problems with different structures and parameters, especially for obtaining the optimal solution for the nonlinear optimal control problem. In conclusion, the efficiency of the algorithm is highly recommended. In future research, it is strongly suggested to investigate the application of optimization techniques in stochastic

The authors would like to acknowledge the Universiti Tun Hussein Onn Malaysia (UTHM) and the Ministry of Higher Education (MOHE) for the financial sup-

port for this study under the research grant FRGS VOT. 1561.

The authors declare no conflict of interest.

differences between the model used and the real plant.

is demonstrated.

**Figure 12.**

**5. Concluding remarks**

*Adjusted parameter γ(k), Example 2.*

*DOI: http://dx.doi.org/10.5772/intechopen.89711*

optimization and control.

**Acknowledgements**

**Conflict of interest**

**95**

**Figure 10.** *Stationary Hu(k), Example 2.*

**Figure 11.** *Adjusted parameter α(k), Example 2.*

*Conjugate Gradient Approach for Discrete Time Optimal Control Problems with Model-Reality… DOI: http://dx.doi.org/10.5772/intechopen.89711*

**Figure 9.**

**Figure 10.**

**Figure 11.**

**94**

*Adjusted parameter α(k), Example 2.*

*Stationary Hu(k), Example 2.*

*Final costate p(k), Example 2.*

*Control Theory in Engineering*

reduced gradually to zero at the terminal time, and **Figure 10** shows the stationary condition, which examines the existing of the optimal solution. The adjusted parameters, which are shown in **Figures 11** and **12**, respectively, measure the differences between the model used and the real plant.

Hence, the correct optimal solution of Problem (P) is approximated successfully by solving the model in Problem (M), and the efficiency of the algorithm proposed is demonstrated.

## **5. Concluding remarks**

The approach, which integrates system optimization and parameter estimation, was discussed in this chapter. The use of the conjugate gradient method in solving the model-based optimal control problem has been examined, and the applicability of the conjugate gradient approach in associating the principle of model-reality differences was identified. Definitely, many computational approaches could be used to solve the model-based optimal control; however, the algorithm proposed in this chapter gives a tractable solution procedure for handling the optimal control problems with different structures and parameters, especially for obtaining the optimal solution for the nonlinear optimal control problem. In conclusion, the efficiency of the algorithm is highly recommended. In future research, it is strongly suggested to investigate the application of optimization techniques in stochastic optimization and control.

### **Acknowledgements**

The authors would like to acknowledge the Universiti Tun Hussein Onn Malaysia (UTHM) and the Ministry of Higher Education (MOHE) for the financial support for this study under the research grant FRGS VOT. 1561.

### **Conflict of interest**

The authors declare no conflict of interest.

*Control Theory in Engineering*
