**3. Implementation of the closed-loop control of anaesthesia**

The main elements that constitute the control system are depicted in figure 5. As can be observed there is a computer that centralizes the monitoring and control task in the system. The BIS monitor is a passive analyser of EEG, that allows monitor the deep of anaesthesia, and has the first objective of adjust in real time the dose of drugs administered to one patient to the actual need. The BIS correlated well with the level of responsiveness and provided an excellent prediction of the level of sedation and loss of consciousness for propofol and midazolam. In this work the Aspect® A-2000 monitor was used. The communication with the computer was implemented via a RS-232 serial interface. Concerning the actuator, the Graseby® infusion pump was used for drug infusion in the patient. The pump is also governed via a RS-232 serial interface.

Fig. 5. Main elements of the closed-loop control system.

Apart from sending commands to the pump, the program in the PC reads continuously its state to detect eventual failures of any of the elements in the control loop, like missing BIS

Closed-Loop Control of Anaesthetic Effect 461

0 1000 2000 3000 4000 5000 6000 7000 8000

0 1000 2000 3000 4000 5000 6000 7000 8000

t(sec.)

0 2000 4000 6000 8000 10000 12000 14000 16000 18000

0 2000 4000 6000 8000 10000 12000 14000 16000 18000

t(sec.)

t(sec.)

t(sec.)

0

20

0

0

20

40

Bis

60

80

10

20

Infusion Rate(mg/Kg/h)

30

Fig. 6. Results of anaesthesia automatic control on patient 1.

Fig. 7. Results of anaesthesia automatic control on patient 2.

40

Bis

60

80

5

10

Infusion Rate mg/Kg/h

15

signal, excessive infusion rate in the pump syringe changes, etc. The program in the computer has all the routines to monitor and control the system.

The goal is to make a manual induction with propofol and remifentanil and maintain the BIS target during the maintenance of anaesthesia. Remifentanil infusion was adjusted manually and rocuronium was administered in bolus as needs. The operation conditions and the population in which the study was performed are explained in next section.

#### **4. PI control**

First control algorithm implemented was a PI controller. This algorithm has been extensively used in several automated closed loop systems. The administration of the drug is made based in the error between the BIS target and the current BIS, and the accumulated error during the operation. Both actions are regulated by gains, which are adjusted in an empirical way trying to get a smooth transitory and a stable response of the patient.

The goal is to make a manual induction with propofol and remifentanil and maintain a BIS target during the maintenance of anaesthesia. Reminfentanil infusion was adjusted manually and rocuronium was administered in bolus as needs. In the real proofs, a BIS target (BISr) of 50 is considered while the measurement and actuation period is 5 seconds. Before starting its operation, the software checks that all the security alarms are programmed.

This study was approved by the Ethical and Research Committee of the Hospital Universitario de Canarias and has written informed consent of the patients. The study was performed on a population of 15 patients of 30-60 years. In the real proofs with patients, a BIS target of 50 is considered while the measurement and actuation period is 5 seconds. Before starting its operation, the software checks that all the security alarms are programmed. In the operating room, the patient was connected to the BIS monitor, and the anaesthesia system was started in monitor mode. After the patient had breathed 100% oxygen for 3 min, the system was switched to manual mode, and anaesthesia was induced by means of nearly 2mg/Kg propofol manual bolus. Once the patient achieves a BIS closed to 50, the system is switched to automatic and the interest control algorithm is responsible to regulate the BIS around the objective.

The adjustment of the controller gains was made in an empirical way trying to get a smooth transitory and a stable response. This task was done following standard procedures in online process control engineering. For this, it was necessary the presence of a control expert together with the anaesthesiologist in the operating theatre. Thus, after several trials adequate values for PI controller where found to be *Kp*=0.67, *Ki*=0.055. This set of values was tested in the whole population of the study with satisfactory results. Figures 6 and 7 present the evolution of the anaesthesia for two different patients: patient 1 and patient 2. As can be observed, in both cases the system remains stabilised around the reference value with an oscillation of near 10 units in the worst case (patient 2).

The results obtained with the population submitted to proofs show the patient remains stabilised around the reference value with an oscillation of near 10 units in the worst case.

signal, excessive infusion rate in the pump syringe changes, etc. The program in the

The goal is to make a manual induction with propofol and remifentanil and maintain the BIS target during the maintenance of anaesthesia. Remifentanil infusion was adjusted manually and rocuronium was administered in bolus as needs. The operation conditions

First control algorithm implemented was a PI controller. This algorithm has been extensively used in several automated closed loop systems. The administration of the drug is made based in the error between the BIS target and the current BIS, and the accumulated error during the operation. Both actions are regulated by gains, which are adjusted in an

The goal is to make a manual induction with propofol and remifentanil and maintain a BIS target during the maintenance of anaesthesia. Reminfentanil infusion was adjusted manually and rocuronium was administered in bolus as needs. In the real proofs, a BIS target (BISr) of 50 is considered while the measurement and actuation period is 5 seconds. Before starting its operation, the software checks that all the security alarms are

This study was approved by the Ethical and Research Committee of the Hospital Universitario de Canarias and has written informed consent of the patients. The study was performed on a population of 15 patients of 30-60 years. In the real proofs with patients, a BIS target of 50 is considered while the measurement and actuation period is 5 seconds. Before starting its operation, the software checks that all the security alarms are programmed. In the operating room, the patient was connected to the BIS monitor, and the anaesthesia system was started in monitor mode. After the patient had breathed 100% oxygen for 3 min, the system was switched to manual mode, and anaesthesia was induced by means of nearly 2mg/Kg propofol manual bolus. Once the patient achieves a BIS closed to 50, the system is switched to automatic and the interest control algorithm is responsible to

The adjustment of the controller gains was made in an empirical way trying to get a smooth transitory and a stable response. This task was done following standard procedures in online process control engineering. For this, it was necessary the presence of a control expert together with the anaesthesiologist in the operating theatre. Thus, after several trials adequate values for PI controller where found to be *Kp*=0.67, *Ki*=0.055. This set of values was tested in the whole population of the study with satisfactory results. Figures 6 and 7 present the evolution of the anaesthesia for two different patients: patient 1 and patient 2. As can be observed, in both cases the system remains stabilised around the reference value with

The results obtained with the population submitted to proofs show the patient remains stabilised around the reference value with an oscillation of near 10 units in the worst

and the population in which the study was performed are explained in next section.

empirical way trying to get a smooth transitory and a stable response of the patient.

computer has all the routines to monitor and control the system.

**4. PI control** 

programmed.

case.

regulate the BIS around the objective.

an oscillation of near 10 units in the worst case (patient 2).

Fig. 6. Results of anaesthesia automatic control on patient 1.

Fig. 7. Results of anaesthesia automatic control on patient 2.

Closed-Loop Control of Anaesthetic Effect 463

In order to make the simulations proofs of the proposed algorithm, a physiology model of the patient dynamics was designed. As it was told, the model has two parts: pharmacokinetics and pharmacodynamics. The parameters adjustment was made in simulation using the real results obtained from a female, 56 years old patient, 84 Kg. weight,

After obtaining a satisfactory manual adjustment, the values for the pharmacokinetics model are k10=0.006, k12=11.0; k21=14.04, k13=10.02, k31=283.50 and ke0=0.0063. The values for the pharmacodynamics model are EC50=610.0, =1.5, BIS0 =100 and BISmax=0.

To validate the model the simulated response is compared with the real one. The obtained

0 500 1000 1500 2000 2500 3000 3500

0 500 1000 1500 2000 2500 3000 3500

Time (sec.)

Fig. 9. a) Simulated BIS output (dotted) and real patient BIS output (solid) obtained under the action of a PI controller. b) PI controlled output (dotted) and PI with delay-time

The proposal here is to improve the performance of the closed-loop system by means of a compensation of the system time-delay. The origin of this time-delay is the period of time between from the infusion pump starts until the drug is distributed along the central compartment. The majority of the works in the literature do not explicitly consider the presence of this time-delay in the proposed models. In fact, in model equations (1)-(4), timedelay is not considered. But in real proofs, some delays between 1 and 2 minutes have to be considered to have a realistic model of the dynamics. Under this hypothesis, a time-delay

Time (sec.)

results, shown in figure 9a), prove the goodness of the model.

**5.1 Model adjustment** 

30

compensation output (solid).

**5.2 Smith predictor (dead-time compensation)** 

BIS

40

50

BIS

60

70

160 cm. height.

The study revealed that although results are satisfactory, eventually the performance of the controller could decrease. There are two main factors that contribute to this. First of all, the variability between patients implies that the nominal PI parameters chosen are not the best choice for all the patients. Together with this, the dead time present in the system also contributes to reduce the phase margin of the closed loop system. The origin of this timedelay is the period of time since the drug is infused until it causes the adequate effect in the patient. The main effect observed is that the evolution of the BIS is quite oscillatory (see figure 7) around the reference value.

#### **5. Dead-time compensation**

In previous section it can be viewed that PI controller usually gives a response with oscillations around the BIS reference value. In this section, the control algorithm is modified in order to compensate these oscillations and get a better transitory. The results shown in this paper are in simulation after having adjusted the patient dynamical model.

The first method implemented to improve the results obtained with the fixed PI controller is to compensate the dead-time present in the system. To do this, a dead-time compensator based on the Smith Predictor theory (Smith, 72) is proposed to act with the PI controller. The basis of the Smith Predictor is to consider the feedback of the controlled variable BIS without delay. As this is variable is not available, the predictor estimates this value and uses this estimation as the feedback signal. To correct the deviations between this estimation and the real value, a correction term, resulting from the error between the estimation and the measured BIS, is added to the feedback signal, as can be seen in the figure 8.

Fig. 8. PI Controller with Smith Predictor for patient hypnosis control.

The basics of this compensation algorithm consider the formulation of the Smith Predictor for linear systems. To apply the Smith Predictor to the nonlinear model of the patient, a firstorder plus a time-delay approximation of the patient model is considered. A delay between 90 and 120 seconds is considered.

#### **5.1 Model adjustment**

462 Pharmacology

The study revealed that although results are satisfactory, eventually the performance of the controller could decrease. There are two main factors that contribute to this. First of all, the variability between patients implies that the nominal PI parameters chosen are not the best choice for all the patients. Together with this, the dead time present in the system also contributes to reduce the phase margin of the closed loop system. The origin of this timedelay is the period of time since the drug is infused until it causes the adequate effect in the patient. The main effect observed is that the evolution of the BIS is quite oscillatory (see

In previous section it can be viewed that PI controller usually gives a response with oscillations around the BIS reference value. In this section, the control algorithm is modified in order to compensate these oscillations and get a better transitory. The results shown in

The first method implemented to improve the results obtained with the fixed PI controller is to compensate the dead-time present in the system. To do this, a dead-time compensator based on the Smith Predictor theory (Smith, 72) is proposed to act with the PI controller. The basis of the Smith Predictor is to consider the feedback of the controlled variable BIS without delay. As this is variable is not available, the predictor estimates this value and uses this estimation as the feedback signal. To correct the deviations between this estimation and the real value, a correction term, resulting from the error between the estimation and the

> Infusion pump

> > BIS Monitor BIS(t)

> > > +

Delay \_

**Patient** 

+

+

BISr(t) U(t) Hypnosis

this paper are in simulation after having adjusted the patient dynamical model.

measured BIS, is added to the feedback signal, as can be seen in the figure 8.

PI Controller

+ \_

Fig. 8. PI Controller with Smith Predictor for patient hypnosis control.

90 and 120 seconds is considered.

The basics of this compensation algorithm consider the formulation of the Smith Predictor for linear systems. To apply the Smith Predictor to the nonlinear model of the patient, a firstorder plus a time-delay approximation of the patient model is considered. A delay between

First-order approximation

figure 7) around the reference value.

**5. Dead-time compensation** 

In order to make the simulations proofs of the proposed algorithm, a physiology model of the patient dynamics was designed. As it was told, the model has two parts: pharmacokinetics and pharmacodynamics. The parameters adjustment was made in simulation using the real results obtained from a female, 56 years old patient, 84 Kg. weight, 160 cm. height.

After obtaining a satisfactory manual adjustment, the values for the pharmacokinetics model are k10=0.006, k12=11.0; k21=14.04, k13=10.02, k31=283.50 and ke0=0.0063. The values for the pharmacodynamics model are EC50=610.0, =1.5, BIS0 =100 and BISmax=0.

To validate the model the simulated response is compared with the real one. The obtained results, shown in figure 9a), prove the goodness of the model.

Fig. 9. a) Simulated BIS output (dotted) and real patient BIS output (solid) obtained under the action of a PI controller. b) PI controlled output (dotted) and PI with delay-time compensation output (solid).

#### **5.2 Smith predictor (dead-time compensation)**

The proposal here is to improve the performance of the closed-loop system by means of a compensation of the system time-delay. The origin of this time-delay is the period of time between from the infusion pump starts until the drug is distributed along the central compartment. The majority of the works in the literature do not explicitly consider the presence of this time-delay in the proposed models. In fact, in model equations (1)-(4), timedelay is not considered. But in real proofs, some delays between 1 and 2 minutes have to be considered to have a realistic model of the dynamics. Under this hypothesis, a time-delay

Closed-Loop Control of Anaesthetic Effect 465

Following this control scheme, the controller parameters are adjusted by an adaptation law that depends on the error between the system output (BIS) and the model reference output defined for this closed-loop. Minimising a certain cost-function involving this error, an adaptation law of the adjustable controller parameters is obtained. In this case, the adjustable parameters are the static gain and the time constant of the approximated first-

Several simulation experiments has been made for the patient simulated in previous sections, choosing as the reference model a second-order model with poles, expressed in the

The results are shown in figure 11a), where the evolution of the BIS under the self-adaptive compensator algorithm is drawn in solid line and compared with the results obtained in figure 9b) –PI and PI+compensator controllers-. In figure 11b) the evolution of the static gain under this self-adaptive scheme is shown. As it can be observed, some extra oscillations are produced with respect to the previous algorithm, which corresponds to the period of time in

<sup>0</sup> <sup>500</sup> <sup>1000</sup> <sup>1500</sup> <sup>2000</sup> <sup>2500</sup> <sup>3000</sup> <sup>3500</sup> <sup>40</sup>

Time (sec.)

<sup>0</sup> <sup>500</sup> <sup>1000</sup> <sup>1500</sup> <sup>2000</sup> <sup>2500</sup> <sup>3000</sup> <sup>3500</sup> -20

Time (sec.)

Fig. 11. a) PI controlled system output (dotted), PI with Smith Predictor controlled output (dashed), and self-adaptive time-delay compensation controlled output (solid) compared for the same patient. b) Evolution of the static gain of the first-order approximation model of

**5.5 Results with the PI controller with self-adaptive dead-time compensation** 

order model used in the Smith Predictor.

that the parameter is adapting.

45


the time-delay compensation.


Static gain of the first-order model


0

<sup>5</sup> x 10-3

50

BIS

55

60

65

z-plane discrete formulation, in z=0.98 and z=-0.75.

compensator based on the Smith Predictor theory has been proposed to be added to the PI controller.

#### **5.3 Results with PI controller with dead-time compensation**

As it is well known, the basics of this compensation algorithm consider the formulation of the Smith Predictor for linear systems. To apply the Smith Predictor to the nonlinear model of the patient, a first-order plus a time-delay approximation of the patient model is considered. Thus, the configuration employed can be seen in figure 8. A delay between 90 and 120 seconds is considered in the simulations. In figure 9b the results obtained with the patient are shown. The evolution of the BIS signal with the Smith Predictor (in solid line) is much better than with the PI controller (in dotted line), and does not show oscillations around the reference BIS value.

#### **5.4 Self-adaptive dead-time compensation**

The main advantage of the time-delay compensation for the PI controller is a better performance in the transitory of the BIS signal. This advantage is conditioned to obtain a good fist-order approximation of the system. However, this model has to be changed in at least two situations. First, when the operation point changes due to a change in the BIS reference for the same patient. Second, when the controller is applied in a different patient, whose physiologic model has to be estimated.

In order to improve the efficiency of that controller, an adaptation of the first-order model patient is added. The aim of this algorithm is to make the time-delay compensator independent of the model assumed for the patient. In order to obtain a simple adaptive algorithm that guarantees the closed-loop stability, model reference adaptive controller – MRAC- (Aström and Wittenmark, 94) is used, as can be seen in figure 10.

Fig. 10. PI with Smith Predictor controller inserted in the adaptive control scheme MRAC. The error between the system output and the model reference output is used to update the parameters of the PI with Smith Predictor controller.

compensator based on the Smith Predictor theory has been proposed to be added to the PI

As it is well known, the basics of this compensation algorithm consider the formulation of the Smith Predictor for linear systems. To apply the Smith Predictor to the nonlinear model of the patient, a first-order plus a time-delay approximation of the patient model is considered. Thus, the configuration employed can be seen in figure 8. A delay between 90 and 120 seconds is considered in the simulations. In figure 9b the results obtained with the patient are shown. The evolution of the BIS signal with the Smith Predictor (in solid line) is much better than with the PI controller (in dotted line), and does not show oscillations

The main advantage of the time-delay compensation for the PI controller is a better performance in the transitory of the BIS signal. This advantage is conditioned to obtain a good fist-order approximation of the system. However, this model has to be changed in at least two situations. First, when the operation point changes due to a change in the BIS reference for the same patient. Second, when the controller is applied in a different patient,

In order to improve the efficiency of that controller, an adaptation of the first-order model patient is added. The aim of this algorithm is to make the time-delay compensator independent of the model assumed for the patient. In order to obtain a simple adaptive algorithm that guarantees the closed-loop stability, model reference adaptive controller –

> Infusion pump

Hypnosis

BIS(t)

BIS Monitor

\_

**Patient** 

+

Fig. 10. PI with Smith Predictor controller inserted in the adaptive control scheme MRAC. The error between the system output and the model reference output is used to update the

Reference Model

MRAC- (Aström and Wittenmark, 94) is used, as can be seen in figure 10.

**5.3 Results with PI controller with dead-time compensation** 

controller.

around the reference BIS value.

**5.4 Self-adaptive dead-time compensation** 

whose physiologic model has to be estimated.

+ \_

BISr(t) U(t)

PI + Smith Predictor Controller

parameters of the PI with Smith Predictor controller.

Following this control scheme, the controller parameters are adjusted by an adaptation law that depends on the error between the system output (BIS) and the model reference output defined for this closed-loop. Minimising a certain cost-function involving this error, an adaptation law of the adjustable controller parameters is obtained. In this case, the adjustable parameters are the static gain and the time constant of the approximated firstorder model used in the Smith Predictor.

#### **5.5 Results with the PI controller with self-adaptive dead-time compensation**

Several simulation experiments has been made for the patient simulated in previous sections, choosing as the reference model a second-order model with poles, expressed in the z-plane discrete formulation, in z=0.98 and z=-0.75.

The results are shown in figure 11a), where the evolution of the BIS under the self-adaptive compensator algorithm is drawn in solid line and compared with the results obtained in figure 9b) –PI and PI+compensator controllers-. In figure 11b) the evolution of the static gain under this self-adaptive scheme is shown. As it can be observed, some extra oscillations are produced with respect to the previous algorithm, which corresponds to the period of time in that the parameter is adapting.

Fig. 11. a) PI controlled system output (dotted), PI with Smith Predictor controlled output (dashed), and self-adaptive time-delay compensation controlled output (solid) compared for the same patient. b) Evolution of the static gain of the first-order approximation model of the time-delay compensation.

Closed-Loop Control of Anaesthetic Effect 467

To obtain the dynamics of the patient, a stochastic recursive least-squares method is employed. It consists of an observer of the discrete system that tries to minimize the

> 1 <sup>2</sup> ( ) <sup>2</sup> *J k*

where *ε(k)* is the residual error between the output of the system, *BIS(k)*, and the observed

Consider the following first order model as an approximation of the patient dynamics, with *u(k)* being the drug infusion rate of applied to the patient, and *BIS(k)* the BIS value obtained:

> () () *<sup>T</sup> y k k*

( ) ( 1) ( 1) *<sup>T</sup>*

 1 1 *T*

In each instant of time, the values of output and input of the system are measured. The

( ) ( ) ( ) ( 1) *<sup>T</sup> ek yk k k* 

( ) ( 1) ( ) ( ) ( ) *k k W k kek*

( ) *<sup>c</sup> W k*

The control law considered for the system is obtained by applying a PI controller. This controller has two gains to adjust: the proportional gain, Kp, and the integer gain, Ki. The

> <sup>1</sup> ( ) <sup>1</sup> *<sup>c</sup> <sup>z</sup> G k*

 

 

1

*z*

iterative process to obtain the best parameters

patient is given by the equations:

where:

being *c* a constant.

**6.2 Pole placement controller** 

discrete version of this controller is given by:

(9)

(11)

that reproduce the performance of the

(14)

1 1 *BIS k a BIS k b u k* ( ) ( 1) ( 1) (10)

*k BIS k u k* (12)

*a b* (13)

(15)

*<sup>k</sup>* (16)

(17)

**6.1 Patient dynamics identification** 

model, being *k* the instant of time.

following cost function:

It can be expressed by:

Once the optimal values for the parameters are reached, the performance of the system is very similar to the previous controller. In that case, the static gain took a value of -0.037. In this case, the stabilising value for this parameter is near the half. However, the performance is also satisfactory. Moreover, no assumptions over the system had to been made, which is the main advantage of this new algorithm.

Comparing the results obtained with the previous algorithm, some extra oscillations are produced, which corresponds to the period of time in that the parameter is adapting. Once the optimal values for the parameters are reached, the performance of the system is very similar to the previous controller. Moreover, no assumptions over the system had to been made, which is the main advantage of this control algorithm.

## **6. PI control with self-adaptive gains**

Another method to avoid the problem of the oscillations around the BIS reference value occurred with the PI controller is using an adaptive scheme to obtain the gains of the PI controller. The method is based on assuring a desired performance of the closed loop, by emplacing its poles as required, in order to obtain a smooth transition to the BIS reference value.

This method is part of a kind of adaptive controllers, known as self-tuning regulators –STR- (Aström and Wittenmark, 94). They are based on two steps. First, an identification of the system is made in order to get the dynamics in each instant of time. This assures that the controller takes into account the variations in the dynamics of the system. Second step is to compute the control law assuming that the identification results are true. The variety of STR schemes differs in the method used to compute the control law. In this work, an adjustable PI controller is used, which gains are tuned by trying that the closed loop performance of the system be as similar as possible to a desired reference model. This is obtained by emplacing the poles of the reference model into the desired values to get a satisfactory response of the system.

STR controllers have two different parts. The most important part is that they provide an observer in order to identify the dynamics of the system to control. This is an essential point due to the results of the system identification are taken into account to compute the adaptive control law. In this case, a recursive least-squares method based is used to provide the parameters of the dynamics of the observed system.

The second part of the controller is to compute the control law to apply to the system. The results of the identification part are taken to obtain the control action. In this way, the controller takes into account the variations of the dynamics of the system, if they occur. Another advantage of this scheme is its flexibility to be applied to different systems, with the same dynamics but with different parameters, because the identification process adapts the controller to the new situation.

In this case, a pole placement controller is used. Considering the dynamics of the patient as a first order system, and using a PI controller with adaptive gains, the closed loop performance results in a second order system, which dynamics are set by emplacing its poles to the desired values for the closed loop system. Next subsections describe these two parts of the proposed controller.

Once the optimal values for the parameters are reached, the performance of the system is very similar to the previous controller. In that case, the static gain took a value of -0.037. In this case, the stabilising value for this parameter is near the half. However, the performance is also satisfactory. Moreover, no assumptions over the system had to been made, which is

Comparing the results obtained with the previous algorithm, some extra oscillations are produced, which corresponds to the period of time in that the parameter is adapting. Once the optimal values for the parameters are reached, the performance of the system is very similar to the previous controller. Moreover, no assumptions over the system had to been

Another method to avoid the problem of the oscillations around the BIS reference value occurred with the PI controller is using an adaptive scheme to obtain the gains of the PI controller. The method is based on assuring a desired performance of the closed loop, by emplacing its poles as required, in order to obtain a smooth transition to the BIS reference

This method is part of a kind of adaptive controllers, known as self-tuning regulators –STR- (Aström and Wittenmark, 94). They are based on two steps. First, an identification of the system is made in order to get the dynamics in each instant of time. This assures that the controller takes into account the variations in the dynamics of the system. Second step is to compute the control law assuming that the identification results are true. The variety of STR schemes differs in the method used to compute the control law. In this work, an adjustable PI controller is used, which gains are tuned by trying that the closed loop performance of the system be as similar as possible to a desired reference model. This is obtained by emplacing the poles of the reference model into the desired values to get a satisfactory

STR controllers have two different parts. The most important part is that they provide an observer in order to identify the dynamics of the system to control. This is an essential point due to the results of the system identification are taken into account to compute the adaptive control law. In this case, a recursive least-squares method based is used to provide the

The second part of the controller is to compute the control law to apply to the system. The results of the identification part are taken to obtain the control action. In this way, the controller takes into account the variations of the dynamics of the system, if they occur. Another advantage of this scheme is its flexibility to be applied to different systems, with the same dynamics but with different parameters, because the identification process adapts

In this case, a pole placement controller is used. Considering the dynamics of the patient as a first order system, and using a PI controller with adaptive gains, the closed loop performance results in a second order system, which dynamics are set by emplacing its poles to the desired values for the closed loop system. Next subsections describe these two

the main advantage of this new algorithm.

**6. PI control with self-adaptive gains** 

parameters of the dynamics of the observed system.

the controller to the new situation.

parts of the proposed controller.

value.

response of the system.

made, which is the main advantage of this control algorithm.

#### **6.1 Patient dynamics identification**

To obtain the dynamics of the patient, a stochastic recursive least-squares method is employed. It consists of an observer of the discrete system that tries to minimize the following cost function:

$$J = \frac{1}{2}\varepsilon^2(k)\tag{9}$$

where *ε(k)* is the residual error between the output of the system, *BIS(k)*, and the observed model, being *k* the instant of time.

Consider the following first order model as an approximation of the patient dynamics, with *u(k)* being the drug infusion rate of applied to the patient, and *BIS(k)* the BIS value obtained:

$$BIS(k) + a\_1 BIS(k-1) = b\_1 \mu(k-1) \tag{10}$$

It can be expressed by:

$$
\hat{y}(k) = \boldsymbol{\phi}^T(k)\boldsymbol{\theta} \tag{11}
$$

$$\boldsymbol{\varphi}(k) = \begin{bmatrix} -BIS(k-1) & \boldsymbol{\mu}(k-1) \end{bmatrix}^T \tag{12}$$

$$\boldsymbol{\theta} = \begin{bmatrix} \boldsymbol{a}\_1 & \boldsymbol{b}\_1 \end{bmatrix}^T \tag{13}$$

In each instant of time, the values of output and input of the system are measured. The iterative process to obtain the best parameters that reproduce the performance of the patient is given by the equations:

$$e(k) = y(k) - \boldsymbol{\phi}^T(k)\boldsymbol{\theta}(k-1)\tag{14}$$

$$
\theta(k) = \theta(k-1) + \mathcal{W}(k)\phi(k)e(k) \tag{15}
$$

where:

$$\mathcal{W}(k) = \frac{c}{k} \tag{16}$$

being *c* a constant.

#### **6.2 Pole placement controller**

The control law considered for the system is obtained by applying a PI controller. This controller has two gains to adjust: the proportional gain, Kp, and the integer gain, Ki. The discrete version of this controller is given by:

$$\mathcal{G}\_c(k) = \frac{\alpha + \beta z^{-1}}{1 - z^{-1}} \tag{17}$$

to 0.005.

Closed-Loop Control of Anaesthetic Effect 469

For the second-order reference model, the poles are located in 0.997±0.0027j that correspond to a system with natural frequency 0.05 rad/sec and delta coefficient 0.75. For initiating the identification algorithm (15), the parameters used in (26) are chosen. *c* constant in (16) is set

Figure 12 shows the evolution of the BIS and the parameters of the PI controller in this case. As it can be seen, after the application of the initial bolus, the PI controller varies its gains and the patient remains its degree of freedom around the desired value of 50, but with oscillations.

Fig. 12. Results of the PI self-adaptive controller. First graph shows the BIS evolution with

Fig. 13. Results of the compensated self-adaptive PI controller. First graph shows the BIS evolution with respect to BISr and second graph are the gains evolution of the PI controller.

respect to BISr and second graph are the gains evolution of the PI controller.

*BIS k BIS k* ( ) 0.812 ( 1) 25.389 ( 1) *u k* (26)

where the relation with the gains is given by:

$$\alpha = \mathbf{K}\_p + \mathbf{K}\_i \frac{T}{2} \tag{18}$$

$$
\beta = -K\_p + K\_i \frac{T}{2} \tag{19}
$$

being *T* the sampling time of the system. Considering for the patient the model (10) and for the controller the expression (17), the poles of the closed loop are given by the roots of the following polynomial:

$$D(z^{-1}) = (1 + a\_1 z^{-1})(1 - z^{-1}) + b\_1 z^{-1} \left(a + \beta z^{-1}\right) \tag{20}$$

Consider the following specification for the closed-loop:

$$Q(z^{-1}) = (1 - q\_1 z^{-1})(1 - q\_2 z^{-1})\tag{21}$$

where *q1* and *q2* are the location of the desired poles for the system. It is easy to obtain from (20) and (21) the resulted gains of the PI controller:

$$\alpha = \frac{-1}{b\_1} (1 + q\_1 + q\_2 + a\_1) \tag{22}$$

$$
\beta = \frac{1}{b\_1} (q\_1 q\_2 + a\_1) \tag{23}
$$

The parameters of the system are obtained in each instant of time form the identification process (15). With the values (22) and (23), the gains of the PI controller are obtained, using (18) and (19), by:

$$K\_p = \frac{\alpha - \beta}{2} \tag{24}$$

$$K\_i = \frac{\alpha + \beta}{T} \tag{25}$$

#### **6.3 Results with the PI self-adaptive controller**

The results shown here correspond to a female of 40 years old, 70 Kg. weight and 170 cm height. The Schnider model presented in section 2 gives an accurate response with the real values obtained for the patient, where the following parameters for (6)-(8) equations are chosen: BIS0=95, BISmax=8.9, EC50=4.94 μg/ml and γ=2.69. Initially, the patient is infused with a bolus of 1.45 mg/Kg during 2 minutes in order to carry the patient near the desired value for the degree of hypnosis. The reference value is BISr=50. After that, the compensated adaptive controller starts controlling the system.

To define the controller, the following assumptions are considered. First, the model for the patient is chosen as:

2 *p i T*

2 *p i T*

 1 111 1 1 1 *Dz az z bz z* ( ) (1 )(1 )

1 11

<sup>121</sup>

12 1

*q q a*

2

The parameters of the system are obtained in each instant of time form the identification process (15). With the values (22) and (23), the gains of the PI controller are obtained, using

> 

*Ki <sup>T</sup>* 

The results shown here correspond to a female of 40 years old, 70 Kg. weight and 170 cm height. The Schnider model presented in section 2 gives an accurate response with the real values obtained for the patient, where the following parameters for (6)-(8) equations are chosen: BIS0=95, BISmax=8.9, EC50=4.94 μg/ml and γ=2.69. Initially, the patient is infused with a bolus of 1.45 mg/Kg during 2 minutes in order to carry the patient near the desired value for the degree of hypnosis. The reference value is BISr=50. After that, the compensated

To define the controller, the following assumptions are considered. First, the model for the

<sup>1</sup> <sup>1</sup> *qqa <sup>b</sup>*

where *q1* and *q2* are the location of the desired poles for the system. It is easy to obtain from

1

1 1

*b* 

*Kp*

being *T* the sampling time of the system. Considering for the patient the model (10) and for the controller the expression (17), the poles of the closed loop are given by the roots of the

*K K* (18)

*K K* (19)

 (20)

1 2 *Q z*( ) (1 )(1 ) *<sup>q</sup> <sup>z</sup> <sup>q</sup> <sup>z</sup>* (21)

(22)

(23)

(24)

(25)

where the relation with the gains is given by:

Consider the following specification for the closed-loop:

(20) and (21) the resulted gains of the PI controller:

**6.3 Results with the PI self-adaptive controller** 

adaptive controller starts controlling the system.

following polynomial:

(18) and (19), by:

patient is chosen as:

$$BIS(k) - 0.812BIS(k-1) = -25.389u(k-1)\tag{26}$$

For the second-order reference model, the poles are located in 0.997±0.0027j that correspond to a system with natural frequency 0.05 rad/sec and delta coefficient 0.75. For initiating the identification algorithm (15), the parameters used in (26) are chosen. *c* constant in (16) is set to 0.005.

Figure 12 shows the evolution of the BIS and the parameters of the PI controller in this case. As it can be seen, after the application of the initial bolus, the PI controller varies its gains and the patient remains its degree of freedom around the desired value of 50, but with oscillations.

Fig. 12. Results of the PI self-adaptive controller. First graph shows the BIS evolution with respect to BISr and second graph are the gains evolution of the PI controller.

Fig. 13. Results of the compensated self-adaptive PI controller. First graph shows the BIS evolution with respect to BISr and second graph are the gains evolution of the PI controller.

Closed-Loop Control of Anaesthetic Effect 471

50

0 max 0 *BIS BIS <sup>r</sup> BIS BIS*

 

 

<sup>1</sup> exp ln <sup>1</sup> *Cen EC*

Finally, to compute the nominal input *un* and the nominal state *<sup>n</sup> x* equation (29) is solved

1 '

11 12 13 1 21 12 23 2 31 32 33 3 41 42 43 4

*A AAB A AAB*

*A AAB A AAB*

*A C A C*

In Fig. 15, a simulation of the BIS in a patient with only this nominal input (*un*) is presented. As can be observed, the BIS tends to the nominal value (BISr=50) if only this input is applied. In practice, several considerations have to be taken into account. The first one is related to the modeling errors in the patient dynamics. In the simulation presented here no modeling errors were considered. In the real implementation, a deviation of the response of the system with respect to this ideal trajectory will be observed. On the other hand, as can be observed, the response exhibits a sluggish behavior that in real practice is undesirable. That is why this action is complemented with an additional term that tries to correct the deviations of the system from the nominal trajectory and also improves the transient

*A C A C*

*en en en en*

*M*

*N*

*T*

*n n n n en x CCCC* , the

0 *Ax Bu n n* (29)

(30)

(31)

*<sup>n</sup> x MN* (32)

(33)

(34)

Assuming that the target state is an equilibrium state 123

nominal input can be obtained by solving:

to obtain the following solution:

where 123 ' *<sup>T</sup>*

response of the BIS curve.

*n n n nn x CCCu* and:

with:

*Cen* is computed by using the EMAX model (6), (7) and (8):

To avoid these oscillations in the stationary, the compensation of the time delay by means of a Smith predictor is done in this case. To do that, the time delay considered is 1 minute. Figure 13 shows the results obtained in this case. The oscillations around the objective value are considerably reduced and the performance of the closed loop systems results better than in previous case.

#### **7. Model Predictive Control (MPC)**

As an alternative to signal based controllers proposed before, it is shown an algorithm that uses explicitly the model of the patient to compute the drug infusion rate. The objective is to improve the performance of other techniques as those based in PI controllers. Figure 14 shows the structure of the proposed controller. As can be observed, the drug infusion rate is computed as a sum of two terms:

$$
\mu(t) = \mu\_n + \delta\mu \tag{27}
$$

The first term (*un*) is obtained by inverting the model of the patient and is computed to take the BIS variable to the nominal value (BIS*r*=50, *x*n). That is, from the target BIS (BIS*r*) and using the model of the patient (PK+PD) it can be obtained the infusion rate that leads the BIS signal to the desired value. To do this, the inverse dynamics of the system model is evaluated, assuming that the system is approximated by (1)-(5). Taking matrix notation for this model, it can be expressed by:

$$
\dot{\mathfrak{x}} = A\mathfrak{x} + Bu \tag{28}
$$

$$\text{where } \mathfrak{x} = \begin{bmatrix} \mathsf{C}\_1 & \mathsf{C}\_2 & \mathsf{C}\_3 & \mathsf{C}\_e \end{bmatrix}^T.$$

Fig. 14. Structure of the proposed model-based predictive controller.

Assuming that the target state is an equilibrium state 123 *T n n n n en x CCCC* , the nominal input can be obtained by solving:

$$A\mathbf{x}\_n + B\boldsymbol{u}\_n = \mathbf{0} \tag{29}$$

*Cen* is computed by using the EMAX model (6), (7) and (8):

$$C\_{en} = \exp\left(\frac{1}{\mathcal{I}} \ln\left(EC\_{50}^{\mathcal{I}} \frac{a}{1-a}\right)\right) \tag{30}$$

with:

470 Pharmacology

To avoid these oscillations in the stationary, the compensation of the time delay by means of a Smith predictor is done in this case. To do that, the time delay considered is 1 minute. Figure 13 shows the results obtained in this case. The oscillations around the objective value are considerably reduced and the performance of the closed loop systems results better than

As an alternative to signal based controllers proposed before, it is shown an algorithm that uses explicitly the model of the patient to compute the drug infusion rate. The objective is to improve the performance of other techniques as those based in PI controllers. Figure 14 shows the structure of the proposed controller. As can be observed, the drug infusion rate is

> ( ) *ut u u <sup>n</sup>*

The first term (*un*) is obtained by inverting the model of the patient and is computed to take the BIS variable to the nominal value (BIS*r*=50, *x*n). That is, from the target BIS (BIS*r*) and using the model of the patient (PK+PD) it can be obtained the infusion rate that leads the BIS signal to the desired value. To do this, the inverse dynamics of the system model is evaluated, assuming that the system is approximated by (1)-(5). Taking matrix notation for

Predictive

BIS

\_ <sup>+</sup>

Propofol

*u* J Constraints

controller A,B

(27)

*x Ax Bu* (28)

Linear Model Identification

infusion rate BIS

in previous case.

**7. Model Predictive Control (MPC)** 

computed as a sum of two terms:

this model, it can be expressed by:

*<sup>e</sup> xCCCC* .

Inverse Dynamics

> Schnider Model

*T*

Fig. 14. Structure of the proposed model-based predictive controller.

+

*un* 

+

BIS*r* **Patient** 

where <sup>123</sup>

$$\alpha = \frac{BIS\_r - BIS\_0}{BIS\_{\text{max}} - BIS\_0} \tag{31}$$

Finally, to compute the nominal input *un* and the nominal state *<sup>n</sup> x* equation (29) is solved to obtain the following solution:

$$\mathbf{x}'\_n = M^{-1}N\tag{32}$$

where 123 ' *<sup>T</sup> n n n nn x CCCu* and:

$$M = \begin{bmatrix} A\_{11} & A\_{12} & A\_{13} & B\_1 \\ A\_{21} & A\_{12} & A\_{23} & B\_2 \\ A\_{31} & A\_{32} & A\_{33} & B\_3 \\ A\_{41} & A\_{42} & A\_{43} & B\_4 \end{bmatrix} \tag{33}$$

$$N = -\begin{bmatrix} A\_{14} \mathbf{C}\_{en} \\ A\_{24} \mathbf{C}\_{en} \\ A\_{34} \mathbf{C}\_{en} \\ A\_{44} \mathbf{C}\_{en} \end{bmatrix} \tag{34}$$

In Fig. 15, a simulation of the BIS in a patient with only this nominal input (*un*) is presented.

As can be observed, the BIS tends to the nominal value (BISr=50) if only this input is applied. In practice, several considerations have to be taken into account. The first one is related to the modeling errors in the patient dynamics. In the simulation presented here no modeling errors were considered. In the real implementation, a deviation of the response of the system with respect to this ideal trajectory will be observed. On the other hand, as can be observed, the response exhibits a sluggish behavior that in real practice is undesirable. That is why this action is complemented with an additional term that tries to correct the deviations of the system from the nominal trajectory and also improves the transient response of the BIS curve.

Closed-Loop Control of Anaesthetic Effect 473

0 10 20 30 40 50 60 70 80 90 100

0 10 20 30 40 50 60

Time (min.)

Fig. 16. Simulation of the model-based controller of figure 14 on a patient. Nominal input

In this chapter both results on modelling and control of patients under general anaesthesia with propofol is presented. First results presented refer to the synthesis of linear models for use in model-based controllers. The anaesthetic process was segmented into several phases, according to the state of the surgery (consciousness, hypnosis, intubation, incision, etc.). The propofol infusion rate in ml/h was used as the input variable *u(t)*, while the BIS represented

For simulation purposes a PK/PD model based on compartmental approaches as obtained. The model was adjusted using information of real data from patients. The obtained model

Concerning hypnosis control, this chapter presents a review of the state of the art of the closed-loop control of anaesthesia. Then, a description of approaches based on signal

The chapter proposed an advanced PI controller with several important features. First, an adaptive module is included that adapts the controller to the specific patient behaviour. On the other hand, the controller incorporates a dead-time compensation system that improves notably the performance of the controller. The performance of this compensation is

the output. Validation of the proposed model was done with real data patients.

was used to simulate the response of the patients with the different controllers.

(dashed line) and nominal+predictive controller (solid line).

feedback and model based controllers are presented.

30

40

50

BIS

**8. Conclusion** 

60

70

Inf. rate (mg/Kg/h)

Fig. 15. Simulation of the BIS in a patient applying only the nominal input *un*.

This action is computed considering that the deviations of the BIS from the nominal value can be described by a linear approximation:

$$
\delta \delta \dot{\mathbf{x}} = A \delta \mathbf{x} + B \delta u \tag{35}
$$

$$
\delta BIS = \mathbf{C} \delta \mathbf{x} + D \delta \mathbf{u} \tag{36}
$$

where *<sup>n</sup> xxx* and *BIS BIS BISr* .

Then the control law *uk* can be obtained from an optimization problem (model based predictive controller). The problem can be formulated as obtaining the control law *uk* so that a specified cost function is minimized under a receding horizon strategy. Consider for example the following index:

$$f\_k = \sum\_{j=1\_1}^{N} \gamma(j) \left[ w(k+j) - \hat{y}(k+j \mid t) \right]^2 + \sum\_{j=1}^{N\mathcal{U}} \mathcal{\lambda}(j) \left[ \delta u(k+j-1) \right]^2 \tag{37}$$

where *N* is the prediction horizon and *NU* is the control horizon. The problem is to find the sequence *uk+j* so that *Jk* is minimized. Assuming a receding horizon strategy, only the first value of the sequence is applied and the procedure is repeated at *k*+1. In this optimization, constraints can also be included, although the computational complexity is greatly increased.

Figure 16 shows a simulation of this strategy. Initial condition was BIS=39. As can be observed, the response of the system is now much faster than that observed in figure 14 and achieves a very acceptable error quite soon.

0 50 100 150 200 250

 

> 

*uk+j* so that *Jk* is minimized. Assuming a receding horizon strategy, only the first

*uk* can be obtained from an optimization problem (model based

 

(37)

2 2

(35)

(36)

*uk* so

t(min.)

This action is computed considering that the deviations of the BIS from the nominal value

 *x Ax Bu* 

*BIS C x D u* 

that a specified cost function is minimized under a receding horizon strategy. Consider for

( ) ( ) ( | ) ( ) ( 1) <sup>ˆ</sup> *<sup>N</sup> NU*

where *N* is the prediction horizon and *NU* is the control horizon. The problem is to find the

value of the sequence is applied and the procedure is repeated at *k*+1. In this optimization, constraints can also be included, although the computational complexity is greatly

Figure 16 shows a simulation of this strategy. Initial condition was BIS=39. As can be observed, the response of the system is now much faster than that observed in figure 14 and

predictive controller). The problem can be formulated as obtaining the control law

*J j wk j yk j t j uk j*

1 1

*j j*

Fig. 15. Simulation of the BIS in a patient applying only the nominal input *un*.

.

30

where *<sup>n</sup>* 

sequence

increased.

Then the control law

example the following index:

can be described by a linear approximation:

 *xxx* and *BIS BIS BISr* 

1

*k*

achieves a very acceptable error quite soon.

40

50

60

BIS

70

80

90

100

Fig. 16. Simulation of the model-based controller of figure 14 on a patient. Nominal input (dashed line) and nominal+predictive controller (solid line).
