**4.2. A decision on beliefs approach**

**•** Positive shifts in the process mean

146 Dynamic Programming and Bayesian Inference, Concepts and Applications

Pr{*xk* |*Out* −*of* −*control*}, is calculated using equation (63).

kind of belief function to ascertain good properties for *B* <sup>+</sup>

*k*

=

Also is Pr{*xk* |*Out* −*of* −*control*} calculated using equation (66).

(*Ok* ) is determined by the following equation,


*k*

*B O*

*B O*

**•** Negative shifts in the process mean

(*Ok* ) is determined by the following equation,

+ -

( ) ( ) ( )

j

+

(*Ok* ), showing the probability of occurring a positive shift in the process mean,

Pr{ } 0.5 *<sup>k</sup> x In control* - = (62)

. Above probabilities are not exact probabilities and they are a

(*Ok* )

(63)

+ - (64)

(66)


will be calculated applying equation (61) recursively. Pr{*xk* | *In* −*control*} is defined by the

For positive shift, the probability of being a positive shift in the process at iteration *k*,

where *φ*(*xk* ) is the cumulative probability distribution function for the normal distribution

( ) ( ) ( ( )) <sup>1</sup> 1 1 0.5 1 *k k*

j

(*Ok* ) denotes the probability of being a negative shift in the process mean that

j

( )( ( )) ( ( )) <sup>1</sup> 1 1

j

*kk k*

1 1 0.5 1 *k k*

*BO x*

*BO x BO*


j

Pr{ } 0.5 *<sup>k</sup> x In control* - = (65)

*k k k*

*BO x*

is calculated using equation (61) recursively. In this case, Pr{*xk* | *In* −*control*} is defined by the

Pr{ } 1 ( ) *k k x Out of control x* - - =-

( ) ( )( ( ))



*BO x BO*

+ + - -

j

Pr{ } ( ) *k k x Out of control x* -- =

The values of *B* <sup>+</sup>

following equation,

with mean μ and variance σ<sup>2</sup>

Therefore *B* <sup>+</sup>

The values of *B* <sup>−</sup>

Thus *B* <sup>−</sup>

following equation,

We present a decision making approach in terms of Stochastic Dynamic Programming approach. Presented approach is like an optimal stopping problem.

Suppose n stages for decision making is remained and two decisions are available.


Decision making framework is as follows:


Suppose new observation *xk* is gathered. (*k* is the number of gathered observations so far). *V* (*n*, *d* +(*n*)) is defined as the probability of correct selection when *n* decision making stages are remained and we follow *d* +(*n*) strategy explained above also *V* (*n*) denotes the maximum value of *V* (*n*, *d* +(*n*)) thus,

$$V\left(n\right) = \max\_{d^+\left(n\right)} \left\{ V\left(n, d^+\left(n\right)\right) \right\} \tag{68}$$

CS is defined as the event of correct selection. *S1* is defined as selecting the out-of-control condition (positive shift) as an optimal solution and *S2* is defined as selecting the in-control condition as an optimal decision and *NS* is defined as not selecting any candidate in this stage.

Hence, using the total probability law, it is concluded that:

( ( )) 11 22 , x{Pr{ }} Pr{ }Pr{ } Pr{ }Pr{ } Pr{ }Pr{ } *V n d n Ma CS CS S S CS S S CS NS NS* <sup>+</sup> == + + (69)

Pr{*CS* |*S*1} denotes the probability of correct selection when candidate *S1* is selected as the optimal candidate and this probability equals to its belief, *B* <sup>+</sup> (*Ok* ), and with the same discus‐ sion, it is concluded that Pr{*CS* |*S*2} =1− *B* <sup>+</sup> (*Ok* )

Pr{*S*1} is the probability of selecting out of control candidate (positive shift) as the solution thus following the decision making strategy, we should have *B* <sup>+</sup> (*Ok* ) =max(*<sup>B</sup>* <sup>+</sup> (*Ok* ), <sup>1</sup><sup>−</sup> *<sup>B</sup>* <sup>+</sup> (*Ok* )) and *B* + (*Ok* ) <sup>&</sup>gt;*<sup>d</sup>* +(*n*) that is equivalent to following,

$$\Pr\left\{\mathcal{S}\_{1}\right\} = \Pr\left\{\mathcal{B}^{+}\left(\mathcal{O}\_{k}\right) > d^{+}\left(n\right)\right\}, d^{+}\left(n\right) \in \left[0.5, 1\right] \tag{70}$$

Calculation method for *V* (*n*, *d* +(*n*)) :

Now equation (73) is rewritten as follows:

(*sm*, *Ok* ) are defined as follows:

( ) { ( ) ( )} ( ) { ( ) ( )}

( ( )) ( ( )) { ( )} ( ( )) { ( )} ( )

+ + + +

*k k*


= -- > +

( , ) 1 Pr ( , ) 1

, ( , ) 1 Pr ( , )

*V n d n B gr O V n B gr O d n B sm O V n B sm O d n V n* a

*k k*

(*gr*, *Ok* ) <sup>−</sup>*α<sup>V</sup>* (*<sup>n</sup>* <sup>−</sup>1) and *<sup>B</sup>* <sup>+</sup>

(*gr*, *Ok* ) <sup>−</sup>*α<sup>V</sup>* (*<sup>n</sup>* <sup>−</sup>1) and *<sup>B</sup>* <sup>+</sup>

(*gr*, *Ok* ) as the solution.

negative coefficient, to maximize *V* (*n*, *d* +(*n*)), optimality methods should be applied.

( ( )) ( )( ( ))

1

(*Ok* )>*<sup>d</sup>* +(*n*)} is determined as follows:

=

1

+ + - + + +

*dn BO*


( ( )) ( )

*d nBO*

1 1



*k k*

In this condition, one of the probabilities in equation (10) has positive coefficient and one has

should have *d* +(*n*)=1 in order to maximize *V* (*n*, *d* +(*n*)). Since *B* <sup>+</sup>

select any candidate in this condition and sampling process continues.

should have *d* +(*n*)=0.5 in order to maximize *V* (*n*, *d* +(*n*)). since *B* <sup>+</sup>

(*gr*, *Ok* )

*hd n*

+ + +

a

*k kk k kk*

<sup>=</sup> - (74)

Using Dynamic Programming Based on Bayesian Inference in Selection Problems

(*sm*, *Ok* ) −*αV* (*n* −1) are negative, thus we

(*sm*, *Ok* ) −*αV* (*n* −1) are positive, thus we

(*gr*, *Ok* ) <sup>&</sup>lt;*<sup>d</sup>* +(*n*)=1, we don't

http://dx.doi.org/10.5772/57423

(*gr*, *Ok* ) <sup>&</sup>gt;*<sup>d</sup>* +(*n*)=0.5, we

(75)

149

 a

, max ,1 , min ,1

+ ++ - ++ = -

*B gr O B O B O B gr O B O B O*

(*gr*, *Ok* ) and *<sup>B</sup>* <sup>+</sup>

There are three conditions:

In this condition, both *B* <sup>+</sup>

In this condition, both *B* <sup>+</sup>

First the value of Pr{*B* <sup>+</sup>

(*gr*, *Ok* ) <*αV* (*n* −1)

(*sm*, *Ok* ) >*αV* (*n* −1)

select the candidate of belief *B* <sup>+</sup>

(*sm*, *Ok* ) <sup>&</sup>lt;*α<sup>V</sup>* (*<sup>n</sup>* <sup>−</sup>1)< *<sup>B</sup>* <sup>+</sup>

**•** *Definition:h* (*d* +(*n*)) is defined as follows:

**1.** *B* <sup>+</sup>

**2.** *B* <sup>+</sup>

**3.** *B* <sup>+</sup>

*B* +

With the same reasoning, it is concluded that,

$$\Pr\left\{S\_{2}\right\} = \Pr\left\{1 - \mathcal{B}^{+}\left(O\_{k}\right) > d^{+}\left(n\right)\right\}, d^{+}\left(n\right) \in \left[0.5, 1\right] \tag{71}$$


By the above preliminaries, the function *V* (*n*) is determined as follows:

( ) ( ) { ( )} { ( )} ( { ( )} { ( )}) ( ) { ( )} ( ) { ( )} ( { ( )} { ( )}) 0.5 1 0.5 1 ( )Pr ( ) (1 ( ))Pr 1 ( ) max Pr{ } 1 Pr ( ) Pr 1 ( ) ( )Pr ( ) 1 ( ) Pr 1 ( ) max ( 1) 1 Pr ( ) Pr 1 ( ) *k k k k d n k k k k k k d n k k V n BO BO d n BO BO d n CS NS B O d n B O d n B O B O d n B O B O dn* a*Vn B O d n B O d n* + + + ++ + ++ ++ ++ < < + ++ + + ++ ++ < < = é ù > +- - > ê ú + - > -- > ë û <sup>é</sup> > +- - > <sup>ê</sup> <sup>=</sup> <sup>ê</sup> + -- > - - > <sup>ë</sup> ù ú <sup>ú</sup> ê úû (72)

In terms of above equation, *V* (*n*, *d* +(*n*)) is obtained as follows:

$$V\left(n,d^{+}\left(n\right)\right) = \begin{bmatrix} \mathcal{B}^{+}\left(\mathcal{O}\_{k}\right)\Pr\left\{\mathcal{B}^{+}\left(\mathcal{O}\_{k}\right) > d^{+}\left(n\right)\right\} + \left(1 - \mathcal{B}^{+}\left(\mathcal{O}\_{k}\right)\right)\Pr\left\{\left(1 - \mathcal{B}^{+}\left(\mathcal{O}\_{k}\right)\right) > d^{+}\left(n\right)\right\} \\\\ + \alpha V\left(n - 1\right)\left(1 - \Pr\left\{\mathcal{B}^{+}\left(\mathcal{O}\_{k}\right) > d^{+}\left(n\right)\right\} - \Pr\left\{1 - \mathcal{B}^{+}\left(\mathcal{O}\_{k}\right) > d^{+}\left(n\right)\right\}\right) \end{bmatrix} \tag{73}$$

Calculation method for *V* (*n*, *d* +(*n*)) :

Pr{*CS* |*S*1} denotes the probability of correct selection when candidate *S1* is selected as the

(*Ok* )

Pr{*S*1} is the probability of selecting out of control candidate (positive shift) as the solution thus

{ } { ( ) ( )} ( ) <sup>1</sup> Pr Pr , 0.5,1 *<sup>k</sup> S BO dn dn* + ++ => Î é ù

{ } { ( ) ( )} ( ) <sup>2</sup> Pr Pr 1 , 0.5,1 *<sup>k</sup> S BO dn dn* + ++ =- > Î é ù

**1.** Pr{*CS* | *NS*} denotes the probability of correct selection when none of candidates has been selected and it means that the maximum value of the beliefs is less than *d* +(*n*) and the process of decision making continues to latter stage. As a result, in terms of Dynamic Programming Approach, the probability of this event equals to maximum of probability of correct selection in latter stage *(n-1), V* (*n* −1), but since taking observations has cost, then the value of this probability in current time is less than its actual value and by using

**2.** Since the entire solution space is partitioned, it is concluded that

( )Pr ( ) (1 ( ))Pr 1 ( )

*k k k k*

*BO BO d n BO BO d n CS NS B O d n B O d n*

é ù > +- - > ê ú

+ ++ + ++

( )Pr ( ) 1 ( ) Pr 1 ( )

*k k k k*

*B O B O d n B O B O dn*

*Vn B O d n B O d n*

<sup>ú</sup> ê úû

( )( { ( )} { ( )})

ë û

*BO BO d n BO BO d n*

é ù > +- - > ê ú <sup>=</sup>

+ ++ + + +

*k k*

++ ++

( 1) 1 Pr ( ) Pr 1 ( )

( ( )) { ( )} ( ) { ( )}

+ -- > - - >

( )Pr ( ) 1 ( ) Pr (1 ( ))

1 1 Pr ( ) Pr 1 ( ) *k k k k*

*Vn B O d n B O d n*

Pr{ } 1 Pr ( ) Pr 1 ( )

+ - > -- > ë û

+ ++ + +

*d n k k*

++ ++ < <

<sup>é</sup> > +- - > <sup>ê</sup> <sup>=</sup> <sup>ê</sup> + -- > - - >

*d n k k*

In terms of above equation, *V* (*n*, *d* +(*n*)) is obtained as follows:

++ ++ < <

{ ( )} { ( )} ( { ( )} { ( )})

{ ( )} ( ) { ( )} ( { ( )} { ( )})

(*Ok* ), and with the same discus‐

(*Ok* ), <sup>1</sup><sup>−</sup> *<sup>B</sup>* <sup>+</sup>

ë û (70)

ë û (71)

ù ú (72)

(73)

(*Ok* )) and

(*Ok* ) =max(*<sup>B</sup>* <sup>+</sup>

optimal candidate and this probability equals to its belief, *B* <sup>+</sup>

148 Dynamic Programming and Bayesian Inference, Concepts and Applications

following the decision making strategy, we should have *B* <sup>+</sup>

sion, it is concluded that Pr{*CS* |*S*2} =1− *B* <sup>+</sup>

(*Ok* ) <sup>&</sup>gt;*<sup>d</sup>* +(*n*) that is equivalent to following,

With the same reasoning, it is concluded that,

the discounting factor α, it equals *αV* (*n* −1)

By the above preliminaries, the function *V* (*n*) is determined as follows:

Pr{*CS* | *NS*} =1−(Pr{*S*1} + Pr{*S*2})

( )

*V n*

,

*V nd n*

+

( )

=

0.5 1

+

max

( )

a

a

ë

0.5 1

+

max

*B* +

*B* + (*gr*, *Ok* ) and *<sup>B</sup>* <sup>+</sup> (*sm*, *Ok* ) are defined as follows:

$$\begin{aligned} \mathcal{B}^{+}\left(\mathcal{g}r,\mathcal{O}\_{k}\right) &= \max\left\{\mathcal{B}^{+}\left(\mathcal{O}\_{k}\right), \mathbf{1} - \mathcal{B}^{+}\left(\mathcal{O}\_{k}\right)\right\} \\ \mathcal{B}^{-}\left(\mathcal{g}r,\mathcal{O}\_{k}\right) &= \min\left\{\mathcal{B}^{+}\left(\mathcal{O}\_{k}\right), \mathbf{1} - \mathcal{B}^{+}\left(\mathcal{O}\_{k}\right)\right\} \end{aligned} \tag{74}$$

Now equation (73) is rewritten as follows:

$$\begin{aligned} V\left(n, d^+\left(n\right)\right) &= \left(B^+\left(gr, O\_k\right) - \alpha V\left(n-1\right)\right) \Pr\left\{B^+\left(gr, O\_k\right) > d^+\left(n\right)\right\} + \\ \left(B^+\left(sm, O\_k\right) - \alpha V\left(n-1\right)\right) \Pr\left\{B^+\left(sm, O\_k\right) > d^+\left(n\right)\right\} + \alpha V\left(n-1\right) \end{aligned} \tag{75}$$

There are three conditions:

**1.** *B* <sup>+</sup> (*gr*, *Ok* ) <*αV* (*n* −1)

In this condition, both *B* <sup>+</sup> (*gr*, *Ok* ) <sup>−</sup>*α<sup>V</sup>* (*<sup>n</sup>* <sup>−</sup>1) and *<sup>B</sup>* <sup>+</sup> (*sm*, *Ok* ) −*αV* (*n* −1) are negative, thus we should have *d* +(*n*)=1 in order to maximize *V* (*n*, *d* +(*n*)). Since *B* <sup>+</sup> (*gr*, *Ok* ) <sup>&</sup>lt;*<sup>d</sup>* +(*n*)=1, we don't select any candidate in this condition and sampling process continues.

$$\mathbf{2.} \quad \mathcal{B}^\*(\text{sm}, \ O\_k) \succeq \alpha V(n-1)$$

In this condition, both *B* <sup>+</sup> (*gr*, *Ok* ) <sup>−</sup>*α<sup>V</sup>* (*<sup>n</sup>* <sup>−</sup>1) and *<sup>B</sup>* <sup>+</sup> (*sm*, *Ok* ) −*αV* (*n* −1) are positive, thus we should have *d* +(*n*)=0.5 in order to maximize *V* (*n*, *d* +(*n*)). since *B* <sup>+</sup> (*gr*, *Ok* ) <sup>&</sup>gt;*<sup>d</sup>* +(*n*)=0.5, we select the candidate of belief *B* <sup>+</sup> (*gr*, *Ok* ) as the solution.

**3.** *B* <sup>+</sup> (*sm*, *Ok* ) <sup>&</sup>lt;*α<sup>V</sup>* (*<sup>n</sup>* <sup>−</sup>1)< *<sup>B</sup>* <sup>+</sup> (*gr*, *Ok* )

In this condition, one of the probabilities in equation (10) has positive coefficient and one has negative coefficient, to maximize *V* (*n*, *d* +(*n*)), optimality methods should be applied.

**•** *Definition:h* (*d* +(*n*)) is defined as follows:

$$h\left(d^+\left(n\right)\right) = \frac{d^+\left(n\right)\left(1 - B^+\left(O\_{k-1}\right)\right)}{\left(1 - d^+\left(n\right)\right)B^+\left(O\_{k-1}\right)}\tag{76}$$

First the value of Pr{*B* <sup>+</sup> (*Ok* )>*<sup>d</sup>* +(*n*)} is determined as follows:

$$\Pr\left\{\mathcal{B}^+(O\_k) > d^+\left(n\right)\right\} = \Pr\left\{\frac{\rho\left(\mathbf{x}\_k\right)\mathcal{B}^+\left(O\_{k-1}\right)}{\rho\left(\mathbf{x}\_k\right)\mathcal{B}^+\left(O\_{k-1}\right) + \left(1 - \mathcal{B}^+\left(O\_{k-1}\right)\right)0.5}\right\} = \begin{cases} \frac{\rho\left(\mathbf{x}\_k\right)\mathcal{B}^+\left(O\_{k-1}\right)}{\rho\left(\mathbf{x}\_k\right)\mathcal{B}^+\left(O\_{k-1}\right) + \left(1 - \mathcal{B}^+\left(O\_{k-1}\right)\right)0.5} \left(\mathbf{x}\_k\right) \\ > d^+\left(n\right) \end{cases} \tag{77}$$
 
$$\Pr\left\{\rho\left(\mathbf{x}\_k\right) > h\left(d^+\left(n\right)\right) 0.5\right\} = 1 - 0.5h\left(d^+\left(n\right)\right)$$

The optimal threshold *d* +(*n*) is determined by the above equation. Since the optimal value of *d* +(*n*) should be in the interval [0.5, 1] thus it is concluded that the optimal value of *d* +(*n*) will

> ( ( ) ( )) ( ( ) ( ))

*B O Vn B O Vn* a

+ +

be adapted for detecting the negative shifts with the same reasoning.

(*Ok* )) =1<sup>−</sup> *<sup>B</sup>* <sup>+</sup>

(*Ok* )) =1<sup>−</sup> *<sup>B</sup>* <sup>−</sup>

The general decision making algorithm is summarized as follows:

1 1 *k k*

The above method is presented for detecting the positive shifts in the process mean and can

(*O*0) =0.5, *<sup>B</sup>* <sup>−</sup>

**3.** If *n* <0, then no shift is occurred in the process mean and decision making stops.

(*Ok* ), *<sup>B</sup>* <sup>+</sup>

(*Ok* )) <sup>&</sup>gt;*α<sup>V</sup>* (*<sup>n</sup>* <sup>−</sup>1), then if *Max*(*<sup>B</sup>* <sup>+</sup>

(*Ok* )) <sup>&</sup>gt;*α<sup>V</sup>* (*<sup>n</sup>* <sup>−</sup>1) then if *Max*(*<sup>B</sup>* <sup>−</sup>

(*Ok* )) <sup>&</sup>gt;*α<sup>V</sup>* (*<sup>n</sup>* <sup>−</sup>1)>*Min*(*<sup>B</sup>* <sup>+</sup>

concluded that a positive shift is occurred in the process mean and decision making stops,

shift and go to stage 2 after checking the occurrence of negative shift in the rest of the

cluded that a negative shift is occurred the process mean and decision making stops and

of *d* +(*n*) (minimum acceptable belief for detecting the positive shift) by the following

a


é ù ê ú

> <sup>1</sup> ,0.5 1

> > (*O*0) =0.5.

1

Using Dynamic Programming Based on Bayesian Inference in Selection Problems

(*Ok* ) by equation (61).

(*Ok* )) <*αV* (*n* −1), then data is not sufficient for detecting the positive

(*Ok* )) <*αV* (*n* −1), then data is not sufficient for detecting the negative

(*Ok* ), <sup>1</sup><sup>−</sup> *<sup>B</sup>* <sup>+</sup>

(*Ok* ), <sup>1</sup><sup>−</sup> *<sup>B</sup>* <sup>−</sup>

(*Ok* ), then no negative shift is occurred in the process

(*Ok* ), <sup>1</sup><sup>−</sup> *<sup>B</sup>* <sup>+</sup>

(*Ok* ), then no positive shift is occurred in the process

(*Ok* )) <sup>=</sup> *<sup>B</sup>* <sup>+</sup>

(*Ok* )) <sup>=</sup> *<sup>B</sup>* <sup>−</sup>

(*Ok* )), then determine the value

(*Ok* ), it is

(*Ok* ) it is con‐

(82)

151

http://dx.doi.org/10.5772/57423

be determined as follows:

( )

+

**1.** Set k=0 and the initial beliefs *B* <sup>+</sup>

**4.** Update the values for the beliefs *B* <sup>−</sup>

(*Ok* ), <sup>1</sup><sup>−</sup> *<sup>B</sup>* <sup>+</sup>

mean and decision making stops.

(*Ok* ), <sup>1</sup><sup>−</sup> *<sup>B</sup>* <sup>+</sup>

(*Ok* ), <sup>1</sup><sup>−</sup> *<sup>B</sup>* <sup>−</sup>

(*Ok* ), <sup>1</sup><sup>−</sup> *<sup>B</sup>* <sup>−</sup>

(*Ok* ), <sup>1</sup><sup>−</sup> *<sup>B</sup>* <sup>−</sup>

(*Ok* ), <sup>1</sup><sup>−</sup> *<sup>B</sup>* <sup>+</sup>

shift and go to stage 2.

mean and decision making stops.

(*Ok* ), <sup>1</sup><sup>−</sup> *<sup>B</sup>* <sup>+</sup>

**5.** If *Min*(*B* <sup>+</sup>

**6.** If *Max*(*B* <sup>+</sup>

**7.** If *Min*(*B* <sup>−</sup>

**8.** If *Max*(*B* <sup>−</sup>

**9.** If *Max*(*B* <sup>+</sup>

equation:

algorithm.

if *Max*(*B* <sup>−</sup>

also if *Max*(*B* <sup>+</sup>

*d n Max*

**2.** Gather an observation and set *k* =*k* + 1, *n* =*n* −1.

=

Since *φ*(*xk* ) is a cumulative distribution function thus it follows a uniform distribution function in interval [0, 1], thus the above equality is concluded.

With the same reasoning, it is concluded that:

$$\Pr\left\{1 - B^{+}(O\_{k}) \ge d^{+}\left(n\right)\right\} = \Pr\left\{1 - d^{+}\left(n\right) \ge B^{+}\left(O\_{k}\right)\right\} = 0.5h\left(1 - d^{+}\left(n\right)\right) \tag{78}$$

Now equation (73) can be written as follows:

$$V(n) = \max\_{0.5 \times d^+(n) \times 1} \begin{bmatrix} \mathcal{B}^+ \left( O\_k \right) \left( 1 - 0.5h\left( d^+ \left( n \right) \right) \right) + \left( 1 - \mathcal{B}^+ \left( O\_k \right) \right) 0.5h \left( 1 - d^+ \left( n \right) \right) \\\\ + \alpha V \left( n - 1 \right) \left( 1 - 0.5 \left( 1 - h \left( d^+ \left( n \right) \right) \right) - 0.5h \left( 1 - d^+ \left( n \right) \right) \right) \end{bmatrix} \tag{79}$$

And equation (79) can be written as follows:

$$\begin{aligned} V\left(n, d^+\left(n\right)\right) &= \left(B^+\left(O\_k\right) - \alpha V\left(n-1\right)\right)\left(1 - h\left(d^+\left(n\right)\right)0.5\right) + \\ V\left(1 - B^+\left(O\_k\right) - \alpha V\left(n-1\right)\right)0.5h\left(1 - d^+\left(n\right)\right) + \alpha V\left(n-1\right) \end{aligned} \tag{80}$$

Since *V* \* (*n*)= *Max* 0.5<*d* +(*n*)<1 *V* (*n*, *d* +(*n*)) thus it is sufficient to maximize the real value function

*V* (*n*, *d* +(*n*)), therefore; we should find the function value in points where first derivative is equated to zero as follows,

$$\begin{split} \frac{\partial V\left(n, d^+\left(n\right)\right)}{\partial d^+\left(n\right)} = 0 &\Rightarrow -\frac{\left(B^+\left(\mathcal{O}\_k\right) - \alpha V\left(n-1\right)\right)}{\left(1 - B^+\left(\mathcal{O}\_k\right) - \alpha V\left(n-1\right)\right)} = \frac{\left(1 - d^+\left(n\right)\right)^2}{d^+\left(n\right)} \\ \Rightarrow d^+\left(n\right) &= \frac{1}{\sqrt{-\frac{\left(B^+\left(\mathcal{O}\_k\right) - \alpha V\left(n-1\right)\right)}{\left(1 - B^+\left(\mathcal{O}\_k\right) - \alpha V\left(n-1\right)\right)}}} \\ \end{split} \tag{81}$$

The optimal threshold *d* +(*n*) is determined by the above equation. Since the optimal value of *d* +(*n*) should be in the interval [0.5, 1] thus it is concluded that the optimal value of *d* +(*n*) will be determined as follows:

$$d^+\left(n\right) = \text{Max}\left[\frac{1}{\sqrt{\frac{\left(\mathcal{B}^+\left(O\_k\right) - \alpha V\left(n-1\right)\right)}{\left(1 - \mathcal{B}^+\left(O\_k\right) - \alpha V\left(n-1\right)\right)}}}, 0.5\right] \tag{82}$$

The above method is presented for detecting the positive shifts in the process mean and can be adapted for detecting the negative shifts with the same reasoning.

The general decision making algorithm is summarized as follows:


{ ( )}

150 Dynamic Programming and Bayesian Inference, Concepts and Applications

*k*

With the same reasoning, it is concluded that:

Now equation (73) can be written as follows:

( )

a

*k*

a

*d n*

And equation (79) can be written as follows:

( ( )) ( )

0

( )

*d n*

Þ =

+

( ) max

*V n*

(*n*)= *Max* 0.5<*d* +(*n*)<1

equated to zero as follows,

Since *V* \*

j

( ) ( ) ( ) ( ) ( ( ))


*k k*

+ -

ì ü ï ï

ï ï <sup>&</sup>gt;î þ

Pr 1 ( ) Pr 1 { *BO d n d n BO h d n k k* ( )} { ( ) ( ) 0.5 1 } ( ( )) + + ++ <sup>+</sup> - ³ =- ³ = - (78)

( )( ( ( ))) ( ( )) ( ( ))

 a

*V* (*n*, *d* +(*n*)) thus it is sufficient to maximize the real value function

( ( )) ( )

2

2

1 0.5 1 0.5 1

ë û

*B O hd n B O h d n*

+ ++ +

1 1 0.5 1 0.5 1

= - -- +

*V* (*n*, *d* +(*n*)), therefore; we should find the function value in points where first derivative is

( ( ) ( )) ( ( ) ( ))

a

a

1 1

1

1

( ( ) ( )) ( ( ) ( ))


a

a

*B O Vn B O Vn*

*k k*

+ +

1 1

1

, 1 1

¶ -- - = Þ- <sup>=</sup> ¶ - --

*k k*

+ + + + + +

*V nd n B O V n d n d n B O Vn d n*

( )( ( ( ( ))) ( ( ))) 0.5 1

< < + + é ù - +- - ê ú <sup>=</sup> + -- - - -

( ( )) ( ( ) ( ))( ( ( )) ) ( ( ) ( )) ( ( )) ( )

1 1 0.5 1 1


+ + +

*B O Vn h d n Vn* a

*k*

+ +

*V nd n B O V n h d n*

, 1 1 0.5

*V n hd n h d n* <sup>+</sup>

*k k*

*xBO*

1

(77)

(79)

(80)

(81)

( )

1 1 Pr ( ) Pr 1 0.5

+

*BO d n xBO BO d n*

+ + + +

j

+ +

*k kk k*

j

Since *φ*(*xk* ) is a cumulative distribution function thus it follows a uniform distribution function

> = í ý + - =

{ ( ) ( ( )) } ( ( ))

> = -

*x hd n hd n*

Pr 0.5 1 0.5

in interval [0, 1], thus the above equality is concluded.


$$\begin{aligned} d^\*\left(n\right) &= \\ & \left( \frac{1}{\sqrt{\frac{1}{\left(\mathcal{B}^+\left(\mathcal{O}\_k\right) - \alpha V\left(n-1\right)\right)}{\left(1 - \mathcal{B}^+\left(\mathcal{O}\_k\right) - \alpha V\left(n-1\right)\right)}}, 0.5}}, 0.5\right) \\ & \left( \frac{1}{\left(1 - \mathcal{B}^+\left(\mathcal{O}\_k\right) - \alpha V\left(n-1\right)\right)} + 1}, 0.5\right) \end{aligned} \tag{83}$$

**Appendix 1**

where, *b***<sup>2</sup>**

and

**'**

obtained as *Σxx***-***ΣxX*

**Appendix 2**

Assume (*μj*

(*μj*

) *<sup>j</sup>*∈{*gr*,*sm*}=0 and (*σ<sup>j</sup>*

equal to its mean, we have

=*ΣxX ΣXX* **-1**

Conditional Mean and Variance of the Variables

(*μsm*, *μgr* |(*μj*

*Σ*: The covariance matrix of the process

*<sup>T</sup> <sup>Σ</sup>XX* **-1** *<sup>Σ</sup>xX* .

Evaluating the Optimal Value of *dgr*,*sm*(*k*)

) *<sup>j</sup>*∈{1,2,...,*m*}=0 and (*σ<sup>j</sup>*

Pr

Conditional mean of variables *gr* and *sm* can be evaluated using the following equation.

é ù ê ú ë û *XX xX xX xx Σ Σ*

*ΣxX* : Submatrix of the covariance matrix *Σ* between variables *j* = *gr*, *sm* and *j* ≠ *gr*, *sm*

) *<sup>j</sup>*∈{1,2,...,*m*}=1. Then,

0.5( ) 0.5( ) ,

*T T gr sm*

( ( ))

<sup>2</sup> <sup>2</sup> , ,

*k gr sm k sm*

ì ü ì ü íí ³ = ýý î þ î þ

{ ( ( )) }

*T hd k T*

*k gr sm gr sm k sm*

2 2 , , ,

Now since (*Tk* ,*sm*, *Tk* ,*gr*|*sm*) follow a standard normal distribution

³ +=

2

follow a *χ* <sup>2</sup>

{ ( ( ))}

Pr 0.5( ) ln ( ) 0.5( )

,

degree of freedom. Then using an approximation, if we assume that (*Tk* ,*sm*)

*e hd k e*

*k sm*


Pr ( ) ( ) 2ln ( )

*k gr sm gr sm*

*T T hd k*

) *<sup>j</sup>*∈{*gr*,*sm*}=1, hence (*Tk* ,*gr*|*sm*)2 and (*Tk* ,*sm*)

2 2 , ,

Further, the conditional covariance matrix of variables *j* = *gr*, *sm* on variables *j* ≠ *gr*, *sm*, is

**'**

((*Xkj*) *<sup>j</sup>*≠*gr*,*sm* −(*μ <sup>j</sup>*) *<sup>j</sup>*≠*gr*,*sm*) (85)

http://dx.doi.org/10.5772/57423

153

(87)

distribution with one

is approximately

2

*Σ Σ* (86)

Using Dynamic Programming Based on Bayesian Inference in Selection Problems

) *<sup>j</sup>*≠*gr*,*sm*) =(*μsm*, *μgr*) + *b***<sup>2</sup>**

*Σ =*

*Σxx*: Submatrix of the covariance matrix *Σ* for variables *j* = *gr*, *sm*

*ΣXX* : Submatrix of the covariance matrix *Σ* for variables *j* ≠ *gr*, *sm*

**10.** If *Max*(*B* <sup>−</sup> (*Ok* ), <sup>1</sup><sup>−</sup> *<sup>B</sup>* <sup>−</sup> (*Ok* )) <sup>&</sup>gt;*α<sup>V</sup>* (*<sup>n</sup>* <sup>−</sup>1)>*Min*(*<sup>B</sup>* <sup>−</sup> (*Ok* ), <sup>1</sup><sup>−</sup> *<sup>B</sup>* <sup>−</sup> (*Ok* )), then determine the value of *d* <sup>−</sup>(*n*) (minimum acceptable belief for detecting the negative shift) by the following equation:

$$d^-\left(n\right) = \tag{4.40}$$

$$\text{Max}\left(\frac{1}{\sqrt{\frac{\left(B^-\left(O\_k\right) - \alpha V\left(n-1\right)\right)}{\left(1 - B^-\left(O\_k\right) - \alpha V\left(n-1\right)\right)}}, 0.5\right) \tag{84}$$

