**Appendix 1**

Conditional Mean and Variance of the Variables

Conditional mean of variables *gr* and *sm* can be evaluated using the following equation.

$$\mathfrak{a}\left(\mu\_{sm'}\,\mu\_{\mathcal{S}^r}\,\middle|\,\left(\mu\_{\}\right)\_{\neq \neq \mathcal{S}^r,sm}\right) = \left(\mu\_{sm'}\,\mu\_{\mathcal{S}^r}\right) + \mathfrak{b}\left(\left(X\_{k\bar{j}}\right)\_{\neq \neq \mathcal{S}^r,sm} - \left(\mu\_{\neq}\right)\_{\neq \neq \mathcal{S}^r,sm}\right) \tag{85}$$

where, *b***<sup>2</sup> '** =*ΣxX ΣXX* **-1**

and

( )

=

*d n*

152 Dynamic Programming and Bayesian Inference, Concepts and Applications

+

*Max*

( )

=

*d n*


*Max*

dynamic programming approach is *α nV* (0).

(*Ok* ), <sup>1</sup><sup>−</sup> *<sup>B</sup>* <sup>−</sup>

**10.** If *Max*(*B* <sup>−</sup>

**1.** If *B* <sup>+</sup>

**2.** If *B* <sup>−</sup>

(1− *B* <sup>+</sup>

(1− *B* <sup>−</sup>

to stage 2.

**5. Conclusion**

equation:

( ( ) ( )) ( ( ) ( ))

*B O Vn B O Vn* a

+ +

(*Ok* )) <sup>&</sup>gt;*α<sup>V</sup>* (*<sup>n</sup>* <sup>−</sup>1)>*Min*(*<sup>B</sup>* <sup>−</sup>

1 1 *k k*

( ( ) ( )) ( ( ) ( ))

*B O Vn B O Vn* a


1 1 *k k*

a

to stage 2 after checking the occurrence of negative shift in rest of the algorithm.

**3.** The approximate value of *αV* (*n* −1) based on the discount factor *α* in the stochastic

In this chapter, we introduced a new approach to determine the best solution out of *m* candidates. To do this, first, we defined the belief of selecting the best solution and explained how to model the problem by the Bayesian analysis approach. Second, we clarified the approach by which we improved the beliefs, and proved that it converges to detect the best solution. Next, we proposed a decision-making strategy using dynamic programming

approach in which there were a limited number of decision-making stages.


æ ö ç ÷

a

of *d* <sup>−</sup>(*n*) (minimum acceptable belief for detecting the negative shift) by the following

<sup>1</sup> ,0.5 1

(*Ok* ) <sup>&</sup>gt;*<sup>d</sup>* +(*n*), then a positive shift is occurred and decision making stops, and if

(*Ok* )) <sup>&</sup>gt;*<sup>d</sup>* +(*n*), then no positive shift is occurred and decision making stops, else go

(*Ok* ) <sup>&</sup>gt;*<sup>d</sup>* <sup>−</sup>(*n*), then a negative shift is occurred and decision making stops, and If

(*Ok* )) <sup>&</sup>gt;*<sup>d</sup>* <sup>−</sup>(*n*), then no negative shift is occurred and decision making stops, else go

1


æ ö ç ÷

> <sup>1</sup> ,0.5 1

1

(*Ok* ), <sup>1</sup><sup>−</sup> *<sup>B</sup>* <sup>−</sup>

(83)

(84)

(*Ok* )), then determine the value

$$
\boldsymbol{\Sigma} = \begin{bmatrix} \boldsymbol{\Sigma}\_{\mathbf{XX}} & \boldsymbol{\Sigma}\_{\mathbf{xX}} \\ \boldsymbol{\Sigma}\_{\mathbf{xX}} & \boldsymbol{\Sigma}\_{\mathbf{xx}} \end{bmatrix} \tag{86}
$$

*Σ*: The covariance matrix of the process

*Σxx*: Submatrix of the covariance matrix *Σ* for variables *j* = *gr*, *sm*

*ΣxX* : Submatrix of the covariance matrix *Σ* between variables *j* = *gr*, *sm* and *j* ≠ *gr*, *sm*

*ΣXX* : Submatrix of the covariance matrix *Σ* for variables *j* ≠ *gr*, *sm*

Further, the conditional covariance matrix of variables *j* = *gr*, *sm* on variables *j* ≠ *gr*, *sm*, is obtained as *Σxx***-***ΣxX <sup>T</sup> <sup>Σ</sup>XX* **-1** *<sup>Σ</sup>xX* .
