**1. Introduction**

[12] Billinton R, Allan R. Reliability evaluation of power systems. Plenum Press, New

[13] RockafellarR, Uryasev S, Zabarankin M. Generalized deviations in risk analysis. Fi‐

[14] Fischer T. Risk capital allocation by coherent risk measures based on one-sided mo‐

[16] Rockafellar T, Uryasev S, ZabarankinM. Deviation measures in risk analysis and op‐ timization. University of Florida, Department of Industrial & Systems Engineering

ments. Insurance: Mathematics and Economics 2003; 32(1) 135-146.

[15] Pavlo AK. Higher moment coherent risk measures. Taylor & Francis, 2007.

York, 1996.

Working Paper 2002.

nance and Stochastics 2006; 10(1) 51-74.

124 Dynamic Programming and Bayesian Inference, Concepts and Applications

An important subject in mathematical science that causes new improvements in data analysis is sequential analysis. In this type of analysis, the number of required observations is not fixed in advance, but is a variable and depends upon the values of the gathered observation. In sequential analysis, at any stage of data gathering process, to determine the number of required observations at the next stage, we analyze the data at hand and with respect to the obtained results, we determine how many more observations are necessary. In this way, the process of data gathering is cheaper and the information is used more effectively. In other words, the data gathering process in sequential analysis, in contrast to frequency analysis, is on-line. This idea caused some researches to conduct researches in various statistical aspects (Basseville and Nikiforov[1]).

In this chapter, using the concept of the sequential analysis approach, we develop an innova‐ tive Bayesian method designed specifically for the best solution in selection problem. The proposed method adopts the optimization concept of Bayesian inference and the uncertainty of the decision-making method in dynamic programming environment. The proposed algorithm is capable of taking into consideration the quality attributes of uncertain values in determining the optimal solution. Some authors have applied sequential analysis inference in combination with optimal stopping problem to maximize the probability of making correct decision. One of these researches is a new approach in probability distribution fitting of a given statistical data that Eshragh and Modarres [2] named it Decision on Belief (DOB). In this decision-making method, a sequential analysis approach is employed to find the best under‐ lying probability distribution of the observed data. Moreover, Monfared and Ranaeifar [3] and Eshragh and Niaki [4] applied the DOB concept as a decision-making tool in some problems.

© 2014 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Since the idea behind the sequential analysis modeling is completely similar to the decisionmaking process of a human being in his life, it may perform better than available methods in decision-making problems. In these problems, when we want to make a decision, first we divide all of the probable solution space into smaller subspaces (the solution is one of the subspaces). Then based on our experiences, we assign a probability measure (belief) to each subspace, and finally we update the beliefs and make the decision.

of success of the *i*

stage, we will have

( )

,

1

*B*

1

*j*

=

*j*

=

=

å

*B*

a b

, ,

*ik ik*

( )

=


 b

ab

a

way to do this is to use

,1 ,1 ,

*ik ik i k*

, Pr ,

*<sup>n</sup> th jk jk jk jk*

,1 ,1 , ,

*B j*


 a

> ab

*p*¯

th population obtained by *αi*,*<sup>k</sup>*

a b

a

to be the best one given *αi*,*k* and *βi*,*k* as

a b=

*<sup>i</sup>*,*<sup>k</sup>* , we take a *Beta* prior distribution with parameters αi,0=0.5 and βi,0=0.5. Then, using Bayesian

, , 0.5 , , 0.5

a

*i k i k ik ik*

At stage *k* of the data gathering process, after taking a sample and observing the numbers of

inference, we can easily show that the posterior probability density function of *<sup>p</sup>*¯

, , ,

*i k i k i k*

( 1) ( ) (1 ) ( 0.5) ( 0.5)

 b

failures and successes, we update the probability distribution function of *<sup>p</sup>*¯

lation. To do this, define *B*(*αi*,*<sup>k</sup>* , *βi*,*<sup>k</sup>* ) as a probability measure (called belief) of the *i*

( ) { } th , , , , , Pr population is the best , *ik ik ik ik B i*

{ ( )} {( ) }

Pr Population is the best , Pr , Population is the best

<sup>é</sup> <sup>ù</sup> <sup>ê</sup> <sup>ú</sup> <sup>ë</sup> <sup>û</sup>

Pr Population is the best , Pr , Population is the best

 b


 b

{( ) }


*th th*

*j j*

a

*th i k*

*i*

( ) {( ) }

é ù ê ú åë û

, Pr , Population is the best

*i i*

a

*<sup>n</sup> th th*

,

b

, ,

a b ,1 ,1 , ,

*ik ik ik ik*

{ ( )} {( ) }

Population is the best

From equation (3) we see that to update the beliefs, we need to evaluate Pr{(*αi*,*<sup>k</sup>* , *<sup>β</sup>i*,*<sup>k</sup>* )|*<sup>i</sup> th* Population is the best} ; *<sup>i</sup>* =1, 2, ..., *<sup>n</sup>* in each decision-making stage. One

*<sup>p</sup> <sup>i</sup>*

, 1

*p*

=

=

*j k j*

å (4)

{( ) } ,

Pr , Population is the best *th i k ik ik n*

,1 ,1 , ,

*jk jk jk jk*

 ab

> ab

We then update the beliefs based on the values of (*αi*,*<sup>k</sup>* , *βi*,*<sup>k</sup>* ) for each population in iteration *k*. If we define *B*(*αi*,*<sup>k</sup>* <sup>−</sup>1, *βi*,*<sup>k</sup>* <sup>−</sup>1) as the prior belief for each population, in order to update the posterior belief *B*(*αi*,*<sup>k</sup>* , *βi*,*<sup>k</sup>* ), since we may assume that the data are taken independently in each

, ,

*i k i k f p p p*

*km* , referring to Jeffrey's prior (Nair et al.[7]), for

b

Using Dynamic Programming Based on Bayesian Inference in Selection Problems


a b *<sup>i</sup>*,*k* is

http://dx.doi.org/10.5772/57423

127

*<sup>i</sup>*,*<sup>k</sup>* for each popu‐

(2)

th population

(3)
