**6.2 Unsupervised case or hyperparameter estimation**

In the previous section, we considered the linear models with Gaussian priors with known parameters *v<sup>ϵ</sup>* and *vf*. In many practical situations these parameters are not known, and we want to estimate them too. For this, we can assign them too prior laws. As the variances are positive quantities and using the concept of conjugate priors, we can assign then Inverse Gamma priors:

*Bayesian Inference for Inverse Problems DOI: http://dx.doi.org/10.5772/intechopen.104467*

$$\begin{cases} p(v\_c) = \mathcal{Z}\mathcal{G}(v\_f|a\_{c\_0}, \boldsymbol{\beta}\_{c\_0}) \\ p(v\_f) = \mathcal{Z}\mathcal{G}\Big(v\_f|a\_{f\_0}, \boldsymbol{\beta}\_{f\_0}\Big) \end{cases} \tag{43}$$

and using the likelihood *p*ð Þ¼ *g*j*f*, *v<sup>ϵ</sup>* N ð Þ *g*j*Hf*, *vϵI* and the prior *p f*j*v<sup>f</sup>* � � <sup>¼</sup> <sup>N</sup> *<sup>f</sup>*j0, *<sup>v</sup>fI* � �, we can easily obtain the expressions of the following conditional posterior laws:

$$\begin{cases} p\left(\mathbf{f}|\mathbf{g}, \hat{v}\_c, \hat{v}\_f\right) = \mathcal{N}\left(\mathbf{f}|\hat{\mathbf{f}}, \hat{\mathbf{2}}\right) \quad \text{with} : \\ \hat{\mathbf{f}} = \left[\mathbf{H}^t \mathbf{H} + \hat{\lambda}\mathbf{I}\right]^{-1} \mathbf{H}^t \mathbf{g} \\ \hat{\boldsymbol{\Sigma}} = \hat{v}\_c \left[\mathbf{H}^t \mathbf{H} + \hat{\lambda}\mathbf{I}\right]^{-1}, \quad \hat{\lambda} = \hat{v}\_c/\hat{v}\_f \end{cases} \tag{44}$$

and

$$\begin{cases} p(v\_\epsilon | \mathbf{g}, f) = \mathcal{Z}\mathcal{G}(v\_\epsilon | \tilde{a}\_\epsilon, \tilde{\beta}\epsilon) \\ p\left(v\_f | \mathbf{g}, f\right) = \mathcal{Z}\mathcal{G}\left(v\_f | \tilde{a}\_f, \tilde{\beta}\_f\right) \end{cases} \tag{45}$$

where all the details and in particular the expressions for *α*~*ϵ*, ~*βϵ*, *α*~ *<sup>f</sup>* , ~*β <sup>f</sup>* can be found in [10].

As we can see, the expressions of ^*f* and **Σ**^ are the same as in the previous case, except that the values of ^*vϵ*, ^*v<sup>f</sup>* and ^*λ* have to be updated. They are obtained from the conditionals *p v*ð Þ *<sup>ϵ</sup>*j*g*,*<sup>f</sup>* and *p vf*j*g*,*<sup>f</sup>* � � which depend on *<sup>f</sup>*. This shows that we can propose an iterative algorithm in two steps: Determine the expression of *p f*j*g*, ^*vϵ*, ^*v<sup>f</sup>* � � and using the values of in the previous iteration, we can propose an estimate for *f*, and then, using *p v*ð Þ *<sup>ϵ</sup>*j*g*,*<sup>f</sup>* and *p vf*j*g*,*<sup>f</sup>* � �, we can give estimates for ^*v<sup>ϵ</sup>* and ^*v<sup>f</sup>* which can again be used in the first step. It is interesting to know that all the three approaches of JMAP, GEM and VBA for this cas follow exactly this same iterative algorithm. The only differences will be in the update values of *α*~*ϵ*, *β*~*ϵ*, *α*~*f*, *β*~*<sup>f</sup>* and the choice of the estimators (MAP or PM) of ^*v<sup>ϵ</sup>* and ^*v <sup>f</sup> :*

This case is also summarized in **Figure 10**.

$$\begin{array}{c} \begin{array}{c} \begin{array}{|c|c|} \hline \begin{array}{c} a\_{f},\beta\_{f} \\ \downarrow \\ \hline \end{array} \end{array} \end{array} \begin{array}{c} \begin{array}{c} \begin{array}{c} \begin{array}{c} a\_{\varepsilon},\beta\_{\varepsilon} \\ \downarrow \\ \hline \end{array} \end{array} \end{array} \end{} \begin{array}{c} \begin{array}{c} \begin{array}{c} \hline \hline a\_{\varepsilon},\beta\_{\varepsilon} \\ \downarrow \\ \hline \end{array} \end{array} \end{} \end{}$$

$$\begin{array}{c} p(v\_{f}|v\_{f}) = \mathcal{N}(f|0,v\_{f}|I) \begin{array}{c} \begin{array}{c} \downarrow \\ \downarrow \\ \hline \end{array} \end{array} \begin{array}{c} \begin{array}{c} \begin{array}{c} \begin{array}{c} v\_{e} \end{array} \\ \downarrow \\ \hline \end{array} \end{array} \end{} \begin{array}{c} p(v\_{e}) = \mathcal{Z}\mathcal{G}(v\_{e}|a\_{e},\beta\_{e}) \\ \downarrow \\ \hline \end{array} \end{array} \begin{array}{c} p(v\_{e}) = \mathcal{Z}\mathcal{G}(v\_{e}|a\_{e},\beta\_{e}) \\ \downarrow \\ \hline \end{array}$$

$$\begin{array}{c} p(v\_{e}) = \mathcal{N}(g|f,v\_{e}) = \mathcal{N}(g|Hf,v\_{e}I) \end{array}$$

**Figure 10.**

*Bayesian inference scheme in linear systems and Gaussian priors. The posterior is also Gaussian, and all the computations can be done analytically.*
