**6.1 Simple supervised case**

Let consider the linear forward model we considered in previous section

$$p(f|g, \upsilon\_f, \upsilon\_\varepsilon) = \mathcal{N}(f|\bar{f}, \Sigma \text{ with } \bar{f} = [H^\prime H + \lambda I]^{-1} H^\prime g \text{ and } \bar{\Sigma} = \upsilon\_\varepsilon [H^\prime H + \lambda I]^{-1}$$

**Figure 9.** *Supervised linear Gaussian case.*

$$\mathbf{g} = \mathbf{H}\mathbf{f} + \mathbf{e},\tag{39}$$

and assign Gaussian laws to *ϵ* and *f* which leads to:

$$\begin{cases} p(\mathbf{g}|\mathbf{f}) = \mathcal{N}(\mathbf{g}|\mathbf{H}\mathbf{f}, v\_{\ell}I) \propto \exp\left[-\frac{1}{2v\_{\varepsilon}}||\mathbf{g} - \mathbf{H}\mathbf{f}||^{2}\right] \\\\ p(\mathbf{f}) = \mathcal{N}(\mathbf{f}|0, v\_{f}I) \propto \exp\left[-\frac{1}{2v\_{f}}\left\|\mathbf{f}\right\|^{2}\right] \end{cases} \tag{40}$$

Using these expressions, we get:

$$\begin{cases} p(\mathbf{f}|\mathbf{g}) \propto \exp\left[ -\frac{1}{2v\_{\epsilon}} \|\mathbf{g} - \mathbf{H}\mathbf{f}\|^2 - \frac{1}{2v\_{f}} \|\mathbf{f}\|^2 \right] \\\\ \quad \propto \exp\left[ -\frac{1}{2v\_{\epsilon}} J(\mathbf{f}) \right] \text{with } J(\mathbf{f}) = \|\mathbf{g} - \mathbf{H}\mathbf{f}\|^2 + \lambda \|\mathbf{f}\|^2, \quad \lambda = \frac{v\_{\epsilon}}{v\_{f}} \end{cases} \tag{41}$$

which can be summarized as:

$$p(\mathbf{f}|\mathbf{g}) = \mathcal{N}\left(\mathbf{f}|\hat{\mathbf{f}}, \hat{\boldsymbol{\Sigma}}\right) \text{ with } \hat{\boldsymbol{f}} = \left[\mathbf{H}^{\prime}\mathbf{H} + \lambda\boldsymbol{I}\right]^{-1}\mathbf{H}^{\prime}\mathbf{g} \text{ and } \hat{\boldsymbol{\Sigma}} = \boldsymbol{v}\_{\epsilon}\left[\mathbf{H}^{\prime}\mathbf{H} + \lambda\boldsymbol{I}\right]^{-1},\tag{42}$$

where *<sup>λ</sup>* <sup>¼</sup> *<sup>v</sup><sup>ϵ</sup> v f :* This case is summarized in **Figure 9**.

This is the simplest case where we know exactly the expression of the posterior law and all the computations can be done explicitly. However, for great dimensional problems, where the vectors *f* and *g* are very great dimensional, we may even not be able to keep in memory the matrix *H* and surely not be able to compute the inverse of the matrix *H*<sup>0</sup> ½ � *H* þ *λI :* In Section 9 on Bayesian computation, We will see how to do these computations.
