**4. Integral-type method**

#### **4.1 Image motion blurring related to depth**

From Eqs. (2) and (3), and the probabilistic characteristics of *r*ð Þ*<sup>j</sup>* , *v* is also a 2-D Gaussian random variable with E½ �¼ *v* **0** and the variance–covariance matrix of **A. Local optimization algorithm**

*DOI: http://dx.doi.org/10.5772/intechopen.97404*

with respect to the depth corresponding to each pixel.

*Stereoscopic Calculation Model Based on Fixational Eye Movements*

ð

*fconv*ð Þ� *x*

*<sup>f</sup> <sup>m</sup>*ð Þ� *<sup>x</sup> <sup>f</sup> conv*ð Þ *<sup>x</sup>* � �<sup>2</sup>

*=∂y*<sup>2</sup> as follows:

*<sup>d</sup>* ¼ � <sup>1</sup> � *<sup>λ</sup>*

≈ 1

∇2

Eq. (23) using ð Þ *i*, *j* as a description of an image position.

<sup>20</sup> *di*þ1,*j*�<sup>1</sup> � *di*,*<sup>j</sup>*

1 � *λ*

*<sup>i</sup>*�1,*<sup>j</sup>* <sup>þ</sup> *<sup>d</sup>*ð Þ *<sup>n</sup> i*,*j*�1

*<sup>∂</sup>d*ð Þ *<sup>x</sup> ∂y* � �<sup>2</sup>

> ð Þ *n <sup>i</sup>*,*<sup>j</sup>* þ

*<sup>i</sup>*,*j*þ<sup>1</sup> <sup>þ</sup> *<sup>d</sup>*ð Þ *<sup>n</sup>*

� � <sup>þ</sup>

*4.2.1 Numerical experiments of integral-type method*

L

ð

R

*JL*ð Þ� *d*ð Þ *x*

**B. Global optimization algorithm**

ð

*<sup>=</sup>∂x*<sup>2</sup> <sup>þ</sup> *<sup>∂</sup>*<sup>2</sup>

*<sup>∂</sup>d*ð Þ *<sup>x</sup> ∂x* � �<sup>2</sup>

þ

þ 1

*d*ð Þ *<sup>n</sup>*þ<sup>1</sup> *<sup>i</sup>*,*<sup>j</sup>* ¼ *d*

be adopted.

*JG*ð Þ¼ *d*ð Þ *x* ð Þ 1 � *λ*

using <sup>∇</sup><sup>2</sup> � *<sup>∂</sup>*<sup>2</sup>

*d* ð Þ *n <sup>i</sup>*,*<sup>j</sup>* <sup>¼</sup> <sup>1</sup> 5 *d*ð Þ *<sup>n</sup> <sup>i</sup>*þ1,*<sup>j</sup>* <sup>þ</sup> *<sup>d</sup>*ð Þ *<sup>n</sup>*

**13**

We assume that a depth value in a local region L around each *x* is constant. Based on the minimum least square criterion, the objective function can be defined

We can recover the depth individually for each pixel by minimizing this function with respect to the depth corresponding to each pixel. Therefore, simultaneous multivariate optimization is not required and one-dimensional numerical search can

By assuming a spatially smooth depth map in the solution, we can define the following objective function based on regularization theory of Poggio et al. [21].

*dx* þ *λ*

where *λ* is a parameter for adjusting the degree of constraint on the smoothness of the depth map, and the integration of Eq. (23) is performed for the entire image. From the variational principle, the Euler–Lagrange equation for *d*ð Þ *x* is derived

For discrete computation, we can approximate the smoothness constraint in

<sup>5</sup> *di*þ1,*<sup>j</sup>* � *di*,*<sup>j</sup>*

� �<sup>2</sup> <sup>þ</sup> *di*þ1,*j*þ<sup>1</sup> � *di*,*<sup>j</sup>* � �<sup>2</sup> n o*:*

Using Eq. (25) and the discrete representation of Eq. (24), we can minimize Eq. (23) by the following iterative formulation with an iteration number of *n*.

*<sup>λ</sup> <sup>f</sup> <sup>m</sup>*,*i*,*<sup>j</sup>* � *<sup>f</sup> conv <sup>d</sup>*ð Þ *<sup>n</sup>*

1 20

The proposed algorithm assumes that the definition of the motion blur image in Eq. (20) holds. To observe such ideal motion blur, it takes enough exposure time for

*d*ð Þ *<sup>n</sup>*

*<sup>λ</sup> <sup>f</sup> <sup>m</sup>* � *fconv* � � *<sup>∂</sup> <sup>f</sup> conv*

� �<sup>2</sup> <sup>þ</sup> *di*,*j*þ<sup>1</sup> � *di*,*<sup>j</sup>* � �<sup>2</sup> n o

*i*,*j* � � � � *<sup>∂</sup> <sup>f</sup> conv <sup>d</sup>*ð Þ *<sup>n</sup>*

*<sup>i</sup>*þ1,*j*þ<sup>1</sup> <sup>þ</sup> *<sup>d</sup>*ð Þ *<sup>n</sup>*

<sup>ð</sup> *<sup>∂</sup>d*ð Þ *<sup>x</sup> ∂x* � �<sup>2</sup>

þ

*<sup>∂</sup>d*ð Þ *<sup>x</sup> ∂y* � �<sup>2</sup> ( )*dx*, (23)

*<sup>∂</sup><sup>d</sup> :* (24)

*i*,*j* � �

*<sup>i</sup>*þ1,*j*�<sup>1</sup> <sup>þ</sup> *<sup>d</sup>*ð Þ *<sup>n</sup>*

� �*:*

*<sup>∂</sup><sup>d</sup>* , (26)

*<sup>i</sup>*�1,*j*�<sup>1</sup> <sup>þ</sup> *<sup>d</sup>*ð Þ *<sup>n</sup>*

*i*�1,*j*þ1

(27)

(25)

*<sup>f</sup> <sup>m</sup> <sup>x</sup>* � *<sup>x</sup>*<sup>00</sup> ð Þ� *<sup>f</sup> conv <sup>x</sup>* � *<sup>x</sup>*<sup>00</sup> ð Þ � �<sup>2</sup>

*gx x*<sup>0</sup> ð Þ ; *d f* <sup>0</sup> *x* � *x*<sup>0</sup> ð Þ*dx*<sup>0</sup>

*dx*00, (21)

*:* (22)

$$\mathbf{V}[\mathbf{v}] = \begin{bmatrix} \varkappa^2 \mathbf{y}^2 + (\mathbbm{1} + \varkappa^2 + Z\_0 d)^2 & 2\varkappa \mathbf{y} \left( \mathbbm{1} + \frac{\varkappa^2 + \mathfrak{y}^2}{2} + Z\_0 d \right) \\\\ 2\varkappa \mathfrak{y} \left( \mathbbm{1} + \frac{\varkappa^2 + \mathfrak{y}^2}{2} + Z\_0 d \right) & \varkappa^2 \mathfrak{y}^2 + (\mathbbm{1} + \mathfrak{y}^2 + Z\_0 d)^2 \end{bmatrix} \sigma\_r^2. \tag{19}$$

This equation can be seen as a function of the inverse depth *d*, so if we know the variance–covariance matrix at each pixel position, we can calculate the depth map.

There are some schemes to obtain the variance–covariance matrix of optical flow defined by Eq. (19) locally at each image position from multiple images observed through random camera rotations imitating tremor. The simplest and most natural way is to first detect the optical flow from the image and then calculate its quadratic statistic. However, here we are considering a case where the intensity pattern is fine with respect to the temporal sampling rate and it is difficult to accurately detect the optical flow. Therefore, we adopt an integral-formed scheme in which the variance–covariance matrix is calculated as the distribution of local image blur.

We define an averaged image brightness *f ave*ð Þ *x* as an arithmetic average of observed *M* images *f <sup>j</sup>* ð Þ *x* n o *<sup>j</sup>*¼1,⋯,*<sup>M</sup>* with fixational eye movements. If *<sup>M</sup>* is asymptotically large, the following convolution holds using locally defined a twodimensional Gaussian point spread functions *gx*ð Þ� and an original image brightness *f* <sup>0</sup>ð Þ *x* .

$$f\_{\text{ave}}(\mathbf{x}) = \int\_{\mathbf{x'} \in \mathcal{R}} \mathbf{g}\_{\mathbf{x}}(\mathbf{x'}) f\_0(\mathbf{x} - \mathbf{x'}) d\mathbf{x'},\tag{20}$$

where *x* indicates the image position, R is a local region around *x*, and *gx*ð Þ� has a vairance-covaiance matrix indicated by Eq. (19). Furthermore, it is assumed that Ð *gx x*<sup>0</sup> ð Þ*dx*<sup>0</sup> ¼ 1 is satisfied.

As explained above, we model *f ave*ð Þ *x* as a blurred image due to fixation eye movements. The difference in the degree of blur of *f ave*ð Þ *x* indicates the difference in depth. In other words, the closer the imaging target is to the camera, the greater this image blur.

#### **4.2 Direct algorithms for integral-type method**

In the two-step algorithm, after detecting the variance–covariance matrix of the image blur shown in Eq. (19), the maximum likelihood estimation of *d* using that as the observable is analytically possible [15]. However, the solution is not optimal because it observes indirect quantities. With optimality in mind, we need to use the direct method of directly estimating the depth map without determining *gx*ð Þ *:*; *d* . This strategy usually requires a numeric search or iterative update [16]. We constructed two algorithms as a direct method, each adopting local optimization and global optimization respectively.

*Stereoscopic Calculation Model Based on Fixational Eye Movements DOI: http://dx.doi.org/10.5772/intechopen.97404*

#### **A. Local optimization algorithm**

**4. Integral-type method**

*Applications of Pattern Recognition*

*V v*½ �¼

depth map.

image blur.

*f* <sup>0</sup>ð Þ *x* .

Ð

**12**

observed *M* images *f <sup>j</sup>*

*gx x*<sup>0</sup> ð Þ*dx*<sup>0</sup> ¼ 1 is satisfied.

this image blur.

**4.1 Image motion blurring related to depth**

2*xy* 1 þ

From Eqs. (2) and (3), and the probabilistic characteristics of *r*ð Þ*<sup>j</sup>* , *v* is also a 2-D Gaussian random variable with E½ �¼ *v* **0** and the variance–covariance matrix of

*<sup>x</sup>*<sup>2</sup> <sup>þ</sup> *<sup>y</sup>*<sup>2</sup>

*<sup>x</sup>*2*y*<sup>2</sup> <sup>þ</sup> <sup>1</sup> <sup>þ</sup> *<sup>y</sup>* ð Þ <sup>2</sup> <sup>þ</sup> *<sup>Z</sup>*0*<sup>d</sup>* <sup>2</sup>

*<sup>j</sup>*¼1,⋯,*<sup>M</sup>* with fixational eye movements. If *<sup>M</sup>* is

*gx x*<sup>0</sup> ð Þ*f* <sup>0</sup> *x* � *x*<sup>0</sup> ð Þ*dx*<sup>0</sup>

<sup>2</sup> <sup>þ</sup> *<sup>Z</sup>*0*<sup>d</sup>* � �

, (20)

*<sup>r</sup>:* (19)

*<sup>x</sup>*2*y*<sup>2</sup> <sup>þ</sup> <sup>1</sup> <sup>þ</sup> *<sup>x</sup>* ð Þ <sup>2</sup> <sup>þ</sup> *<sup>Z</sup>*0*<sup>d</sup>* <sup>2</sup> <sup>2</sup>*xy* <sup>1</sup> <sup>þ</sup>

<sup>2</sup> <sup>þ</sup> *<sup>Z</sup>*0*<sup>d</sup>* � �

This equation can be seen as a function of the inverse depth *d*, so if we know the variance–covariance matrix at each pixel position, we can calculate the

There are some schemes to obtain the variance–covariance matrix of optical flow defined by Eq. (19) locally at each image position from multiple images observed through random camera rotations imitating tremor. The simplest and most natural way is to first detect the optical flow from the image and then calculate its quadratic statistic. However, here we are considering a case where the intensity pattern is fine with respect to the temporal sampling rate and it is difficult to accurately detect the optical flow. Therefore, we adopt an integral-formed scheme in which the variance–covariance matrix is calculated as the distribution of local

We define an averaged image brightness *f ave*ð Þ *x* as an arithmetic average of

asymptotically large, the following convolution holds using locally defined a twodimensional Gaussian point spread functions *gx*ð Þ� and an original image brightness

where *x* indicates the image position, R is a local region around *x*, and *gx*ð Þ� has a vairance-covaiance matrix indicated by Eq. (19). Furthermore, it is assumed that

In the two-step algorithm, after detecting the variance–covariance matrix of the image blur shown in Eq. (19), the maximum likelihood estimation of *d* using that as the observable is analytically possible [15]. However, the solution is not optimal because it observes indirect quantities. With optimality in mind, we need to use the direct method of directly estimating the depth map without determining *gx*ð Þ *:*; *d* . This strategy usually requires a numeric search or iterative update [16]. We constructed two algorithms as a direct method, each adopting local optimization

As explained above, we model *f ave*ð Þ *x* as a blurred image due to fixation eye movements. The difference in the degree of blur of *f ave*ð Þ *x* indicates the difference in depth. In other words, the closer the imaging target is to the camera, the greater

*<sup>x</sup>*<sup>2</sup> <sup>þ</sup> *<sup>y</sup>*<sup>2</sup>

ð Þ *x* n o

*f ave*ð Þ¼ *x*

**4.2 Direct algorithms for integral-type method**

and global optimization respectively.

ð

*x*<sup>0</sup> ∈ R

We assume that a depth value in a local region L around each *x* is constant. Based on the minimum least square criterion, the objective function can be defined with respect to the depth corresponding to each pixel.

$$J\_L(d(\mathfrak{x})) \equiv \int\_{\mathcal{L}} \left( f\_{\,\,m}(\mathfrak{x} - \mathfrak{x}'') - f\_{\,\,conv}(\mathfrak{x} - \mathfrak{x}'') \right)^2 d\mathfrak{x}'',\tag{21}$$

$$f\_{conv}(\mathbf{x}) \equiv \int\_{\mathcal{R}} \mathbf{g} \mathbf{x}(\mathbf{x}'; d) f\_0(\mathbf{x} - \mathbf{x}') d\mathbf{x}'.\tag{22}$$

We can recover the depth individually for each pixel by minimizing this function with respect to the depth corresponding to each pixel. Therefore, simultaneous multivariate optimization is not required and one-dimensional numerical search can be adopted.

#### **B. Global optimization algorithm**

By assuming a spatially smooth depth map in the solution, we can define the following objective function based on regularization theory of Poggio et al. [21].

$$J\_G(d(\mathfrak{x})) = (1 - \lambda) \left[ \left( f\_m(\mathfrak{x}) - f\_{conv}(\mathfrak{x}) \right)^2 d\mathfrak{x} + \lambda \left[ \left\{ \left( \frac{\partial d(\mathfrak{x})}{\partial \mathfrak{x}} \right)^2 + \left( \frac{\partial d(\mathfrak{x})}{\partial \mathfrak{y}} \right)^2 \right\} d\mathfrak{x}, \tag{23} \right]$$

where *λ* is a parameter for adjusting the degree of constraint on the smoothness of the depth map, and the integration of Eq. (23) is performed for the entire image. From the variational principle, the Euler–Lagrange equation for *d*ð Þ *x* is derived using <sup>∇</sup><sup>2</sup> � *<sup>∂</sup>*<sup>2</sup> *<sup>=</sup>∂x*<sup>2</sup> <sup>þ</sup> *<sup>∂</sup>*<sup>2</sup> *=∂y*<sup>2</sup> as follows:

$$
\nabla^2 d = -\frac{1-\lambda}{\lambda} \left( f\_m - f\_{conv} \right) \frac{\partial f\_{conv}}{\partial d}.\tag{24}
$$

For discrete computation, we can approximate the smoothness constraint in Eq. (23) using ð Þ *i*, *j* as a description of an image position.

$$\begin{split} \left(\frac{\partial d(\mathbf{x})}{\partial \mathbf{x}}\right)^2 + \left(\frac{\partial d(\mathbf{x})}{\partial \mathbf{y}}\right)^2 &\approx \frac{1}{5} \left\{ \left(d\_{i+1j} - d\_{ij}\right)^2 + \left(d\_{ij+1} - d\_{ij}\right)^2 \right\} \\ &+ \frac{1}{20} \left\{ \left(d\_{i+1j-1} - d\_{ij}\right)^2 + \left(d\_{i+1j+1} - d\_{ij}\right)^2 \right\}. \end{split} \tag{25}$$

Using Eq. (25) and the discrete representation of Eq. (24), we can minimize Eq. (23) by the following iterative formulation with an iteration number of *n*.

$$d\_{ij}^{(n+1)} = \overline{d}\_{ij}^{(n)} + \frac{1-\lambda}{\lambda} \left( f\_{m,ij} - f\_{conv} \left( d\_{ij}^{(n)} \right) \right) \frac{\partial f\_{conv} \left( d\_{ij}^{(n)} \right)}{\partial d},\tag{26}$$

$$\overline{d}\_{i,j}^{(n)} = \frac{1}{5} \left( d\_{i+1,j}^{(n)} + d\_{i,j+1}^{(n)} + d\_{i-1,j}^{(n)} + d\_{i,j-1}^{(n)} \right) + \frac{1}{20} \left( d\_{i+1,j+1}^{(n)} + d\_{i+1,j-1}^{(n)} + d\_{i-1,j-1}^{(n)} + d\_{i-1,j+1}^{(n)} \right). \tag{27}$$

#### *4.2.1 Numerical experiments of integral-type method*

The proposed algorithm assumes that the definition of the motion blur image in Eq. (20) holds. To observe such ideal motion blur, it takes enough exposure time for

#### *Applications of Pattern Recognition*

imaging. Here, we use artificial images to examine the characteristics of the proposed algorithm with respect to the relationship between the size of image motion and the fineness of intensity texture.

We artificially generate motion blur images. First, a true depth map is set up, and a large number of images are generated by a computer graphics technique that randomly samples *r* according to the Gaussian distribution in Eq. (4). By averaging these images, we can artificially generate motion blur images. Input motion blur images averaged 10, 000 images to mimic analog motion blur, **Figure 4** shows an example of a reference (original) image and a true inverse depth map. The image size is 256 � 256 pixels, which is equivalent to �0*:*5≤*x*, *y*≤0*:*5 measured in focal length units.

0 10 20 30 40 50 60 70 0

0 10 20 30 40 50 60 70 0

0 10 20 30 40 50 60 70 0

0 10 20 30 40 50 60 70 0

 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.24

(a) (b)

(c) (d)

 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.24

(a) (b)

 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.24

(c) (d)

*Example of motion-blurred image and recovered inverse depth maps with σ<sup>r</sup>* ¼ 0*:*01*: (a) motion-blurred*

*Example of motion-blurred image and recovered inverse depth maps with σ<sup>r</sup>* ¼ 0*:*008*: (a) motion-blurred*

 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.24

0 10 20 30 40 50 60 70 0

0 10 20 30 40 50 60 70 0

 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.24

**Figure 7.**

**15**

 10 20 30 40 50 60 70

*image; (b) local optimization; (c) λ* ¼ 0*:*2*; (d) λ* ¼ 0*:*6 *(reprinted from [16]).*

 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.24 0.26

**Figure 6.**

 10 20 30 40 50 60 70

*Stereoscopic Calculation Model Based on Fixational Eye Movements*

*DOI: http://dx.doi.org/10.5772/intechopen.97404*

*image; (b) local optimization; (c) λ* ¼ 0*:*2*; (d) λ* ¼ 0*:*6 *(reprinted from [16]).*

 10 20 30 40 50 60 70

> 10 20 30 40 50 60 70

 10 20 30 40 50 60 70

 10 20 30 40 50 60 70

**Figure 4.**

*Example of the artificial data used in the experiments: (a) original image; (b) true inverse depth map used for generating the blurred image (reprinted from [16]).*

**Figure 5.**

*Example of motion-blurred image and recovered inverse depth maps with σ<sup>r</sup>* ¼ 0*:*006*: (a) motion-blurred image; (b) local optimization; (c) λ* ¼ 0*:*2*; (d) λ* ¼ 0*:*6 *(reprinted from [16]).*

## *Stereoscopic Calculation Model Based on Fixational Eye Movements DOI: http://dx.doi.org/10.5772/intechopen.97404*

**Figure 6.**

imaging. Here, we use artificial images to examine the characteristics of the proposed algorithm with respect to the relationship between the size of image motion

> 0.08 0.1 0.12 0.14 0.16 0.18 0.2

(a) (b)

*Example of the artificial data used in the experiments: (a) original image; (b) true inverse depth map used for*

 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.24 0.26 0.28 0.3

(a) (b)

(c) (d)

*Example of motion-blurred image and recovered inverse depth maps with σ<sup>r</sup>* ¼ 0*:*006*: (a) motion-blurred*

 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.24 0.26 0.28 0.3

We artificially generate motion blur images. First, a true depth map is set up, and a large number of images are generated by a computer graphics technique that randomly samples *r* according to the Gaussian distribution in Eq. (4). By averaging these images, we can artificially generate motion blur images. Input motion blur images averaged 10, 000 images to mimic analog motion blur, **Figure 4** shows an example of a reference (original) image and a true inverse depth map. The image size is 256 � 256 pixels, which is equivalent to �0*:*5≤*x*, *y*≤0*:*5 measured in focal

0 10 20 30 40 50 60 70 0

0 10 20 30 40 50 60 70 0

0 10 20 30 40 50 60 70 0

 10 20 30 40 50 60 70

 10 20 30 40 50 60 70

> 10 20 30 40 50 60 70

and the fineness of intensity texture.

*Applications of Pattern Recognition*

*generating the blurred image (reprinted from [16]).*

0 10 20 30 40 50 60 70 0

 10 20 30 40 50 60 70

*image; (b) local optimization; (c) λ* ¼ 0*:*2*; (d) λ* ¼ 0*:*6 *(reprinted from [16]).*

length units.

**Figure 4.**

 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.24 0.26 0.28 0.3

**Figure 5.**

**14**

*Example of motion-blurred image and recovered inverse depth maps with σ<sup>r</sup>* ¼ 0*:*008*: (a) motion-blurred image; (b) local optimization; (c) λ* ¼ 0*:*2*; (d) λ* ¼ 0*:*6 *(reprinted from [16]).*

**Figure 7.**

*Example of motion-blurred image and recovered inverse depth maps with σ<sup>r</sup>* ¼ 0*:*01*: (a) motion-blurred image; (b) local optimization; (c) λ* ¼ 0*:*2*; (d) λ* ¼ 0*:*6 *(reprinted from [16]).*

Local optimization algorithms (LOA) are computationally expensive, and global optimization algorithms (GOA) are slow to converge. Therefore, we considered a hybrid algorithm that uses LOA sparsely to obtain the initial value of GOA. For the initial value of LOA, use the plane corresponding to the background of **Figure 4(b)**. To make a rough estimate, LOA uses blocks of 41 � 41 pixels as L in Eq. (21) and applies LOA once to each block. On the other hand, the size of R in Eq. (22) was adaptively determined according to the depth value updated by the optimization process. Therefore, R took a different size at each position in the image. We assumed a square area for R. The length of that side was 10 times the larger of the two deviations of *gx*ð Þ �; *d* along *x* axis and *y* axis. These can be evaluated using Eq. (19).

smooth depth tends to be recovered from the recognized smooth motion blur from **Figure 8(c)**, it can be confirmed that the smoothness constraint of Eq. (23) is an

When applying the difference-type method to an actual image and checking the actual performance, the performance was improved by selecting the image pair used for depth restoration. We have adopted a scheme that excludes image pairs that are expected to have large approximation errors in the gradient equation on a pixel-by-pixel basis. We can use the inner product of the spatial gradient vectors of consecutive image pairs to select image pairs that do not cause aliasing problems.

*<sup>s</sup>* ¼ *f*

*<sup>x</sup>* <sup>þ</sup> *<sup>f</sup> yyv*<sup>2</sup>

ð Þ *i*,*j <sup>y</sup>* � *f*

ð Þ *i*,*j <sup>y</sup>* � *f*

ð Þ *i*,*j <sup>y</sup> v* ð Þ *i*,*j*

*<sup>s</sup>* � *<sup>f</sup>*ð Þ *<sup>i</sup>*, *<sup>j</sup>*�<sup>1</sup>

*<sup>s</sup>* � *<sup>f</sup>*ð Þ *<sup>i</sup>*, *<sup>j</sup>*�<sup>1</sup>

*<sup>s</sup>* <sup>∣</sup>*=*∣*f*ð Þ *<sup>i</sup>*,*<sup>j</sup>*

*<sup>s</sup>* is perpendicular to that of optical flow, the equation error becomes

*<sup>s</sup>* <sup>∣</sup>*=*∣*f*ð Þ *<sup>i</sup>*,*<sup>j</sup>*

We built the camera hardware system for examining the practical performance of our camera model shown in **Figure 1**. The implemented camera system is shown

n o*:* (29)

In the next step, from the image pairs remained by the above decision, we additively

After discarding a bad image pair, the higher-order terms can be considered small. In this case, the quadratic term in Eq. (28) can be estimated for each pixel *i* as follows:

*<sup>x</sup>* þ *f*

ð Þ *<sup>i</sup>*,*<sup>j</sup> <sup>x</sup>* <sup>þ</sup> *<sup>f</sup>*

ð Þ *<sup>i</sup>*,*<sup>j</sup> <sup>x</sup>* <sup>þ</sup> *<sup>f</sup>*

This measurement depends on the direction of the optical flow but is invariant with respect to the amplitude of the optical flow. To calculate the value of *J*, we need to know the true value of the optical flow. By examining the details of *J*, even

We can define a measure for estimating the equation error as the ratio of this

select the suitable image pairs at each pixel by estimating the amount of the higher

<sup>2</sup> *<sup>f</sup> xxv*<sup>2</sup>

ð Þ *i*, *j*�1 *x* � �*v*ð Þ *<sup>i</sup>*,*<sup>j</sup>*

ð Þ *i*, *j*�1 *x* � �*<sup>v</sup>*

> 2∣*f* ð Þ *i*,*j <sup>x</sup> v*

*<sup>s</sup>* � *<sup>f</sup>*ð Þ *<sup>i</sup>*, *<sup>j</sup>*�<sup>1</sup>

threshold value are selected at each pixel to be used for depth recovery.

ð Þ *i*,*j <sup>x</sup>* , *f* ð Þ *i*,*j y* h i<sup>T</sup>

.

. *ft* is exactly represented as follows:

*<sup>y</sup>* þ 2 *f xyvxvy* n o <sup>þ</sup> <sup>⋯</sup>*:* (28)

> ð Þ *i*, *j*�1 *y* � �*v*ð Þ *<sup>i</sup>*,*<sup>j</sup>*

ð Þ *i*, *j*�1 *y* � �*<sup>v</sup>*

*y*

ð Þ *i*,*j <sup>y</sup>* ∣

*<sup>y</sup>* <sup>∣</sup> *:* (30)

*<sup>s</sup>* is large, when the direction of

*<sup>s</sup>* ∣ can be used as the worst value. In

*<sup>s</sup>* ∣ is less than the certain

*s* T *f*ð Þ *<sup>i</sup>*, *<sup>j</sup>*�<sup>1</sup> *<sup>s</sup>* is

For each pixel, the image pairs of which the sign of the inner product *f*ð Þ *<sup>i</sup>*,*<sup>j</sup>*

**5. Real image experiments of differential-type method**

**5.1 Selective use of image pairs to improve accuracy**

*Stereoscopic Calculation Model Based on Fixational Eye Movements*

negative are discarded. It is noted that *f*ð Þ *<sup>i</sup>*,*<sup>j</sup>*

order terms included in the observation of *ft*

� 1 <sup>2</sup> *<sup>f</sup>*

higher order term to the first order term.

∣ *f* ð Þ *i*,*j <sup>x</sup>* � *f*

if the difference of the spatial gradient *f*ð Þ *<sup>i</sup>*,*<sup>j</sup>*

the following, the image pairs for which ∣*f*ð Þ *<sup>i</sup>*,*<sup>j</sup>*

*J* ¼

small. Therefore, the value ∣*f*ð Þ *<sup>i</sup>*,*<sup>j</sup>*

**5.2 Camera system implementation**

*f*ð Þ *<sup>i</sup>*,*<sup>j</sup>*

*<sup>s</sup>* � *<sup>f</sup>*ð Þ *<sup>i</sup>*, *<sup>j</sup>*�<sup>1</sup>

in **Figure 9**.

**17**

*ft* ¼ �*<sup>f</sup> <sup>x</sup>vx* � *<sup>f</sup> <sup>y</sup>vy* � <sup>1</sup>

ð Þ *i*,*j <sup>x</sup>* � *f*

obstacle to the reduction of RMSE.

*DOI: http://dx.doi.org/10.5772/intechopen.97404*

The depth restoration simulation was executed while changing the camera rotation size *σr*. The recovered inverse depth map is shown in **Figures 5**–**7**. GOA uses various values of *λ*. The relationship between the root mean square error (RMSE) of the recovered depth map and the value of *λ* is also shown in **Figure 8**. From **Figure 5**, we can see that it is not suitable for depth recovery because it is difficult to accurately measure the motion blur of the image position with a small rotation of the camera. From **Figure 8(a)**, small rotations do not provide sufficient measurement information, so to reduce the RMSE of the recovered depth map, the smoothness constraint indicated by *λ* is strongly required. On the contrary, when the rotation is large, the point image distribution function becomes wider than the spatial change of the target shape. Therefore, the Gaussian function with the variance–covariance matrix in Eq. (19) is inappropriate, the motion blur recognized by this model is smoother than the actual blur of the image, and a depth recovery error occurs. This can be confirmed from the RMSE value in **Figure 8(c)**. Since the
