2. Fisheye projection models

Pinhole projection is so called because it preserves the rectilinearity of the projected scene (i.e. straight lines in the scene are projected as straight lines on the image plane). The Pinhole (perspektife) projection is shown in Figure 1. The Pinhole (perspektife) projection mapping function is given in Eq. (1).

$$\mathbf{r}\_u = \mathbf{f}. \tan\left(\theta\right) \text{(respectively projection)}\tag{1}$$

where f is the distance between the principal point and the image plane, θ is the incident angle (in radians) of the projected ray to the optical axis of the camera and ru is the projected radial distance from the principal point on the image plane. However, for wide field of view (FOV) cameras, under rectilinear projection, the size of the projected image becomes very large, increasing to infinity at an FOV of 180� [62].

Figure 1. Pinhole (perspektife) projection representation.

Interior orientation parameters (IOPs) can be estimated by a procedure called camera calibration. The perspective bundle, which generated the image, can be reconstructed by this procedure. The principal point co-ordinates, focal length and coefficient for systematic errors correction (lens distortion: symmetric radial and decentring and affinity) are the IOPs of digital cameras. When additional parameters (IOPs) in Eq. (2) [75] are examined, collinearity equations are the most popular camera calibration method [63, 76].

more important in the photogrammetric measurement assessment. This study suggests a precalibration process of these kinds of hardware for the photogrammetric process in the test field. In the literature, although there are many geometric camera calibration publications, none of them compares the mobile phone fisheye lens kit with conventional fisheye lens on the fundamentals of photogrammetric measurement assessment. The results of this photogrammetric process are also compared with conventional wide-angle hardware in this paper. The second section of this chapter briefly describes fisheye projection models. The third section of this chapter briefly describes equidistant model. The fourth section reports an empirical study for calibration of the combination of iPhone 4S camera with Olloclip 3 in one fisheye lens and Nikon Coolpix8700 camera FC-09 fisheye lens combination by using equidistant model. The fifth section interprets the results that resulted from the experiment process. The sixth

Pinhole projection is so called because it preserves the rectilinearity of the projected scene (i.e. straight lines in the scene are projected as straight lines on the image plane). The Pinhole (perspektife) projection is shown in Figure 1. The Pinhole (perspektife) projection mapping

where f is the distance between the principal point and the image plane, θ is the incident angle (in radians) of the projected ray to the optical axis of the camera and ru is the projected radial distance from the principal point on the image plane. However, for wide field of view (FOV) cameras, under rectilinear projection, the size of the projected image becomes very large,

ru ¼ f: tan ð Þð θ perspective projectionÞ (1)

section concludes the study.

function is given in Eq. (1).

2. Fisheye projection models

32 Smartphones from an Applied Research Perspective

increasing to infinity at an FOV of 180� [62].

Figure 1. Pinhole (perspektife) projection representation.

$$\begin{aligned} \mathbf{x}\_{\text{f}} &= \mathbf{x}' - \mathbf{x}\_0 - \Delta \mathbf{x} = -\mathbf{f} \frac{\mathbf{X}\_{\text{c}}}{\mathbf{Z}\_{\text{c}}}\\ \mathbf{y}\_{\text{f}} &= \mathbf{y}' - \mathbf{y}\_0 - \Delta \mathbf{y} = -\mathbf{f} \frac{\mathbf{Y}\_{\text{c}}}{\mathbf{Z}\_{\text{c}}} \end{aligned} \tag{2}$$

where, f represents the focal length, and (Xc, Yc, Zc) shows the 3D point co-ordinates of photogrammetric reference system in Eq. (3); point co-ordinates of the image are (xf, yf); image point co-ordinates of the reference system parallel to photogrammetric system are represented as (x', y'), this element originates from image centre and principal point (pp) of the co-ordinates are (xo, yo).

$$\begin{aligned} \mathbf{X\_{c}} &= \mathbf{r\_{11}} \cdot (\mathbf{X} - \mathbf{X\_{CP}}) + \mathbf{r\_{12}} \cdot (\mathbf{Y} - \mathbf{Y\_{CP}}) + \mathbf{r\_{13}} \cdot (\mathbf{Z} - \mathbf{Z\_{CP}}) \\ \mathbf{Y\_{c}} &= \mathbf{r\_{21}} \cdot (\mathbf{X} - \mathbf{X\_{CP}}) + \mathbf{r\_{22}} \cdot (\mathbf{Y} - \mathbf{Y\_{CP}}) + \mathbf{r\_{23}} \cdot (\mathbf{Z} - \mathbf{Z\_{CP}}) \\ \mathbf{Z\_{c}} &= \mathbf{r\_{31}} \cdot (\mathbf{X\_{c}} - \mathbf{X\_{CP}}) + \mathbf{r\_{32}} \cdot (\mathbf{Y} - \mathbf{Y\_{CP}}) + \mathbf{r\_{33}} \cdot (\mathbf{Z - Z\_{CP}}) \end{aligned} \tag{3}$$

where rij (i and j from 1 to 3) represents rotation matrix elements and with rij, the object can be used in relation to the image reference system; (X, Y, Z) shows any point's co-ordinates in the object reference system and (Xcp, Ycp, Zcp) shows perspective centre (PC) in object reference system [63]. Pinhole (perspektife) projection model is not suitable for fisheye lenses. Fisheye lenses instead are usually designed to obey one of the following projections [68]:

$$\mathbf{r} = \mathbf{2.f.tan}(\mathbf{0}/2) \text{ (stereographic projection)}\tag{4}$$

$$\mathbf{r} = \mathbf{f}. \boldsymbol{\Theta} \text{ (equidistant projection)}\tag{5}$$

$$\mathbf{r} = 2.\mathbf{f}.\sin(\theta/2) \text{ (equisolid angle projection)}\tag{6}$$

$$\mathbf{r} = \mathbf{f}. \sin \left( \theta \right) \left( \text{orthogonal projection} \right) \tag{7}$$

In Eqs. (1) and (4)–(7), the angle between optical axis and incoming ray is shown with θ symbol; the distance between image point and principal point is represented with r, and focal length is represented with f. Equidistance projection can be accepted as the most wide-spread used fisheye lens model. Figure 2a illustrates the schematic description of different projections for the fisheye lens. Figure 2b shows the difference between pinhole lens and fisheye lens. The images acquired with non-perspective projection are more near to principal point when the results are compared to the results of perspective projection. Therefore, the view angle of fisheye lens is wider than conventional lens. Moreover, actual image surface of fisheye lens presents a hemisphere in accordance with a pinhole lens plane. Thus, projecting the image on surface of the hemisphere into an actual imaging plane results in a deformation of the fisheye lens [77].

Figure 2. The principles for various lenses: (a) shows different lens projections, p, p1, p2, p3 and p4 are respectively perspective projection, stereographic projection, equidistance projection, equisolid angle projection and orthogonal projection; the corresponding distances between image points and the principal point are represented with r, r1, r2, r3 and r4; (b) shows the difference between pinhole lens and fisheye lens. In terms of fisheye lens, perspective image's projection on the hemisphere surface into the image plane is the actual image.

Figure 3. Radial distortion in the 2D imaging plane: O represents the image centre, Pu, Pd ∈ R<sup>2</sup> are pixel co-ordinate vectors in the input undistorted and output distorted images, respectively, ru, rd ∈ R are the distance of Pu and Pd from centre, and θ ¼ θ<sup>u</sup> ¼ θdis the angle of OPu or, equivalently, OPd.

A wide-angle lens produces geometric distortion in the radial direction called the barrel distortion, since it compresses the peripheral region to contain a wide angle of view in the image plane. Considering this problem, many researchers have proposed various models to correct the barrel distortion of the wide-angle lens. A two-dimensional (2D) approximated barrel distortion model is shown in Figure 3, where an original pixel Pu moves towards the centre at Pd along the radial direction in the image plane [65]. A polynomial model was proposed to approximate various types of wide-angle lenses using the distortion coefficients. The distance of the distorted pixel Pu is determined by the polynomial equation [65].
