**Color Restoration of Aerial Photographs**

Daniel Carneiro da Silva and Ana Lúcia Bezerra Candeias *Federal University of Pernambuco Brazil* 

#### **1. Introduction**

A non-uniform distribution of illumination on a negative is provoked by direct and indirect illumination, atmospheric factors and construction of lenses. The effect in aerial photographs can be perceived more easily in photo-indices and mosaics. Often it is attributed to vignetting. However, the direction of illumination due to the position of the sun and atmospheric factors provoke an additional effect that is not radially symmetric to the center of the photograph. This compound effect can appear in any photograph scale.

These problems that were well resolved with the use of filters for haze or anti-vignetting in black&white photographs, are now becoming more critical with the current wide use of color photographs and the increasing use of orthophoto-maps produced with those photographs.

There was also the adaptation of the production methodology of the orthophoto-maps, which in the past demanded that each sheet be produced with just one photograph, but which nowadays, by using the techniques of digital image processing, can allow mosaics of two or more photographs. There is a demand nowadays for seams not to be apparent because of the common differences in tones that exist between neighboring photographs.

The techniques that are already available can resolve various problems, such as the reduction of clearness because of haze from sun reflection, bright areas (hot spots) and vignetting effect; with digital image processing for commercial programs (Nobrega & and Quintanilha, 2004; Li et al., 2004a; Wu & Campbell, 2004; Paparodis et al., 2006), but the results are not always acceptable, because the seams are visible or because artifacts appear. On the other hand, there are a lot of research studies and methods being developed that have presented good results and can be incorporated into commercial programs. Some of these methods are discussed in LI et al. (2004a).

This chapter is divided into three main parts: the first one shows the causes of non-uniform illumination in aerial photographs; the second one shows the practical applications of some methods; and the third one presents the results.

Initially, the correction of the vignetting effect is presented by a simple formula and the haze effect in high altitude photographs with color transformation. After that, a method is developed based on masks that is intended to correct the combined illumination degradation effect of vignetting with the bi-directional reflectance distribution function

Color Restoration of Aerial Photographs 3

As the effect is symmetrical in relation to the center of the photograph, it can be

 *I*(*b*) = cos(*b*)*n* (1)

Equations like the one above were used also by HOMMA et al. (2000); Homma et al. (2000);

The effects of atmospheric radiance in aerial photographs are complex and they are caused by camera altitude, type, size, concentration and distribution of the atmospheric aerosol, sighting angle, height and azimuth in relation to the sun (Slater, 1983). They can be uniform or non-uniform in the whole area of the photograph. Normally, for photographs from great heights the uniform effect is attributed to the haze and the non-uniform area is related to BRDF (Bi-directional Reflectance Distribution Function) and variations in the type of haze

Figure 2a shows the geometric elements of an aerial photograph that will be useful for the discussion of this study: the EC (Exposition Center), the Nadir, the solar height angle, sun rays indicated by arrows and the camera viewing angle. In Figure 2b, the solar azimuth and

In the next section we will analyze the effects of atmospheric radiance in aerial photographs

A well known effect of haze in photogrammetry is the reduction of contrast in photographs taken at high altitudes and in the case of colored photographs the appearance of a uniform

mathematically corrected. Equation 1 is a very used function to correct this problem

Lamparelli (2006) for correction of the vignetting effect in aerial photographs.

Where: *I* is illumination which reaches the negative  *b* angle between optic axis and light ray

(Paparoditis et al., 2006; Wu & Campbell, 2004).

of low and high altitude.

**2.2.1 Haze effect** 

 *n* varies from 2.5 to 4 (Slater, 1983; Kraus, 1997).

**2.2 Effects of atmospheric radiance in aerial photographs** 

the direction of the illumination in relation to the EC are shown.

 a) b) Fig. 2. Geometry of solar illumination on vertical aerial photograph

(BRDF) in color aerial photographs. The manipulation of histograms and the Kries hypothesis are showed for color transformation applied to aerial photographs.

The results are discussed in terms of visual quality and processing time.

## **2. Non uniform illumination in aerial photographs**

The non uniform illumination in aerial photographs originates in the vignetting effects, directional scattering of solar illumination in the presence of haze and the surface bidirectional reflectance.

Moreover there are other factors that can reduce the quality of aerial photographs during their execution, such as clouds or shadows from clouds, shadows from topographic elevations or buildings, reflection from the sun in water bodies, smoke, haze and the quality of the optical system and the film. Analogical aerial photogrammetry developed ways to avoid or partially correct those problems with the use of devices like filters and special films, aside from adequate flight planning for each region and season of the year.

Nowadays, some of these problems, like the reduction of clearness by haze, reflexions from sunlight, shiny areas (*hot spot)* and vignetting, can be solved, at least partially, with digital image processing programs (Nobrega & Quintanilha, 2004; Lamparelli et al, 2004; Silva & Candeias, 2008; Li et al., 2004a). Other aspects are more complex, like the elimination of cloud shadows, and they are still being studied (LI et al, 2004b).

#### **2.1 Vignetting effects**

The vignetting effects come from the non-uniform illumination that passes through a lens system until it reaches the negative, where the amount of light is greater in the center and diminishes at the borders. The effect is radial and symmetrical in the center of the photograph, the borders become darker, and in the case of colored photographs, they also become bluer. This problem is greater for wide angle cameras. Figure 1 presents an example of the vignetting effect in black and white and color photograph.

Fig. 1. Photograph with vignetting effect obtained with wide angle camera. (Photograph: Base Engenharia)

(BRDF) in color aerial photographs. The manipulation of histograms and the Kries

The non uniform illumination in aerial photographs originates in the vignetting effects, directional scattering of solar illumination in the presence of haze and the surface

Moreover there are other factors that can reduce the quality of aerial photographs during their execution, such as clouds or shadows from clouds, shadows from topographic elevations or buildings, reflection from the sun in water bodies, smoke, haze and the quality of the optical system and the film. Analogical aerial photogrammetry developed ways to avoid or partially correct those problems with the use of devices like filters and special

Nowadays, some of these problems, like the reduction of clearness by haze, reflexions from sunlight, shiny areas (*hot spot)* and vignetting, can be solved, at least partially, with digital image processing programs (Nobrega & Quintanilha, 2004; Lamparelli et al, 2004; Silva & Candeias, 2008; Li et al., 2004a). Other aspects are more complex, like the elimination of

The vignetting effects come from the non-uniform illumination that passes through a lens system until it reaches the negative, where the amount of light is greater in the center and diminishes at the borders. The effect is radial and symmetrical in the center of the photograph, the borders become darker, and in the case of colored photographs, they also become bluer. This problem is greater for wide angle cameras. Figure 1 presents an example

a) Colored b) Black and white

Fig. 1. Photograph with vignetting effect obtained with wide angle camera.

films, aside from adequate flight planning for each region and season of the year.

hypothesis are showed for color transformation applied to aerial photographs.

The results are discussed in terms of visual quality and processing time.

**2. Non uniform illumination in aerial photographs** 

cloud shadows, and they are still being studied (LI et al, 2004b).

of the vignetting effect in black and white and color photograph.

bidirectional reflectance.

**2.1 Vignetting effects** 

(Photograph: Base Engenharia)

As the effect is symmetrical in relation to the center of the photograph, it can be mathematically corrected. Equation 1 is a very used function to correct this problem

$$I(b) \equiv \cos(b)^n \tag{1}$$

Where: *I* is illumination which reaches the negative

 *b* angle between optic axis and light ray

 *n* varies from 2.5 to 4 (Slater, 1983; Kraus, 1997).

Equations like the one above were used also by HOMMA et al. (2000); Homma et al. (2000); Lamparelli (2006) for correction of the vignetting effect in aerial photographs.

#### **2.2 Effects of atmospheric radiance in aerial photographs**

The effects of atmospheric radiance in aerial photographs are complex and they are caused by camera altitude, type, size, concentration and distribution of the atmospheric aerosol, sighting angle, height and azimuth in relation to the sun (Slater, 1983). They can be uniform or non-uniform in the whole area of the photograph. Normally, for photographs from great heights the uniform effect is attributed to the haze and the non-uniform area is related to BRDF (Bi-directional Reflectance Distribution Function) and variations in the type of haze (Paparoditis et al., 2006; Wu & Campbell, 2004).

Figure 2a shows the geometric elements of an aerial photograph that will be useful for the discussion of this study: the EC (Exposition Center), the Nadir, the solar height angle, sun rays indicated by arrows and the camera viewing angle. In Figure 2b, the solar azimuth and the direction of the illumination in relation to the EC are shown.

In the next section we will analyze the effects of atmospheric radiance in aerial photographs of low and high altitude.

Fig. 2. Geometry of solar illumination on vertical aerial photograph

#### **2.2.1 Haze effect**

A well known effect of haze in photogrammetry is the reduction of contrast in photographs taken at high altitudes and in the case of colored photographs the appearance of a uniform

Color Restoration of Aerial Photographs 5

illustrates the effects of direct illumination in uniformly spaced trees, which causes an illumination gradient. The sun is on the left in relation to the center of the photograph., so the illumination is darker. On the opposite side, on the right, the BRDF effect occurs and the

The BRDF evaluates the reflectance of a surface and depends on the direction of the irradiant flux and the direction of the reflected flux detected (Slater, 1980). This evaluation considers the height angles and the sun azimuth, the angles of the surface on which the flux focuses, the orientation angles and the wave length of the visible light. The calculation of the BRDF is complex and is often used in illumination models, the greater difficulty being the need for information about the reflectance and shape of the objects in the terrain, that are not easily available. As an alternative, simplified or empirical functions are used to estimate the effects of

The effects of the BRDF have an impact in the quality of the images in the same magnitude

The shape of the hot spot is normally reported as a bright circular area or a peak (Asrar, 1989; Beisl & Woodhouse, 2004), but as can be seen in the illustration of Figure 4 and the examples in Figure 5, the lighter areas do not have a regular circular shape. In vertical images it always

The black circles on figures 5a and 5b are plotters of viewing angles with a 10° interval and zero at center, showing the subjective visualization of the BRDF effect. The sun position was known from flight data so that the arrow is pointing to the sun and the center of the white circle coincides with the solar zenith angle. These figures also help the visualization of the directional spreading of the haze effect. As will be seen in section 2.2.4 the haze increases the

The bright peaks, as seen in left side of Figure 5b, can be caused by the specular reflection of the sun in water bodies or metallic rooftops. They are also some times called hot spots in the literature, but they occur in the same side of the sun in the image. That is why in this chapter

Specular reflection points appear when the solar high angle is bigger than half of the opening angle of the camera and the projection of the sun rays reaches a reflective surface. Using the geometry in Figure 2 this reflection would occur in point A. When using a large angular camera there can be reflection points with the sun height at over 45º, which is why it is necessary to take special care on photogrammetric flights done around midday, when the sun is higher. In the photograph of figure 5b there are both specular reflection points (pointed by arrow) and hot spots (neighborhood of the white circle), positioned on the

Some authors noted that there are other factors besides the BRDF that can change the illumination in photographs. Kraus (1989) shows that the photographs taken in the Southern

order as the atmospheric effects (Beisl & Woodhouse, 2004), including the haze.

appears when the solar zenithal angle is smaller than the sight angle camera.

the BRDF in aerial photographs (Beisl & Woodhouse, 2004).

illumination is brighter.

BRDF effect.

**2.2.3 Points of specular reflection** 

we will call them of points of specular reflection.

opposite side in relation to the center of the photograph.

**2.2.4 Directional spreading of light of the haze effect** 

blue tone (Figure 3). This effect is produced by light scattering in the atmosphere even with a clear sky and it is increased in the presence of a dry or humid haze. As a blue light has a higher index of refraction, its scattering is greater and it becomes more visible (Slater, 1983; Fiete, 2004). The reduction of the contrast is significant and reduces the visualization of details of the images (Kraus, 1992).

Fig. 3. Bluish color Photograph due to the presence of haze in the atmosphere (Photograph: TOPOCART S/C)

#### **2.2.2 Bright areas**

Bright areas, more known as *hot spots,* are the effect caused by the non-visualization of object shadows due to the position of the observer in relation to the sun. This type of bright area does not have to be confound to specular reflection which will be discussed in the next section.

When the sun is directly behind the EC or the observer, a great portion of the landscape will be visualized with direct lighting, and the reflectance will tend to be greater. At the same time the shadows are covered by the height itself of objects like buildings and trees. This effect is due to Paparoditis et al., 2006; Beisl & Woodhouse, 2004). Figure 4

Fig. 4. The brighter area, in the right side of the figure, is in opposite side to the sun due to BRDF. (Tuominen & Pekkarinen, 2004).

blue tone (Figure 3). This effect is produced by light scattering in the atmosphere even with a clear sky and it is increased in the presence of a dry or humid haze. As a blue light has a higher index of refraction, its scattering is greater and it becomes more visible (Slater, 1983; Fiete, 2004). The reduction of the contrast is significant and reduces the visualization of

Fig. 3. Bluish color Photograph due to the presence of haze in the atmosphere

Bright areas, more known as *hot spots,* are the effect caused by the non-visualization of object shadows due to the position of the observer in relation to the sun. This type of bright area does not have to be confound to specular reflection which will be discussed in the next section.

When the sun is directly behind the EC or the observer, a great portion of the landscape will be visualized with direct lighting, and the reflectance will tend to be greater. At the same time the shadows are covered by the height itself of objects like buildings and trees. This effect is due to Paparoditis et al., 2006; Beisl & Woodhouse, 2004). Figure 4

Fig. 4. The brighter area, in the right side of the figure, is in opposite side to the sun due to

details of the images (Kraus, 1992).

(Photograph: TOPOCART S/C)

BRDF. (Tuominen & Pekkarinen, 2004).

**2.2.2 Bright areas** 

illustrates the effects of direct illumination in uniformly spaced trees, which causes an illumination gradient. The sun is on the left in relation to the center of the photograph., so the illumination is darker. On the opposite side, on the right, the BRDF effect occurs and the illumination is brighter.

The BRDF evaluates the reflectance of a surface and depends on the direction of the irradiant flux and the direction of the reflected flux detected (Slater, 1980). This evaluation considers the height angles and the sun azimuth, the angles of the surface on which the flux focuses, the orientation angles and the wave length of the visible light. The calculation of the BRDF is complex and is often used in illumination models, the greater difficulty being the need for information about the reflectance and shape of the objects in the terrain, that are not easily available. As an alternative, simplified or empirical functions are used to estimate the effects of the BRDF in aerial photographs (Beisl & Woodhouse, 2004).

The effects of the BRDF have an impact in the quality of the images in the same magnitude order as the atmospheric effects (Beisl & Woodhouse, 2004), including the haze.

The shape of the hot spot is normally reported as a bright circular area or a peak (Asrar, 1989; Beisl & Woodhouse, 2004), but as can be seen in the illustration of Figure 4 and the examples in Figure 5, the lighter areas do not have a regular circular shape. In vertical images it always appears when the solar zenithal angle is smaller than the sight angle camera.

The black circles on figures 5a and 5b are plotters of viewing angles with a 10° interval and zero at center, showing the subjective visualization of the BRDF effect. The sun position was known from flight data so that the arrow is pointing to the sun and the center of the white circle coincides with the solar zenith angle. These figures also help the visualization of the directional spreading of the haze effect. As will be seen in section 2.2.4 the haze increases the BRDF effect.

#### **2.2.3 Points of specular reflection**

The bright peaks, as seen in left side of Figure 5b, can be caused by the specular reflection of the sun in water bodies or metallic rooftops. They are also some times called hot spots in the literature, but they occur in the same side of the sun in the image. That is why in this chapter we will call them of points of specular reflection.

Specular reflection points appear when the solar high angle is bigger than half of the opening angle of the camera and the projection of the sun rays reaches a reflective surface. Using the geometry in Figure 2 this reflection would occur in point A. When using a large angular camera there can be reflection points with the sun height at over 45º, which is why it is necessary to take special care on photogrammetric flights done around midday, when the sun is higher. In the photograph of figure 5b there are both specular reflection points (pointed by arrow) and hot spots (neighborhood of the white circle), positioned on the opposite side in relation to the center of the photograph.

#### **2.2.4 Directional spreading of light of the haze effect**

Some authors noted that there are other factors besides the BRDF that can change the illumination in photographs. Kraus (1989) shows that the photographs taken in the Southern

Color Restoration of Aerial Photographs 7

The product is added to this spectral radiance upward along the route. Thus the formulation presented by Hall contains simplifications and does not take into account the multiscattering. However, their approach is very enlightening and shows that the flow or final radiance is a function of the scattering coefficient and the distance traversed by the flow.

Hall (1954), using G. B. Harrison's formulas, relates the haze factor to the height of the sun. To do that is need consider the brightness of alight pulse sweeping a vertical plane as in the Figure 6. The geometric elements of figure 6 involved in the formulas are: the sun zenith angle *θ*, the a angle *Φ* that makes a haze cone with the sun ray, the viewing angle *β*, the exposition center (*EC)*; altitude of flying *h*. The points on the ground A and B have viewing

angles *βa* and *βb,* respectively, and point N is in Nadir.

Fig. 6. Geometrical elements involved in analysis of haze factor.

the vertical line or nadir

on the ground intercepted by the cone.

The brightness of a cone of haze at angle *β* is given by:

Where: *σ* is the coefficient of total scattering in the ft-1 unit.

 

The percentage of haze factor at *β* angle can be estimated in aerial photographs by:

Where: *HBβ* is the brightness of an elemental cone of haze which makes an angle *β* with

*GBβ* is the brightness as seen through the haze of a horizontal white diffuse reflector

cos cos *<sup>B</sup> <sup>h</sup> H Af e* 

cos (sec sec ( ) <sup>1</sup>

    

(3)

*Hβ = 100*x *(HBβ)/GBβ* (2)

and Northern hemispheres present systematically brighter areas in the North and South, respectively. Asarar (1989) shows that there is more radiation spread in the direction close to the incident light that must not be confused with the hot spot effect.

a)

Fig. 5. White circle on center of hot spot and arrow pointing to sun. a) Only hot spot, b) Also with specular reflection on side of sun. (Photograph: Base Engenharia)

Silva & Candeias (2009) have reproduced the work of Hall (1954), which allows the identification of the center of bright areas which is the effect of haze scattering and a function of the angle of the sun light and viewing angle of camera.

The radiance reaching the sensor is the sum of spectral reflectance and thermal radiant surface, multiplied by the spectral transmittance of the path in the atmosphere (Slater, 1980).

and Northern hemispheres present systematically brighter areas in the North and South, respectively. Asarar (1989) shows that there is more radiation spread in the direction close to

a)

b) Fig. 5. White circle on center of hot spot and arrow pointing to sun. a) Only hot spot, b) Also

Silva & Candeias (2009) have reproduced the work of Hall (1954), which allows the identification of the center of bright areas which is the effect of haze scattering and a

The radiance reaching the sensor is the sum of spectral reflectance and thermal radiant surface, multiplied by the spectral transmittance of the path in the atmosphere (Slater, 1980).

with specular reflection on side of sun. (Photograph: Base Engenharia)

function of the angle of the sun light and viewing angle of camera.

the incident light that must not be confused with the hot spot effect.

The product is added to this spectral radiance upward along the route. Thus the formulation presented by Hall contains simplifications and does not take into account the multiscattering. However, their approach is very enlightening and shows that the flow or final radiance is a function of the scattering coefficient and the distance traversed by the flow.

Hall (1954), using G. B. Harrison's formulas, relates the haze factor to the height of the sun. To do that is need consider the brightness of alight pulse sweeping a vertical plane as in the Figure 6. The geometric elements of figure 6 involved in the formulas are: the sun zenith angle *θ*, the a angle *Φ* that makes a haze cone with the sun ray, the viewing angle *β*, the exposition center (*EC)*; altitude of flying *h*. The points on the ground A and B have viewing angles *βa* and *βb,* respectively, and point N is in Nadir.

Fig. 6. Geometrical elements involved in analysis of haze factor.

The percentage of haze factor at *β* angle can be estimated in aerial photographs by:

$$
\hbar \mathsf{H} \mathfrak{\beta} = 100 \times \left( \mathsf{H}\_{\mathsf{B}} \mathfrak{\beta} \right) \mathsf{C} \mathsf{B} \mathfrak{\beta} \tag{2}
$$

Where: *HBβ* is the brightness of an elemental cone of haze which makes an angle *β* with the vertical line or nadir

 *GBβ* is the brightness as seen through the haze of a horizontal white diffuse reflector on the ground intercepted by the cone.

The brightness of a cone of haze at angle *β* is given by:

$$\mathbf{H}^{B}\boldsymbol{\beta} = A\boldsymbol{f}(\boldsymbol{\phi})\frac{\cos\theta}{\cos\beta+\cos\theta}\left[1-e^{-\sigma\hbar(\sec\beta+\sec\theta)}\right] \tag{3}$$

Where: *σ* is the coefficient of total scattering in the ft-1 unit.

Color Restoration of Aerial Photographs 9

Fig. 7. Brightness curve of the haze (*HBβ*) with *A*=1, θ=40°, *σ*=0.00005 ft-1, *h*= 16,500 ft

a)

b)

Fig. 8. Curves of the factor haze percentages with *σ =* 0.00005 ft-1, *σ* = 0.00008 ft-1,

h = 16,000ft. a) *θ* = 40º b) *θ* = 60º

*f*(*Φ*), is the scattering phase function, which describes the angular distribution of scattered radiation (*Φ* here is the scattering angle). The scattering phase function *f*(*Φ*) depends on the density and physical characteristics of the particles and the wavelength. As you increase the size of the particles there is a stronger forward scattering (*Φ* = 0 °) and a smaller but still significant back-scattering (*Φ* = 180°).

Doing the second part of left side of (3) as:

$$G = \frac{\cos \theta}{\cos \beta + \cos \theta} \Big[ 1 - e^{-\sigma h (\sec \beta + \sec \theta)} \Big] \tag{4}$$

We now have the brightness of the haze as:

$$H^{B}\beta = A.f(\phi).G\tag{5}$$

The brightness of a white reflector is given by:

$$\mathbf{G}^{\mathcal{B}}\boldsymbol{\beta} = \boldsymbol{A} \frac{\cos \theta}{\pi} \left[ e^{-\sigma \hbar \mathbf{\hat{n}} \cdot \mathbf{\hat{s}} + \mathbf{s} \mathbf{e} \cdot \boldsymbol{\theta}} \right] \tag{6}$$

The haze factor in percentage is therefore:

$$H\beta = 100\pi f(\phi) \frac{e^{\sigma h(\sec\beta + \sec\theta)} - 1}{\cos\beta + \cos\theta} \tag{7}$$

Where *A* is a constant and depends of sun illumination.

The locus of *G* with respect to *β* is a smooth curve with minimum at *β* = 0 and symmetrical of about *OY*.

The curve of HB *β* for *θ* = 40 °, *σ* = 0.00005 ft-1, *h* = 16000 ft is shown on Figure 7. The value of σ, estimated at 5x 10-1 ft-1 corresponds to the daytime visibility of 20 miles. The curve of *f*(*Φ*) against Φ is generally U shaped and the values used by Hall are in Table 1.


Table 1. Values of scattering phase function *f*(Φ) against Φ used by Hall(1954).

The strong variation over a short range of *β* in the neighbourhood of *β* =-*θ* and = 180 ° means a variation which is proportional to the haze lightness that illuminates the corresponding region of the photograph (Hall, 1954).

Figures 8a and 8b show graphs of the haze factor percentage calculated for *σ* = 0.00005 ft-1, *σ* =0.00008 ft -1, *h* = 16,000 ft, angles *θ* = 40 ° and *θ* = 60°. The haze factor is always greater on the side opposite the sun (*β* = -40º in figure 8a). There is a sharp peak when *σ* increases around *θ* = 40º. If the semi-lens viewing angle (*β)* is 45° and *θ* = 60° the maximum haze factor would be out of the image. (Figure 8b).

*f*(*Φ*), is the scattering phase function, which describes the angular distribution of scattered radiation (*Φ* here is the scattering angle). The scattering phase function *f*(*Φ*) depends on the density and physical characteristics of the particles and the wavelength. As you increase the size of the particles there is a stronger forward scattering (*Φ* = 0 °) and a smaller but still

cos (sec sec <sup>1</sup>

 

*<sup>h</sup> e*

The locus of *G* with respect to *β* is a smooth curve with minimum at *β* = 0 and symmetrical

The curve of HB *β* for *θ* = 40 °, *σ* = 0.00005 ft-1, *h* = 16000 ft is shown on Figure 7. The value of σ, estimated at 5x 10-1 ft-1 corresponds to the daytime visibility of 20 miles. The curve of *f*(*Φ*)

Φ 0 10 20 30 40 50 60 70 80 90 *f*(*Φ*) 0.470 0.400 0.310 0.220 0.160 0.110 0.070 0.050 0.040 0.035

The strong variation over a short range of *β* in the neighbourhood of *β* =-*θ* and = 180 ° means a variation which is proportional to the haze lightness that illuminates the

Figures 8a and 8b show graphs of the haze factor percentage calculated for *σ* = 0.00005 ft-1, *σ* =0.00008 ft -1, *h* = 16,000 ft, angles *θ* = 40 ° and *θ* = 60°. The haze factor is always greater on the side opposite the sun (*β* = -40º in figure 8a). There is a sharp peak when *σ* increases around *θ* = 40º. If the semi-lens viewing angle (*β)* is 45° and *θ* = 60° the maximum haze factor

Φ 100 110 120 130 140 150 160 170 180 *f*(*Φ*) 0.030 0.030 0.040 0.045 0.060 0.080 0.100 0.160 0.030

Table 1. Values of scattering phase function *f*(Φ) against Φ used by Hall(1954).

 

 

(sec sec ) 1

 

cos cos

   (4)

(6)

(7)

*f G* (5)

cos cos *<sup>h</sup> G e* 

 

. ( ). *<sup>B</sup> H A* 

*<sup>B</sup>* cos *<sup>h</sup>*(sec sec *GA e* 

100 ( )

 

against Φ is generally U shaped and the values used by Hall are in Table 1.

*H f*

Where *A* is a constant and depends of sun illumination.

corresponding region of the photograph (Hall, 1954).

would be out of the image. (Figure 8b).

significant back-scattering (*Φ* = 180°).

Doing the second part of left side of (3) as:

We now have the brightness of the haze as:

The brightness of a white reflector is given by:

The haze factor in percentage is therefore:

of about *OY*.

Fig. 7. Brightness curve of the haze (*HBβ*) with *A*=1, θ=40°, *σ*=0.00005 ft-1, *h*= 16,500 ft

Fig. 8. Curves of the factor haze percentages with *σ =* 0.00005 ft-1, *σ* = 0.00008 ft-1, h = 16,000ft. a) *θ* = 40º b) *θ* = 60º

Color Restoration of Aerial Photographs 11

Peaks of sun light create white spots or white areas on digital images when the amount of concentrated light exceeds the dynamic range of sensitivity of the CCD sensor. This problem can be avoided with proper planning of the photogrammetric flight, knowing of sun ephemeris and, nowadays, it can be mitigated with the use of digital images with more than eight bits per color channel. Ashikhmin (2002) and Reinhard et al. (2002) can make details visible in parts that are too light or too dark, using algorithms that adapt contrast intervals, to map levels of the original image to the levels likely to be played on the monitor screen or

Among the processes that can correct non-uniform illumination in digital images, that are available in commercial programs, the most common is the one that carries out the balancing of histograms between blocks, through homogenization of statistical parameters. The correction is done by dividing the original images into blocks, or into sub-images, calculating diverse statistics as global minimum and maximum averages of each block; and processing the histograms, so that they balance the differences of brightness intensity among the blocks. This method can be used both among photographs of a mosaic andin isolated photographs for correcting vignetting effects, bright areas and BRDF. A description of this method, together with some examples of variation, can be found in Li et al., (2004);

Fig. 10. Photograph processed using histograms blocks balancing, showing transitions between areas with significant tone differences. (Photograph: TOPOCART S/A)

**3.2 Corrections of specular reflection** 

**3.3 Correction by manipulation of histogram blocks** 

Paparodis et al. (2006) and Nobrega & Quintanilha (2004).

in print.

With the given formulas it is possible to estimate the positions of the brighter areas and the gradient of change of lighting in the photographs, using data from flying reports and sun ephemeris.

## **3. Restoration of colors**

The colors of objects and landscapes in the photographs altered by any of the various factors discussed above can be restored, at least in part, with the help of processing image methods, so that they are more realistic, more uniform between adjacent photos and more uniform with respect to the frame itself.

In this section some of these procedures will be applied. In photogrammetry and remote sensing the methods used in general are: application of functions to correct vignetting (Homma et al, 2000), models of atmospheric radiation and illumination (Beisl&Woodhouse, 2004), manipulation of histograms (Kraus, 1997; Tuominen & Pekkarinen, 2004; Nobrega & Quintanilha, 2004), restoration with the use of RGB color space, HSI-RGB (Guo & Moore, 1993), use of masks (Li et al, 2004a, Silva & Candeias, 2008), burning and dodging (Reinhard et al, 2002; Li et al, 2004a).

#### **3.1 Restoration of haze**

During the taking of the photographs the attenuation of haze effects can be done using a yellow filter placed in front of the lense that absorbs the excess blue light. In digital photogrammetry the degradation, in digitalized images or those obtained directly from a digital camera, can be corrected using radiometric processing functions, color correction or radiometric atmospheric models. Figure 9 shows an original photograph and the result after color casting correction using Kries method explained in section 3.6.

Fig. 9. a) Photograph with uniform haze, b) Photograph processed using Kries method.

With the given formulas it is possible to estimate the positions of the brighter areas and the gradient of change of lighting in the photographs, using data from flying reports and sun

The colors of objects and landscapes in the photographs altered by any of the various factors discussed above can be restored, at least in part, with the help of processing image methods, so that they are more realistic, more uniform between adjacent photos and more uniform

In this section some of these procedures will be applied. In photogrammetry and remote sensing the methods used in general are: application of functions to correct vignetting (Homma et al, 2000), models of atmospheric radiation and illumination (Beisl&Woodhouse, 2004), manipulation of histograms (Kraus, 1997; Tuominen & Pekkarinen, 2004; Nobrega & Quintanilha, 2004), restoration with the use of RGB color space, HSI-RGB (Guo & Moore, 1993), use of masks (Li et al, 2004a, Silva & Candeias, 2008), burning and dodging (Reinhard

During the taking of the photographs the attenuation of haze effects can be done using a yellow filter placed in front of the lense that absorbs the excess blue light. In digital photogrammetry the degradation, in digitalized images or those obtained directly from a digital camera, can be corrected using radiometric processing functions, color correction or radiometric atmospheric models. Figure 9 shows an original photograph and the result after

a) b)

Fig. 9. a) Photograph with uniform haze, b) Photograph processed using Kries method.

color casting correction using Kries method explained in section 3.6.

ephemeris.

**3. Restoration of colors** 

with respect to the frame itself.

et al, 2002; Li et al, 2004a).

**3.1 Restoration of haze** 

#### **3.2 Corrections of specular reflection**

Peaks of sun light create white spots or white areas on digital images when the amount of concentrated light exceeds the dynamic range of sensitivity of the CCD sensor. This problem can be avoided with proper planning of the photogrammetric flight, knowing of sun ephemeris and, nowadays, it can be mitigated with the use of digital images with more than eight bits per color channel. Ashikhmin (2002) and Reinhard et al. (2002) can make details visible in parts that are too light or too dark, using algorithms that adapt contrast intervals, to map levels of the original image to the levels likely to be played on the monitor screen or in print.

#### **3.3 Correction by manipulation of histogram blocks**

Among the processes that can correct non-uniform illumination in digital images, that are available in commercial programs, the most common is the one that carries out the balancing of histograms between blocks, through homogenization of statistical parameters. The correction is done by dividing the original images into blocks, or into sub-images, calculating diverse statistics as global minimum and maximum averages of each block; and processing the histograms, so that they balance the differences of brightness intensity among the blocks. This method can be used both among photographs of a mosaic andin isolated photographs for correcting vignetting effects, bright areas and BRDF. A description of this method, together with some examples of variation, can be found in Li et al., (2004); Paparodis et al. (2006) and Nobrega & Quintanilha (2004).

Fig. 10. Photograph processed using histograms blocks balancing, showing transitions between areas with significant tone differences. (Photograph: TOPOCART S/A)

Color Restoration of Aerial Photographs 13

The calculated values are used in the interpolations to get the new gray value image

 <sup>255</sup> ' min max min *GL DN GL*

This algorithm provides good results but has high computational cost and some variations have been proposed that use only mean and standard deviation compared with a global mean value. In general, all promise gain of brightness for darkened areas and darkening of light regions. This type has been used in commercial programs for balancing color aerial photographs and satellite images and digital mosaics in general. For more details consult

The more common methods of manipulations of histogram are equalization and stretching (or normalization). They are applied for enhance contrast of images. Histogram Equalization seeks to produce a balanced image with a uniform distribution of grey tones. Histogram normalization adopts the normal distribution as the transfer function for the

Furthermore is possible to change the histogram of an image using the histogram of another image as model. This procedure is called histogram matching and figure 12 shows a example. The histogram of figure 12a is of desired image, used as reference, in figure 12b is the histogram of image to be changed and in figure 12c is the histogram of new image. The

a) b) c)

Fig. 12. Example of histogram matching. a) Histogram of reference image, b) Histogram to

*GL GL* (9)

Where: *GLmin*= minimum value of brightness of the pixel. *GLmax*= maximum value of brightness of the pixel

 *x* = column for the position of the pixel in the block *y* = row for the position of the pixel in the block

Where: *GL'* = new value of brightness to be applied to pixel.

 *X* = dimension of the block columns *Y* = dimension of the block in rows

*DN* = value of brightness of the pixel

Nobrega & Quintanilla (2001).

desired image.

**3.4 Global histogram manipulation** 

final result is very similar to figure 12a.

be changed, c) Histogram changed.

produced:

The method, although well diffused, does not always have good results, principally if the photographs, or areas of photographs, have significant intensity differences or abrupt variations in tonalities. This problem can be seen in examples shown in (Nobrega & Quintanilha,2004; Li et al., 2004a; Wu & Campbell, 2004; Paparodis et al., 2006). Figure 10 shows an example of a photograph processed with this type of manipulation of histograms. The amplified details show the color gradients which appear in the transition zones between clear and dark features in the images, and among different kinds of vegetation.

The algorithms to manipulating of histograms blocks divide the image into sub-images, and after that it calculates parameters such as mean, standard deviation, minimum and maximum gray levels of each color channel and as individual histograms. There is a variation called LRM (Local Range Modification) and the definition is presented in Schowendgert (1997). In this process, after the standardization of histograms, a smoothing process near the edges of each sub-image is adopted to avoid the appearance of breaks in continuity. Thus the maximum (MAX) and minimum (MIN) values are calculated for each vertex of the block, the minimum average (LA, LB, LD, L*E*) and average maximum values (*HA*, *HB*, *HD*, *HE*) of neighboring blocks, as shown in Figure 11.

Fig. 11. Blocks and parameters used in the LRM (Schowengerdt, 1997).

The interpolation for the *xy* position in the above figure uses the maximum and minimum values of the vertices 6, 7, 10 and 11 and provides the values given by the equations below:

$$\text{GL.min} = \left[\frac{\text{x}}{\text{X}} \text{MIN}\_{\text{T}} + \left(\frac{\text{X} - \text{x}}{\text{X}}\right) \text{MIN}\_{6}\right] \left(\frac{\text{Y} - \text{y}}{\text{Y}}\right) + \left[\frac{\text{x}}{\text{X}} \text{MIN}\_{11} + \left(\frac{\text{X} - \text{x}}{\text{X}}\right) \text{MIN}\_{10}\right] \frac{\text{y}}{\text{Y}}$$

$$\text{GL.max} = \left[\frac{\text{x}}{\text{X}} \text{MAX}\_{\text{T}} + \left(\frac{\text{X} - \text{x}}{\text{X}}\right) \text{MAX}\_{6}\right] \left(\frac{\text{Y} - \text{y}}{\text{Y}}\right) + \left[\frac{\text{x}}{\text{X}} \text{MAX}\_{11} + \left(\frac{\text{X} - \text{x}}{\text{X}}\right) \text{MAX}\_{10}\right] \frac{\text{y}}{\text{Y}}\tag{8}$$

The method, although well diffused, does not always have good results, principally if the photographs, or areas of photographs, have significant intensity differences or abrupt variations in tonalities. This problem can be seen in examples shown in (Nobrega & Quintanilha,2004; Li et al., 2004a; Wu & Campbell, 2004; Paparodis et al., 2006). Figure 10 shows an example of a photograph processed with this type of manipulation of histograms. The amplified details show the color gradients which appear in the transition zones between

The algorithms to manipulating of histograms blocks divide the image into sub-images, and after that it calculates parameters such as mean, standard deviation, minimum and maximum gray levels of each color channel and as individual histograms. There is a variation called LRM (Local Range Modification) and the definition is presented in Schowendgert (1997). In this process, after the standardization of histograms, a smoothing process near the edges of each sub-image is adopted to avoid the appearance of breaks in continuity. Thus the maximum (MAX) and minimum (MIN) values are calculated for each vertex of the block, the minimum average (LA, LB, LD, L*E*) and average maximum values

clear and dark features in the images, and among different kinds of vegetation.

(*HA*, *HB*, *HD*, *HE*) of neighboring blocks, as shown in Figure 11.

Fig. 11. Blocks and parameters used in the LRM (Schowengerdt, 1997).

The interpolation for the *xy* position in the above figure uses the maximum and minimum values of the vertices 6, 7, 10 and 11 and provides the values given by the equations below:

> min 76 11 10 *x Xx <sup>Y</sup> y y x Xx GL MIN MIN MIN MIN*

> max <sup>7</sup> <sup>6</sup> <sup>11</sup> <sup>10</sup> *x Xx <sup>Y</sup> y y x Xx GL MAX MAX MAX MAX*

*X X YX X Y* 

*X X YX X Y* 

(8)

Where: *GLmin*= minimum value of brightness of the pixel.

*GLmax*= maximum value of brightness of the pixel


The calculated values are used in the interpolations to get the new gray value image produced:

$$GL = \frac{255}{GL.\max - GL.\min} \left( DN - GL.\min\right) \tag{9}$$

Where: *GL'* = new value of brightness to be applied to pixel.

*DN* = value of brightness of the pixel

This algorithm provides good results but has high computational cost and some variations have been proposed that use only mean and standard deviation compared with a global mean value. In general, all promise gain of brightness for darkened areas and darkening of light regions. This type has been used in commercial programs for balancing color aerial photographs and satellite images and digital mosaics in general. For more details consult Nobrega & Quintanilla (2001).

#### **3.4 Global histogram manipulation**

The more common methods of manipulations of histogram are equalization and stretching (or normalization). They are applied for enhance contrast of images. Histogram Equalization seeks to produce a balanced image with a uniform distribution of grey tones. Histogram normalization adopts the normal distribution as the transfer function for the desired image.

Furthermore is possible to change the histogram of an image using the histogram of another image as model. This procedure is called histogram matching and figure 12 shows a example. The histogram of figure 12a is of desired image, used as reference, in figure 12b is the histogram of image to be changed and in figure 12c is the histogram of new image. The final result is very similar to figure 12a.

Fig. 12. Example of histogram matching. a) Histogram of reference image, b) Histogram to be changed, c) Histogram changed.

Color Restoration of Aerial Photographs 15

The correction or alteration of colors in digital images car be carried out through several methods using operations in color space and transformations between them, however, a good experience of the human operator is necessary. A method that could be more

The Von Krie hypothesis method considers that the primary stimuli of *RGB* color in the retina can be linked to the imaginary stimuli (*XYZ*) by a linear transformation with matrix

> *R X G MY B Z*

This hypothesis is commonly used in image acquisition devices (such as cameras and scanners) to correct image lighting (sunlight, incandescent or fluorescent lamps) in a

> 0 0 0 0 0 0

*R kR R G kG G B kB B* 

Where *R'G'B'* is image desired, and *RGB* is the original image. The coefficients are obtained

' '' ; ; *RGB kR kG kB*

(11)

(12)

*RGB* (13)

M (Wyszecki & Stiles, 1982). Equation 11 shows the relation of *RGB* with *XYZ*.

'

'

'

from the sample initially and then applied to the entire image

Fig. 13. Examples of masks to correcting illumination intensities.

**3.6 Corrections using Kries method** 

simplified manner, in two ways:

A) white balancing

independent is based on the Von Krie hypothesis.

This method is known as image white balancing

Most of image processing software has the capacity of handle with histograms and change them. More details about theory can be found in (Kraus, 1997; Pratt, 1991).

#### **3.5 Correction with masks**

The masks in image processing are binary images or gray tones that are used for delimiting areas where certain operations can be carried out, or to control the degree of processing they can go through. Examples of binary image masks are the delimiting polygons of each of the images that form mosaics. Masks in gray tones are already used by photographers and the graphic industry for attenuating shadows of scenery and environment.

However, as these resources have not been used in photogrammetry, an analysis of the viability of the use of masks is carried out to correct non-uniform illumination in aerial photographs. For this, initially, a complete sequence of a mask construction will be detailed.

A mask should represent the mean illumination intensities of the combined effects of vignetting and DBRF. Considering that the variation in luminosity which reaches the plan of the negative does not vary significantly among the photographs of the same area or when taken in a short period of time, a mask should suffice for processing a group of neighboring aerial photographs.

For the construction of the masks (Figure 13) only one photograph would be used, but the mask can get better if two or more are used, so that the result can be free of differences in the tonality of the scenery. The process can start by two ways: a) profiles of gray values of the image pixels in lines and columns, along the borders, through the center and intermediate regions; b) or using averages values of sub-blocks. This latter process eliminates better the high frequencies. Therefore, when using polynomial regression of a cubic function, as in equation 10, an adequate fitting for a uniform surface is obtained without abrupt changes:

$$\mathbf{Z(x,y) = A + By + Cy^2 + Dy^3 + Ex + Fxy + Gxy^2 + Hx^2 + Ix^2 + Ix^3} \tag{10}$$

Where: *x*,*y* = coordinates of the points equivalent to the positions of the pixels or centers of sub-blocks

 *Z*(*x*,*y*) = the ordinate, or grey value, in the *x*,*y* position

 *A* to *J* = the coefficients obtained with a sample entry.

The masks can be constructed in accordance with the available resources and the ones used here were obtained by following the steps below:


Most of image processing software has the capacity of handle with histograms and change

The masks in image processing are binary images or gray tones that are used for delimiting areas where certain operations can be carried out, or to control the degree of processing they can go through. Examples of binary image masks are the delimiting polygons of each of the images that form mosaics. Masks in gray tones are already used by photographers and the

However, as these resources have not been used in photogrammetry, an analysis of the viability of the use of masks is carried out to correct non-uniform illumination in aerial photographs. For this, initially, a complete sequence of a mask construction will be

A mask should represent the mean illumination intensities of the combined effects of vignetting and DBRF. Considering that the variation in luminosity which reaches the plan of the negative does not vary significantly among the photographs of the same area or when taken in a short period of time, a mask should suffice for processing a group of neighboring

For the construction of the masks (Figure 13) only one photograph would be used, but the mask can get better if two or more are used, so that the result can be free of differences in the tonality of the scenery. The process can start by two ways: a) profiles of gray values of the image pixels in lines and columns, along the borders, through the center and intermediate regions; b) or using averages values of sub-blocks. This latter process eliminates better the high frequencies. Therefore, when using polynomial regression of a cubic function, as in equation 10, an adequate fitting for a uniform surface is obtained

 Z(*x*,*y*) =*A*+*By*+*Cy*²+*Dy*3+*Ex*+*Fxy*+*Gxy*²+*Hx*²+*Ix*²+*Jx*3 (10) Where: *x*,*y* = coordinates of the points equivalent to the positions of the pixels or centers

The masks can be constructed in accordance with the available resources and the ones used

Division of image forming a grid of 5x5 (25 blocks), trying to eliminate the margins and

Calculation of mean intensity of each of the 25 blocks and registration of the minimum

 *Z*(*x*,*y*) = the ordinate, or grey value, in the *x*,*y* position  *A* to *J* = the coefficients obtained with a sample entry.

Interpolation of *Z* values with a cubic or quadratic function.

Transformation of colored images into gray tones at levels of 0 to 255.

here were obtained by following the steps below:

them. More details about theory can be found in (Kraus, 1997; Pratt, 1991).

graphic industry for attenuating shadows of scenery and environment.

**3.5 Correction with masks** 

detailed.

aerial photographs.

without abrupt changes:

of sub-blocks

the fiducial marks.

and maximum levels.

conversion to negative image

#### **3.6 Corrections using Kries method**

The correction or alteration of colors in digital images car be carried out through several methods using operations in color space and transformations between them, however, a good experience of the human operator is necessary. A method that could be more independent is based on the Von Krie hypothesis.

The Von Krie hypothesis method considers that the primary stimuli of *RGB* color in the retina can be linked to the imaginary stimuli (*XYZ*) by a linear transformation with matrix M (Wyszecki & Stiles, 1982). Equation 11 shows the relation of *RGB* with *XYZ*.

$$
\begin{bmatrix} R \\ G \\ B \end{bmatrix} = M \begin{bmatrix} X \\ Y \\ Z \end{bmatrix} \tag{11}
$$

This hypothesis is commonly used in image acquisition devices (such as cameras and scanners) to correct image lighting (sunlight, incandescent or fluorescent lamps) in a simplified manner, in two ways:

#### A) white balancing

This method is known as image white balancing

$$
\begin{bmatrix}
\boldsymbol{R'} \\
\boldsymbol{G'} \\
\boldsymbol{B'} \\
\end{bmatrix} = \begin{bmatrix}
k\boldsymbol{R} & \boldsymbol{0} & \boldsymbol{0} \\
\boldsymbol{0} & k\boldsymbol{G} & \boldsymbol{0} \\
\boldsymbol{0} & \boldsymbol{0} & k\boldsymbol{B}
\end{bmatrix} \begin{bmatrix}
\boldsymbol{R} \\
\boldsymbol{G} \\
\boldsymbol{B} \\
\end{bmatrix} \tag{12}
$$

Where *R'G'B'* is image desired, and *RGB* is the original image. The coefficients are obtained from the sample initially and then applied to the entire image

$$kR = \frac{R}{R} ; kG = \frac{G}{G} ; kB = \frac{B}{B} \tag{13}$$

Color Restoration of Aerial Photographs 17

Fig. 14. Correction of vignetting effect of left image showed in the right.

145, average= 143 , factor = 1.161. The result is shown on Figure 15b.

For the color photograph (Figure 15a) the calculation by bands was performed: Red, center 200, edges 173, 155, 157 and, 161, average= 161, factor = 1.238; Green, center 184, edges 143, 142, 153 and 140, average = 144.5, factor = 1.273; Blue, Center 166 , edges 147, 141, 139 and

a) b) Fig. 15. Vigneting in color photograph a) Original image, b) With Vignetting correction.

Another way of using *fcor* is to create a mask that can be applied to all images in a range of line flight, for example. Thus processing is faster, compared to methods of manipulating

histograms, because you just add the masks of each band to the bands of the images.

Where *R*, *G*, *B* are the average of the desired area. *R'*= *G'* = *B'*= 255, because they correspond to a white area and to images of eight bits per channel.

B) When the lighting is not known

In this case the process is applied in a representative area of the image and the maximum level of intensity in each band is determined. With this procedure *R* = max (*R*), *G* = max (*G*) and *B* = max (*B*) and *R* ', *G*' and *B* 'are also equal to 255.

In this study the method was applied to change the colors of an image with undesirable color based on another image that shows the desired tone, or even the same image in different areas.

## **4. Results and discussion**

#### **4.1 Vignetting correcting**

Vignetting corrretion was presented in Equation 1. The image in grey format is sized for 1/8 of original size, and applied a filter like Gaussian to blur the image and to get the grey levels in the center and the borders. The final calculations use Equations 14 and 15 with the following adaptations: a) consideration of the maximum radius equal to diagonal of image frame and each pixel radius equal distance between pixel and image centre; b) for simplicity, cos(*b*) calculation uses the focal length equal to the width of the image; c) the maximum correction occurs at border and at center it is null.

During the calculation the radius is obtained and the correction for each pixel is given by *fcor*. The radius is the distance pixel-center. The gray level correction is:

$$fcor = corr \max^\* \frac{(1 - \cos \beta)}{(1 - \cos \beta \max)} \tag{14}$$

Where: cos=cos(b) at any point cosmax=cos(b) at border The pixel color is corrected by:

$$\mathbf{I}(\mathbf{i}, \mathbf{j}) = f \alpha r + \mathbf{I}(\mathbf{i}, \mathbf{j}) \tag{15}$$

An example of this final processing is shown on Figure 14b. The calculation of the average gray in a reduced image was 190 in the center and at the edges it was 148, 157, 145 and 147, with an average of 149; factor 190/149 = 1.275. After applying the corrections the dark border is eliminated.

The Minimum (Min), Maximum (Max), Mean and Standard deviation (Stdev) of the Figure 14 are showed in table 2. The global effect with the vignetting correction is to grow up the minimum and mean and reduce the standard deviation of the result image.


Table 2. Basic Statistics of Figure 14

Where *R*, *G*, *B* are the average of the desired area. *R'*= *G'* = *B'*= 255, because they correspond

In this case the process is applied in a representative area of the image and the maximum level of intensity in each band is determined. With this procedure *R* = max (*R*), *G* = max (*G*)

In this study the method was applied to change the colors of an image with undesirable color based on another image that shows the desired tone, or even the same image in

Vignetting corrretion was presented in Equation 1. The image in grey format is sized for 1/8 of original size, and applied a filter like Gaussian to blur the image and to get the grey levels in the center and the borders. The final calculations use Equations 14 and 15 with the following adaptations: a) consideration of the maximum radius equal to diagonal of image frame and each pixel radius equal distance between pixel and image centre; b) for simplicity, cos(*b*) calculation uses the focal length equal to the width of the image; c) the

During the calculation the radius is obtained and the correction for each pixel is given by

 I(i,j)= *fcor* + I(i,j) (15) An example of this final processing is shown on Figure 14b. The calculation of the average gray in a reduced image was 190 in the center and at the edges it was 148, 157, 145 and 147, with an average of 149; factor 190/149 = 1.275. After applying the corrections the dark

The Minimum (Min), Maximum (Max), Mean and Standard deviation (Stdev) of the Figure 14 are showed in table 2. The global effect with the vignetting correction is to grow up the

**Basic Stats Min** Max **Mean Stdev** Fig. 14(a) 22 255 153.014712 34.643259 Fig. 14(b) 77 255 178.653428 30.067176

minimum and mean and reduce the standard deviation of the result image.

(1 cos ) max\* (1 max) *fcor corr co*

(14)

to a white area and to images of eight bits per channel.

and *B* = max (*B*) and *R* ', *G*' and *B* 'are also equal to 255.

maximum correction occurs at border and at center it is null.

*fcor*. The radius is the distance pixel-center. The gray level correction is:

B) When the lighting is not known

**4. Results and discussion** 

Where: cos=cos(b) at any point cosmax=cos(b) at border The pixel color is corrected by:

Table 2. Basic Statistics of Figure 14

border is eliminated.

**4.1 Vignetting correcting** 

different areas.

Fig. 14. Correction of vignetting effect of left image showed in the right.

For the color photograph (Figure 15a) the calculation by bands was performed: Red, center 200, edges 173, 155, 157 and, 161, average= 161, factor = 1.238; Green, center 184, edges 143, 142, 153 and 140, average = 144.5, factor = 1.273; Blue, Center 166 , edges 147, 141, 139 and 145, average= 143 , factor = 1.161. The result is shown on Figure 15b.

Fig. 15. Vigneting in color photograph a) Original image, b) With Vignetting correction.

Another way of using *fcor* is to create a mask that can be applied to all images in a range of line flight, for example. Thus processing is faster, compared to methods of manipulating histograms, because you just add the masks of each band to the bands of the images.

Color Restoration of Aerial Photographs 19

a) Original. b) With mask. c) With histogram blocks

The basics statistics in Table 4 show that the strip with pre-processed images is brighter (Figure 17a) and it has lesser standard deviation than with rough image (Figure 17b).

Band: R

Min: 4.00 Max: 255.00 Mean: 128.19 Standard Deviation:

46.73

45.96

35.74

Min: 4.00 Max: 255.00 Mean: 112.01 Standard Deviation:

Min: 4.00 Max: 255.00 Mean: 127.36 Standard Deviation:

Band: G

Band: B

In Figure 17a some difference of tonality can still be seen in the stick zones. This could be due to the use of only a single photograph for preparing the mask and the influence of the features of the land as the vegetation. Ongoing tests show that an average mask obtained from more than one photograph represents, with more accuracy, the combined illumination

Fig. 16. Comparing processed images with mask and histogram manipulation

With pre-processed images With rough images

Qualitatively it can be seen in Figure 16.

Band: R

Basic Statistics

33.79

31.68

27.23

Band: G

Band: B

effects of the BRDF, haze and vignette.

Table 4. Basic Statistics

Min: 57.00 Max: 255.00 Mean: 164.81 Standard Deviation:

Min: 57.00 Max: 255.00 Mean: 162.85 Standard Deviation:

Min: 51.00 Max: 255.00 Mean: 145.13 Standard Deviation:

#### **4.2 Application of masks**

One of the processing results with an individually added mask to the *RGB* bands of Figure 16a, is shown on Figure 16b. The very dark tonality in the inferior part of this figure was corrected. The colors of the darker parts are not well recuperated because they were saturated. This problem will be resolved in section 4.3.

The block balancing method was also applied to the same photograph and the result is shown on Figure 16c, where the mean tonality of the image is more uniform, but the inferior part is still very dark, aside from presenting the artifacts already shown on Figure 10

Table 2 shows the basic statistics of original image, image balanced with mask and with histogram blocks images of Figure 16. Observing the standard deviation (Table 3), it is possible to see that there is more variability of colors in original image and this effect is corrected in the other ones.

The method of mask addition, although it does not introduce artifacts and undesired color transitions, it is computationally more efficient, since the same mask can be used in many photographs taken at a certain interval of flight time, as in the strip (Figure 17). Moreover, the process of adding original image with the mask is quicker than the processing of the multiplications and divisions involved in other methods. For an image of 11,500 x 11,500 (pixels) and blocks of 100 x 100, the total time was 3min 40s with manipulation of histogram blocks, and only 1min 10s with the mask, on the same computer. The preparation time for the mask depends on the program used, but it can be totally automated.


Table 3. Basic Statistics

One of the processing results with an individually added mask to the *RGB* bands of Figure 16a, is shown on Figure 16b. The very dark tonality in the inferior part of this figure was corrected. The colors of the darker parts are not well recuperated because they were

The block balancing method was also applied to the same photograph and the result is shown on Figure 16c, where the mean tonality of the image is more uniform, but the inferior

Table 2 shows the basic statistics of original image, image balanced with mask and with histogram blocks images of Figure 16. Observing the standard deviation (Table 3), it is possible to see that there is more variability of colors in original image and this effect is

The method of mask addition, although it does not introduce artifacts and undesired color transitions, it is computationally more efficient, since the same mask can be used in many photographs taken at a certain interval of flight time, as in the strip (Figure 17). Moreover, the process of adding original image with the mask is quicker than the processing of the multiplications and divisions involved in other methods. For an image of 11,500 x 11,500 (pixels) and blocks of 100 x 100, the total time was 3min 40s with manipulation of histogram blocks, and only 1min 10s with the mask, on the same computer. The preparation time for

Basic Statistics

**Original With mask With block balancing** 

Band: R

Min: 4.00 Max: 255.00 Mean: 135.82 Standard Deviation:

37.63

36.87

30.90

Min: 0.00 Max: 255.00 Mean: 116.43 Standard Deviation:

**Band: B** 

Min: 0.00 Max: 255.00 Mean: 133.27 Standard Deviation:

**Band: G** 

part is still very dark, aside from presenting the artifacts already shown on Figure 10

the mask depends on the program used, but it can be totally automated.

**Band: R** 

Min: 4.00 Max: 255.00 Mean: 149.52 Standard Deviation:

34.82

33.64

28.97

Min: 4.00 Max: 255.00 Mean: 136.21 Standard Deviation:

**Band: B** 

Min: 4.00 Max: 255.00 Mean: 148.28 Standard Deviation:

**Band: G** 

**4.2 Application of masks** 

corrected in the other ones.

**Band: R** 

Min: 4.00 Max: 255.00 Mean: 139.31 Standard Deviation:

58.64

56.47

46.76

Table 3. Basic Statistics

Min: 0.00 Max: 255.00 Mean: 117.58 Standard Deviation:

**Band: B** 

Min: 0.00 Max: 255.00 Mean: 136.70 Standard Deviation:

**Band: G** 

saturated. This problem will be resolved in section 4.3.

a) Original. b) With mask. c) With histogram blocks

Fig. 16. Comparing processed images with mask and histogram manipulation

The basics statistics in Table 4 show that the strip with pre-processed images is brighter (Figure 17a) and it has lesser standard deviation than with rough image (Figure 17b). Qualitatively it can be seen in Figure 16.


Table 4. Basic Statistics

In Figure 17a some difference of tonality can still be seen in the stick zones. This could be due to the use of only a single photograph for preparing the mask and the influence of the features of the land as the vegetation. Ongoing tests show that an average mask obtained from more than one photograph represents, with more accuracy, the combined illumination effects of the BRDF, haze and vignette.

Color Restoration of Aerial Photographs 21

Various tests of mask use in colored aerial photographs were done with rural and urban

 The correction works well for images with sceneries without significant variations of texture and tonality. If strong variations occur the masks must be made using an

The saturated colors in the borders of the original images are not recuperated and a pre

 In case the mask is added to the HSV decomposition component V, the recomposed image keeps the tone of the areas of more altered saturated hues (alteration of hue),

The general color aspects of an image should be adjusted to a color standard of another image using the Von Kries hypothesis. Similar procedure can also be applied using only color band adjustment present in most graphic processing software, but some practical

The image for this kind of color correction must not be saturated. The saturation can be indicated by the presence of too many pixels (peak) at the ends of the histogram, in one or more bands. The histogram of image in Figure 19 shows that it has not color saturation.

a) b)

The image in figure 19 was processed with Kries method and using color cast of figure 20 resulting in image showed in figure 21. The coefficients were KR=1.0759, KG=1.1751,

Fig. 19. a) Image example without color saturation, b) Histograms of the RGB bands.

sceneries in varied scales, and the following was observed:

average of three or more photographs.

than when processing with *RGB* bands.

experience from the operator is necessary.

processing is need.

**4.3 Correction of full frame 4.3.1 Using Kries method** 

KB=1.1451

Fig. 17. Example of a mosaic strip. a) Pre-processed with masks, b) With rough images. (Photograph: TOPOCART S/A).

However, the greater uniformity of tones observed in the images processed with masks is already enough to significantly improve the quality of a mosaic that is reprocessed with block balancing, as shown on Figure 18a, while on Figure 18b there are still areas with more shadows.

Fig. 18. Mosaics processed with block balancing. a) Photographs pre-processed with masks, b) original photographs. (Photographs: TOPOCART S/A ).

a)

b)

However, the greater uniformity of tones observed in the images processed with masks is already enough to significantly improve the quality of a mosaic that is reprocessed with block balancing, as shown on Figure 18a, while on Figure 18b there are still areas with more

a)

b) Fig. 18. Mosaics processed with block balancing. a) Photographs pre-processed with masks,

b) original photographs. (Photographs: TOPOCART S/A ).

Fig. 17. Example of a mosaic strip. a) Pre-processed with masks, b) With rough images.

(Photograph: TOPOCART S/A).

shadows.

Various tests of mask use in colored aerial photographs were done with rural and urban sceneries in varied scales, and the following was observed:


#### **4.3 Correction of full frame**

#### **4.3.1 Using Kries method**

The general color aspects of an image should be adjusted to a color standard of another image using the Von Kries hypothesis. Similar procedure can also be applied using only color band adjustment present in most graphic processing software, but some practical experience from the operator is necessary.

The image for this kind of color correction must not be saturated. The saturation can be indicated by the presence of too many pixels (peak) at the ends of the histogram, in one or more bands. The histogram of image in Figure 19 shows that it has not color saturation.

Fig. 19. a) Image example without color saturation, b) Histograms of the RGB bands.

The image in figure 19 was processed with Kries method and using color cast of figure 20 resulting in image showed in figure 21. The coefficients were KR=1.0759, KG=1.1751, KB=1.1451

Color Restoration of Aerial Photographs 23

a)

Fig. 22. Histogram of red band figure 15a. a) original saturated, b) After change.

figure 23 with a ton greener more real.

Fig. 23. Image of figure 15a with de-saturation of Red band

The reconstruction of the RGB image using the new red band results in the image of the

b)

Fig. 20. Image with a color cast desired.

Fig. 21. Image processed with Kries method.

An example of image with color saturation in the red band is in figure 15a (histogram in figure 22a). The practical solution to eliminate the saturation is to manipulate the luminance or intensity, in order to remove the peak value occurs at 255. In this example firstly was reduced the intensity range using gamma correction setting it to 0.60 and then all the intensity levels was reduced in 10 units. These values were determined by trial and error, but it can also be used image statistics to see the saturation and the need intensity shift. The final result of the histogram of red band is shown in Figure 22b.

An example of image with color saturation in the red band is in figure 15a (histogram in figure 22a). The practical solution to eliminate the saturation is to manipulate the luminance or intensity, in order to remove the peak value occurs at 255. In this example firstly was reduced the intensity range using gamma correction setting it to 0.60 and then all the intensity levels was reduced in 10 units. These values were determined by trial and error, but it can also be used image statistics to see the saturation and the need intensity shift. The

Fig. 20. Image with a color cast desired.

Fig. 21. Image processed with Kries method.

final result of the histogram of red band is shown in Figure 22b.

Fig. 22. Histogram of red band figure 15a. a) original saturated, b) After change.

The reconstruction of the RGB image using the new red band results in the image of the figure 23 with a ton greener more real.

Fig. 23. Image of figure 15a with de-saturation of Red band

Color Restoration of Aerial Photographs 25

The image of figure 15b was also processed with the Kries method using as color standard the same figure 21. The final corrected image (Figure 26) looks similar to image above but

Fig. 25. Final Image of histogram matching on image 15b

Fig. 26. Using Kries method in image 14b

more greenish and their histograms have some differences (Figure 27).

#### **4.3.2 Using histogram matching**

The use of the histogram matching (section 3.4) can also change the color cast of an image. The figure 24b shows the histograms of the image with strong reddish cast to be changed, the figure 24a shows the histograms of the image 21 (template) and figure 24c the final matching histogram. The figure 25 shows the new image with the new histograms.

Fig. 24. Histogram matching a) of image template, b) of image to be changed, c) result of the matching

The use of the histogram matching (section 3.4) can also change the color cast of an image. The figure 24b shows the histograms of the image with strong reddish cast to be changed, the figure 24a shows the histograms of the image 21 (template) and figure 24c the final

Band Red Band Green Band Blue

Reference image Figure 21

Image to change (Figure 15b)

Results of histogram matching

Fig. 24. Histogram matching a) of image template, b) of image to be changed, c) result of the

matching histogram. The figure 25 shows the new image with the new histograms.

**4.3.2 Using histogram matching** 

matching

Fig. 25. Final Image of histogram matching on image 15b

The image of figure 15b was also processed with the Kries method using as color standard the same figure 21. The final corrected image (Figure 26) looks similar to image above but more greenish and their histograms have some differences (Figure 27).

Fig. 26. Using Kries method in image 14b

Color Restoration of Aerial Photographs 27

Drago, F; Myszkowsky, K; Annen, T; Chiba, N. (2003). Adaptative Logarithmic Mapping for

Fiete, Robert D. (2004). Elements of Photogrammetric Optics. In: *Manual Of Photogrammetry* 

Gonçalves, Glauber A. (2006). *Detecção Automática de Alterações na Cartografia Cadastral com* 

Homma, Kohzo; Yamamoto Hiromichi; Iwata, Yoshitaka. (2000). A Study on Image

Kraus, Karl. (1992) *Photogrammetr, Volume 1, Fundamentals and Standard Processes*. Bonn:

Kraus, Karl.(1997). *Photogrammetry, Volume 2, Adavanced Methods and Applications*.. Bonn:

Lamparelli, Rubens Augusto C; Rocha, Jansle V; Jaime, Andrés L. G. (2004). Correção

Li, Zh J; Zhang, Z.X; Zhang J. Q. (2004a) Dodging In Photogrammetry And Remote Sensing.

*Information Sciences*. V. 34 Part B5. Proceedings XIXth Congress Instanbul. Li, Yan; Sasagawa, T; Gong, Peng. (2004b). A System Of The Shadow Detection And Shadow

Mcglone, J. C.(Ed). (2004). *Manual Of Photogrammetry* (5th Edition). Bethesda, Maryland, Eua: American Society for Photogrammetry and Remote Sensing. 1151 P. Nóbrega, Rodrigo A. A; Quintanilha José A. (2004). Comparative Analysis Of Automatic

Reinhard, E; Stark, M; Shirley P; Ferwerda J. (2002). Photographic Tone Reproduction For

Schowengerdt, R. A. (1997). Spectral Transforms. In: *Remote Sensing Models And Methods For* 

Silva, D. C., Candeias, A. L. B (2009). Causas da Iluminação não Uniforme em Fotografias Aéreas Coloridas. Revista Brasileira de Cartografia. v.61, p.105 - 115.

Graduação em Ciências Geodésica, Universidade Federal do Paraná. Hall, R. (1954). The Effect of Haze and High Solar Altitude on the Density of Air Survey

Negatives. *Photogrammetric Record*. V. 1. N. 4. P. 20-37.

Displaying High Contrast Scenes. *Eurographics 2003*. V. 22, N. 3.

Isprs Instambul Cd-Rom.

And Remote Sensing.

Amsterdam Cd-Rom.

*Cartografia*. N. 56. V. 01. P. 65-74.

Proceedings XIXth Congress Instanbul.

Digital Images. *Preccedins of The Siggraph 2002.*

*Image Processing*. 2 Ed. San Diego, Ca, Cap. 5, P. 202-219.

Dümmler. 397 P.

Dümmler. 466 P.

400-412.

*The Photogrammetry, Remote Sensing and Spatial Information Sciences*. V. 34 Part XXX.

*(5th Edition)*. Bethesda, Maryland, USA: American Society For Photogrammetry

*Base em Imagens de Câmaras Digitais*. Doctoral Thesis. Curitiba: Curso de Pós-

Restoration for Airborne Cameras. In: *The International Archives Of The Photogrammetry, Remote Sensing*. V. 33 Part B7. Proceedings XiXth Congress Isprs

Radiométrica de Imagens Digitais Aéreas para o Efeito Vinhete. *Revista Brasileira de* 

In: *The International Archives of the Photogrammetry, Remote Sensing and Spatial* 

Removal For High Resolution City Aerial Photo. In: *International Archives Of Photogrammetry And Remote Sensing and Spatial Information Sciences.* V.35 Part B3.

Digital Image Balancing And Standard Histogram Enhancement Techniques In Remote Sensing Imagery. *Revista Brasileira de Cartografia*. N. 56. V. 01. P. 55-64. Paparodis, N; Souchou, J. P.; Martinoty, G; Deseilliny, M. P..(2006). High-End Aerial Digital

Cameras and their Impact on the Automation and Quality of the Production Workflow. Isprs Journal of Photogrammetry and Remote Sensing. V. 60, N. 6. P.

Fig. 27. Histograms after restoratios a) of figure 25 by match, b) of figure 26 by Kries.

In a general way these two methods of changing the color cast have minor computational cost than the homogenization of histogram blocks. The best choice and the efficiency in practical applications will be defined by the software implementation. In some cases only one method would give the desired result and in others a combination of both histogram matching and the Kries method would be better fitting.

## **5. Conclusions**

This chapter discusses first, the causes of undesirable color changes in aerial photographs due to atmospheric factors and geometric factors of imaging, like the haze effect, position of the sun and aperture angle of cameras, and in second place some procedures for restorations of those colors.

The combination of the vignetting effect, and of the backscattering of light in haze, results in a non-uniform intensity on photographic frame with an irregular format, different for each photogrammetric flying, so that is not possible to correct it using only the simple model based in the cosine law.

The methods of color restoration in aerial photographs using mask, manipulation of histograms and the Kries hypothesis are discussed and applied with several examples. These methods have minor computational cost than the homogenization among histogram blocks used in commercial softwares.

#### **6. References**


a) b)

In a general way these two methods of changing the color cast have minor computational cost than the homogenization of histogram blocks. The best choice and the efficiency in practical applications will be defined by the software implementation. In some cases only one method would give the desired result and in others a combination of both histogram

This chapter discusses first, the causes of undesirable color changes in aerial photographs due to atmospheric factors and geometric factors of imaging, like the haze effect, position of the sun and aperture angle of cameras, and in second place some procedures for restorations

The combination of the vignetting effect, and of the backscattering of light in haze, results in a non-uniform intensity on photographic frame with an irregular format, different for each photogrammetric flying, so that is not possible to correct it using only the simple model

The methods of color restoration in aerial photographs using mask, manipulation of histograms and the Kries hypothesis are discussed and applied with several examples. These methods have minor computational cost than the homogenization among histogram

Ashikhmin, M. (2002). A Tone Mapping Algorithm for High Contrast Images. In:

Asrar, Ghassem (Ed). (1989). *Theory and Applications of Optical Remote Sensing*. New York:

Beisl, U; Woodhouse, N. (2004). Correction Of Atmospheric And Bidirectional Effects In

*Eurographics Workshop on Rendering.* Paul Debevec and Simon Gibson (Editors) Proceedings of the 13th Eurographics Workshop on Rendering Techniques, Pisa,

Multi-Spectral Ads40 Images For Mapping Purposes. *The International Archives Of* 

Fig. 27. Histograms after restoratios a) of figure 25 by match, b) of figure 26 by Kries.

matching and the Kries method would be better fitting.

**5. Conclusions** 

of those colors.

**6. References** 

based in the cosine law.

blocks used in commercial softwares.

Italy, June 26-28, 2002.

John Wiley & Sons. 734 P.

*The Photogrammetry, Remote Sensing and Spatial Information Sciences*. V. 34 Part XXX. Isprs Instambul Cd-Rom.


**2** 

*USA* 

*Earthmine Inc., Berkeley,* 

**High-Quality Seamless Panoramic Images** 

Jaechoon Chon, Jimmy Wang, Tom Slankard and John Ristevski

Image mosaicing is the creation of a larger image by stitching together smaller images. Image mosaicing has many different applications, such as satellite imagery mosaics (Chon et al., 2010,) the creation of virtual reality environments (Szeliski, 1996; Chon et al., 2007; Brown and Lowe, 2007,) medical image mosaics (Chou et al., 1997,) and video compression (Irani et al., 1995; Irani and Anandan, 1998; Kumar et al., 1995; Lee et al., 1997; Teodosio and Bender, 1993). Image mosaicing has four steps: 1) estimating relative pose among smaller images to project on a plane or specific defined surfaces, 2) projecting those images on the surface, 3) correcting photometric parameters among projecting images, and 4) blending overlapping images.

The first step has two categories: 2D plane images (For example, scanned maps in sections and orthoimages) based methods and perspective image based methods. Orthoimages are generated by correcting each image pixel from a perspective image with a help of a Digital Elevation Model (DEM,) produced by a stereo camera system, a laser/radar aerial system or obtained of photogrammetric works. To align the images with the DEM, we need to estimate absolute orientation with ground control points (GCP.) In the case of perspective images, we need to estimate relative pose using homography, affine transformation, colinear conditions, and coplanner condition with feature correspondences. (Hartley and Zisserman

The second step projects all images onto a specific defined surface, such as 2D plane, cylinder (Chen, 1995; Szeliski, 1996), sphere (Coorg and Teller, 2000; Szeliski and Shum, 1997), and multiple projection planes (Chon et al., 2007.) In the case of cylinder and sphere, we assume that images are captured by fixing the position of a camera and rotating it at the

In the third step, we have to balance photometric parameters among projected images because exposure affects the photometric parameters of each image. At the last step, we

Ideally each sample (pixel) along a ray would have the same intensity in every image that it intersects, but in reality this is not the case. Even after gain compensation some image edges are still visible due to a number of un-modeled effects, such as vignetting (the decrease of intensity towards the edge of the image,) parallax effects due to unwanted motion of the optical center, registration errors due to an incorrect or approximate camera model, radial

& Books, 2003; McGlone et al. & Books, 2004; Szeliski & Books, 2011)

position. And all images are projecting onto a cylinder or a sphere surface.

have to create one image from multiple overlapping images.

**1. Introduction** 

distortion, and so on.

