**3.2 Fourier filtering**

To improve the previous works, one of the first solution has been to represent each image in a colour-texture hybrid space, and to use mathematical morphology tools (Cointault et al., 2008a) in order to extract and count the number of wheat ears.

Although this previous method gives satisfying results (around 10% of well detection, compared to manual counting), the hybrid space construction method is a supervised one and is limited by the objectivity of the operator. Moreover the statistical methods are dependent on the choice of the direction for the process, need an important computing time and are lightning-dependent.

The objective has been thus to propose new detection algorithms more rapids, robusts, and invariants according to image acquisition conditions, based on Fourier filtering and two dimensional discrete fast Fourier transform (FFT) (Cooley &Tukey, 1965). This approach includes three important steps: high-pass filtering, thresholding and cleaning of the image based on mathematical morphology operations (Serra, 1982).

## **3.2.1 The three important steps**

**For the high pass Fourier filtering,** a two dimensional FFT is performed on the target image (eq. 8).

$$F(k\_{x'}k\_y) = \frac{1}{\sqrt{N\_x N\_y}} \sum\_{n\_x=0}^{N\_x - 1} \sum\_{n\_y=0}^{N\_y - 1} f(n\_{x'}n\_y) e^{\alpha \frac{k\_x n\_x}{N\_x}} e^{\alpha \frac{k\_y n\_y}{N\_y}} \tag{8}$$

Based on the centered Fourier image, a high pass filter is applied in order to eliminate low frequencies in the FFT image (figure 8). The cut off frequency is empirically sized by a 10 pixels width disk mask as it is shown in the figure 8.

**The thresholding of the resulting image** is based on anInverse FFT and a predetermined threshold is applied in order to eliminate low pixel values which do not correspond to wheat objects (ground, leaf,…) (figure Fig. 9).

**The cleaning step** aims at eliminate remaining "non wheat" pixel groups, which are small and scattered. It lies on mathematical morphology operation and is performed with three sub steps:


For the whole set of sample images, 1 to 3 segmentation errors have been found for each image. As the images show an average of 20 wheatears, the error rate can be estimated around 10%. Most of the errors encountered are identified as badly lightened wheatears. This kind of error could be easily overcome by using artificial light instead of natural light conditions, or

To improve the previous works, one of the first solution has been to represent each image in a colour-texture hybrid space, and to use mathematical morphology tools (Cointault et al.,

Although this previous method gives satisfying results (around 10% of well detection, compared to manual counting), the hybrid space construction method is a supervised one and is limited by the objectivity of the operator. Moreover the statistical methods are dependent on the choice of the direction for the process, need an important computing time

The objective has been thus to propose new detection algorithms more rapids, robusts, and invariants according to image acquisition conditions, based on Fourier filtering and two dimensional discrete fast Fourier transform (FFT) (Cooley &Tukey, 1965). This approach includes three important steps: high-pass filtering, thresholding and cleaning of the image

**For the high pass Fourier filtering,** a two dimensional FFT is performed on the target image

1 1

*y y x x <sup>y</sup> <sup>x</sup>*

*k n <sup>N</sup> k n <sup>N</sup>*

*y x*

*N N*

(8)

<sup>1</sup> (,) (,)

*x y x y x y n n Fk k fn n e e N N* 

0 0

Based on the centered Fourier image, a high pass filter is applied in order to eliminate low frequencies in the FFT image (figure 8). The cut off frequency is empirically sized by a 10

**The thresholding of the resulting image** is based on anInverse FFT and a predetermined threshold is applied in order to eliminate low pixel values which do not correspond to

**The cleaning step** aims at eliminate remaining "non wheat" pixel groups, which are small and scattered. It lies on mathematical morphology operation and is performed with three sub steps: First, a dilatation, which aims at making bigger and closer pixel groups in the image. Then a blurring convolution with a Gaussian smoothing operator, followed by a thresholding, which eliminate too small groups of pixels. These small groups are considered as miss. This step makes smaller the pixel groups that correspond to wheat

Finally, another dilatation is performed which aims at regenerate size of pixel groups

*x y*

using color images instead of grey level images, as it will be shown in the next section.

2008a) in order to extract and count the number of wheat ears.

based on mathematical morphology operations (Serra, 1982).

pixels width disk mask as it is shown in the figure 8.

wheat objects (ground, leaf,…) (figure Fig. 9).

and then justify a third step.

corresponding to wheat ears (figure 10).

**3.2 Fourier filtering** 

and are lightning-dependent.

**3.2.1 The three important steps** 

(eq. 8).

Fig. 8. Example of wheat image (a), (b) its associated FFT projection, (c) cut off disk, (d) zoom of cut off disk.

Fig. 9. Image after inverse FFT(a) and threshold image (b).

Fig. 10. Threshold image (a) and cleaned image of wheat detection (b).

Texture, Color and Frequential Proxy-Detection

Image Processing for Crop Characterization in a Context of Precision Agriculture 59

Original images Fourier results

Image Counting Difference (%)

Manual image processing

1 139 142 2,11

2 36 34 -5,88

3 90,5 94 3,72

4 116,5 122 4,51

5 136,5 142 3,87

Within the image set, a high variability of ear's number can be observed. With most of the images, high pass Fourier filtering method returns slightly higher counts. Based on the whole image sample, absolute difference between manual counting and image processing counting is contained under the value of 6%. The mean error obtained is 4,02% for this set of

The development of a high pass Fourier filtering approach method aims at creating an easily usable and adaptable method while obtaining a best wheat ear detection. Therefore, it is essential to compare this approach with previously used methods such as hybrid space method (Cointault et al., 2008b). For a visual comparison, high pass Fourier filtering has been applied onto images that have been previously treated with hybrid approach

Table 2. Specifications of five experimental drying runs used for validation.

images.

(figure 13).

Fig. 12. Results of wheat ear detection using high pass Fourier Filtering.

### **3.2.2 Wheat ear counting estimation**

The first image processing by thresholding and high pass Fourier filtering gives a binary image, composed of several pixel groups. Each group represents one or more wheat ears to be counted. In order to estimate this number we analyze the shape of each group considering two possible configurations:


Fig. 11. Pixel groups with nearly convex (a) and concave shapes (b), convex hull of a concave shaped pixel group (c).

In order to quantify the number of wheat ears, we estimate a shape index based on two features extracted from each pixel group:

$$Q = \frac{S\_c - S}{S} - 1\tag{9}$$

With *S* the surface and *SC* the convex hull surface of the pixel group (Figure 6c).

Coming from this index, a number*X* of ears is attributed for the group, considering this interval:

$$\left(\left(X - 1\right)/10 < Q < X \,/\, 10\right) \tag{10}$$

Another approach, based on skeleton analysis can be found in (Germain et al, 1995).

#### **3.2.3 Results and discussion**

A fast visual observation (figure 12) shows that only a small amount of groups corresponds to non-wheat things in image. More precisely, the biggest wheat ears are well detected and well separated from surrounded leaves.

Ears that lie the nearest of the ground, that are partially hidden or that are a little bit over exposed in the image are not well detected. Small amount of very big leaves also remain after cleaning step.

In order to test our image processing we performed algorithm on 40 images and compare the results with the mean of manual counting done by several experts. For the example, five images sample have been randomly chosen (table 2).

The first image processing by thresholding and high pass Fourier filtering gives a binary image, composed of several pixel groups. Each group represents one or more wheat ears to be counted. In order to estimate this number we analyze the shape of each group

Pixel group presents a convex or nearly convex shape pattern: it is considered that only

Pixel group presents a concave shape pattern (figure 11b): it is consider that several ears

(a) (b) (c) Fig. 11. Pixel groups with nearly convex (a) and concave shapes (b), convex hull of a concave

In order to quantify the number of wheat ears, we estimate a shape index based on two

<sup>1</sup> *S S <sup>c</sup> <sup>Q</sup> <sup>S</sup>*

Coming from this index, a number*X* of ears is attributed for the group, considering this

A fast visual observation (figure 12) shows that only a small amount of groups corresponds to non-wheat things in image. More precisely, the biggest wheat ears are well detected and

Ears that lie the nearest of the ground, that are partially hidden or that are a little bit over exposed in the image are not well detected. Small amount of very big leaves also remain

In order to test our image processing we performed algorithm on 40 images and compare the results with the mean of manual counting done by several experts. For the example, five

With *S* the surface and *SC* the convex hull surface of the pixel group (Figure 6c).

Another approach, based on skeleton analysis can be found in (Germain et al, 1995).

(9)

( 1) /10 /10 *X QX* (10)

**3.2.2 Wheat ear counting estimation** 

considering two possible configurations:

features extracted from each pixel group:

**3.2.3 Results and discussion** 

after cleaning step.

well separated from surrounded leaves.

images sample have been randomly chosen (table 2).

are presents.

shaped pixel group (c).

interval:

one ear is present in this kind of group (figure 11a).

Fig. 12. Results of wheat ear detection using high pass Fourier Filtering.


Table 2. Specifications of five experimental drying runs used for validation.

Within the image set, a high variability of ear's number can be observed. With most of the images, high pass Fourier filtering method returns slightly higher counts. Based on the whole image sample, absolute difference between manual counting and image processing counting is contained under the value of 6%. The mean error obtained is 4,02% for this set of images.

The development of a high pass Fourier filtering approach method aims at creating an easily usable and adaptable method while obtaining a best wheat ear detection. Therefore, it is essential to compare this approach with previously used methods such as hybrid space method (Cointault et al., 2008b). For a visual comparison, high pass Fourier filtering has been applied onto images that have been previously treated with hybrid approach (figure 13).

Texture, Color and Frequential Proxy-Detection

the whole field whatever the infestation rate.

sufficiently robusts to lightning conditions

where Lf is the observed luminance for that pixel.

multiplicative factor of the light, and hence Rf

resulting in an additive constant of Rf.

phenomena are to be considered, namely:

**hyperspectral imagery** 

localized weed control.

(Slaughter et al., 2008):

**4.1 Spectral pre-processing** 

Image Processing for Crop Characterization in a Context of Precision Agriculture 61

**4. Proxy-detection image processing for weed-wheat crop discrimination by** 

In the domain of weed control, since the environmental and economical stakes are particularly great, the herbicides are largely spread in order to assure sufficient yields for

The reason is essentially a technological one. Even if some low-cost devices are currently available to assure a localised spraying of herbicides on bare soil (vegetation detection by photoelectric cells), no commercial product allows a reliable and localised post-emergence treatment. Indeed, a such apparatus needs a sophisticated perception system, based on digital vision and allowing to distinguish between weeds and crops. The identification of varieties inside the vegetation is nowadays the principal lock to the development of

The corresponding research are numerous and can be divided into two main approaches

 the spectral approach, in which the plant reflectance is the main parameter, using hyper- or multispectral images (Feyaerts & Van Gool, 2001; Vrindts, 2002; De Baerdemaeker et al., 2002). The difficulty is then to propose spectral differences

 the spatial approach, based on spatial criteria such as plant morphology (Chi et al., 2003; Manh et al., 2001), plant texture (Burks et al., 2000) … The main difficulty is tied

The study proposed in this chapter is tied to the previous approach: hyperspectral images of the wheat crop are acquired during the weed control period, and associated to specific preprocessing to avoid illumination conditions. After, the possible spectral discrimination between wheat and dicotyledonous weeds by means of chimiometric tools has been evaluated.

The images acquired by the camera (after a first internal processing taking into account the spectral sensitivity of the sensor) are images of luminance, which by definition depend on both the reflectance of the scene and lighting conditions. This is why a reference surface (gray ceramic), for which the reflectance Rc was measured in the laboratory, was systematically placed in each scene (figure 14). The average luminance Lc observed in the

It is important to note that the Rf value obtained is a "apparent" reflectance. Other

The inclination of the observed leaf from the direction of the solar source, introducing a

the possibility of specular reflection, adding a component to the spectrum colorless, and

Rf = Rc \* (Lf/Lc) (11)

image to the reference can then be corrected reflectance for each pixel of vegetation :

here to the complexity and the natural variability of the scenes.

Fig. 13. Visual comparison between Fourier and Hybrid approach for two different images.

Visual comparisons with hybrid space results show that high pass Fourier filtering approach eliminate more non wheat objects within the images. Fourier approach separates more efficiently ear groups. Calculation time for Fourier approach is few sec per image while its few minutes for hybrid space, with the same operating system.

High pass Fourier filtering gives global satisfying results. Although a close range of settings has been determined, inverse FFT remains a parameter that has to be adjusted according to input image. An empirical value has been found and gives good results for most of images but it could be optimized with an automatic threshold selection such as k-means methods (MacQueen, 1967).

In the context of wheat detection, it has been observed that some ear objects are eliminated after cleaning step. These non-detections mainly correspond to near ground ears or ears massively hidden under leaves, hence, it should be relativized as too low ears may have development problems and may not be considerate within wheat yield. Ears that are located in over or under exposed part of image are not well detected but it is not due to the algorithm but to the quality of the acquisition, which is limited by the natural conditions. Small amount of very big leaves also remain after cleaning step and eliminate these artifacts constitute a further axis of development, including shape analysis in cleaning step.

In the context of wheat ear counting, it is observed that counting error percentage decrease with number of ears in images, hence, best results may be obtained with images representing more area and yet, more ears. Actually, worst error, 5,56%, is obtained with only 36 wheat ears in the image.

It is important to note that in most cases, Fourier approach returns slightly higher counts than manually counts. It should be due to missing detection, such as remaining leaves or over exposed part of images. Counts should be more precise with the including of shape analysis in cleaning step. In all cases, whatever the method used, the only way to obtain a right detection is to use 3-Dimensional information.
