**6. Shape feature analysis**

If the process of image vegetative/background classification is to be useful, the separated plant region of interest (ROI) must provide important canopy or leaf shape feature or property information to at least discriminate between broadleaf and grass species (Woebbecke et al., 1995b; Meyer et al., 1998a; Meyer et al., 1998b).

Supervised leaf and single plant canopy shape feature analysis has been studied the most. Petry and Kuhbauch (1989) found shape parameters using five canonical indices found distinctly different for several weed species. Guyer et al. (1986, 1993) used image shape feature analysis on individual leaves to distinguish between weed species and corn. Guyer et al. (1993) using only leaf and canopy shapes, reported a 69% correct identification rate for 40 weeds and agricultural crop species. Guyer found that no single shape feature alone could distinguish corn from all other species. Franz et al. (1991 a,b) identified plants based on individual leaf shape at two growth stages using the Fourier-Mellin correlation. Woebbecke et al. (1995a, b) used basic image shape feature analysis to discriminate between broadleaf and grassy plant canopies. Woebbecke found that broadleaf and grass shape features best appeared to a vision system at early stages of growth or within a specific window of time, from 1-4 weeks after emergence. Downey, et al (2004) described a field canopy shape identification system which used a binary canopy erosion technique to discriminate between grasses and broadleaf plants. Yonekawa et al. (1996) presented a set of classical shape features for a leaf taxonomy database. Chi, et al (2002) fitted Bezier curves to different leaf boundary shapes. Mclellan and Endler (1998) compared several morphometric methods for describing complex shapes. They found that approximately 20 harmonics of the elliptic Fourier method accurately depicted shapes of *Acer saccharinum*, *Acer saccharum*, and *Acer palmatum* leaves. A leaf shape image retrieval systems was also reported by Wang, et al (2003).

Du et al (2005, 2006, 2007) proposed the Douglas-Peucker approximation algorithm for leaf shapes and the shape representation was used to form the sequence of invariant attributes.

themselves (non-concealed) against the soil-residue background. Others will have vegetation from occluded leaves around them, which we will call concealed leaves. The latter would represent a difficult image processing problem, not easily solved by traditional

Deformable templates using active contours were used by Manh, et al. (2001) to locate boundaries of green foxtail leaves. Manh's process attempted to combine color separation and shape feature analysis into a single operation. The procedure began with identification of a leaf tip, and followed by shape analysis across the rest of the green material. However, a manually selected energy level or color was needed. Segmentation accuracy for a single

Individual, whole, and fragments of leaves were isolated using the Gustafson-Kessel fuzzy clustering method over bare soil, corn stalks, and wheat straw color images (Hindman, 2001, Meyer et al., 2004b, Gustafson and Kessel, 1979). Zadeh intensification of the fuzzy cluster membership functions resulted in definitive green canopy areas, but not individual leaves. However, Camargo Neto, et al (2006) used the Gustafsen-Kessel fuzzy leaf cluster fragmentation method on green canopy regions of interest. He also developed a reassembling method of the green cluster fragments resulting in individual leaves using a

If the process of image vegetative/background classification is to be useful, the separated plant region of interest (ROI) must provide important canopy or leaf shape feature or property information to at least discriminate between broadleaf and grass species

Supervised leaf and single plant canopy shape feature analysis has been studied the most. Petry and Kuhbauch (1989) found shape parameters using five canonical indices found distinctly different for several weed species. Guyer et al. (1986, 1993) used image shape feature analysis on individual leaves to distinguish between weed species and corn. Guyer et al. (1993) using only leaf and canopy shapes, reported a 69% correct identification rate for 40 weeds and agricultural crop species. Guyer found that no single shape feature alone could distinguish corn from all other species. Franz et al. (1991 a,b) identified plants based on individual leaf shape at two growth stages using the Fourier-Mellin correlation. Woebbecke et al. (1995a, b) used basic image shape feature analysis to discriminate between broadleaf and grassy plant canopies. Woebbecke found that broadleaf and grass shape features best appeared to a vision system at early stages of growth or within a specific window of time, from 1-4 weeks after emergence. Downey, et al (2004) described a field canopy shape identification system which used a binary canopy erosion technique to discriminate between grasses and broadleaf plants. Yonekawa et al. (1996) presented a set of classical shape features for a leaf taxonomy database. Chi, et al (2002) fitted Bezier curves to different leaf boundary shapes. Mclellan and Endler (1998) compared several morphometric methods for describing complex shapes. They found that approximately 20 harmonics of the elliptic Fourier method accurately depicted shapes of *Acer saccharinum*, *Acer saccharum*, and *Acer palmatum* leaves. A leaf shape image retrieval systems was also reported by Wang, et al

Du et al (2005, 2006, 2007) proposed the Douglas-Peucker approximation algorithm for leaf shapes and the shape representation was used to form the sequence of invariant attributes.

species of foxtail leaves was reported to be 84%. No other species were studied.

algorithms such as edge detection, erosion, dilation, and such.

(Woebbecke et al., 1995b; Meyer et al., 1998a; Meyer et al., 1998b).

genetic algorithm (Holland, 1975).

**6. Shape feature analysis** 

(2003).

A modified dynamic programming (MDP) algorithm for shape matching was proposed for the plant leaf recognition. Oide and Ninomiya (2000) used the Elliptic Fourier (EF) method to classify soybean varieties, using a normalized leaf shape. The EF method using a chaincoded, closed contour, invariant to scale, translation, and rotation was first introduced by Kuhl and Giardina (1982). EF has been used in recent studies to describe the shape of objects. Innes and Bates (1999) used an Elliptical Fourier descriptor to demonstrate an association between genotype and morphology of shells. Chen et al. (2000) used Elliptic Fourier descriptors to describing shape changes in the human mandible for male and female at different ages. Most methods previously investigated ignore leaf edge serration. Leaf serration or edgeness is an important morphologic feature used for identifying plant species. For example, the curvature functions developed by Franz et al. (1991b) were found generally inadequate where leaflet serration was quite pronounced. Camargo Neto, et al, 2006b applied the Elliptic Fourier shape feature analysis to extracted leaves of velvet leaf *Abutilon theophrasti*, pig weed *Amaranthus retroflexus L.*, sunflower *Helianthus annus*, and soybean *Glycine max*. A velvet leaf example is shown in Figure 3.

Hearn (2009) used a database of 2,420 leaves from 151 plant species for a plant leaf shape analysis. Using metrics derived during Fourier and Procrustes analyses, it was found that a minimum of ten leaves for each species, 100 margin points, and ten Fourier harmonics were required to develop any accuracy using the leaf shape of a species. His results indicated a success rate of 72% correct species identification for all 151 species used. This may mean that more than leaf shape is needed for classification.

#### **7. Textural feature analysis**

Color and/or leaf shape features alone may not be sufficient to consistently distinguish between young weed and crop plant species. Textural features may supply some additional botanical information, such as leaf venation, leaf pubescence, but also leaf disease and insect damage. The color or tonal detail for texture was first described by quantification of cooccurrence of tonal pairs or contrast also known as spatial tonal frequency (Haralick, 1978 and 1979). Wavelet analysis and energy have been recently suggested as a frequency based textural analysis for segmenting weeds imbedded in canopies (Chang and Kuo, 1993, Strickland and Hahn, 1997, Tang, et al, 2003). Shearer and Holmes (1990) used color cooccurrence matrix method to identify the textural features of isolated plants. Shearer and Jones (1991) proposed a texture-alone plant detection system based upon hue-saturationintensity (HSI) images. Oka and Hinata (1989) used side view images of rice to distinguish between old and new Japanese rice cultivars. Zhang and Chaisattapagon (1995) tested a combination color, shape, and texture approach for detecting weeds in wheat fields and found that leaf –surface- coarseness indices defined by Fourier spectra may be effective in differentiating wheat from broad-leaf weeds. Meyer et al. (1999) showed that combined color, shape, and textural statistical discriminate analysis system could separate grasses from broadleaf canopies against bare soil backgrounds. Major problems for obtaining botanical textural detail involve image resolution, leaf orientation or rotation, shadows, bidirectional reflectance of leaf surfaces, and background lighting. Uneven lighting for example, could obscure venation - mesophyll leaf detail. Diffuse lighting could provide more even illumination than direct-beam lighting. Fu and Chi (2006) presented an algorithm for extracting leaf vein details from detached leaves under artificial light. Park, et al (2008) described a prototype system for classifying plants based on leaf venation features. Their

Machine Vision Identification of Plants 411

Most studies in the last 20-years have addressed the classification of only two crop-weed classes or general cases of broad leaf versus grasses and in other cases, crop row versus between crop row (Tang, et al, 2003). However, to precisely classify a plant species that may be imbedded within other different species of plants in an image is a botanically challenging

Agarval, et al (2006) described an ongoing project to digitize information about plant specimens that would become available to field botanists and crop managers. They indicated that the first step required acquisition of digital images and possibly plant architectural models, along with an effective retrieval method and mobile computing mechanisms for accessing this information. At that time they had indicated progress in developing a digital archive of the collection of various plant specimens at the Smithsonian

Analytical tools are improving for classifying plant species. The artificial neural network (ANN) has been proposed for many classification activities. Plotze and Bruno (2009) have also proposed a plant taxonomy system. Yang et al., (2000, 2002, 2003) used RGB pixel intensities as inputs for a fuzzy artificial neural network (ANN) for distinguishing weeds from corn, with success rates as high as 66% for corn and 85% for weeds. To encompass the uncertainty of image classification processes, fuzzy set theory (FST) has been proposed for plant classification (Gottimukkala et al., 1999). FST provides a possibilistic alternative (different, but in many cases complementary) to the probabilistic or statistical approaches. FST embraces virtually all (except one) of the definitions, precepts, and axioms that define classical sets that supports common mathematics, (Ross, 2004). It uses variables in the form of membership functions with degrees of support for fuzziness, incorporating uncertainty (Zadeh, 1965; Mamdani, 1976; Li and Yen, 1995). Pal, et al 1981, 1994, Bezdek, 1973, 1993) summarized the use of a FST neural network for pattern recognition, generating membership functions, performing fuzzy logic (FL) operations, and then deriving inference rule sets. Jang (1993) invented the artificial neural fuzzy inference system (ANFIS) for training membership functions and rule sets that could be

Fuzzy logic machine vision classification systems are intended to imitate human perception or vision and to handle uncertainty. In the weed discrimination example, expert human perception or scouting validation is required for ground truthing. Bhutani and Battou (1995) and Tizhoosh (1998, 2000) provide computational overviews and various examples of fuzzy logic applied to image processing. Incorporating unsupervised fuzzy logic clustering and image analysis into site-specific technologies has tremendous potential (Kuhl and Giardina, 1982, Gath and Geva, 1989, De and Chatterji, 1998, Babuska, 1998, Manthalkar, at al, 2003, Meyer, et al. 2004). The very nature of site-specific data collection, image analysis, decision-making, etc., is characterized by uncertainty, ambiguity, and vagueness, which

Hindman and Meyer (2000) demonstrated a prototype fuzzy inference system for plant detection. Jones et al. (2000) used remotely sensed data with FL classification to detect crop status, resulting in a fuzzy description of crop phenology based upon spectral data. Yang et al., (2003) also presented potential herbicide savings in weed control with a fuzzy logic system. Heming and Rath (2001) proposed a fuzzy weed classifier that yielded correct classification accuracies between 51 and 95%. The potential fallacy of any regression, ANN,

**8. Plant species classification** 

used for classification (Figure 4).

may be over overcome with these techniques.

exercise.

Institution.

Fig. 3. Elliptic Fourier shape approximations for velvetleaf (*Abutilon theophrasti Medicus*), a) original leaf image, b) 1st EF harmonic, c) 1st + 2nd + 3rd + 4th EF, d) 1st + … + 8th EF harmonics, e) 1st + … + 16th EF harmonics, and f) 1st + … + 30th EF harmonics.

method detected the differences between tree and parallel venations in leaves, and thus could be considered as an enhancement to the classification tool set.

De Oliveira Plotze (2009) combined computer vision techniques and plant taxonomy protocols, these methods are capable of identifying plant species. The biometric measurements are concentrated in leaf internal forms, specifically in the venation system. The methodology was tested with eleven species of passion fruit of the genus *Passiflora*. The features extracted from the leaves were then applied to a neural network system to develop a classification of species. The results were very accurate in correctly differentiating among species with 97% of success. Zheng and Wang (2009, 2010) presented the results of mathematical morphology used on images of single leaf samples. Mathematical morphology provides four fundamental operations of dilation, erosion, opening, and closing in image processing. Their goal was to extract only leaf veins using hue and intensity information. Camargo Neto and Meyer (2005) classified the plant species botanically sing additional leaf venation textural features acquired during the previous steps. One thing is clear, lack of care in the photography of a leaf may affect image textural properties and classification.

a) b)

c) d)

e) f) Fig. 3. Elliptic Fourier shape approximations for velvetleaf (*Abutilon theophrasti Medicus*), a)

method detected the differences between tree and parallel venations in leaves, and thus

De Oliveira Plotze (2009) combined computer vision techniques and plant taxonomy protocols, these methods are capable of identifying plant species. The biometric measurements are concentrated in leaf internal forms, specifically in the venation system. The methodology was tested with eleven species of passion fruit of the genus *Passiflora*. The features extracted from the leaves were then applied to a neural network system to develop a classification of species. The results were very accurate in correctly differentiating among species with 97% of success. Zheng and Wang (2009, 2010) presented the results of mathematical morphology used on images of single leaf samples. Mathematical morphology provides four fundamental operations of dilation, erosion, opening, and closing in image processing. Their goal was to extract only leaf veins using hue and intensity information. Camargo Neto and Meyer (2005) classified the plant species botanically sing additional leaf venation textural features acquired during the previous steps. One thing is clear, lack of care

original leaf image, b) 1st EF harmonic, c) 1st + 2nd + 3rd + 4th EF, d) 1st + … + 8th EF harmonics, e) 1st + … + 16th EF harmonics, and f) 1st + … + 30th EF harmonics.

in the photography of a leaf may affect image textural properties and classification.

could be considered as an enhancement to the classification tool set.
