Overview of Hyperspectral Imaging

#### **Chapter 1**

## Perspective Chapter: Hyperspectral Imaging for the Analysis of Seafood

*Samuel Ortega, Stein-Kato Lindberg, Kathryn E. Anderssen and Karsten Heia*

#### **Abstract**

Hyperspectral imaging technology is able to provide useful information about the interaction between electromagnetic radiation and matter. This information makes possible chemical characterization of materials in a non-invasive manner. For this reason, the technology has been of great interest for the food industry in recent decades. In this book chapter, we provide a survey of the current status of the use of hyperspectral technology for seafood evaluation. First, we provide a brief description of the optical properties of tissue and an introduction to the instrumentation used to capture these images. Then, we survey the main applications of hyperspectral imaging in the seafood industry, including the quantification of different chemical components, the estimation of freshness, the quality assessment of seafood products, and the detection of nematodes, among others. Finally, we provide a discussion about the current state of the art and the upcoming challenges for the application of this technology in the seafood industry.

**Keywords:** hyperspectral imaging, food quality, seafood industry, spectroscopy, fish

#### **1. Introduction**

Hyperspectral imaging is a technology able to measure simultaneously both the spectral and the spatial features of objects or materials under examination. The spectral properties are produced by the interaction between the electromagnetic radiation and the different constituents in a sample, which produces distinct absorption, reflection, and scattering effects on the incident light [1]. The aforementioned optical properties of the different materials are related to their chemical composition and physical properties. Hyperspectral technology for food quality inspection has two main advantages. First is its non-invasive nature, which makes it possible to perform a chemical analysis of the samples without the need to handle them in any way. Secondly, the measurement is very quick to perform as data can be obtained for an entire sample in the matter of seconds. These aspects make the technology easy to integrate with a conveyor belt, which makes it possible to analyze every sample individually. This is preferable to random screening, where the properties of a small batch of subsamples are analyzed, and it is assumed that their chemical properties are the same for the whole population.

For these reasons, in recent years, hyperspectral imaging has awakened the interest of many researchers for the analysis of food products. According to Scopus, the total number of scientific articles related to studies on hyperspectral imaging for food applications is 1305 in the past 22 years (from 2000 to 2022), with an increasing trend in the number of publications (**Figure 1**).

The range of applications within the food industry is wide and has been extensively covered in the literature by several literature reviews. Those studies cover a wide range of applications including wheat-based products [2], dairy products [3, 4], cereals [5], fruits and vegetables [6, 7], meat [8–10], or condiments [11]. Additionally, it has also been applied to detect adulteration [12] or fraud [13].

Furthermore, other researchers have analyzed the potential of hyperspectral imaging for food microbiology inspection [14] or for the optimization of agricultural procedures [15].

The common motivation for all of these research efforts is to find new technologies able to determine quality parameters on food products, with the goal of avoiding the use of traditional characterization techniques, which are usually destructive, time consuming, and, in certain cases, subjective.

In this book chapter, we provide a survey of the current status on the use of hyperspectral imaging technology in the seafood industry as well as potential future applications. It is worth noting that the workflow for the investigation of hyperspectral imaging in this field requires an appropriate experimental design, the use of adequate instrumentation to carry out data acquisition campaigns, the collection of reference data, and finally the image processing of the hyperspectral images. For this reason, the research performed in this field usually requires a close multidisciplinary collaboration of skilled professionals from different fields, such as biologists, physicists, and engineers, among others.

This book chapter is organized as follows. First, a brief description of the optical properties of different tissue constituents is provided. Second, we discuss the most relevant factors about the instrumentation that should be considered for food inspection applications. Then, we provide a survey about the specific proposed solutions for

*Perspective Chapter: Hyperspectral Imaging for the Analysis of Seafood DOI: http://dx.doi.org/10.5772/intechopen.108726*

the use of hyperspectral imaging evaluation of seafood products. This survey is not technical, and it has been focused on the goal of providing a description of the wide range of applications that have been covered in the literature until now. We also provide the readers with a summary table containing more specific details of the different research works presented in this book chapter. Finally, we discuss the current limitations of the technology and the potential future trends for hyperspectral imaging use in the seafood industry.

#### **2. Optical properties of biological tissue**

The quantification of the chemical constituents of biological tissue is possible due to the optical properties of light when propagating within it. The three types of interactions between electromagnetic radiation and tissue that can be measured are absorption, refraction, and scattering [16]. Light absorption is related to the amount of electromagnetic radiation that is transformed into energy by tissue molecules. The different molecules will present specific absorption peaks, which are related to the transitions between two energy levels by light at specific wavelengths.

The absorption peaks of different biological tissue constituents in the visible and near-infrared regions of the electromagnetic spectrum have been widely characterized in the literature. For that reason, the absorption spectra of water, lipids, proteins, collagen, and hemoglobin in its different oxygenation states are known [17, 18]. A representation of those absorption peaks in the spectral range from 500 to 1600 nm is presented in **Figure 2** [19].

#### **3. Instrumentation**

Every hyperspectral acquisition system is composed of a lens, an optical element employed to perform the spectral sampling, an electronic sensor, and a light source. There are different types of hyperspectral systems depending on how the sampling of

#### **Figure 2.**

*Absorption peaks of different tissue constituents in the spectral range from 500 to 1600 nm. Reproduced from [19]; creative commons BY 4.0; published by SPIE (2011).*

the electromagnetic spectra is accomplished. This information is beyond the scope of this book chapter, but readers who are interested in this can refer to different reviews on hyperspectral imaging hardware in the literature [20].

A relevant characteristic of hyperspectral imaging instrumentation that is extensively mentioned in this book chapter is the spectral range. The spectral range defines the region of the electromagnetic spectrum that a hyperspectral camera is able to measure. In commercial hyperspectral systems, there are standard definitions for the spectral range. Visible and near-infrared (VNIR) refers to the spectral range from 400 to 1000 nm, while near-infrared (NIR) and short-wave infrared (SWIR) are used for the ranges 1000–1700 nm and 1000–2500 nm, respectively. Other key parameters in hyperspectral imaging instrumentation are the spectral resolution and the spatial resolution, but these concepts will not be used in this book chapter.

Although the details of hyperspectral cameras are not relevant for this book chapter, the selection of the illumination type to produce the appropriate light–tissue interactions within the sample is relevant. Using a diffuse reflectance illumination scheme, the light is evenly delivered to the sample, and it is measured by a hyperspectral camera after being reflected off its surface. With this illumination mode, the interaction of light and matter is only measured from the surface of the sample. In some cases, the diffuse light can penetrate a small distance into the sample depending on its translucency. However, in complex and inhomogeneous samples, this type of illumination is not enough for accurate characterization of their chemical composition [21, 22]. For this reason, some researchers have proposed the use of interactance (also known as transflectance) illumination, where the light is able to penetrate deeper into the sample. This illumination mode consists of a focused light illuminating the sample in a different spatial location to where the spectral information is captured, allowing the hyperspectral camera to measure the light interaction after multiple internal reflections have occurred inside the sample [23]. In the applications mentioned in this book chapter, both types of light illumination schemes are used.

#### **4. Applications of hyperspectral imaging in the seafood industry**

#### **4.1 Chemical composition**

The analysis of the chemical composition of seafood products is important for the determination of their overall quality or nutritional value, among others. However, conventional chemical analysis techniques are destructive and time consuming. For that reason, in recent years, hyperspectral imaging has been foreseen as a technology suitable for providing a non-invasive measurement of those chemical properties.

For example, in Atlantic salmon, moisture and fat content are considered to be closely related to the overall quality of the product. The fat content has consequences for both the customers and the industry. For the customers, the amount of fat present in a fresh fillet determines the flavor and texture of the product. For the industry, it is important to quantify the amount of fat in a salmon fillet to determine its target market. For example, the optimum fat content for smoked salmon is between 8 and 12% [24], while salmons with higher fat content and marbling are preferred for sushi and sashimi [25, 26]. Similarly, the moisture is related to the shelf-life of seafood products.

#### *Perspective Chapter: Hyperspectral Imaging for the Analysis of Seafood DOI: http://dx.doi.org/10.5772/intechopen.108726*

Several research studies have been focused on non-invasive determination of moisture and fat using hyperspectral imaging. Several authors have proposed using NIR spectroscopy to estimate fat and moisture in Atlantic salmon. Zhu *et al.* obtained accurate models using only the spectral information of the samples [27]. However, fat content is not uniform throughout a sample, and Zhang *et al.* demonstrated that more robust models for fat and moisture can be obtained if texture features extracted from characteristic spectral bands are used as predictors [28]. Using the aforementioned approaches, the authors not only predicted the overall fat and moisture content for the samples but also provided their spatial distribution within the salmon fillets (**Figure 3**). In a more technical approach, Dixit *et al.* performed a comparison between two different hyperspectral technologies (line scan and snapshot) working in different spectral ranges for the determination of fat in Atlantic salmon [29]. The authors concluded that the spectral range from 670 to 950 nm was able to provide an equivalent performance in the prediction of fat compared to the spectral range from 550 to 1700 nm, which may lead to the use of cheaper instrumentation for this application due to the narrower spectral range needed.

Another important quality indicator for fish is blood content. During capture, fish are, as a rule, drained of their blood by cutting through the gills. This is mainly done in order to kill the fish quickly, but it also has the effect of preventing the blood from settling in the muscle and changing its color. The appearance of a fish fillet impacts its perceived quality, and a red hue in a whitefish fillet can be off-putting to the consumer. In the case of smoked products, any remaining blood turns brown and can, for instance, be perceived as dark spots in a smoked salmon fillet.

Skjelvareid *et al.* demonstrated that hyperspectral imaging can detect and quantify blood in whitefish fillets [30]. The hemoglobin in the blood absorbs light very strongly in a specific region of the visual spectrum and therefore stands out against the white fish muscle. The different oxidation states of the hemoglobin can also be distinguished

#### **Figure 3.**

*Spatial mapping of moisture (a) and fat (b). Reprinted by permission from springer nature customer service Centre GmbH: Springer nature, food and bioprocess technology [27] [COPYRIGHT: Springer nature] (2013).*

**Figure 4.**

*Quantification of blood in cod fillets using hyperspectral imaging. a) Calibrated color image based on diffuse reflectance hyperspectral imaging. b) Estimated blood concentration based on diffuse reflectance hyperspectral imaging. c) Estimated blood concentration based on interactance hyperspectral imaging [31].*

by their spectral signature, which makes it possible to do a pixelwise spectral unmixing by using the known reference spectra for the hemoglobin. An example of the quantification of blood in Atlantic cod fillets can be observed in **Figure 4**.

The same method has been applied to salmon fillets as well. The pigments in the salmon muscle absorb light in the same spectral region as the hemoglobin but with a different spectral profile. It is therefore possible to distinguish the blood from the pigments by taking both of them into account.

Two illumination setups are presented in the above publications. The first one is a diffuse illumination for reflectance imaging, while the other is an interactance. The idea is that surface reflection does not give enough information about the internal properties of the fillet, such that the focused light source of the interactance setup is necessary to penetrate further into the muscle. To ensure that the light recorded by the camera has propagated through the muscle and been attenuated by it, the focused light source is placed a certain distance from the field of view of the camera, which reduces surface reflection in the camera field of view while providing a good signal from the inside of the fillet [31]. This technique has also been shown to work for quantifying blood in whole whitefish through the skin, which, at the time of writing, is being developed into a commercial quality control method [32].

#### **4.2 Analysis of freshness**

Technologies able to non-invasively estimate the freshness of seafood products are in demand for the industry. There are currently different techniques for the

#### *Perspective Chapter: Hyperspectral Imaging for the Analysis of Seafood DOI: http://dx.doi.org/10.5772/intechopen.108726*

estimation of freshness in seafood products; however, such methods are labor intensive and usually destructive and cannot be applied to every specimen in the product line. The possibility of technology able to perform rapid freshness analysis for every sample could bring to the industry new alternatives for decision making with the goal to improve the processing and sorting of the raw materials.

Several researchers have investigated the estimation of the freshness of seafood products using hyperspectral imaging. Usually, the approaches followed by those researchers consist of the utilization of spectral data together with multivariate analysis methods to predict the values of different reference measurements related to the freshness.

A basic common reference method for the estimation of freshness is the storage time. Some researchers have successfully estimated storage time as a freshness indicator for fillets from different fish species using hyperspectral imaging, for example, pearl gentian grouper [33], Atlantic salmon [34], and Atlantic cod [35]. Kimiya *et al.* [34] and Sivertsen *et al.* [35] attributed the spectral changes between the different storage times to the oxidation of hemoglobin and myoglobin proteins during the chilled storage, which enables the successful estimation of the storage time based on the spectral information.

The total volatile basic nitrogen (TVB-N) is often used as a biomarker of protein and amine degradation and is considered a proxy freshness of fresh meat and fish products [36]. TVB-N has been widely used as a reference value for freshness estimation using hyperspectral imaging. In the literature, TVB-N estimation in fillets from different species can be found, including rainbow trout [37], grass carps [38, 39], or tilapia [40]. **Figure 5** shows the spatial distribution of TVB-N values within grasp carp fillets. All the above research presented accurate models for predicting TVB-N values using the VNIR spectral range. However, Yu *et al.* demonstrated that combining the VNIR and NIR spectral ranges resulted in improved estimations [40].

Although storage time and TVB-N methods have been the more common reference methods for determining freshness using hyperspectral imaging, other researchers have used alternative methods with successful results. Zhang *et al.* used electrical conductivity on largemouth bass fillets [41], while sensory evaluation of the shelf-life was used as a reference method for the estimation of freshness by Khoshnoudi-Nia *et al.* [42].

#### **Figure 5.**

*Spatial distribution of TVB-N values for freshness estimation. (a), (b) and (c) shows the TVB-N for different fillets (8.26, 12.98, and 15.69 mg N/100 g, respectively). Reprinted from innovative Food Science & Emerging Technologies, 21, Jun-Hu Cheng, Da-Wen Sun, Xin-An Zeng, Hong-Bin Pu, non-destructive and rapid determination of TVB-N content for freshness evaluation of grass carp (Ctenopharyngodon idella) by hyperspectral imaging [38], page 9, Copyright (2014), with permission from Elsevier.*

#### **4.3 Quality characterization**

Quality evaluation of seafood products, and food products in general, is mainly determined by how the odor, color, and texture of the product is perceived by customers. Traditionally, this quality evaluation has been addressed by sensory evaluation panels, who are a group of people trained to perform a quality judgment of seafood products. In recent years, some solutions based on hyperspectral imaging have been investigated to produce objective measurements of these quality parameters for different seafood products to help the industry stakeholders optimize their production.

Texture is a significant feature for the quality perception of seafood products by customers. For the texture evaluation, there are instruments that allow one to perform objective measurements, which are more repeatable than the subjective opinion of a sensory panel. However, the use of texture analyzers is time consuming and destructive. For this reason, some researchers have proposed the use of hyperspectral imaging for the characterization of texture features in seafood products. In those studies, the reference texture data are usually collected using a variety of mechanical instruments able to measure the force needed to compress or tear a sample. Wang *et al.* developed multivariate regression models based on the spectral data from commercial crisp grass carp (*Ctenopharyngodon idellus*) fillets to predict their hardness attributes using the spectral information in the VNIR spectral range [43]. Another research study demonstrated that the use of hyperspectral images in the SWIR spectral range is also suitable for the estimation of texture features in rainbow trout (*Oncorhynchus mykiss*) fillets [44]. The results of these studies showed high correlation between the predicted texture values from the spectral data and the texture measurements. In another innovative study, the authors also obtained promising models for the estimation of texture parameters of fish by using spectral and textural data from eyes and gills [45]. This approach has the advantage of being able to predict the texture of the fish before it is cut into fillets.

Wang *et al.* proposed the use of artificial neural networks together with VNIR spectral data for the characterization of color in large yellow croaker (*Larimichthys crocea*) fillets [46]. In this study, the color variations in the samples were produced by storing the samples in different conditions and acquiring hyperspectral data. The corresponding reference measurements used a colorimeter to quantify the color parameters of the sample. The results of this study showed that hyperspectral imaging is a potential tool for color characterization of samples, with some advantages over the colorimeters. Colorimeters require point measurements, which present two main disadvantages: there is a need for physical contact with the sample, and the measurements are performed in a limited number of spots on the sample.

#### **4.4 Detection of nematodes**

Parasites in fish are a significant problem for seafood producers and consumers, presenting both quality and health concerns. Typically, the presence of parasites in products leads to rejection of the product by both purchasers and sellers. Parasites, such as *Anisakis simplex* and *Pseudoterranova decipiens,* are commonly present in whitefish fillets [47]. Today, every single fillet is inspected by transillumination on candling tables [48], and nematodes are removed manually. The detection rate using candling tables has been reported as low as 23% in a recent study by Mercken *et al.* [49]. Manual screening for parasites is an expensive operation previously reported to account for half of the production cost for Pacific cod from the Bering Sea and the Gulf of Alaska [50]. Several different instrumental methods have been evaluated for nematode detection:

#### *Perspective Chapter: Hyperspectral Imaging for the Analysis of Seafood DOI: http://dx.doi.org/10.5772/intechopen.108726*

fluorescence [51], ultrasonic waves [52], X-ray and computer tomography [31], and multispectral imaging [53]. The first conceptualization on the use of spectroscopic techniques for nematode detection was proposed by Pau *et al.* in 1991, where the spectral differences between the parasites and the fish muscle were shown [54]. The chemical differences between nematodes and fish muscle were documented by Stormo *et al.* [55], and a later work discussed the impact of selecting a limited number of wavelengths based on such chemical differences [56]. In Sigernes *et al.*, the authors developed a custom spectral imager targeting a wide variety of seafood industry applications [57]. In that work, the authors showed as a proof of concept that the spectral information can be potentially used to identify nematodes in fish samples. Using the same instrumentation, Heia *et al.* conducted the first research study on the detection of nematodes with hyperspectral images [58]. Using the transmittance illumination mode, this work served as a proof of concept to show the potential of spectral imaging for nematode detection. The goal of using transillumination was to be able to detect nematodes deeply embedded in the fish flesh. However, this preliminary work was limited by a low number of samples and ideal laboratory conditions. With the goal of making the system more suitable for an industrial setting, Sivertsen *et al.* further investigated this research line. First, a transillumination setup based on a commercial hyperspectral camera with a higher number of samples was evaluated [59]. However, despite the promising results in the detection of nematodes, the transmittance setup still presented obstacles for implementation in industry, for example, a low imaging speed and challenges regarding the optimization of the light conditions. For those reasons, in a subsequent study, Sivertsen *et al.* proposed for the first time the use of interactance hyperspectral imaging for the detection of nematodes [60]. In this work, the authors were able to satisfy industrial needs for fast acquisition and processing of the images. However, although the detection rate of nematodes was comparable with the human manual inspection, the detection rate was still low and the false positive rate too high to meet industrial requirements.

In an ongoing research project funded by the Norwegian Seafood Research Fund (FHF), entitled *Commercial Nematode Detection in Whitefish Fillets* (901614), a solution is being developed to perform nematode detection using hyperspectral imaging. The project is being conducted by the Norwegian Institute of Food, Fisheries and Aquaculture Research (Nofima) and Maritech, a company commercializing a hyperspectral solution for seafood inspection called Maritech Eye™. In the previous approaches, only the spectral information from the nematodes and the fish muscle was exploited. In this project, a solution based on a deep learning neural network, where both the spatial and the spectral features of the data are utilized to detect the nematodes, is proposed. **Figure 6** shows the manual annotation of the nematodes as well as the automatic detection of the nematodes using hyperspectral image analysis with a deep neural network. The experiment to demonstrate the feasibility of this approach was tested under industrial conditions in a cod production factory belonging to the company Maredeus (Portugal). The results of the proposed approach were accurate, with a high detection rate and almost no false positives. In addition, the system was able to operate at industrial speed (400 mm/second), including both the image acquisition and the data analysis, which would make it possible to use this approach as an industrial solution for the detection of nematodes.

#### **4.5 Identification of different species**

Another challenge for seafood production lines is the automatic sorting of different species when they are processed simultaneously. Additionally, the use of imaging

#### **Figure 6.**

*Manual annotation of nematodes (blue) and automatic prediction (yellow) of their location on cod fillets based on hyperspectral image analysis.*

technologies able to identify different fish species is attractive both for the consumer and for the industry, since they can help to mitigate fraud in fish mislabeling [61]. In a research study performed by Chauvin *et al.*, the authors evaluated the potential of the spectral information of fillets from different species in order to correctly classify them [62, 63]. A total of 22 fish species were recorded using diffuse reflectance illumination (VNIR and SWIR spectral ranges) and fluorescence excitation (VIS). Using this data, different supervised classifiers based on the spectral data from the different species were trained. The results obtained in this study suggest that the combination of spectral channels from the different spectral ranges and imaging modalities improve the classification compared to single-mode data (i.e., only VNIR, only SWIR, or only fluorescence). Finally, the authors investigated reducing the number of spectral bands needed for species identification without compromising on the performance of the classifier. The outcome of this research was a selection of 7 spectral bands that can be potentially used for the identification of species. This finding paves the way for the future development of cheap instruments based on LED illumination using such specific wavelengths to perform the species sorting.

Beyond the seafood industry, hyperspectral imaging has also been investigated for species identification with the goal of using this technology as a complementary tool to existing molecular and morphological techniques for faunal biodiversity assessment. Kolmann *et al.* performed a study in South American fish species that are difficult to distinguish even under controlled conditions: piranhas and pacus (both from the family Serrasalmidae) [64]. The authors were able to successfully discriminate between 47 different species and subspecies, using only their spectral information (**Figure 7**). The outcomes of this study demonstrated hyperspectral imaging as a potential technology for biodiversity screening.

#### **4.6 Damage detection**

One of the main pretreatments applied to freshwater fish is scale removal; however, methods to do this can produce damage to the product. With the goal of better characterizing the damages caused by the different physical scale removal methods, Wang *et al.* proposed the utilization of VNIR spectral data as a tool to visualize such

*Perspective Chapter: Hyperspectral Imaging for the Analysis of Seafood DOI: http://dx.doi.org/10.5772/intechopen.108726*

**Figure 7.**

*Comparative spectral signatures from four different body regions of a pacu (*Myloplus schomburgkii*) (right) and a piranha (Serrasalmus geryi) (left). Reproduced from [64]; creative commons BY 4.0; published by springer nature limited (2021).*

damages [65]. The results of this study were positive, showing an accurate identification of the damaged areas based only on the spectral information.

Another type of damage occurs when fish are caught. Jensen *et al.* proposed the use of a catch damage index based on VNIR hyperspectral information to characterize the catch damage when different trawling strategies are used [66]. The method is based on the estimation of the residual blood in fish muscle by using constrained spectral unmixing [30]. Using this application of hyperspectral image processing, it was possible to conduct an experiment to evaluate the effect of different trawling strategies on fish damage.

#### **4.7 Detection of contamination agents**

Plastic contamination in marine environments leads to the ingestion of microplastics by fish. There is evidence that indicates that microplastics intake causes harmful effects to fish health [67]. In recent years, research has shown an increasing trend in the presence of microplastics in seafood products [68]. However, the methods to accurately quantify the presence of microplastics are complex and expensive, which complicates the experimental trials required to quantify the effect of this problem. For this reason, Zhang *et al.* proposed the use of hyperspectral imaging in the range from 900 to 1700 nm for the identification of microplastics [69]. With the goal of training a supervised classifier based on the spectral information, the intestinal tract contents of different fish were contaminated with plastic polymers of different chemical composition, size, and color. Then, the accuracy of the proposed methodology was evaluated, both in prepared samples and in fish samples from three different species. The results of this experiment indicated that hyperspectral imaging can be a suitable technology to detect the presence of microplastics in the intestinal tract of fish. However, the precision in the detection is affected by the size of the plastic particles, which makes it necessary to increase the dataset to improve the machine learning models to improve the detection of small plastic particles.

From the food safety perspective, the detection of harmful microorganisms present in fish is a relevant topic. With the goal of developing imaging technologies for the detection of *Enterobacteriaceae* contamination in Atlantic salmon (Salmo salar) flesh, He *et al.* investigated the use of the NIR spectra for monitoring the presence of such bacteria [70]. After capturing hyperspectral images of salmon contaminated with *Enterobacteriaceae* at different storage periods, the authors were able to quantify the presence and severity of the bacterial contamination. It is worth noticing that hyperspectral imaging technology is not able to measure the bacteria presence by itself; however, there are differences between the spectra from contaminated and non-contaminated salmon flesh.

#### **4.8 Applications in aquaculture**

Aquaculture production has significantly grown during the past 20 years [71]. This is mainly due to the increasing demand for seafood products, together with the goal of the seafood industry to increase productivity. Thus, there is a current demand for novel information and digital technologies that can be applied in aquaculture to improve the productivity of fish farms [72]. Nowadays, the use of hyperspectral imaging technologies in aquaculture is limited to a few contributions.

In Atlantic salmon (S. salar) farming, the transition from juvenile freshwater fish (parr) to seawater adapted fish (smolt) is called smoltification. Smoltification involves changes in the morphology, physiology, and biochemistry of juvenile salmon. From a fish farmer perspective, it is important to monitor the smoltification process for two principal reasons. On the one hand, an incomplete smoltification process at the time the salmon is transferred to seawater leads to poor salmon welfare and an increased risk of mortality. On the other hand, a late transition to seawater generates negative consequences for the farmer since the production chain is not optimized, which induces economic losses. With the goal of providing the aquaculture industry with a solution to this problem, Svendsen *et al.* studied the relationship between the spectral information and the physiological changes in juvenile salmons [73]. After analyzing more than 300 fish from three different farms, the authors were able to perform an accurate discrimination between parr and smolt with high sensitivity and specificity. The classification was performed using a machine learning classifier (Support Vector Machine) using only three specific spectral channels.

Salmon lice (*Lepeophtheirus salmonis*) are parasites that live on salmonid fishes. The salmon lice represent a huge problem for both farmed and wild salmon because they can produce severe fin damage, bleeding, and open wounds in the host. Salmon affected by these parasites are likely sensitive to other pathogens, leading to increased sickness. Therefore, salmon lice are a problem that generates negative effects in salmon welfare and leads to significant economic losses for the farmers and suffering for the fish. Early warnings for lice could help farmers to take action to eliminate the infestation. However, the identification and counting of these parasites are challenging tasks even for skilled staff. With the goal of providing an automatic solution to this problem, Pettersen *et al.* conducted an experiment where underwater hyperspectral imaging was applied for the detection of salmon lice [74]. First, the authors recorded and characterized the spectral signature of different salmon lice subtypes in laboratory conditions, in both air and underwater conditions. Finally, they tested the method to identify the different lice subtypes in salmon using the underwater hyperspectral imaging system. Although the research suggested underwater hyperspectral imaging as a promising technology for the detection of salmon lice in sea cages, it can be considered *Perspective Chapter: Hyperspectral Imaging for the Analysis of Seafood DOI: http://dx.doi.org/10.5772/intechopen.108726*

as a proof of concept, and more research needs to be performed to optimize the instrumentation for use as a final product in aquaculture farms.

The use of hyperspectral imaging has also been applied in an indirect manner with the goal to improve the quality of the fish feed in aquaculture. Marine fishmeal powder is added as protein supplement in fish feed in aquaculture, but recently, the adulteration of this product with cheaper alternatives with lower nutritional value has become a common trend. To address this fraud, Kong *et al.* proposed the use of NIR hyperspectral imaging and convolutional neural networks for the identification of adulterants in marine fishmeal [75].

The aforementioned examples suggest that hyperspectral imaging technology can contribute to improvements in aquaculture in the near future.

#### **5. Summary table**

In this section, we provide a summary of the main research works that have been covered in this book chapter. In **Table 1**, the information about each research work is specified. This information includes the type of application, the fish species, the type of samples, the number of specimens, the illumination modality, the spectral range, and the image-processing method used to retrieve information from the hyperspectral images.




*N: Number of samples, Illum: Illumination Mode (DR: Diffuse Reflectance, IA: Interactance, FL: Fluorescence). PLSR: Partial Least Squares Regression, SVM: Support Vector Machines, PLS-DA: Partial Least Squares Discriminant Analysis, SAM: Spectral Angle Mapper.*

#### **Table 1.**

*Summary table.*

#### **6. Conclusions**

In this book chapter, we have surveyed the main applications of hyperspectral imaging for seafood industry-related problems. The main goal in most of the research carried out in this field is to provide an alternative to the expensive, time-consuming, and invasive reference methods that are currently employed for the characterization of seafood products. Additionally, the advantage of hyperspectral technology is its applicability to industrial production chains, where the analysis can be performed individually for every sample, which can lead to the optimization of production and decision making for the industry.

Although the application field of this technology is wide and promises to address actual problems for both the industry and the consumers, there are still challenges that must be carefully investigated in the upcoming years.

As far as the instrumentation is concerned, there are still uncertainties about which type of illumination (diffuse reflectance or interactance) is more appropriate for each application. Additionally, there is no strict criterion for the selection of the most adequate spectral range for each application. In this sense, more comparative research should be carried out in order to have clearer arguments on which spectral range should be used for different applications.

The number of processing methods used to extract information from hyperspectral data is huge and diverse. An appropriate evaluation of these methods should be carefully carried out to gain a better understanding of their limitations and advantages for each scenario. Additionally, most of the methods covered in the literature are based exclusively on the spectral information, while the spatial information is usually underrated. However, the trend in hyperspectral image analysis in other fields is to try to exploit simultaneously both the spatial and the spectral features of the data, especially with the rise of sophisticated deep learning architectures to this end [76, 77].

Regarding the future of this research line, the upcoming challenges should be focused on the transfer of knowledge to industry, where this technology could be employed to improve production chains and decision making. In this sense, commercial products consisting of industrial-grade spectral imaging systems have been recently launched, such as the QMonitor (QVision AS, Oslo, Norway) or the Maritech Eye (Maritech Systems AS, Molde, Norway). Both systems are based on interactance illumination mode. The QMonitor is a multispectral NIR system, while the Maritech Eye is a hyperspectral system in the VNIR spectral range. These devices have been proven to be useful for different food quality applications [78–82] and are currently used in food industry production facilities.

### **Acknowledgements**

This project is supported by DigiFoods, a Norwegian Strategic Research Initiative (project number 309259), and is also part of NFR funding grant 294805.

#### **Conflict of interest**

The authors declare no conflict of interest.

### **Author details**

Samuel Ortega\*, Stein-Kato Lindberg, Kathryn E. Anderssen and Karsten Heia Nofima, Norwegian Institute of Food Fisheries and Aquaculture Research, Tromsø, Norway

\*Address all correspondence to: samuel.ortega@nofima.no

© 2022 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### **References**

[1] Lu R, editor. Light Scattering Technology for Food Property, Quality and Safety Assessment. London: CRC Press Taylor & Francis Group; 2017

[2] Badaró AT, Hebling e Tavares JP, Blasco J, Aleixos-Borrás N, Barbin DF. Near infrared techniques applied to analysis of wheat-based products: Recent advances and future trends. Food Control. 2022;**140**:109115. DOI: 10.1016/ j.foodcont.2022.109115

[3] Hebling e Tavares JP, da Silva Medeiros ML, Barbin DF. Near-infrared techniques for fraud detection in dairy products: A review. Journal of Food Science. 2022;**87**(5):1943-1960. DOI: 10.1111/1750-3841.16143

[4] Bittante G, Patel N, Cecchinato A, Berzaghi P. Invited review: A comprehensive review of visible and near-infrared spectroscopy for predicting the chemical composition of cheese. Journal of Dairy Science. 2022; **105**(3):1817-1836. DOI: 10.3168/ jds.2021-20640

[5] An D, Zhang L, Liu Z, Liu J, Wei Y. Advances in infrared spectroscopy and hyperspectral imaging combined with artificial intelligence for the detection of cereals quality. Critical Reviews in Food Science and Nutrition. 2022:1-31. DOI: 10.1080/10408398.2022.2066062

[6] He Y, Xiao Q, Bai X, Zhou L, Liu F, Zhang C. Recent progress of nondestructive techniques for fruits damage inspection: A review. Critical Reviews in Food Science and Nutrition. 2022;**62**(20):5476-5494. DOI: 10.1080/ 10408398.2021.1885342

[7] Lu Y, Saeys W, Kim M, Peng Y, Lu R. Hyperspectral imaging technology for quality and safety evaluation of

horticultural products: A review and celebration of the past 20-year progress. Postharvest Biology and Technology. 2020;**170**:111318. DOI: 10.1016/j.posth arvbio.2020.111318

[8] Antequera T, Caballero D, Grassi S, Uttaro B, Perez-Palacios T. Evaluation of fresh meat quality by hyperspectral imaging (HSI), nuclear magnetic resonance (NMR) and magnetic resonance imaging (MRI): A review. Meat Science. 2021;**172**:108340. DOI: 10.1016/j.meatsci.2020.108340

[9] Jia W, van Ruth S, Scollan N, Koidis A. Hyperspectral imaging (HSI) for meat quality evaluation across the supply chain: Current and future trends. Current Research in Food Science. 2022; **5**:1017-1027. DOI: 10.1016/j. crfs.2022.05.016

[10] Falkovskaya A, Gowen A. Literature review: Spectral imaging applied to poultry products. Poultry Science. 2020; **99**(7):3709-3722. DOI: 10.1016/j. psj.2020.04.013

[11] Mei J, Zhao F, Xu R, Huang Y. A review on the application of spectroscopy to the condiments detection: From safety to authenticity. Critical Reviews in Food Science and Nutrition. 2022;**62**(23):6374-6389. DOI: 10.1080/10408398.2021.1901257

[12] Faith Ndlovu P, Samukelo Magwaza L, Zeray Tesfay S, Ramaesele Mphahlele R. Destructive and rapid noninvasive methods used to detect adulteration of dried powdered horticultural products: A review. Food Research International. 2022;**157**:111198. DOI: 10.1016/j.foodres.2022.111198

[13] Nobari Moghaddam H, Tamiji Z, Akbari Lakeh M, Khoshayand MR, Haji Mahmoodi M. Multivariate analysis of food fraud: A review of NIR based instruments in tandem with chemometrics. Journal of Food Composition and Analysis. 2022;**107**: 104343. DOI: 10.1016/j.jfca.2021.104343

[14] Soni A, Dixit Y, Reis MM, Brightwell G. Hyperspectral imaging and machine learning in food microbiology: Developments and challenges in detection of bacterial, fungal, and viral contaminants. Comprehensive Reviews in Food Science and Food Safety. 2022; **21**(4):3717-3745. DOI: 10.1111/ 1541-4337.12983

[15] Khan A, Vibhute AD, Mali S, Patil CH. A systematic review on hyperspectral imaging technology with a machine and deep learning methodology for agricultural applications. Ecological Informatics. 2022;**69**:101678. DOI: 10.1016/j.ecoinf.2022.101678

[16] Tuchin VV. Tissue Optics: Light Scattering Methods and Instruments for Medical Diagnosis. 3rd ed. Bellingham WA: SPIE; 2015

[17] Wilson RH, Nadeau KP, Jaworski FB, Tromberg BJ, Durkin AJ. Review of short-wave infrared spectroscopy and imaging methods for biological tissue characterization. Journal of Biomedical Optics. 2015;**20**(3):030901. DOI: 10.1117/1.JBO.20.3.030901

[18] Khodabux K, Lomelette M, Jhaumeerlaulloo S, Ramasami P, Rondeau P. Chemical and near-infrared determination of moisture, fat and protein in tuna fishes. Food Chemistry. 2007;**102**(3):669-675. DOI: 10.1016/j. foodchem.2006.05.057

[19] Nachabe R et al. Diagnosis of breast cancer using diffuse optical spectroscopy from 500 to 1600 nm: Comparison of classification methods. Journal of

Biomedical Optics. 2011;**16**(8):087010. DOI: 10.1117/1.3611010

[20] Li Q, He X, Wang Y, Liu H, Xu D, Guo F. Review of spectral imaging technology in biomedical engineering: Achievements and challenges. Journal of Biomedical Optics. 2013;**18**(10):100901. DOI: 10.1117/1.JBO.18.10.100901

[21] Wold JP et al. Non-contact Transflectance near infrared imaging for representative on-line sampling of dried salted coalfish (Bacalao). Journal of Near Infrared Spectroscopy. 2006;**14**(1): 59-66. DOI: 10.1255/jnirs.587

[22] Wu D, Sun D-W. Advanced applications of hyperspectral imaging technology for food quality and safety analysis and assessment: A review — Part I: Fundamentals. Innovative Food Science and Emerging Technologies. 2013;**19**:1-14. DOI: 10.1016/j. ifset.2013.04.014

[23] Sivertsen AH, Chu C-K, Wang L-C, Godtliebsen F, Heia K, Nilsen H. Ridge detection with application to automatic fish fillet inspection. Journal of Food Engineering. 2009;**90**(3):317-324. DOI: 10.1016/j.jfoodeng.2008.06.035

[24] Downey G. Non-invasive and nondestructive percutaneous analysis of farmed salmon flesh by near infra-red spectroscopy. Food Chemistry. 1996; **55**(3):305-311. DOI: 10.1016/0308-8146 (95)00118-2

[25] Hsin-I Feng C. The tale of sushi: History and regulations. Comprehensive Reviews in Food Science and Food Safety. 2012;**11**(2):205-220. DOI: 10.1111/j.1541-4337.2011.00180.x

[26] Richardsen R and Østli J. Norwegian Trout in Japan. Consumer Preferences, Perceptions and Competitors. 2003.

*Perspective Chapter: Hyperspectral Imaging for the Analysis of Seafood DOI: http://dx.doi.org/10.5772/intechopen.108726*

[Online]. Available: https://nofima. brage.unit.no/nofima-xmlui/handle/ 11250/282799

[27] Zhu F, Zhang H, Shao Y, He Y, Ngadi M. Mapping of fat and moisture distribution in Atlantic Salmon using near-infrared hyperspectral imaging. Food and Bioprocess Technology. 2014; **7**(4):1208-1214. DOI: 10.1007/ s11947-013-1228-z

[28] Zhang H et al. Non-destructive determination of fat and moisture contents in Salmon (Salmo salar) fillets using near-infrared hyperspectral imaging coupled with spectral and textural features. Journal of Food Composition and Analysis. 2020;**92**: 103567. DOI: 10.1016/j.jfca.2020.103567

[29] Dixit Y, Reis MM. Hyperspectral imaging for assessment of total fat in salmon fillets: A comparison between benchtop and snapshot systems. Journal of Food Engineering. 2023;**336**:111212. DOI: 10.1016/j.jfoodeng.2022.111212

[30] Skjelvareid MH, Heia K, Olsen SH, Stormo SK. Detection of blood in fish muscle by constrained spectral unmixing of hyperspectral images. Journal of Food Engineering. 2017;**212**:252-261. DOI: 10.1016/j.jfoodeng.2017.05.029

[31] Heia K, Washburn KE, and Skjelvareid MH. Automatic Quality Control of Internal Defects in cod - Results from Hyperspectral, Ultrasound and X-ray Imaging. 2017. [Online]. Available: https:// nofima.brage.unit.no/nofima-xmlui/hand le/11250/2480578

[32] Maritech AS. Maritech Eye. 2022. https://maritech.com/our-solutions-seaf ood-production/maritech-eye/

[33] Chen Z, Wang Q, Zhang H, Nie P. Hyperspectral imaging (HSI)

Technology for the non-Destructive Freshness Assessment of pearl gentian grouper under different storage conditions. Sensors. 2021;**21**(2):583. DOI: 10.3390/s21020583

[34] Kimiya T, Sivertsen AH, Heia K. VIS/NIR spectroscopy for nondestructive freshness assessment of Atlantic salmon (Salmo salar L.) fillets. Journal of Food Engineering. 2013; **116**(3):758-764. DOI: 10.1016/j. jfoodeng.2013.01.008

[35] Sivertsen AH, Kimiya T, Heia K. Automatic freshness assessment of cod (Gadus morhua) fillets by Vis/Nir spectroscopy. Journal of Food Engineering. 2011;**103**(3):317-323. DOI: 10.1016/j.jfoodeng.2010.10.030

[36] Bekhit AE-DA, Holman BWB, Giteru SG, Hopkins DL. Total volatile basic nitrogen (TVB-N) and its role in meat spoilage: A review. Trends in Food Science and Technology. 2021;**109**: 280-302. DOI: 10.1016/j.tifs.2021.01.006

[37] Moosavi-Nasab M, Khoshnoudi-Nia S, Azimifar Z, Kamyab S. Evaluation of the total volatile basic nitrogen (TVB-N) content in fish fillets using hyperspectral imaging coupled with deep learning neural network and meta-analysis. Scientific Reports. 2021;**11**(1):5094. DOI: 10.1038/s41598-021-84659-y

[38] Cheng J-H, Sun D-W, Zeng X-A, Pu H-B. Non-destructive and rapid determination of TVB-N content for freshness evaluation of grass carp (Ctenopharyngodon idella) by hyperspectral imaging. Innovative Food Science and Emerging Technologies. 2014;**21**:179-187. DOI: 10.1016/j. ifset.2013.10.013

[39] Cheng J-H, Sun D-W, Wei Q. Enhancing visible and near-infrared hyperspectral imaging prediction of TVB-N level for fish fillet freshness evaluation by filtering optimal variables. Food Analytical Methods. 2017;**10**(6): 1888-1898. DOI: 10.1007/s12161-016- 0742-9

[40] Yu H-D et al. Hyperspectral imaging in combination with data fusion for rapid evaluation of tilapia fillet freshness. Food Chemistry. 2021;**348**: 129129. DOI: 10.1016/j. foodchem.2021.129129

[41] Zhang W, Cao A, Shi P, Cai L. Rapid evaluation of freshness of largemouth bass under different thawing methods using hyperspectral imaging. Food Control. 2021;**125**:108023. DOI: 10.1016/ j.foodcont.2021.108023

[42] Khoshnoudi-Nia S, Moosavi-Nasab M. Prediction of various freshness indicators in fish fillets by one multispectral imaging system. Scientific Reports. 2019;**9**(1):14704. DOI: 10.1038/ s41598-019-51264-z

[43] Wang QX, Su LH, Zou J, Chen NX, Wu T, Yang L. Research on hardness detection method of crisped grass carp based on visible - near infrared hyperspectral technology. Journal of Physics Conference Series. 2021;**1757**(1): 012002. DOI: 10.1088/1742-6596/1757/1/ 012002

[44] Khoshtaghaza MH, Khojastehnazhand M, Mojaradi B, Goodarzi M, Saeys W. Texture quality analysis of rainbow trout using hyperspectral imaging method. International Journal of Food Properties. 2016;**19**(5):974-983. DOI: 10.1080/ 10942912.2015.1042111

[45] Wang X, Shan J, Han S, Zhao J, Zhang Y. Optimization of fish quality by evaluation of Total volatile basic nitrogen (TVB-N) and texture profile analysis (TPA) by near-infrared (NIR)

hyperspectral imaging. Analytical Letters. 2019;**52**(12):1845-1859. DOI: 10.1080/00032719.2019.1571077

[46] Wang S, Das AK, Pang J, Liang P. Real-time monitoring the color changes of large yellow croaker (Larimichthys crocea) fillets based on hyperspectral imaging empowered with artificial intelligence. Food Chemistry. 2022;**382**: 132343. DOI: 10.1016/j. foodchem.2022.132343

[47] McClelland G. Spatial and temporal distributions of larval sealworm (Pseudoterranova decipiens, Nematoda: Anisakinae), in Hippoglossoides platessoides (Pleuronectidae) in eastern Canada from 1980 to 1990. ICES Journal of Marine Science. 2000;**57**(1):69-88. DOI: 10.1006/jmsc.1999.0518

[48] Hafsteinsson H, Rizvi SSH. A review of the Sealworm problem: Biology, implications and solutions. Journal of Food Protection. 1987;**50**(1):70-84. DOI: 10.4315/0362-028X-50.1.70

[49] Mercken E et al. Sensitivity of candling as routine method for the detection and recovery of ascaridoids in commercial fish fillets. Scientific Reports. 2022;**12**(1):1358. DOI: 10.1038/ s41598-022-05235-6

[50] Bublitz CG, Choudhury GS. Effect of light intensity and color on worker productivity and parasite detection efficiency during candling of cod fillets. Journal of Aquatic Food Product Technology. 1993;**1**(2):75-89. DOI: 10.1300/J030v01n02\_08

[51] Pippy JHC. Use of ultraviolet light to find parasitic nematodes in situ. Journal of the Fisheries Board of Canada. 1970; **27**(5):963-965. DOI: 10.1139/f70-107

[52] Hafsteinsson H, Parker K, Chivers R, Rizvi SSH. Application of ultrasonic

*Perspective Chapter: Hyperspectral Imaging for the Analysis of Seafood DOI: http://dx.doi.org/10.5772/intechopen.108726*

waves to detect Sealworms in fish tissue. Journal of Food Science. 1989;**54**(2): 244-247. DOI: 10.1111/j.1365-2621.1989. tb03053.x

[53] Wold JP, Westad F, Heia K. Detection of parasites in cod fillets by using SIMCA classification in multispectral images in the visible and NIR region. Applied Spectroscopy. 2001; **55**(8):1025-1034. DOI: 10.1366/ 0003702011952929

[54] Pau LF. Fish Quality Control by Computer Vision. Abingdon, Oxfordshire: Routledge; 2017

[55] Stormo SK, Ernstsen A, Nilsen H, Heia K, Sivertsen AH, Elvevoll E. Compounds of parasitic roundworm absorbing in the visible region: Target molecules for detection of roundworm in Atlantic cod. Journal of Food Protection. 2004;**67**(7):1522-1525. DOI: 10.4315/ 0362-028X-67.7.1522

[56] Stormo SK, Sivertsen AH, Heia K, Nilsen H, Elvevoll E. Effects of single wavelength selection for Anisakid roundworm larvae detection through multispectral imaging. Journal of Food Protection. 2007;**70**(8):1890-1895. DOI: 10.4315/0362-028X-70.8.1890

[57] Sigernes F, Lorentzen DA, Heia K, Svenøe T. Multipurpose spectral imager. Applied Optics. 2000;**39**(18):3143. DOI: 10.1364/AO.39.003143

[58] Heia K, Sivertsen AH, Stormo SK, Elvevoll E, Wold JP, Nilsen H. Detection of nematodes in cod (Gadus morhua) fillets by imaging spectroscopy. Journal of Food Science. 2007;**72**(1):E011-E015. DOI: 10.1111/j.1750-3841.2006.00212.x

[59] Sivertsen AH, Heia K, Stormo SK, Elvevoll E, Nilsen H. Automatic nematode detection in cod fillets (Gadus Morhua) by Transillumination

hyperspectral imaging. Journal of Food Science. 2011;**76**(1):S77-S83. DOI: 10.1111/j.1750-3841.2010.01928.x

[60] Sivertsen AH, Heia K, Hindberg K, Godtliebsen F. Automatic nematode detection in cod fillets (Gadus morhua L.) by hyperspectral imaging. Journal of Food Engineering. 2012;**111**(4):675-681. DOI: 10.1016/j.jfoodeng.2012.02.036

[61] Hassoun A et al. Fraud in animal origin food products: Advances in emerging spectroscopic detection methods over the past five years. Food. 2020;**9**(8):1069. DOI: 10.3390/ foods9081069

[62] Chauvin J et al. Simulated annealingbased hyperspectral data optimization for fish species classification: Can the number of measured wavelengths Be reduced? Applied Sciences. 2021;**11**(22): 10628. DOI: 10.3390/app112210628

[63] Qin J et al. Detection of fish fillet substitution and mislabeling using multimode hyperspectral imaging techniques. Food Control. 2020;**114**: 107234. DOI: 10.1016/j. foodcont.2020.107234

[64] Kolmann MA et al. Hyperspectral data as a biodiversity screening tool can differentiate among diverse Neotropical fishes. Scientific Reports. 2021;**11**(1): 16157. DOI: 10.1038/s41598-021-95713-0

[65] Wang H, Qiu X, Zeng F, Shao W, Ma Q, Li M. Detection of physical descaling damage in carp based on hyperspectral images and dimension reduction of principal component analysis combined with pixel values. Journal of Food Science. 2022;**87**(6): 2663-2677. DOI: 10.1111/1750-3841.16144

[66] Jensen TK et al. Effect of the T90 codend on the catch quality of cod

(Gadus morhua) compared to the conventional codend configuration in the Barents Sea bottom trawl fishery. Fisheries Research. 2022;**250**:106277. DOI: 10.1016/j.fishres.2022.106277

[67] Jovanović B. Ingestion of microplastics by fish and its potential consequences from a physical perspective. Integrated Environmental Assessment and Management. 2017; **13**(3):510-515. DOI: 10.1002/ieam.1913

[68] Wootton N, Reis-Santos P, Gillanders BM. Microplastic in fish – A global synthesis. Reviews in Fish Biology and Fisheries. 2021;**31**(4):753-771. DOI: 10.1007/s11160-021-09684-6

[69] Zhang Y et al. Hyperspectral imaging based method for rapid detection of microplastics in the intestinal tracts of fish. Environmental Science & Technology. 2019;**53**(9): 5151-5158. DOI: 10.1021/acs.est.8b07321

[70] He H-J, Sun D-W. Selection of informative spectral wavelength for evaluating and Visualising Enterobacteriaceae contamination of Salmon flesh. Food Analytical Methods. 2015;**8**(10):2427-2436. DOI: 10.1007/ s12161-015-0122-x

[71] Naylor RL et al. A 20-year retrospective review of global aquaculture. Nature. 2021;**591**(7851): 551-563. DOI: 10.1038/s41586-021- 03308-6

[72] Yue K, Shen Y. An overview of disruptive technologies for aquaculture. Aquaculture and Fisheries. 2022;**7**(2): 111-120. DOI: 10.1016/j.aaf.2021.04.009

[73] Svendsen E et al. Identification of spectral signature for in situ real-time monitoring of smoltification. Applied Optics. 2021;**60**(14):4127. DOI: 10.1364/ AO.420347

[74] Pettersen R, Lein Braa H, Gawel BA, Letnes PA, Sæther K, Aas LMS. Detection and classification of Lepeophterius salmonis (Krøyer, 1837) using underwater hyperspectral imaging. Aquacultural Engineering. 2019;**87**: 102025. DOI: 10.1016/j.aquaeng.2019. 102025

[75] Kong D et al. Hyperspectral imaging coupled with CNN: A powerful approach for quantitative identification of feather meal and fish by-product meal adulterated in marine fishmeal. Microchemical Journal. 2022;**180**:107517. DOI: 10.1016/j.microc.2022.107517

[76] Jaiswal G, Sharma A, Yadav SK. Critical insights into modern hyperspectral image applications through deep learning. WIREs: Data Mining and Knowledge Discovery. 2021; **11**(6):e1426. DOI: 10.1002/widm.1426

[77] Audebert N, Le Saux B, Lefevre S. Deep learning for classification of hyperspectral data: A comparative review. IEEE Geoscience Remote Sensor Management. 2019;**7**(2):159-173. DOI: 10.1109/MGRS.2019.2912563

[78] Wold JP, Veiseth-Kent E, Høst V, Løvland A. Rapid on-line detection and grading of wooden breast myopathy in chicken fillets by near-infrared spectroscopy. PLoS One. 2017;**12**(3): e0173384. DOI: 10.1371/journal. pone.0173384

[79] Wold JP, Kermit M, Woll A. Rapid nondestructive determination of edible meat content in crabs ( cancer Pagurus) by near-infrared imaging spectroscopy. Applied Spectroscopy. 2010;**64**(7): 691-699. DOI: 10.1366/ 000370210791666273

[80] O'Farrell M, Wold JP, Høy M, Tschudi J, Schulerud H. On-line fat *Perspective Chapter: Hyperspectral Imaging for the Analysis of Seafood DOI: http://dx.doi.org/10.5772/intechopen.108726*

content classification of inhomogeneous pork trimmings using multispectral near infrared Interactance imaging. Journal of Near Infrared Spectroscopy. 2010;**18**(2): 135-145. DOI: 10.1255/jnirs.876

[81] Wold JP, Solberg LE, Gaarder MØ, Carlehøg M, Sanden KW, Rødbotten R. In-line estimation of fat marbling in whole beef striploins (longissimus lumborum) by NIR hyperspectral imaging. A closer look at the role of myoglobin. Food. 2022;**11**(9):1219. DOI: 10.3390/foods11091219

[82] Wold JP, O'Farrell M, Høy M, Tschudi J. On-line determination and control of fat content in batches of beef trimmings by NIR imaging spectroscopy. Meat Science. 2011;**89**(3):317-324. DOI: 10.1016/j.meatsci.2011.05.001

### Section 2

Preprocessing and Feature Extraction of Hyperspectral Imaging Data for Machine-Learning and Deep-Learning Analysis

#### **Chapter 2**

## Useful Feature Extraction and Machine Learning Techniques for Identifying Unique Pattern Signatures Present in Hyperspectral Image Data

*Jeanette Hariharan, Yiannis Ampatzidis, Jaafar Abdulridha and Ozgur Batuman*

#### **Abstract**

This chapter introduces several feature extraction techniques (FETs) and machine learning algorithms (MLA) that are useful for pattern recognition in hyperspectral data analysis (HDA). This chapter provides a handbook of the most popular FETs that have proven successful. Machine learning algorithms (MLA) for use with HDA are becoming prevalent in pattern recognition literature. Several of these algorithms are explained in detail to provide the user with insights into applying these for pattern recognition. Unsupervised learning applications are useful when the system is provided with the correct set of independent variables. Various forms of linear regression assay adequately solve hyperspectral pattern resolution for identifying phenotypes. K-means is an unsupervised learning algorithm that is used for systematically dividing a dataset into K number of pattern groups. Supervised and unsupervised neural networks (NNs) are used to discern patterns in hyperspectral data with features as inputs and in large datasets where little *a priori* knowledge is applied. Other supervised machine learning procedures derive valuable feature detectors and descriptors through support vector machine. Several methods using reduced sets for extracting patterns from hyperspectral data are shown by discretized numerical techniques and transformation processes. The accuracy of these methods and their usefulness is generally assessed.

**Keywords:** pattern signature, hyperspectral data, data reduction, power spectral density, biomarker

#### **1. Introduction**

Hyperspectral imaging and data analysis have recently received considerable attention since the representative data is ultra-high resolution and informative [1–7]. Hyperspectral data is collected by using a precision imaging device which emits light energy at wavelengths below, within, and above the visible range. Other cameras such as RGB (Red-Green-Blue sensitive filters) and multispectral, have more limited sets of data. These passive sensors only collect one to five reflective values from ambient light that is present (**Figure 1**). It then scans each pixel location sequentially for reflectance values from each wavelength of light emitted. Each wavelength is spaced about 2–5 nms. Apart. The data collected for each pixel in an image then represents the complete spectrum returns for the hyperspectral bands presented (see **Figures 2**–**4**). Within this data, patterns of information exist that have never been detected before, thus allowing the explorer to glean relevant and new highlights from this collected data set.

**Figure 1.** *Various camera operational bands.*

*Useful Feature Extraction and Machine Learning Techniques for Identifying Unique Pattern… DOI: http://dx.doi.org/10.5772/intechopen.107436*

**Figure 3.** *Hyperspectral imaging process (NASA – Public domain).*

**Figure 4.** *Hyperspectral imaging analysis.*

Hyperspectral cameras use a line scanning sensor (mostly, push broom type), that emits light at varying frequencies and then collects the reflected signal through a narrow slit. The narrower the slit, the higher the resolution of the camera, until it begins interfering with the light wave signal itself. The reflected light enters the slit and coincides with a concave mirror (**Figure 5**, M1) where the light is collimated. M1 redirects the collimated light from the scan to the optical grating. Here the light is divided or dispersed into its component frequencies. M2 acts to expand the beams and redirect the light to a reimaging lens array in the sensory unit.

The hyperspectral camera can be embedded on UAVs (e.g., **Figure 6**) to enhance aerial views for many image pursuits for agriculture, marine studies, search and rescue, surveillance, military activities, and construction site safety and management. Once hyperspectral images are stored, the data can be acquired per pixel per wavelength to reconstruct the image or study the reflected signatures. For instance, in agriculture, crops in a region can be surveyed by studying the map of the pure signature spectrum (**Figure 7**). Other pattern recognition algorithms can be used to understand how normal spectrums represent specific species of plants. Detecting plant diseases and stress factors in early stages of disease development is essential for selective and effective management of crop production. Through other data analysis procedures, such as feature extraction, statistical prediction, and reduced signature spectrums, pinpointing where the spectrums differ between a range of normal signatures and abnormal feature spectrums will reveal variations in the species [3, 8, 9]. Realizing how these spectrums differ because of diseases, abiotic stressors, nutritional deficiencies, or other factors, will give more useful information to the farmers.

**Figure 5.** *Hyperspectral imaging hardware operation.*

*Useful Feature Extraction and Machine Learning Techniques for Identifying Unique Pattern… DOI: http://dx.doi.org/10.5772/intechopen.107436*

**Figure 6.** *Hyperspectral imaging device as payload on UAV.*

The focus of this chapter is on extrication of features and pattern recognition algorithms that can be used in hyperspectral data analysis to obtain useful information. Common preprocessing and analysis applications include normalization and derivative spectra enhancement using finite differencing [10], complex step derivative [11], and derivative spectral shape equation [9]. Wavelet Transform has been used and compared to derivative spectra enhancement and shown to be very successful in spectral regions of interest; it is becoming more commonly used as an alternative to spectral derivative methods [12]. Polynomial interpolations are also used to smooth the (spectral) data and better represent enhanced spectra. Multivariate analysis can be used to gain a better understanding of spectral variance between feature data [2, 4]. Recently, autonomous ground and unmanned aerial vehicles with hyperspectral camera payloads have been used to collect data for agricultural purposes [13, 14]. Along with this method of data collection, deep and transfer learning artificial intelligence applications have been developed for pest and plant stress detection [5, 6]. These techniques required a high-quality training dataset for accurate development of the prediction models [5].

Hyperspectral Imaging is gaining widespread use in drone applications for agriculture and water safety. Agricultural applications include landscaping crop regions, analysis of crop health, understanding nutritional status of plants, harvest studies, flowering index, growth cycles comparison, trait discrimination, breeding information, and soil performance. Associated AI and machine learning applications are the mainstay of these informational systems. In water quality analysis, various hyperspectral algorithms such as partial least-squares, fully connected neural networks with backpropagation (FCNN-BP), Support vector machine (SVM) and Random Forest (RF) procedures have been successfully used and compared for quantitative investigation [15]. Other assessments have been implemented using FETs and AI for detecting water contamination in rivers [16], forest fire assessment, and automated drone team hyperspectral fusion.

Machine learning algorithms usually require some preprocessing of the data. Segmentation and feature extraction often use spatial filters, Laplacian of Gaussian with orientation filters [17], and other traditional methods that detect spectral discontinuities or similarities to adequately obtain prediction models for patterns. Gradient magnitude algorithms form ridges at high valued pixels, noted as watersheds, that are used to segment regions. Adaptive thresholding is applied to image data that has nonuniform background. This thresholding approach calculates local thresholds based on specified properties of pixel neighborhoods to segment regions of interest.

Active contours are another avenue of interest in obtaining features. Snakes were developed [18] as parametric curves that uncover boundary regions by minimizing an energy function. This optimization works well with hyperspectral data since it is relational to the spectra and energy reflected. Level sets use iterative solutions to find intersecting boundaries between features by optimizing a formulated level set equation. Using active contours, segmented bounded regions of interest can be brought to the forefront and presented as features in ML algorithms.

Features can be categorized as being primarily applicable to boundaries, regions, coded areas, spectral features or whole images. These are not mutually exclusive and can be used as feature map sections for convolutional neural networks. Hyperspectral signatures of various region features are often used to train neural networks and as inputs to MLA. Features in unsupervised learning environments are realized by the system.

*Useful Feature Extraction and Machine Learning Techniques for Identifying Unique Pattern… DOI: http://dx.doi.org/10.5772/intechopen.107436*

#### **Figure 8.**

*Machine learning algorithms: Categories of methods to be considered when applying ML to hyperspectral data enhancement of soil data intensity range.*

A branched diagram of popular MLAs is given by **Figure 8**. Supervised and unsupervised methods have their own distinct advantages and are dependent on the context of the application. This chapter reviews some useful methods in both categories and clarifies some of the subtle differences between these two types of algorithms. Other useful techniques for MLAs, such as fuzzy logic and quadratic nonlinear methods are depicted in the diagram of **Figure 8**. The reader is encouraged to explore these other methods and compare and contrast how these techniques can be used efficiently to enhance machine learning purposes.

Detecting plant diseases and stress factors in early stages of disease development is essential for selective and effective management of crop production. Laboratory analysis of plant samples for disease detection is time-consuming and labor-intensive. For that reason, several disease detection methods have been developed utilizing advanced and sophisticated hyperspectral data analysis approaches [3, 8, 9] and MLAs. These unique applications will be reviewed.

The rest of the chapter is divided into sections for:


#### **2. Preprocessing methods for hyperspectral data analysis**

Preprocessing of hyperspectral data involves numerical and statistical methods to filter noise and vibration more generally, as well as radiative transfer and empirical models used for airborne applications [19]. Some preprocessing methods include normalization of data, data smoothing, intensity transformation, histogram matching and histogram equalization, adaptive histogram equalization, correlation and convolution using spectral or spatial filters, and the use of fuzzy sets. This section reviews the most common and useful preprocessing methods for general purpose applications in hyperspectral data processing.

#### **2.1 Normalization of hyperspectral data**

Normalizing data taken from using similar sensors and similar methods is a common practice for purposes of understanding and clarifying data. This ensures the integrity of data presented for analysis, prediction and classification. It also provides some smoothing of the data using a standard normal variate transformation. The Standard Normal Variate (SNV) transformation is counter the effects of skewness of the data related to the reflectance spectra. The SNV was found to reduce error in the approximation which could be due to interferences caused by scattering and particle size differences. The probability distribution function that can be used for this standard normal variance transformation is given by

$$f(\mathbf{x}) = \frac{1}{\sigma\sqrt{2\pi}}e^{-\zeta} \tag{1}$$

Where

$$
\zeta = \frac{(\mathbf{x} - \boldsymbol{\mu})^2}{2\sigma^2}
$$

represents the SNV with mean, μ = 0, and standard deviation, σ = 1.

Other distributions can be used for normalizing the data, if the camera manufacturer recommends or if there are other anomalies in the data that need to be taken into consideration. For example, if the data is non-Gaussian, the Z-score standardization can be used to force the data into standard normal distribution. Z-score standardization involves using the data mean and standard deviation to adjust all level points in the data such that

$$\varkappa\_{ii} = \frac{\varkappa\_i - \mu}{\sigma} \tag{2}$$

Then the*xii* values can be used as normally distributed data.

#### **2.2 Hyperspectral data smoothing**

After transforming the data into the SNV domain, the next steps might include resolving the data by curve fitting to smooth the spectral data enough so as to be able *Useful Feature Extraction and Machine Learning Techniques for Identifying Unique Pattern… DOI: http://dx.doi.org/10.5772/intechopen.107436*

to reliably calculate the finite difference approximations or other numerical analysis of the data. A convolution method, such as the Savitzky–Golay Filter (SGF, [20]) can be used to approximate the spectra of the data.

For each pixel location, *i*, of the data, a signal spectrum of the data can be smoothed by:

$$\mathcal{S}\_i = \sum\_N c\_N f\_i + n\_C \tag{3}$$

Where.

*Si* = smoothed pixel value per wavelength.

N = pixel neighborhood.

*cN* ¼ Coefficient of curve fit*:*

*fi* ¼ pixel value at each wavelength

*nC* ¼ center pixel value per wavelength

Determining the coefficients can be done by a box filter, applying Gaussian smoothing for the data set, or other filtering methods. For the SGF, the data is point transformed by polynomial approximation and the coefficients are found by a least square fit.

SGF takes into consideration the order (*n*) of the polynomial to which the data is being fitted, and the size of the window (*m*) inscribing the real data points which are being incorporated for the smoothing at each data point.

The SGF conversion process at the pixel level can be described by:

$$y(i) = \sum\_{n=0}^{k} a\_n i^n |i| \le m \tag{4}$$

Where

y(i) = transformed output of Savitzky–Golay filter

n = term number of polynomials

*an* ¼ coefficients of polynomial

*k* ¼order of polynomial fit

To obtain the values of the *n* þ 1 coefficients, a least squares criterion is used for solving Eq. (4). By taking the partial derivative of the wrt *an* and setting the result equal to zero to minimize the error:

$$\frac{\partial}{\partial a\_n} \left[ \sum\_{i=-m}^{m} \left( y(i) - s\_i \right)^2 \right] = \mathbf{0} \tag{5}$$

Using Eq. (4) at i = 0 the first term can be found. Then using back-substitution and the criterion Eq. (5), the higher coefficients can be found

An example (**Figure 9**) shows the smoothing effect of the SGF applied to a sample hyperspectral signal in real-time data.

#### **2.3 Intensity transformations**

Intensity transformations can be used for data that is skewed because of background or foreground attenuations or ambient light interference. Stretching or

**Figure 9.** *Filter smoothing of hyperspectral Spectrum via Savitzky-Golay filter.*

compressing the value range of pixels that are converted to data values can be done with point transformation functions. The curves used for transformation can be gamma, exponential, power law, adaptive, piecewise linear, etc. The domain transform technique uses an operator for each pixel location:

$$f(\mathbf{x}, \mathbf{y}) = \mathbf{T}[I(\mathbf{x}, \mathbf{y})] \tag{6}$$

$$\mathfrak{s} = \mathbf{T}(r) \tag{7}$$

Where

*<sup>s</sup>* <sup>¼</sup> Intensity of f x, y

*r* ¼ Intensity of Image data

These types of image transformations are sometimes called "mappings" since they use a point to point mapping of the data to express hidden quality features that misrepresented the original data. Common transforms are the gamma, power, logarithmic, contrast stretching, and exponential mapping.

#### **2.4 Histogram equalization**

For hyperspectral images that are digitized by special mapping of the spectral to intensity domains, histogram matching functions can be applied to obtain image data *Useful Feature Extraction and Machine Learning Techniques for Identifying Unique Pattern… DOI: http://dx.doi.org/10.5772/intechopen.107436*

that is uniformly distributed on an interval [0,I]. A histogram of image data is defined by Eq. (8).

$$h(r\_k) = n\_k \tag{8}$$

where

*rk* ¼ *the kth intensity represented by the mapping nk* ¼ #*of pixels in image whose intensity is rk* For an image that is M x N pixels:

$$\sum\_{k=0}^{I} h(r\_k) = \text{MN} \tag{9}$$

We can then obtain an expression for the probability of the occurrence of a pixel of intensity level *rk* by dividing the histogram by the number of pixels in the image:

$$p(r\_k) = \frac{h(r\_k)}{\text{MN}} = \frac{n\_k}{\text{MN}} \tag{10}$$

Then the sum of (10), which is one.

$$\sum\_{k=0}^{I} p(r\_k) = 1\tag{11}$$

The cumulative distribution function can be found for an intensity value, *rk*, as:

$$\mathbb{C}(r\_k) = \sum\_{i=0}^{k} p(r\_i) \tag{12}$$

By using the transformation expression given in Eq. (7) to remap the intensity value of a pixel of *ri* intensity to *si* intensity by a scalar:

$$s\_k = T(r\_k) = \kappa \sum\_{i=0}^k p(r\_i) \tag{13}$$

Where *κ* represents the maximum range of intensities (for integer values, Max (range)-1).

An example of the usefulness of hyperspectral data histogram equalization is shown in **Figure 10**. A hyperspectral landscape image of a section of the bison basin is shown in **Figure 10**(left). The image shows the hyperspectral mapped data before histogram equalization and afterwards (**Figure 10**-right). The spread of the image forested area is washed out since the degree of green is saturated by the equalization. However, the biocrust map is more enhanced by the allowance of the greater spread in the lower and higher spectra pixel intensity region (refer to **Figure 11**).

**Figure 10.**

*Biocrust data (left) before histogram equalization; (right) after histogram equalization; note enhancement of soil.*

**Figure 11.**

*Biocrust data (left) before histogram equalization; (right) after histogram equalization; note the broadening of the intensity range after equalization.*

#### **3. Feature extraction techniques for supervised machine learning**

In order to find regions of interests or embedded patterns in the data, feature extraction methods for hyperspectral data is used to reduce the learning time and amount of data necessary for MLAs. When there is a priori knowledge in data, it is useful to extract this information so that the MLA used for pattern recognition is built on worthwhile information. In ordinary image data, finding lines, edges and corners is usually an advantageous effort since locating areas with sharp transitions is quite often associated with a pattern feature descriptor. With hyperspectral data, a more common application designed by Lowe [17] is scale-invariant feature transforms (SIFT).

*Useful Feature Extraction and Machine Learning Techniques for Identifying Unique Pattern… DOI: http://dx.doi.org/10.5772/intechopen.107436*

There have been many adaptions of this SIFT transform over the years, but the robustness of this algorithm to find patterns in embedded frame data, real-time advancing frame data (such as moving target or moving platform), and for maximally stable extremal regions has become the supreme standard method used for keypoints feature extraction.

The first item that this transform addresses is scale invariance. By applying Gaussian filters to a stack of image(s) and increasing the smoothing by*k<sup>n</sup> σ* for each image that is stacked up to an octave (n = 4), these stacks of images are transformed by

$$
\sigma = \sigma\_1$$

$$
\sigma\_2 = 2\sigma\_1$$

$$
\sigma\_n = 2\sigma\_{n-1} \tag{14}$$

To find the keypoints, a difference operation is performed as given by Eq. (15):

$$\mathcal{L}(\mathbf{x}, \mathbf{y}, \sigma) = [\mathcal{g}(\mathbf{x}, \mathbf{y}, k\sigma) - \mathcal{g}(\mathbf{x}, \mathbf{y}, k\sigma)] \mathfrak{sl}(\mathbf{x}, \mathbf{y}) \tag{15}$$

Where

Lð Þ¼ *x*, *y*, *σ Laplacian operator ℊ*ð Þ¼ *x*, *y*, *σ Gaussian transformed images* I(x,y) = original image data

After this Laplacian of Gaussian operation is performed on the octave stacked smoothed image data, the extrema regions of the data begin to emerge. This scaleinvariant region become the keypoint features of the data.

To build more robustness into this algorithm, invariance to rotation and other affine transformations are accounted for by applying orientation invariant gradient directional operators at the keypoints extracted from Eq. (13). These operators are 4<sup>2</sup> directional histogram matrices where each rotational element is 22.5 degrees differenced and weighted about bins that are multiples of 45o . After correlating this directional filter at the keypoints, keypoint descriptors are indicated and also used as feature directives for the keypoint features. With this collection of extrema data labeled as features, any number of machine learning methods can now be applied with the feature keypoints and descriptors provided as inputs.

The SIFT method was applied for soil biocrust data taken from US geological society biocrust data. Three band Electro-Optical (EO) imaging system - collected on June 2, 2018 using a Ricoh GR II camera (18.3 mm lens) mounted on 3DR Solo quadrotor aerial vehicle (9:45 AM MDT) were collected [21].

The data before and after the SIFT procedure was applied is given in **Figure 12**. After applying a fully connected neural network with two layers, backpropagation and labeling the data, the supervised MLA was able to locate nine distinct areas of the terrain, including the two areas of biocrust.

#### **4. A feature extraction technique for unsupervised machine learning**

While many variations for linear discriminant analysis (LDA) exist, the focus on K-means and K-medoids has gotten less attention. The main emphasis of these methods is to use clustering of data traits to classify features. Clustering algorithms are vital knowledge acquisition tools [22]. Numerical clustering algorithms generally use a

**Figure 12.** *Hyperspectral image data with features extracted by SIFT.*

Euclidean distance measure or geometric distance such as derived by cosine angle to classify new data that is provided to the system. K-medoids rather than calculate cluster centers by distance, it places data points or exemplars as data centroids and classifies by maximizing similarities (or by contrast minimizing dissimilarities) in data point features. The tested classification is given a goodness of fit parameter to test the choice for the number of clusters (mostly uses a "silhouette" function). If this figure of merit is best for a particular data set, the data is placed in the least dissimilar cluster until all data points are accounted for with the least change in the cost (or minimal dissimilarity). The algorithm consists of the following two layers – the "build" and the "swap"

In BUILD:


In SWAP: (while Cost is decreasing)

1.For each cluster, Swap the medoid, *m*, with data point, *d*


2. Swap *mmin* and *dmin* for each cluster until overall cost is minimized.

An application using k-medoids for selecting a set of data features from a hyperspectral image to be associated with decreased nitrogen (**Figure 13**) in an *Useful Feature Extraction and Machine Learning Techniques for Identifying Unique Pattern… DOI: http://dx.doi.org/10.5772/intechopen.107436*

**Figure 13.** *Healthy avocado (left); nitrogen deficient (right).*

avocado plant was used to observe the classification pattern that would occur. The cluster associated with nitrogen deficient plants form a specific spectral signature in the hyperspectral data cube. Using this algorithm provided nearly 97% accuracy for random selected leaves of this signature compared to healthy and less nitrogendeprived avocado plants as can be seen in **Figures 14**–**17**.

The model "build" mode is given by the first five iterations in **Figure 14** and shows how the model converges to the minimal cost analysis (minimum error hyper parameters). Swap mode continues in iterations 3–30. **Figure 15** shows the overall metrics for this algorithm. It took 166 K swap configurations for convergence. The distance metric used for identifying data point similarity was a Chebyshev distance metric. It also used a weighted function for decision by squared inverse. **Figure 16** shows the cluster classification plot of the signature data points. The validation confusion matrix of **Figure 17** resolves the classification accuracy [23].

**Figure 14.** *Model convergence.*

#### *Hyperspectral Imaging - A Perspective on Recent Advances and Applications*


**Figure 15.** *K-medoid model metrics.*

**Figure 16.** *K-medoid cluster plot.*

#### **5. Innovative methods for hyperspectral analysis in agriculture**

The method of finite differencing has been found to work well with hyperspectral data to find dissimilarities in first and second derivative metrics. Using this deterministic method, it is reasonable to find regions-of-interest where the derivative max-min and inflections differ between spectral signature data (**Figure 18**). These are key features that can be shown on a parallel coordinates plot. These 2-D patterns that emerge in high dimensional data can help discern features and provide useful predictors for classification purposes.

*Useful Feature Extraction and Machine Learning Techniques for Identifying Unique Pattern… DOI: http://dx.doi.org/10.5772/intechopen.107436*

**Figure 17.** *K-medoid validation confusion matrix.*

**Figure 18.** *Parallel coordinates plot – Shows areas of discrimination in finite differences.*

#### *Hyperspectral Imaging - A Perspective on Recent Advances and Applications*

**Figure 19.**

*Best practices for Hyperspectral Data Preprocessing & Feature Extraction Procedures for use in machine learning (ML).*

Other methods such as Karhounen-Loeve expansion of the data will provide discernment as to where the data has the highest variance [24]. If these points are used as input features to neural networks, supervised learning will enhance the prediction model convergence and accuracy.

#### **6. Best practices for working with hyperspectral data and machine learning**

When working with hyperspectral data for machine learning, minimizing the amount of data in the signature content is the first order of business. Doing so without losing important feature data is the goal of precision feature extraction techniques. Preprocessing of the data enhances the features to gain a clear understanding of pattern properties.

There are many areas of machine learning to explore (**Figure 8**), to discover the best solution for the context of the problem at hand. Incorporating several models and contrasting and comparing them will bring the best comprehension to what the data is revealing.

Decide what type of information is at the forefront of the problem presented and if unsupervised or supervised learning with feature extraction methods are appropriate.

Designing an accurate set of predictors, features, classifier methods and training data are the most important areas to consider when using machine learning with hyperspectral data. Determining which machine learning technique provides the most accurate solution for classifying data will help build a solution database that can be used for diagnostic purposes. Data fusion in the post processing area can be used with the classified features to acquire exclusive signatures. These unique pattern identifiers can then be stored in a database and used for identification and diagnostic purposes. A flowchart of best practices of data preprocessing and feature extraction procedures is given in **Figure 19**.

#### **7. Conclusions**

This chapter presents an overview of preprocessing and feature extraction methods that are useful when working with hyperspectral data. Examples are shown *Useful Feature Extraction and Machine Learning Techniques for Identifying Unique Pattern… DOI: http://dx.doi.org/10.5772/intechopen.107436*

for applications of these methods using supervised, unsupervised machine learning techniques and neural networks. Emphasis is placed on the context of the problem, development of accurate features and training sets, enhancement to features using weighting functions and decision parameters, and realizing reduced data signatures through preprocessing and feature map expansion.

A branched diagram of the various supervised and unsupervised methods that are popular in machine learning was given in **Figure 8**. This chapter provided a summary of selected techniques given in **Figure 8** as well as provided insights on preprocessing for enhanced machine learning success. Through correct use of feature extraction in building the training and test data sets, machine learning algorithms can provide more accurate results. Machine learning is usually part of an embedded systems as shown in **Figure 19**. This chapter has provided insights into the aspects of feature extraction for enhanced machine learning success, and has examined some of the best algorithms to produce reliable machine learning results for use in diagnostic databases, robotics, factory automation, and other applications where decision and classification are necessary processes.

#### **Author details**

Jeanette Hariharan<sup>1</sup> \*, Yiannis Ampatzidis<sup>2</sup> , Jaafar Abdulridha<sup>3</sup> and Ozgur Batuman<sup>4</sup>

1 Bioengineering Department, Florida Gulf Coast University, Fort Myers, FL, USA

2 Agricultural and Biological Engineering Department, Southwest Florida Research and Education Center, University of Florida, Immokalee, FL, USA

3 Bioproducts and Biosystems Engineering Department, Minneapolis, MN, USA

4 Plant Pathology Department, Southwest Florida Research and Education Center, University of Florida, Immokalee, FL, USA

\*Address all correspondence to: jhariharan@fgcu.edu

© 2022 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

### **References**

[1] Abdulridha J, Batuman O, Ampatzidis Y. UAV-based remote sensing technique to detect Citrus canker disease utilizing hyperspectral imaging and machine learning. Remote Sensing. 2019;**11**(11):1373

[2] Abdulridha J, Ehsani R, Ampatzidis Y, de Castro A. Evaluating the performance of spectral features and multivariate analysis tools to detect Laurel wilt disease and nutritional deficiency in avocado. Computers and Electronics in Agriculture. 2018;**155**: 203-211

[3] Abdulridha J, Ampatzidis Y, Qureshi J, Roberts P. Identification and classification of downy mildew development stages in watermelon utilizing aerial and ground remote sensing and machine learning. Frontiers in Plant Science. 2022;**13**: 791018. DOI: 10.3389/fpls.2022. 791018

[4] Ampatzidis Y, De Bellis L, Luvisi A. iPathology: Robotic applications and management of plants and plant diseases. Sustainability. 2017;**9**(6):1010. DOI: 10.3390/su9061010

[5] Ampatzidis Y, Partel V. UAV-based high throughput phenotyping in Citrus utilizing multispectral imaging and artificial intelligence. Remote Sensing. 2019;**11**(4):410. DOI: 10.3390/ rs11040410

[6] Ampatzidis Y, Partel V, Costa L. Agroview: Cloud-based applications to process, analyze and visualize uAVcollected data for precision agriculture applications utilizing artificial intelligence. Computers and Electronics in Agriculture. 2020;**174**(July): 105157. DOI: 10.1016/j.compag.2020. 105457

[7] Harakannanavar S, Rudagi J, Puranikmath V, Siddiqua A, Pramodhini R. Plant leaf disease detection using computer vision and machine learning algorithms. Global Transitions Proceedings. 2022;**3**: 305-310

[8] Hariharan J, Fuller J, Ampatzidis Y, Abdulridha J, Lerwill A. Finite difference analysis and bivariate correlation of hyperspectral data for detecting Laurel wilt disease and nutritional deficiency in avocado. Remote Sens. 2019;**11**(15):1748. DOI: 10.3390/rs11151748

[9] Paine E, Slonecker E, Simon N, Rosen B, Resmini R, Allen D. Optical characterization of two cyanobacteria genera, Aphanizomenon and Microcystis, with hyperspectralmicroscopy. Journal of Applied Remote Sensing. 2018;**12**(3)

[10] Wang S, Celebi M, Zhang Y, Yu X, Lu S, Yao X, et al. Advances in data Preprocesing for biomedical data fusion: An overview of the methods, challenges, and prospects. Information Fusion. 2021; **76**:376-421

[11] Kiran R, Khandelwal K. Complex step derivative approximation for numerical evaluation of tangent moduli. Computers & Structures. 2014;**140**:1-13

[12] Susic N, Zibrat U, Sirca S, Strajnar P, Razinger J, Knapic M, et al. Discrimination between abiotic and biotic drought stress in tomatoes using hyperspectral imaging. Sensors and Actuators B: Chemical. 2018;**273**:842-852

[13] Poudyal C, Costa L, Sandhu H, Ampatzidis Y, Odero DC, Arbelo OC, et al. Sugarcane yield prediction and genotype selection using UAV-based

*Useful Feature Extraction and Machine Learning Techniques for Identifying Unique Pattern… DOI: http://dx.doi.org/10.5772/intechopen.107436*

hyperspectral imaging and machine learning. Agronomy Journal. 2022:**114**: 2320–2333. DOI: 10.1002/agj2.21133

[14] Vijayakumar V, Ampatzidis Y, Costa L. Tree-level Citrus yield prediction utilizing ground and aerial machine vision and machine learning. Smart Agricultural Technology. 2022: 100077. DOI: 10.1016/j.atech.2022. 100077

[15] Liu H, Yu T, Hu B, Hou X, Qian B. UAV-borne hyperspectral imaging remote sensing system based on Acousto-optic tunable filter for water quality monitoring. Remote Sensing. 2021;**13**(20):4069. DOI: 10.3390/ rs13204069

[16] Scott N, Moore I. Nonnegative matrix factorization-based feature selection analysis for hyperspectral imagery of sediment-laden riverine flow. SPIE. 2018:1063114. DOI: 10.1117/ 12.2301273

[17] Lowe D. Distinctive image features from scale-invariance Keypoints. International Journal of Computer Vision. 2004;**60**(2):91–110. Corpus ID: 174065

[18] Kass M, Witkin A, Terzo Poulos D. Snakes: Active contour models. International Journal of Computer Vision. 1988:321-331

[19] Minu P, Lotliker A, Shaju S, Ashraf P, Kumar TS, Meenakumari B. Performance of operational satellite bio-optic algorithms in different water types in the southeastern Arabian Sea. Oceanologia. 2016;**58**: 317-313

[20] Jardim R, Morgado-Dias F. Savitzky-Golay filtering as image noise reduction with sharp color reset. Microprocessors and Microsystems. 2020;**74**:103006

[21] Savitzky A, Golay MJE. Smoothing and differentiation of data by simplified least squares procedures. Analytical Chemistry. 1964;**36**(8):1627-1639. DOI: 10.1021%2Fac60214a047

[22] Hartigan JA, Wong MA. Algorithm AS 136: A k-means clustering algorithm. Journal of the Royal Statistical Society, Series C. 1979;**28**(1):100-108 JSTOR 2346830

[23] Kaufman L, Rousseeuw PJ. Partitioning around Medoids (program PAM). In: Wiley Series in Probability and Statistics. Hoboken, NJ, USA: John Wiley & Sons, Inc.; 1990. pp. 68-125. DOI: 10.1002/9780470316801.ch2

[24] Fontanella L, Ippoliti L. Time series analysis: Methods and applications. Handbook of Statistics. 2012;**30**(17): 497-520

### Section 3
