Machine Learning and Vision

#### **Chapter 6**

## Challenges and Trends of Machine Learning in the Myoelectric Control System for Upper Limb Exoskeletons and Exosuits

*Jirui Fu, Zubadiah Al-Mashhadani, Keith Currier, Al-Muthanna Al-Ani and Joon-Hyuk Park*

#### **Abstract**

Myoelectric control systems as the emerging control strategies for upper limb wearable robots have shown their efficacy and applicability to effectively provide motion assistance and/or restore motor functions in people with impairment or disabilities, as well as augment physical performance in able-bodied individuals. In myoelectric control, electromyographic (EMG) signals from muscles are utilized, improving adaptability and human-robot interactions during various motion tasks. Machine learning has been widely applied in myoelectric control systems due to its advantages in detecting and classifying various human motions and motion intentions. This chapter illustrates the challenges and trends in recent machine learning algorithms implemented on myoelectric control systems designed for upper limb wearable robots, and highlights the key focus areas for future research directions. Different modalities of recent machine learning-based myoelectric control systems are described in detail, and their advantages and disadvantages are summarized. Furthermore, key design aspects and the type of experiments conducted to validate the efficacy of the proposed myoelectric controllers are explained. Finally, the challenges and limitations of current myoelectric control systems using machine learning algorithms are analyzed, from which future research directions are suggested.

**Keywords:** myoelectric control, upper limb exoskeleton, upper limb exosuit, pattern recognition, machine learning, reinforcement learning

#### **1. Introduction**

In the past few decades, the demand for upper limb exoskeletons and exosuits has grown substantially due to their promising applications across industry, medical and military sectors. The exoskeletons consists of rigid links and joints attached to the human body, whereas the exosuits use soft and flexible materials (such as fabric or soft polymer) to interact with the user's body [1]. The applications of exoskeletons and exosuits include: (i) power augmentation to enhance physical performance or the capabilities of able-boded individuals during strenuous physical tasks [2], and (ii) assisting individuals with disabilities in performing activities of daily living (ADLs) [3].

The exoskeletons and exosuits can be controlled by many different schemes, such as the kinematics control based on the inertia measurement unit sensor (IMU) or encoder [4, 5], the force control based on load cell or torque sensor [6, 7], and the myoelectric control based on the electromyographic sensor (EMG) [8]. Among these control schemes, the myoelectric control systems have gained increasing attention over recent years [9–11]. The myoelectric control systems of the upper limb exoskeletons and exosuits use surface electromyography (EMG) signals, the electric potentials directly measured from the skeletal muscle as input of the control system for exoskeletons and exosuits (**Figure 1**). The surface EMG signals are generated from the motor unit activation, controlled by the human brain, and regulated by the motor neurons in the spinal cord. The mechanism for generating surface EMG signals offers surface EMG signals to detect human movement intention [12]. The critical advantage of a myoelectric control system over other control systems is its timely detection of the user's motion intention leveraging electromechanical delay (EMD); the onset of motion can be detected about 50–100 ms earlier than the physical motion [13, 14]. Moreover, the exoskeletons and exosuits equip with myoelectric control systems have a more adaptive and intelligent interface with the users as the exoskeleton and exosuits can timely and proactively engage assistance through detecting the users' movement intention [15].

Myoelectric control systems for upper limb exoskeletons and exosuits initially used on-off/finite state control and proportional control, as described in Refs. [16, 17]. Although these methods are simple and easy to implement, their ability to accommodate a wide range of different movements is limited, as noted in Ref. [18]. Consequently, their primary use have been limited to a single joint function such as elbow flexion/extension or hand grip. To allow for more complex movements across multiple degrees of freedom (DOFs), machine learning (ML) and deep learning (DL) algorithms have been utilized in the myoelectric control systems. However, the myoelectric control systems with ML or DL algorithms generally require considerable computational power, which imposes practical limitations on the portability of exoskeletons and exosuits [19]. In recent years, with the advancements in more powerful and compact embedded computers, myoelectric control systems with ML or DL algorithms became feasible to implement on upper limb exoskeletons and exosuits. Compared to the early staged myoelectric control modalities, the ML or DL-based myoelectric control systems have shown superior performance and better results in complex, multi-DOF upper limb motions; yet, there still exist challenges and limitations which will be discussed in detail in the subsequent sections.

Given the growing interest in machine learning and deep learning-based myoelectric control systems for upper limb exoskeletons and exosuits, the number of publications in the relevant field has rapidly increased over the past decade. Therefore, it is imperative to understand the latest trend and challenges in machine learning and deep learning-based myoelectric control system for upper limb exoskeletons and exosutis. A systematic review that provides a comprehensive overview of the myoelectric control system for upper limb exoskeletons and exosuits [8] was published by the authors. However, the focus of that review was not specifically machine learning and deep learning-based myoelectric control system of upper limb exoskeletons and exosuits, and it does not discuss current challenges and future directions. This chapter *Challenges and Trends of Machine Learning in the Myoelectric Control System for Upper… DOI: http://dx.doi.org/10.5772/intechopen.111901*

**Figure 1.**

*The generation of electromyography signal and the workflow of myoelectric control systems on upper limb exoskeletons and exosuits.*

is designed to share the extensive review of the machine learning-based myoelectric control system for upper limb exoskeletons and exosuits, particularly from scientific articles published between 2011 and 2023. The identified challenges in implementing machine learning algorithms in the myoelectric control system and future directions are suggested. In the following section, the process of machine learning-based myoelectric control system is summarized (Section 2), and the state-of-the-art implementation of machine learning algorithms in upper limb exoskeletons and exosuits is presented (Section 3). Finally, the remaining unaddressed research questions and tasks are discussed as future research directions (Section 4).

#### **2. The procedure of machine learning-based myoelectric control system**

Similar to any other types of myoelectric control systems, machine learning-based myoelectric control systems include key procedural steps: signal acquisition, preprocessing, feature extraction, and motion intention detection through the trained machine learning model (**Figure 2**). In this section, the process of a machine learningbased myoelectric control system will be presented in detail.

#### **2.1 Data acquisition and signal processing**

The acquisition of the EMG signal is critical to the myoelectric control system of the upper limb exoskeleton, as the accuracy of the myoelectric controller primarily depends on the quality of the EMG signal. It is, therefore, important to acquire quality and accurate EMG signals. Three essential components of EMG data acquisition systems are the electrodes for EMG, the sampling rate and signal filtering.

**Figure 2.** *The process of machine learning based myoelectric control system.*


*Challenges and Trends of Machine Learning in the Myoelectric Control System for Upper… DOI: http://dx.doi.org/10.5772/intechopen.111901*

signal, then using a low-pass Butterworth filter to acquire an envelope indicating the magnitude of the surface EMG signal as it changes over time. However, the selection of the order and cut-off frequency of the Butterworth filter could be optimized for different muscle segments; for example, [24] presents the filter selection for surface EMG signal to remove the drift and artifacts. After high and low pass (or band pass) filters, the EMG signals need full wave rectification then a low pass filter for further processing and feature extraction.

#### **2.2 Feature extraction**

The pre-processed surface EMG signal is presented as a time sequence that includes a large number of randomness. Therefore, directly feeding the pre-processed surface EMG signal to the machine learning model is impractical. To feed the preprocessed surface EMG signal to the machine learning model, the sequence of preprocessed surface EMG signals must be mapped into a smaller dimension vector called a feature vector [25]. The process of extracting feature vectors from the pre-processed surface EMG signal is called feature extraction. In applying a myoelectric control system for upper limb exoskeletons and exosuits, feature extraction includes two types of methods: feature selection and dimensionality reduction algorithms.


#### **3. Taxonomy of machine learning-based myoelectric control systems**

The previous section presents an overview of the process of machine learningbased myoelectric control system. This section will summarize the machine learning algorithms used in the reviewed research articles. These algorithms are categorized into three: (1) classification-based models, which are used to detect the type of movement from the surface EMG features, (2) regression-based models, which can make a continuous prediction of the human subject's joint kinematics or torques, and (3) the reinforcement learning models which optimize the model through the interaction of human subjects and machine learning model. As shown in **Figure 3**, among the included research articles, the regression-based models and classification models are the most widely used modalities in the machine learning-based myoelectric control systems, while not much work has been done in the implementation and validation of reinforcement learning models.

#### **3.1 Classification-based myoelectric control system**

The classification-based Myoelectric Control System uses the classification model to detect the movement from the statistical features of the human subject's surface EMG signal. In the classification-based myoelectric control, the labels are pre-defined by the human subjects, which includes types of upper limb movement such as elbow flexion/extension (diversified labels), and the onset of upper limb movement such as in motion or still (binary labels). To train the classification models, surface EMG data corresponding to the labeled motion must be collected from human subject. Then, the classification model can be trained by various machine learning algorithms such as the Support Vector Machine (SVM) [28], Linear Discriminant Analysis (LDA) [29], Knearest neighbors (KNN) [30], etc. According to our literature review, the support vector machine algorithm is the most popular choice in the classification models of machine learning-based myoelectric control systems. Compared to other machine learning algorithms, the support vector machine algorithm provides better computational efficiency that makes it feasible to run on embedded computers than other types of machine learning algorithms. The support vector machine algorithm can train classification models with either diversified or binary labels. For example, [31] trained

#### *Challenges and Trends of Machine Learning in the Myoelectric Control System for Upper… DOI: http://dx.doi.org/10.5772/intechopen.111901*

a SVM model to classify if the human subject's finger is in motion or not. In [31], a classification-based myoelectric control system trained by the SVM algorithm was proposed for a (#DOF) finger exoskeleton using binary labels. Additionally, [32] compared the accuracy of the classification-based myoelectric control systems for a hand exoskeleton trained by different machine learning algorithms, i.e., SVM, artificial neural network with backpropagation algorithm (ANN), and K-nearest neighbors (KNN). The classification model in [32] includes five labels that correspond to five different types of hand motion, then used the classification output to trigger the predefined assistive mode in the hand exoskeleton. According to [32], the classification model trained by SVM showed the best accuracy among those compared. Moreover, Cheon et al. proposed a myoelectric interface based on the musculotendinous junctions (MTJs) of the flexor digitorum superficialis (FDS) for reliable control of a robotic glove with a single EMG sensor by identifying power grasp intentions [33] and the support vector machine (SVM) algorithm was used to optimize the classification model. Other machine learning algorithms have also been utilized to train the classification model in the machine learning-based myoelectric control systems. For example, [34] utilized the MCLPBoost – a type of decision-tree algorithm to classify the flexion and extension of elbow and wrist joints. Compared to the SVM algorithm, they showed that the MCLPBoost had better robustness against the noised training data.

Many research articles reviewed targeted to improve the performance of classification models. For instance, [16, 35] studied the impact of feature extraction on the accuracy of classification model where two types of feature extraction techniques were explored. The type 1 feature extraction technique converted the single-channel EMG signal to 14 different statistical features; the type 2 feature extraction technique converted five channel EMG signal to a single statistical feature. Both type 1 and 2 feature extraction techniques were designed for the same upper limb exoskeleton and the classification models were trained by the same machine learning algorithm. The experimental result indicated the type 1 feature extraction technique outperformed the type 2 feature extraction from which they suggested that when training the classification-based myoelectric control systems, higher dimensional training set gives better performance. Moreover, [36, 37] implemented the sensor fusion method by combining the EMG and electroencephalography (EEG) signal to improve the accuracy of the classification-based myoelectric control system trained by the artificial neural network with a backpropagation algorithm. Additionally, to prevent the misclassification caused by the unfiltered noise in EMG signals such as crosstalk and motion artifacts, [38] utilized a threshold method in which the amplitude of filtered EMG signal must be greater than a specific value to be an input to the classification model. Twardowski et al. used the machine learning algorithm to convert the motor unit firings from the sEMG signals into biomechanically informed signals that drive the actuation [39]. The resulting signal provides a smoother control scheme with less delay versus using the MAV and RMS response to modulate the actuation. The EMG signal in the study [31, 32, 34, 39] used the root mean square (RMS) as statistical features, while [40] used integrated EMG (iEMG) to train the classification model. Compared to the RMS feature, the iEMG feature requires less computational power. The classification model presented in [40] plotted the output data onto a 2D Cartesian plane that can be distinguished in real-time using a Point-in Polygon algorithm commonly used in computer graphics. This algorithm determines whether the sample in the plane belongs in or out of a given polygonal area which is the area of each given label. Among the tested classifiers, this method provided the highest classification accuracy (94%) when classifying hand grasp motions.

The abovementioned articles utilized the statistical features of EMG data as input for the classification-based myoelectric control systems. However, the raw EMG signal can also be used as input for the classification-based myoelectric control system, as demonstrated by [41], which successfully implemented a vision transformer model to classify two datasets using raw multichannel EMG data. The transformer model is commonly used in natural language processing, but the encoder-decoder network can be applied to determine the underlying characteristics of the input data without manual feature extraction or signal pre-processing. The resulting model achieved a higher classification accuracy versus a convolution neural network model and an LSTM network.

#### **3.2 Regression-based myoelectric control system**

The regression-based myoelectric control system implements regression analysis techniques. In statistics, regression analysis estimates the relationship between a dependent variable (output of regression-based myoelectric control system) and an independent variable (usually the EMG features in the regression-based myoelectric control system) by using a regression model. Compared to the classification-based myoelectric control system, the regression model can output continuous variables such as joint torque and joint angle. The regression model can be trained by various machine learning algorithms. However, there are two regression models found in our literature review, artificial neural network with backpropagation algorithm and Kalman Filters.

Among the research articles reviewed, the artificial neural network was the most widely used method to train the regression model. For example, [42] implemented a regression model to estimate the joint angle from the statistical feature of the human subject's EMG signal. The regression model is trained by artificial neural network with a back propagation algorithm, and the results showed that the regression model could accurately estimate the joint angle of human. Additionally, the regression-based myoelectric control systems have also been widely used in the bilateral training of hand exoskeletons. Because the bilateral training focuses on using the unimpaired hand to help the impaired hand restore its motor control capability, the myoelectric control scheme must accurately estimate the joint kinematics or joint torque of the unimpaired hand which complies with the characteristics of regression-based myoelectric control systems. For example, [38, 43–48] implemented the regression model to estimate the joint angle or joint torque from the unimpaired hand to help the impaired hand to restore its motor control capability. On the other hand, Kalman Filter is another approach used in the regression model for myoelectric control of upper limb wearable robots. Compared to the artificial neural network with backpropagation method, Kalman filter does not need much time and extensive datasets to train the model. Moreover, tuning Kalman filter requires less computational power than tuning the artificial neural network which makes it easier to run on an embedded computer. The studies [49, 50] utilized the Kalman filter to compute the joint torque based on the EMG signal whose regression models offered better accuracy when compared to the regression model trained by artificial neural network with backpropagation algorithm. Another method proposed by Kopke et al. used 6 DOF loadcells and EMG sensors to acquire the training data and the linear discriminate analysis (LDA) algorithm to train the regression model [51]. The experiment demonstrated a 92% accuracy in estimating the joint torque of human subjects'shoulder and elbow joint.

*Challenges and Trends of Machine Learning in the Myoelectric Control System for Upper… DOI: http://dx.doi.org/10.5772/intechopen.111901*

Furthermore, some studies focused on improving the accuracy of regression models. For example, Sierotowice et al. [52] utilized a ridge regression algorithm and a feature selection algorithm called Random Fourier Features to improve the accuracy of the regression model to estimate the hand-grasping force. The regression algorithm of the controller achieved a higher classification accuracy when determining the target forces versus the random Fourier features algorithm (80% versus 73%, respectively). Moreover, the work by Meattini et al. used a soft dynamic time warping (soft-DTW) method to improve the accuracy of the neural network based regression model [53] and the result of this study shows comparable performance to the conventional neural network regression model.

#### **3.3 Reinforcement learning based myoelectric control system**

The reinforcement learning algorithm is another type of machine learning algorithm which are used as a machine learning based myoelectric control system. Different from the classification and regression models, the reinforcement learning model trains an agent to choose the optimal action under a specific state in an environment. The process of reinforcement learning can be divided into several steps; in each step, the smart agent executes an action based on a specific state and receive a reward signal as feedback. The objective of the smart agent is to find the optimal action to maximize the accumulative reward.

Compared to the other two types of machine learning myoelectric control systems, only a few included research literature implemented the reinforcement learning algorithm. Hamaya et al. [54] utilized an elbow exoskeleton and applied the Probabilistic Inference for Learning Control (PILCO) reinforcement learning algorithm. The state vector included elbow joint kinematics and EMG signals, and the reward was based on the deviation between the intended and actual trajectory. PILCO employed the Gaussian process to learn the probabilistic dynamic model of the interface between the human and the exoskeleton. The learned model was then used to assess the control policy, which was optimized using the policy gradient method [55]. This approach proved to be more efficient than other machine learning myoelectric control systems, leading to a shorter training period.

#### **4. Discussion**

This section outlines several research questions and tasks that need to be addressed in future studies, including the robustness of machine learning-based myoelectric control system, the incorporation of safety requirements in machine learning-based myoelectric control systems, and the clinical assessment of assistive and rehabilitative upper limb exoskeletons and exosuits with machine learning-based myoelectric control systems. These research questions point out crucial barriers to the effective use of machine learning-based myoelectric control systems in upper limb exoskeletons and exosuits which warrant further investigations.

#### **4.1 Robustness of machine learning-based myoelectric control systems**

The myoelectric control system's ability to withstand disturbance from both internal and external sources within the environment, as measured by its resistance to electromyography signals [56], is referred to as its robustness. This type of

disturbance is typically caused by muscle fatigue [57], electrode displacement [58], and changes in EMG patterns over time [59]. Over the past decade, there has been a significant increase in studies employing machine learning-based myoelectric control systems, which have shown promising results in preliminary or pilot testing in laboratory settings. However, none of these systems have explored methods to enhance their robustness. To bridge the gap between experimental research and commercial or clinical applications, machine learning-based myoelectric control systems should concentrate on creating a precise control scheme under well-controlled laboratory conditions while also improving robustness in real-world scenarios.

The review of research articles that utilized machine learning-based myoelectric control systems found that these systems face common issues, such as varying characteristics of sEMG signals in different physiological conditions, noise/artifacts, muscle fatigue that causes variance in sEMG signals, and electrode shift during or between sessions. However, none of the studies focused mainly on addressing these issues. Existing studies have investigated these issues in the context of myoelectric control of prosthetics, teleoperate robotic arms, and pattern recognition of sEMG signals. Potential approaches to improve the robustness of machine learning-based myoelectric control systems include using more efficient features, reducing the impact of EMG electrode shift, and improving the data collection protocol or signal processing method. However, these methods have not been studied in the included research articles. Therefore, further investigations are needed to evaluate the performance of machine learning-based myoelectric control systems with these robustness-improving methods and their performance on the upper limb exoskeleton.

In future studies, it is suggested to investigate the performance of upper limb exoskeletons with machine learning-based myoelectric control systems using different time-domain and frequency-domain features. The selection of EMG features should be expanded to account for larger time-domain and frequency-domain features, and the performance of the human-exoskeleton system with the improved myoelectric control system should be evaluated. Additionally, during laboratory research, the causes of error, such as EMG electrode shift and muscle fatigue that could affect the robustness of machine learning-based myoelectric control systems in clinical applications, should be emulated. Novel training protocols should also be investigated because using the EMG signal collected within a short period to train the machine learning-based myoelectric control system will affect its robustness. Therefore, future studies of machine learning myoelectric control systems of upper limb exoskeletons should focus on developing novel control schemes, investigating effective training protocols, and evaluating them on the upper limb exoskeletons. In the research articles reviewed, there were several common issues that were reported. These issues included differences in the characteristics of EMG signals across various physiological conditions, the presence of noise and artifacts, muscle fatigue leading to variations in EMG signals, and electrode movement during or between sessions. However, none of the included research articles specifically addressed these issues by focusing on improving the robustness of machine learning-based myoelectric control systems.

Furthermore, one of our studies explored the implementation of a variational autoencoder to improve the robustness of the classification model in using the EMG signal to recognize the motion performed by the human subject. An autoencoder is a neural network model that is trained to compress and uncompress inputted data while reducing the error between the input data and the reconstructed output data as much as possible [60]. The restrictive architecture of the autoencoder creates a model that

#### *Challenges and Trends of Machine Learning in the Myoelectric Control System for Upper… DOI: http://dx.doi.org/10.5772/intechopen.111901*

can act as a dimensionality reduction method to perform unsupervised feature learning. Implementing autoencoder networks or more advanced encoder-decoder networks can further reduce the complexity of input myoelectric signal data or multimodal sensor data at the compressed latent layer while learning the hidden characteristics that define the system. Autoencoder networks can effectively denoise incoming EMG signal data [61], and the encoder-decoder model framework can be reused using a transfer learning-based model approach [56]. Once the model is trained offline using collected experimental data, the myoelectric control scheme can be readily implemented with little calibration time for the end user. Autoencoder models have already been implemented in research to improve the pattern recognition of myoelectric control schemes in the presence of electrode shift [62], but more research is needed to test the viability of using encoder-decoder networks in myoelectric control schemes.

#### **4.2 Safety requirements in machine learning based myoelectric control systems**

The active and powered upper limb exoskeletons and exosuits require high levels of safety to ensure that they do not pose any risks to human users for assistive or rehabilitative purposes. Previous research has primarily focused on incorporating safety measures in the mechanical design of exoskeletons by implementing mechanical stops, rotation limits, and force limits to prevent any excessive range of motion or force from being applied to the user [3]. However, these mechanisms may not always guarantee the user's safety when there are unknown parameter variances, hardware failures, or actuator malfunctions [63]. Therefore, control strategies that can compensate for various uncertainties and external load disturbances may significantly enhance user safety when wearing the robotic exoskeleton during tasks and movements. According to state-of-the-art research articles, one potential approach to improve safety is to apply data fusion techniques to EMG signals, considering their inherent variability arising from changes in arm posture, electrode repositing, fatigue, etc. [64]. By fusing EMG data, potential errors in motion estimation can be minimized. In [64], two data-fusing algorithms, Variance Weighted Average (VWA) and Decentralized Kalman Filter, were presented as potential methods to improve safety in robotic exoskeletons.

Additionally, other works also utilize the deep reinforcement learning-controlled neuromusculoskeletal simulator (NMMS) to validate the machine learning-based myoelectric control system. The neuromusculoskeletal simulation modulates a wide range of control schemes and parameters to test the efficacy and performance of different control methods while observing model outcomes, such as muscle force, joint kinematics and power using EMG signals [65]. Compared to the conventional control schemes of the NMMS which are broadly classified as forward-type or inversetype. The deep reinforcement learning (DRL) based NMMS controller learns the neuromusculoskeletal system dynamics by interacting with its environment without the experimental data collected from big samples with varying anthropometrics and biomechanics characteristics. For example, [66] implemented a lower limb NMMS with DRL-based locomotion controller to validate a reinforcement learning-based myoelectric control system for a lower limb orthosis. In this work, a deep reinforcement algorithm called Soft-Actor-Critic (SAC) was used to learn the dynamics of the lower-limb NMMS and served as its locomotion controller; meanwhile, a myoelectric control scheme was trained by imitation learning through interacting with the

lower-limb NMMS. Compared to validating the novel myoelectric control scheme on the human body, using the RL-based NMMS can guarantee safety while maximumly emulating the feedback from real human. However, the abovementioned research only deals with the lower-limb neuromusculoskeletal simulator, while the reinforcement learning-based neuromusculoskeletal simulator for upper limb is still extensively unexplored. To address this gap, one of our recent studies utilized the MyoSuite – a Mujoco-based neuromusculoskeletal simulation kit [67] – to simulate the flexion/ extension of human's elbow joint controlled by the deep deterministic policy gradient algorithms (DDPG) – a variant of deep reinforcement learning algorithms [68]. In that work, we compared the performance between two types of action spaces – the PD-based internal model of the central neuron system, and the direct muscular activation output. The result indicated the PD-based internal model has better learning performance than the direct muscular activation output. Additionally, we also simulated the proportional myoelectric control [17] in the NMMS to validate its feasibility in validating the myoelectric control system. However, the result of muscle activation is different from the result in [17]. Therefore, further studies should focus on making the NMMS become more human-like.

#### **4.3 Implementation of reinforcement learning algorithms**

Only 3 percent of the research articles from our survey utilized the reinforcement learning algorithms for the machine learning-based myoelectric control system (**Figure 3**). However, as a branch of machine learning, reinforcement learning has some exclusive advantages if implemented in a control system. For example, reinforcement learning can inherently reflect how humans learn a skill in the real world, which is actively exploring the unknown environment and finding the long-term optimal solutions [69]. More importantly, the reinforcement learning algorithm can learn the optimal solution without the predefined knowledge about the dynamics of environment [70]. Due to these advantages, an increasing number of biomechanical studies implemented reinforcement learning, such as using reinforcement learning to control a lower-limb musculoskeletal model for obstacle avoidance [71], to control a functional electrical stimulation to assist movement [72], and to control an upper limb prothesis [73]. One literature implemented Probabilistic Inference for Learning Control Algorithm (PILCO) – a reinforcement learning algorithm to train a myoelectric control system – on an elbow exoskeleton and achieved a satisfactory result [54]. However, the PILCO algorithm depends on the dynamic model of the environment which increases the difficulty of computation and restricts the usage of trained myoelectric controllers in a single task. Different from [54], the reinforcement learning algorithm used in [72, 73] does not require the dynamic model of the environment, which called model-free reinforcement learning algorithms. The model-free reinforcement learning algorithms have the advantage of less computational difficulty, and wider applicability. There are many model-free reinforcement learning algorithms, for example, the Asynchronous Advantage Actor-Critic (A3C) algorithm [74] which was used to estimate the elbow joint torque from the surface EMG signal [75]. To explore the application of reinforcement learning algorithms in myoelectric control systems, further research is needed to validate these reinforcement learning algorithms (e.g., deep deterministic policy gradient (DDPG), proximal policy optimization (PPO), and asynchronous advantage actor critic (A3C)) used in the field of myoelectric control systems for upper limb wearable robots.

*Challenges and Trends of Machine Learning in the Myoelectric Control System for Upper… DOI: http://dx.doi.org/10.5772/intechopen.111901*

### **5. Conclusion**

This chapter shares a review of the recent implementation of machine learning algorithms in the myoelectric control systems for upper limb exoskeletons and exosuits. The types of machine learning algorithms used in the myoelectric control systems include classification model, regression model and reinforcement learning model. Also, this chapter provides information on the methods, performance, and limitations of each myoelectric control modality. The machine learning algorithms in the myoelectric control systems have shown promising outcomes, including improved human-robot interactions, robot intelligence, and adaptiveness to the user, task and environment compared to traditional myoelectric control systems that did not use learning-based controls. Several challenges and limitations are identified which need to be addressed in future studies related to machine learning -based myoelectric control system for upper limb exoskeletons and exosuits, particularly in narrowing the gap between laboratory studies and clinical applications.

### **Acknowledgements**

Article processing charges were provided in part by the UCF College of Graduate Studies Open Access Publishing Fund.

### **Author details**

Jirui Fu1 , Zubadiah Al-Mashhadani<sup>2</sup> , Keith Currier<sup>1</sup> , Al-Muthanna Al-Ani1 and Joon-Hyuk Park1 \*

1 Department of Aerospace and Mechanical Engineering, University of Central Florida, Orlando FL, USA

2 Department of Electrical and Computer Engineering, University of Central Florida, Orlando FL, USA

\*Address all correspondence to: joonpark@ucf.edu

© 2023 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

### **References**

[1] Bardi E, Gandolla M, Braghin F, Resta F, Pedrocchi A, Ambrosini E. Upper limb soft robotic wearable devices: A systematic review. Journal of NeuroEngineering And Rehabilitation. 2022;**19**:87

[2] Fu J, Hosseini S, Simpson R, Brooks A, Huff R, Park J. A bilateral six degree of freedom cable-driven upper body exosuit. In: 2022 IEEE International Conference on Mechatronics and Automation (ICMA). Guilin, Guangxi, China: IEEE; 2022. pp. 617-623

[3] Gull M, Bai S, Bak T. A review on design of upper limb exoskeletons. Robotics. 2020;**9**:16

[4] Ding S, Ouyang X, Liu T, Li Z, Yang H. Gait event detection of a lower extremity exoskeleton robot by an intelligent IMU. IEEE Sensors Journal. 2018;**18**:9728-9735

[5] Wang S, Wang L, Meijneke C, Van Asseldonk E, Hoellinger T, Cheron G, et al. Design and control of the MINDWALKER exoskeleton. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2014;**23**: 277-286

[6] Massardi S, Rodriguez-Cianca D, Pinto-Fernandez D, Moreno J, Lancini M, Torricelli D. Characterization and evaluation of human–exoskeleton interaction dynamics: A review. Sensors. 2022;**22**:3993

[7] Kim J, Shim M, Ahn D, Son B, Kim S, Kim D, et al. Design of a knee exoskeleton using foot pressure and knee torque sensors. International Journal of Advanced Robotic Systems. 2015;**12**:112

[8] Fu J, Choudhury R, Hosseini S, Simpson R, Park J. Myoelectric control Systems for Upper Limb Wearable Robotic Exoskeletons and Exosuits—A Systematic Review. Sensors. 2022;**22**: 8134

[9] Dupont A, Morin E. A myoelectric control evaluation and trainer system. IEEE Transactions on Rehabilitation Engineering. 1994;**2**: 100-107

[10] Englehart K, Hudgin B, Parker P. A wavelet-based continuous classification scheme for multifunction myoelectric control. IEEE Transactions on Biomedical Engineering. 2001;**48**: 302-311

[11] Englehart K, Hudgins B. A robust, real-time control scheme for multifunction myoelectric control. IEEE Transactions on Biomedical Engineering. 2003;**50**:848-854

[12] Robertson D, Caldwell G, Hamill J, Kamen G, Whittlesey S. Research Methods in Biomechanics. Champaign, IL, USA: Human Kinetics; 2014

[13] Bi L, Feleke A, Guan C. A review on EMG-based motor intention prediction of continuous human upper limb motion for human-robot collaboration. Biomedical Signal Processing and Control. 2019;**51**:113-127

[14] Rechy-Ramirez E, Hu H. Bio-signal based control in assistive robots: A survey. Digital Communications and Networks. 2015;**1**:85-101

[15] Lotti N, Xiloyannis M, Missiroli F, Chiaradia D, Frisoli A, Sanguineti V, et al. Intention-detection strategies for upper limb exosuits: Model-based myoelectric vs dynamic-based control. In: 2020 8th IEEE RAS/EMBS International Conference for Biomedical *Challenges and Trends of Machine Learning in the Myoelectric Control System for Upper… DOI: http://dx.doi.org/10.5772/intechopen.111901*

Robotics and Biomechatronics (BioRob). New York, NY, USA: IEEE; 2020. pp. 410-415

[16] Accogli A, Grazi L, Crea S, Panarese A, Carpaneto J, Vitiello N, et al. EMG-based detection of User's intentions for human-machine shared control of an assistive upper-limb exoskeleton. In: Wearable Robotics: Challenges and Trends. Gewerbestrasse, Switzerland: Springer, Cham; 2016. pp. 181-185

[17] Lenzi T, Rossi S, Vitiello N, Carrozza M. Intention-based EMG control for powered exoskeletons. IEEE Transactions on Bio-medical Engineering. 2012;**59**:2180-2190

[18] Roche A, Rehbaum H, Farina D, Aszmann O. Prosthetic myoelectric control strategies: A clinical perspective. Current Surgery Reports. 2014;**2**:1-11

[19] Bayat N, Rastegari E, Li Q. Human Gait Recognition Using Bag of Words Feature-Representation Method. In: Ahram T, Falcão C, editors, Human Factors and Wearable Technologies. AHFE International Conference. Vol. 29. USA: AHFE Open Access, AHFE International; 2022. DOI: 10.54941/ ahfe1001481

[20] Mesin L, Merletti R, Rainoldi A. Surface EMG: The issue of electrode location. Journal of Electromyography and Kinesiology. 2009;**19**:719-726

[21] Merletti R, Farina D. Analysis of intramuscular electromyogram signals. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences. 2008;**367**: 357-368

[22] Li G, Li Y, Zhang Z, Geng Y, Zhou R. Selection of sampling rate for EMG pattern recognition based prosthesis

control. In: 2010 Annual International Conference of the IEEE Engineering In Medicine and Biology. Buenos Aires, Argentina: IEEE; 2010. pp. 5058-5061

[23] Li G, Li Y, Yu L, Geng Y. Conditioning and sampling issues of EMG signals in motion recognition of multifunctional myoelectric prostheses. Annals of Biomedical Engineering. 2011; **39**:1779-1787

[24] De Luca C, Donald Gilmore L, Kuznetsov M, Roy S. Filtering the surface EMG signal: Movement artifact and baseline noise contamination. Journal of Biomechanics. 2010;**43**: 1573-1579

[25] Oskoei M, Hu H. Myoelectric control systems—A survey. Biomedical Signal Processing and Control. 2007;**2**:275-294

[26] Phinyomark A, Phukpattaranont P, Limsakul C. Feature reduction and selection for EMG signal classification. Expert Systems with Applications. 2012; **39**:7420-7431

[27] Lashgari E, Maoz U. Dimensionality reduction for classification of object weight from electromyography. PLoS One. 2021;**16**:e0255926

[28] Oskoei M, Hu H. Support vector machine-based classification scheme for myoelectric control applied to upper limb. IEEE Transactions on Biomedical Engineering. 2008;**55**:1956-1965

[29] Scheme E, Hudgins B, Englehart K. Confidence-based rejection for improved pattern recognition myoelectric control. IEEE Transactions on Biomedical Engineering. 2013;**60**: 1563-1570

[30] Scheme E, Englehart K, Hudgins B. Selective classification for improved

robustness of myoelectric control under nonideal conditions. IEEE Transactions on Biomedical Engineering. 2011;**58**: 1698-1705

[31] Jana M, Barua B, Hazarika S. Design and development of a finger exoskeleton for motor rehabilitation using electromyography signals. In: 2019 23rd International Conference on Mechatronics Technology (ICMT). Salerno, Italy: IEEE; 2019. pp. 1-6

[32] Arteaga M, Castiblanco J, Mondragon I, Colorado J, Alvarado-Rojas C. EMG-based adaptive trajectory generation for an exoskeleton model during hand rehabilitation exercises. In: 2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob). New York, NY, USA: IEEE; 2020. pp. 416-421

[33] Cheon S, Kim D, Kim S, Kang B, Lee J, Gong H, et al. Single EMG sensordriven robotic glove control for reliable augmentation of power grasping. IEEE Transactions on Medical Robotics and Bionics. 2021;**3**:179-189

[34] Su H, Li Z, Li G, Yang C. EMG-based neural network control of an upper-limb power-assist exoskeleton robot. In: Advances In Neural Networks–ISNN 2013: 10th International Symposium on Neural Networks, Dalian, China, July 4– 6, 2013, Proceedings, Part II 10. Dalian, China: Springer, Berlin, Heidelberg; 2013. pp. 204-211

[35] Trigili E, Grazi L, Crea S, Accogli A, Carpaneto J, Micera S, et al. Detection of movement onset using EMG signals for upper-limb exoskeletons in reaching tasks. Journal of Neuroengineering and Rehabilitation. 2019;**16**:45

[36] Kirchner E, Tabie M, Seeland A. Multimodal movement prediction –

towards an individual assistance of patients. PLoS One. 2014;**9**:e85060

[37] Wu Q, Chen B, Wu H. Neuralnetwork-enhanced torque estimation control of a soft wearable exoskeleton for elbow assistance. Mechatronics. 2019;**63**: 102279

[38] Zeng H, Li K, Wei N, Song R, Tian X. A sEMG-controlled robotic hand exoskeleton for rehabilitation in poststroke individuals. In: 2018 IEEE International Conference on Cyborg and Bionic Systems (CBS). Shenzhen, China: IEEE; 2018. pp. 652-655

[39] Twardowski M, Roy S, Li Z, Contessa P, De Luca G, Kline J. Motor unit drive: A neural interface for real-time upper limb prosthetic control. Journal of Neural Engineering. 2019;**16**:3-5

[40] Secciani N, Topini A, Ridolfi A, Meli E, Allotta B. A novel pointin-polygon-based sEMG classifier for hand exoskeleton systems. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2020;**28**: 3158-3166

[41] Godoy R, Dwivedi A, Liarokapis M. Electromyography based decoding of dexterous, in-hand manipulation motions with temporal multichannel vision transformers. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2022;**30**:2207-2216

[42] Tang Z, Zhang K, Sun S, Gao Z, Zhang L, Yang Z. An upper-limb powerassist exoskeleton using proportional myoelectric control. Sensors. 2014;**14**: 6677-6694

[43] Pang M, Guo S, Song Z. Study on the sEMG driven upper limb exoskeleton rehabilitation device in bilateral rehabilitation. Journal of Robotics and Mechatronics. 2012;**24**:585

*Challenges and Trends of Machine Learning in the Myoelectric Control System for Upper… DOI: http://dx.doi.org/10.5772/intechopen.111901*

[44] Loconsole C, Leonardis D, Barsotti M, Solazzi M, Frisoli A, Bergamasco M, et al. An emg-based robotic hand exoskeleton for bilateral training of grasp. In: 2013 World Haptics Conference (WHC). Daejeon, South Korea: IEEE; 2013. pp. 537-542

[45] Ngeo J, Tamei T, Shibata T, Orlando M, Behera L, Saxena A, et al. Control of an optimal finger exoskeleton based on continuous joint angle estimation from EMG signals. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). Osaka, Japan: IEEE; pp. 338-341

[46] Leonardis D, Barsotti M, Loconsole C, Solazzi M, Troncossi M, Mazzotti C, et al. An EMG-controlled robotic hand exoskeleton for bilateral rehabilitation. IEEE Transactions on Haptics. 2015;**8**:140-151

[47] Burns M, Pei D, Vinjamuri R. Myoelectric control of a soft hand exoskeleton using kinematic synergies. IEEE Transactions on Biomedical Circuits and Systems. 2019;**13**: 1351-1361

[48] Cisnal A, Pérez-Turiel J, Fraile J, Sierra D, Fuente E. RobHand: A hand exoskeleton with real-time EMG-driven embedded control. Quantifying hand gesture recognition delays for bilateral rehabilitation. IEEE Access. 2021;**9**: 137809-137823

[49] Li Z, Wang B, Sun F, Yang C, Xie Q, Zhang W. sEMG-based joint force control for an upper-limb power-assist exoskeleton robot. IEEE Journal of Biomedical and Health Informatics. 2014;**18**:1043-1050

[50] Lu L, Wu Q, Chen X, Shao Z, Chen B, Wu H. Development of a sEMGbased torque estimation control strategy

for a soft elbow exoskeleton. Robotics and Autonomous Systems. 2019;**111**: 88-98

[51] Kopke J, Hargrove L, Ellis M. Applying LDA-based pattern recognition to predict isometric shoulder and elbow torque generation in individuals with chronic stroke with moderate to severe motor impairment. Journal of Neuroengineering and Rehabilitation. 2019;**16**:4-6

[52] Sierotowicz M, Lotti N, Nell L, Missiroli F, Alicea R, Zhang X, et al. EMG-driven machine learning control of a soft glove for grasping assistance and rehabilitation. IEEE Robotics and Automation Letters. 2022;**7**:1566-1573

[53] Meattini R, Bernardini A, Palli G, Melchiorri C. sEMG-based minimally supervised regression using soft-DTW neural networks for robot hand grasping control. IEEE Robotics and Automation Letters. 2022;**7**: 10144-10151

[54] Hamaya M, Matsubara T, Noda T, Teramae T, Morimoto J. Learning assistive strategies for exoskeleton robots from user-robot physical interaction. Pattern Recognition Letters. 2017;**99**: 67-76

[55] Deisenroth M, Rasmussen C. PILCO: A model-based and data-efficient approach to policy search. In: Proceedings of the 28th International Conference on Machine Learning (ICML-11). 2011. pp. 465-472

[56] Phinyomark A, Quaine F, Charbonnier S, Serviere C, Tarpin-Bernard F, Laurillau Y. EMG feature evaluation for improving myoelectric pattern recognition robustness. Expert Systems with Applications. 2013;**40**: 4832-4840

[57] Geng Y, Samuel O, Wei Y, Li G. Improving the robustness of real-time myoelectric pattern recognition against arm position changes in Transradial amputees. BioMed Research International. 2017;**2017**:5090454

[58] Tkach D, Huang H, Kuiken T. Study of stability of time-domain features for electromyographic pattern recognition. Journal of NeuroEngineering and Rehabilitation. 2010;**7**:21

[59] Boschmann A, Kaufmann P, Platzner M, Winkler M. Towards multimovement hand prostheses: Combining adaptive classification with high precision sockets. In: Proceedings of the 2nd European Conference Technically Assisted Rehabilitation. 2009

[60] Bank D, Koenigstein N, Giryes R. Autoencoders. CoRR. 2020. DOI: abs/ 2003.05991

[61] Teh Y, Hargrove L. Using latent representations of muscle activation patterns to mitigate myoelectric interface noise. In: 2021 10th International IEEE/EMBS Conference On Neural Engineering (NER). pp. 1148-1151

[62] Lv B, Sheng X, Zhu X. Improving myoelectric pattern recognition robustness to electrode shift by autoencoder. In: 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). pp. 5652-5655

[63] Kang H, Wang J. Adaptive control of 5 DOF upper-limb exoskeleton robot with improved safety. ISA Transactions. 2013;**52**:844-852

[64] López N, Sciascio F, Soria C, Valentinuzzi M. Robust EMG sensing system based on data fusion for myoelectric control of a robotic arm.

BioMedical Engineering OnLine. 2009;**8**:5

[65] Zhuang Y, Yao S, Ma C, Song R. Admittance control based on EMGdriven musculoskeletal model improves the human–robot synchronization. IEEE Transactions on Industrial Informatics. 2019;**15**:1211-1218

[66] Han J, Lee J, Choi H, Kim J, Choi J. Policy Design for an Ankle-Foot Orthosis Using Simulated Physical Human–Robot Interaction via deep reinforcement learning. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2022;**30**:2186-2197

[67] Caggiano V, Wang H, Durandau G, Sartori M, Kumar V. MyoSuite–A contact-rich simulation suite for musculoskeletal motor control. ArXiv Preprint ArXiv:2205.13600. 2022

[68] Fu J, Choudhury R, Park J. Deep reinforcement learning based upper limb neuromusculoskeletal simulator for modelling human motor control. IEEE. 2023, Under Review

[69] Mnih V, Kavukcuoglu K, Silver D, Rusu A, Veness J, Bellemare M, et al. Human-level control through deep reinforcement learning. Nature. 2015; **518**:529-533

[70] Silver D, Schrittwieser J, Simonyan K, Antonoglou I, Huang A, Guez A, et al. Mastering the game of go without human knowledge. Nature. 2017;**550**:354-359

[71] Kidziński Ł, Mohanty S, Ong C, Huang Z, Zhou S, Pechenko A, et al. Learning to run challenge solutions: Adapting reinforcement learning methods for neuromusculoskeletal environments. In: The NIPS '17 Competition: Building Intelligent

*Challenges and Trends of Machine Learning in the Myoelectric Control System for Upper… DOI: http://dx.doi.org/10.5772/intechopen.111901*

Systems. Gewerbestrasse, Switzerland: Springer, Cham; 2018. pp. 121-153

[72] Wannawas N, Subramanian M, Faisal A. Neuromechanics-Based Deep Reinforcement Learning of Neurostimulation Control in FES Cycling. Italy: IEEE; 2021

[73] Pilarski P, Dawson M, Degris T, Fahimi F, Carey J, Sutton R. Online human training of a myoelectric prosthesis controller via actor-critic reinforcement learning. In: 2011 IEEE International Conference on Rehabilitation Robotics. 2011. pp. 1-7

[74] Mnih V, Badia A, Mirza M, Graves A, Lillicrap T, Harley T, et al. Asynchronous methods for deep reinforcement learning. In: International Conference on Machine Learning. New York, NY, USA: JMLR; 2016. pp. 1928-1937

[75] Wu W, Saul K, Huang H. Using reinforcement learning to estimate human joint moments from electromyography or joint kinematics: An alternative solution to musculoskeletal-based biomechanics. Journal of Biomechanical Engineering. 2021;**143**:2-3

#### **Chapter 7**

## The Use of Machine Vision in the Diagnosis of Ripening Strawberries

*Tamara Oleshko, Dmytro Kvashuk and Iryna Heiets*

#### **Abstract**

The problem of image recognition using methods based on statistical solutions can be divided into two classes: parametric and nonparametric. In the first case there is a set of data on the selection of image criteria, in the second such criteria must be found. Taking into account, the possibilities of obtaining initial data for the identification of agricultural objects, the first option is considered in the article. Possibilities of diagnostics of strawberry growth at a stage of its cultivation in hothouse conditions are presented. Several image classifiers that can be used to solve such problems are investigated. An experiment was performed, which has shown the accuracy of the applied algorithms for diagnosis and recognition of berries, which allowed to set thresholds for the detection of ripe berries. A number of berry recognition methods were compared in order to study the accuracy of berry recognition and response to berry growth.

**Keywords:** image recognition, strawberries, machine vision, ripening, cultivation

#### **1. Introduction**

Image processing is an effective tool in the process of studying crops, the care of which largely depends on the observations that are made at all stages of the plant life cycle. However, when part of the crop is in the process of ripening and part is already ripe, especially in greenhouses, there is a need to control this process. Remontant strawberries, which can yield continuously throughout the year, are among the types of crops that need to be identified for ripening. Therefore, you can use machine vision to control the cultivation of such a crop. The use of such systems in agricultural work has a huge potential for accounting automation and harvesting control. Therefore, an important aspect of such tasks is the development of algorithms for recognizing the characteristics of plants and their classification, as well as the choice of existing methods.

Research in this area shows that the most common methods of machine learning for image classification include neural networks and other methods of machine learning [1–3]. However, due to the high demand for computing power, machine vision systems require both energy and time consumption for image processing. Therefore, depending on the tasks, the identification algorithm should be maximally adapted,

because the criteria for selecting the quality of image recognition differ depending on the applicability to different subject areas.

Thus, to solve the narrow problems of identifying the number of ripe berries, approaches such as identifying an object by color, contour, and shape, which require only matrix calculations, can give a quite acceptable result. Taking into account that, obstacles such as brightness and image scale also play an important role, stationary camcorder installation, and artificial lighting can alleviate these problems.

The main stages, in this case, can be considered as follows:


Increasing the number of selected objects will be an indicator of the beginning of the harvest.

Thus, it is necessary:


As a result, it is possible to diagnose the right time for harvesting, which will allow farmers to harvest ripe berries in time, without losses. Popular algorithms are used for image analysis: SURF; ORB; FAST; SIFT. In order to store data for further comparison, you can use a simple Sqlite database.

#### **2. Literature analysis and problem statement**

The image recognition algorithms in the field of agriculture have become very popular, which have differences and certain classifications [4], a large part of which is implemented using a number of methods of machine learning and neural networks. However, the use of neural networks requires huge arrays of data for training, which cannot always be obtained. In addition, the significant capacity for training and the time spent on processing such arrays is quite significant. It is difficult to determine the logic of the distribution of the weights rate. All this and many other requirements for big data processing impose certain restrictions on the choice of computing tools, obtaining training data, and the difficulty of understanding how the algorithm itself works. However, depending on the image identification tasks, the algorithm for determining the desired image can be simplified in some way. In particular, if you use color detectors in combination with contour detectors, for example, which allow under certain conditions to obtain image descriptors, you can identify the object with high accuracy.

There is a considerable amount of work devoted to the identification of objects in agriculture by color and geometric shapes. Thus, the recognition is performed [5] on the basis of statistical characteristics and signs of the set of obtained ranges of descriptor values with the help of the minimum distance classifier. The results of experiments on a database containing about 2635 fruits, made it possible to establish 15 different classes, which has confirmed the effectiveness of the proposed approach.

A similar study to that proposed in the article aims to analyze automated harvesting methods based on fruit categorization. It in turn offers a new approach that separates the fruit in the foreground from the background, which in addition to color spectrum characteristics also takes into account the identification of texture images. The classification model is built using reference vectors (SVM) and learns using function descriptors taken from the training data set. The proposed approach allows the use of small computing power and can run on single-board computers, such as RaspberryPi [6].

Some works describe the possibility of identifying fruits and vegetables by such criteria as the characteristics of the plant color [7], as well as the structure of their leaves [8]. Thus, the combination of these two approaches gives certain results. However, simple ways to identify images have a number of disadvantages, in particular noise generated by shadows or changes in light. Due to the use of a number of filters, the use of neural networks in this case gives a more accurate result [9]. Therefore, such filters are carefully selected. For example, the Otsu method is often used to reduce noise by low-pass filters [10].

Median filters are used to smooth inconspicuous spots [11]. Spectral analysis is used to obtain a clear identification by color. For example, researchers at the University of Oklahoma studied the use of hyper-spectral methods of image analysis to assess the quality, safety, and classification of fruits and vegetables, which resulted in determining the most effective approaches to estimating the color spectrum in the image [12]. Typically, to highlight RGB color saturation, the image classification is converted to HSV or YCbCr. To ensure image clarity, anti-aliasing filters are used,

such as the Gaussian filter [13]. The step-by-step selection of individual image contours can be represented using the OpenCV image recognition library, which contains the possibilities of software implementation of the respective stages. Thus, to identify strawberries, by means of this library, you can select separate pixels by color. In other words, you can create a mask that characterizes only the berries of strawberries. Then you can reduce the noise by smoothing with the help of the Gaussian filter [13].

To determine the area of a selected object, you can count its pixel IDs, which characterize such a selection. To do this, you can use the threshold method of the OpenCV library, which turns the image, where all the pixels that are darker (less) than 127 are replaced by 0, and everything that is brighter (more) than 127 - by 255.

In this case, it is possible to set criteria for strawberry growth by comparing the specific gravity of the selected color spectrum at different times. To do this, strict conditions must be met. Photo fixation should be performed each time in the same position of the photo recorder and in the same lighting.

In this case, it is possible to observe changes in the size of the strawberry and changes in its color, which during ripening begins to correspond to the established range of identification. In **Figure 2** the corresponding range in the HSV format was set within the following limits: lower threshold—(0,50,20)–(5,255,255); upper threshold—(175,50,20) –(180,255,255).

Taking into account that the photos were taken in different conditions if error and noise in the images a3-d3 are minimal, this approach can be used to assess the growth

**Figure 2.** *The process of identifying strawberries by color.*

of strawberries. In addition, if one of the most complex image processing processes is the Gaussian filter, the corresponding tasks can be solved using single-board computers such as RaspberryPi [14].

The stages of identification presented in **Figure 2** are highlighted in the following sequence:


The OpenCV library was used to solve them. In the Python programming language, it looks like this:

```
import cv2
import os
im = cv2.imread('image.jpg')
im = cv2.resize(im, (400, 300))
img_hsv = cv2.cvtColor(im, cv2.COLOR_BGR2HSV)
m1 = cv2.inRange(img_hsv, (0,50,20), (5,255,255))
m2 = cv2.inRange(img_hsv, (175,50,20), (180,255,255))
mask = cv2.bitwise_or(m1, m2 )
cr = cv2.bitwise_and(im, im, mask=mask)
m1 = cv2.GaussianBlur(mask, (7,7), 3)
cr1 = cv2.GaussianBlur(cr, (3,3), 0)
print (len(cr1[cr1==0]), 'black')
print (len(cr1[cr1>0]), 'white')
cv2.imshow("msk", m1)
cv2.imshow("cr", cr1)
cv2.waitKey()
```
Thus, the ability to assess the growth of strawberries using machine vision, without using complex algorithms for deep learning and neural networks, puts forward the admissibility of our hypothesis. Is it possible to create a strawberry harvest indicator with a low-power computer and simple image operations? The main idea of this hypothesis is, taking into account the studied approaches, to obtain a growth curve of strawberries, based on a simple comparative method. Such a comparison can characterize the dynamics of changes in the area of selected objects in the images, as relative indicators of the specific weight of selected objects in the image. For example, for images a3-d3, presented in **Figure 2**, this ratio will have the following characteristics (**Figure 3**).

However, the difficulty lies in the video recording itself, as it must be carried out under certain conditions of image periodicity and with the same angle and lighting, which requires further adjustment (namely, data acquisition, storage, and subsequent processing).

**Figure 3.** *Percentage of color-coded objects in images a1-d1 Figure 2.*

#### **3. Ways to identify the growth of strawberries based on singular point detectors**

In order to timely diagnose the yield of strawberries, in addition to identifiers by color, you should pay attention to other methods of pattern recognition. In the framework of the task, detecting the increase in strawberries, you can consider detectors of special points in the image, the number of which can also serve as an indicator of growth. The most common are SIFT and SURF.

The use of the SIFT detector is more efficient but requires a significant amount of computation. Its advantages are that it is possible to identify special points in the image regardless of scale and offset, as well as changes in brightness. However, the invariance of the last three transformations cannot be fully obtained. Basis for his work is the construction of the Gaussian pyramid and the differences of the Gaussians (Difference of Gaussian, DoG). Gaussian is an image that is blurred by a Gaussian filter.

$$L(\mathbf{x}, \mathbf{y}, \sigma) = L(\mathbf{x}, \mathbf{y}, \sigma) \* L(\mathbf{x}, \mathbf{y}), \tag{1}$$

where:

L - the Gaussian value at a point with coordinates (x, y);

σ - blur radius;

G - Gaussian core;

I - the value of the original image,

\* - convolution operation.

In this case, the Gaussian difference is the image obtained by subtracting each pixel of one Gaussian source image from Gaussian with a different blur radius.

$$\begin{split} \mathbf{D}(\mathbf{x}, \mathbf{y}, \sigma) &= \left( \mathbf{G}(\mathbf{x}, \mathbf{y}, \mathbf{k}\sigma) - \mathbf{G}(\mathbf{x}, \mathbf{y}, \sigma) \right) \* I(\mathbf{x}, \mathbf{y}) = \\ &= \mathbf{L}(\mathbf{x}, \mathbf{y}, \mathbf{k}\sigma) - \mathbf{L}(\mathbf{x}, \mathbf{y}, \sigma), \end{split} \tag{2}$$

To determine a singular point, together with the construction of the Gaussians pyramid, a pyramid of Gaussians differences is constructed, consisting of the differences of adjacent images in the Gaussians pyramid. Accordingly, the number of images in this pyramid will be N + 1.

*The Use of Machine Vision in the Diagnosis of Ripening Strawberries DOI: http://dx.doi.org/10.5772/intechopen.110894*

**Figure 4.** *Pyramid of Gaussians and their differences.*

**Figure 5.** *Defining a singular point among the differences between Gaussians.*

**Figure 4** shows the Gaussians pyramid, which displays their differences, formed on the basis of two neighboring Gaussians. The number of such differences is always one less than the number of Gaussians, because when you move to the next octave, the size of the images is halved. The point that is the local extremum of the Gaussian difference is then determined. If the value of the Gaussians difference at a certain point is bigger or less than all the values of the neighboring Gaussians differences, such a point is considered an extremum point. In **Figure 5** this point is represented in shaded.

Such points are refined using an approximation of the Gaussian difference function using a second-order Taylor polynomial, which is taken at the point of a definite extremum.

Then, based on the neighborhood of the singular point, a descriptor is constructed, which is presented in the form of a gradient matrix in accordance with the pixels that are around the singular point.

Then a histogram of the descriptor is created, which is used for further correlation of the image with others. Therefore, SIFT detectors need to increase the computational speed, because the image scaling process takes some time. For greater speed, SURF detectors are used which, in order to find singular points, use the Hesse matrix, the determinant of which reaches the extremum at the points of maximum change of the brightness gradient. Thus, if the original image is given by the intensity matrix I, then the current pixel, which is analyzed for changes in color intensity, can be denoted by X = (x, y) and the scale of the filter σ. In this case, the Hesse matrix will look like this:

$$\mathbf{H}(\mathbf{x},\sigma) = \begin{bmatrix} L\_{\mathbf{x}\mathbf{x}}(\mathbf{X},\sigma) & L\_{\mathbf{x}\mathbf{y}}(\mathbf{X},\sigma) \\ L\_{\mathbf{x}\mathbf{y}}(\mathbf{X},\sigma) & L\_{\mathbf{y}\mathbf{y}}(\mathbf{X},\sigma) \end{bmatrix},\tag{3}$$

where: *Lxx*ð Þ *x*, *σ* , *Lxy*ð Þ *x*, *σ* , *Lyy*ð Þ *x*, *σ* - approximation convolution of the second derivative of the Gaussian cores.

The determinant of the Hesse matrix reaches its extremum at the points of maximum change of the brightness gradient. In turn, the SURF method uses a filter with a Gaussian core throughout the image, while finding singular points at which the maximum value of the determinant of the Hesse matrix is achieved. Thanks to this search, both dark spots on a white background and vice versa stand out. Unlike SIFT, the SURF method does not have a mechanism for refining points, but immediately forms descriptors. Like SIFT, SURF is invariant to rotation and scale. Using the capabilities of these detectors, you can use them to establish singular points on the structure of the image of strawberries and with the help of their number observe the yield. This statement is due to the fact that the shape of the strawberry has some common features (**Figure 6**), the identification of which with these detectors can lead to a positive result.

In order to do this, we will determine the training image, which will help to obtain descriptors of singular points (**Figure 6B**). We will also compare the number of singular points for a number of test images, which will establish which method to identify ripening strawberries is more effective, identify color changes, or identify changes in singular points. The following assumption is based on the fact that during the strawberries growth, their structure will expand, which has singular points.

Comparing these two approaches, it should be noted that the use of the SIFT detector will not allow to assess the ripening by color change, but only by size, so it has some limitations.

**Figure 6.** *Common features in the structure of the shape of strawberries.*

*The Use of Machine Vision in the Diagnosis of Ripening Strawberries DOI: http://dx.doi.org/10.5772/intechopen.110894*

According to the results of testing the training sample of descriptors of singular points, which were obtained from the image (**Figure 6b**), a small number of matches were identified on the original images using SIFT and SURF detectors (**Figure 7**). In addition, as you can see in **Figure 6**, there is a fairly large error, which indicates the inexpediency of using such detectors in the process of strawberries ripening identification.

It is presented in **Figure 7**:




**Figure 8.**

*Comparison of the effectiveness of strawberry recognition methods using color filters and SIFT and SURF detectors on samples of test images (Figure 7a1–d1), (%).*

**Figure 9.**

*Position of the infrared distance sensor (created using Google SketchUp).*

#### **4. Identification of strawberry ripening processes by color**

Detection of changes in strawberry images can be implemented on the basis of a mobile platform, which is controlled by a single-board computer RaspberryPi. Infrared proximity sensors can be used to identify shooting locations, which can be used to obtain a signal to stop a moving platform at the necessary location for photo fixing of strawberries. Standard infrared distance sensors allow you to respond to approximations at a distance of up to 30 cm, which is enough to identify the stopping point of the platform. The placement of such sensors can be carried out according to the scheme in **Figure 9**.

After that, at the set time, the platform will start moving again for the next video recording.

The implementation of the software algorithm can be performed on the basis of Python interpreter and libraries: OpenCV, image processing, sqlite3, database, RPi, RaspberryPi COM port, which will be used to provide a control signal to drive the transport platform. Thus, the algorithm will be as follows (**Figure 9**).

*The Use of Machine Vision in the Diagnosis of Ripening Strawberries DOI: http://dx.doi.org/10.5772/intechopen.110894*

**Figure 10.** *Algorithm of mobile platform operation for photo-fixation of the strawberry ripening process.*

By implementing this algorithm in Python [15] and using it for a test sample of strawberry images, you can get the following result (**Figure 10**). Thus, you can see how strawberries grow in **Figure 10**.

The sequence of sampling images in the order of ripening (*a*1, *b*1,*c*1, *d*1, *a*3, *b*3,*c*3, *d*3) and the number of obtained points, respectively (*a*2, *b*2,*c*2, *d*2, *a*4, *b*4,*c*4, *d*4). The growth dynamics of strawberries can be obtained in the form of the ratio of identified points by color to unidentified ones (**Figure 12**). However, a small percentage of errors can be observed in **Figures 11** and **12**, which are caused by the concentration of light on strawberries.

**Figure 11.** *Identification of strawberries by color during ripening.*

#### **Figure 12.**

*The dynamics of ripening strawberries is obtained as a result of the ratio of selected points by color to the total quantity for a conditional period of time.*

### **5. Conclusion**

The most important criterion for strawberry growth is its color and size. Recognition of changes in these criteria in the images can be obtained using conventional color filters, with the specified parameters of the color range. This approach is easy to

implement in practice because it does not use complex computational algorithms, which is quite important for use in agriculture.

To identify the ripening of strawberries, it is enough to use an inexpensive singlechamber computer and a mobile platform with a photo recorder, which at a given time can capture changes in the image of strawberries.

Taking into account the slight error due to the formation of the light concentration on strawberries, the experiment resulted in growth dynamics, which characterizes such an approach on the positive side. However, a number of conditions must be met for its application. In particular, detailed images require the use of means of moving the photo-fixator and the invariability of the angle of photo-fixation. This method, in contrast to the detection of singular points in the image using SIFT and SUFR detectors, has shown higher efficiency, because the most important criterion for ripening strawberries is the change of its color.

#### **Acknowledgements**

The authors thank our colleagues from School of Engineering, Aerospace Engineering and Aviation team at RMIT University (Australia) and Department of Economic Cybernetics at National Aviation University (Ukraine), who provided insight and expertise that greatly assisted the research, although they may not agree with all the interpretations and conclusions of this research.

#### **Author details**

Tamara Oleshko, Dmytro Kvashuk and Iryna Heiets\* National Aviation University, Kyiv, Ukraine

\*Address all correspondence to: iryna.heiets@rmit.edu.au

© 2023 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

### **References**

[1] He K et al. Deep residual learning for image recognition. Computer Vision and Pattern Recognition. 2015;**2015**:770-778

[2] Sa I, Ge Z, Dayoub F, Upcroft B, Perez T, McCool C. A fruit detection system using deep neural networks. Sensors (Basel). 2016;**16**(8):1222. DOI: 10.3390/s16081222

[3] Rocha A, Hauagge DC, Wainer J, Goldenstein S. Automatic fruit and vegetable classification from images. Computers and Electronics in Agriculture. 2010;**70**:96-104. DOI: 10.1016/j.compag.2009.09.002

[4] Burgos-Artizzu X, Ribeiro A, Guijarro M, Pajares G. Real-time image processing for crop/weed discrimination in maize fields. Computers and Electronics in Agriculture. 2011;**75**:337-346

[5] Selvaraj A, Shebiah N, Nidhyananthan S, Ganesan L. Fruit recognition using color and texture features. Journal of Emerging Trends in Computing and Information Sciences. 2010;**1**:90-94

[6] Jana S, Basak S, Parekh R. 2017: Automatic fruit recognition from natural images using color and texture features. 2017. DOI: 10.1109/DEVIC.2017.807 4025

[7] Anami BS, Pujari JD,

Yakkundimath R. Identification and classification of normal and affected agriculture/horticulture produce based on combined color and texture feature extraction. International Journal of Computer Applications in Engineering Sciences. 2011;**1**(3):356-360

[8] El-Helly M, Rafea A, El-Gammal S. An integrated image processing system for leaf disease detection and diagnosis. In: Proceedings of the 1st Indian International Conference on Artificial Intelligence (December 18-20, 2003). Hyderabad, India; 2003. pp. 1182-1195

[9] Kai S, Zhikun L, Hang S, Chunhong G. A research of maize disease image recognition of corn based on BP networks. In: Third International Conference on Measuring Technology and Mechatronics Automation. Shanghai, China: IEEE; 2011. pp. 246-249. DOI: 10.1109/ ICMTMA.2011.66

[10] Nunik Noviana Kurniawati, Siti Norul Huda Sheikh Abdullah, Salwani Abdullah, Saad Abdullah, "Investigation on image processing techniques for diagnosing paddy diseases", International Conference of Soft Computing and Pattern Recognition. Bangi, Malaysia: IEEE; 2009. pp. 272–277 (2009. 10.1109/SoCPaR.2009.62

[11] Ying G, Miao L, Zelin H. A study on the method of image pre-processing for recognition of crop diseases. In: International Conference on Advanced Computer Control. Singapore: IEEE; 2008. pp. 202-206

[12] Wang N, Elmasry G, Sevakarampalayam S, Qiao J. Spectral imaging techniques for food quality evaluation. Stewart Postharvest Review. 2007;**3**:1-8. DOI: 10.2212/spr.2007.1.1

[13] Haddad RA, Akansu AN. A class of Fast Gaussian binomial filters for speech and image processing. IEEE Transactions on Acoustics, Speech, and Signal Processing. 1991;**39**:723-727

[14] Gibbs S. Raspberry Pi becomes Best Selling British Computer. The Guardian. 2015. [Accessed: 28 December 2016]

*The Use of Machine Vision in the Diagnosis of Ripening Strawberries DOI: http://dx.doi.org/10.5772/intechopen.110894*

[15] Kvashuk D, Oleshko T, Heiets I. Moving Platforms Source Code (Version 1.0) [Source code]. 2020. Available from: https://www.pythona nywhere.com/user/Dmitro/shares/ c084641e09ba45ccbf4fb89851c121d8/

### *Edited by Serdar Küçük*

Robotics is an important part of modern engineering involving electricity and electronics, computers, mathematics, and mechanism design. In recent years, in addition to serial robots, multi-robot systems have begun to attract the attention of students, academics, and industry workers. This interest has directly impacted the development of novel theoretical research areas and products. This book explores new developments in multi-robot systems, such as trajectory planning, control algorithms, and programming.

*Andries Engelbrecht, Artificial Intelligence Series Editor*

Published in London, UK © 2023 IntechOpen © your\_photo / iStock

Multi-Robot Systems - New Advances

IntechOpen Series

Artificial Intelligence, Volume 20

Multi-Robot Systems

New Advances

*Edited by Serdar Küçük*