**Meet the editors**

Professor Jun Zhou is one of the top experts in agricultural robots in China and full professor at Nanjing Agricultural University (NJAU). He is also the director of the Department of Science and Technology at NJAU. He received his PhD degree in Mechanical Engineering from NJAU in 2005. Over the past 10 years Professor Zhou has published more than 50 SCI/EI cited articles in

national and international journals on agricultural robots and vision-based control problems in agriculture. His research interests include agricultural robot control, robotic grasping, navigation and guidance, robotic sensing and control, object recognition and location, and mapping and localization.

Dr. Baohua Zhang is an associate professor at the College of Engineering, Nanjing Agricultural University, China. Dr. Zhang received his BS degree in Mechanical and Electronic Engineering from Northwest A&F University in July 2010 and his PhD degree in Mechanical Engineering from Shanghai Jiao Tong University in June 2016. His research interests include harvesting robots,

robot vision, robotic grasping, spectral analysis and modeling, and robotic systems and their applications in agriculture, food, and biosystem engineering. Dr. Zhang has published more than 30 SCI cited articles and 10 EI cited articles in national and international journals.

Contents

**Preface VII**

**Agricultural Robots 3** Baohua Zhang and Jun Zhou

**Algorithms 31**

**Pyralidae Insects 45**

Rakun

Santos

**Section 1 Overview of the Recent Developments and Applications 1**

**Section 2 Agricultural Robots for Field and Greenhouse Farming 9**

Chapter 3 **An Evaluation of Three Different Infield Navigation**

Chapter 4 **Agricultural Robot for Intelligent Detection of**

Zhuhua Hu, Boyi Liu and Yaochi Zhao

Chapter 2 **Development of a Field Robot Platform for Mechanical Weed Control in Greenhouse Cultivation of Cucumber 11**

Shamshiri, Siva K. Balasundram and Muhammad Yamin

Chapter 5 **Hybrid-Powered Autonomous Robots for Reducing Both Fuel**

Amid Heravi, Desa Ahmad, Ibrahim A. Hameed, Redmond Ramin

Peter Bernad, Peter Lepej, Črtomir Rozman, Karmen Pažek and Jurij

**Consumption and Pollution in Precision Agriculture Tasks 61** Mariano Gonzalez-de-Soto, Luis Emmi and Pablo Gonzalez-de-

Chapter 1 **Introductory Chapter: Recent Development and Applications of**

## Contents

**Preface XI**


#### Chapter 6 **Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations 91** Redmond Ramin Shamshiri, Ibrahim A. Hameed, Siva K. Balasundram, Desa Ahmad, Cornelia Weltzien and Muhammad Yamin

Preface

bots, and fruit grading robots.

**Overview of the recent developments and applications**

bots, phenotyping robots, unmanned aerial vehicles, etc. **Agricultural robots for field and greenhouse farming**

The robotics industry was originally developed to supplement or replace humans by doing dull, repetitive, dirty, or dangerous work. Robot systems have broad application prospects in industry, agriculture, defense, and other fields. Over the past few decades, extensive research has been conducted on the applications of agricultural robots and automation to a variety of field and greenhouse operations, and technical fundamentals and their feasibility have been also widely demonstrated. Due to the unstructured environment, adverse interference, and complicated and diversified operation process are the key of blocking its commercialization in robotic agricultural operations. Because of the development of automation techniques, smart sensors, and information techniques, some types of agricultural robots have achieved considerable success in recent years. Mapping and localization, navigation and guidance, ro‐ botic grasping, human/robot interaction, object recognition and location, as well as multiro‐ bot systems and collaboration operation are also research hotspots in agricultural robots.

*Agricultural Robots: Fundamentals and Applications* aims to present the recent developments and applications of agricultural robots, and to publish original articles from a wide variety of interdisciplinary perspectives concerning the theory and practice of robots in the fields of agriculture, food, and biosystem engineering. This book can also serve as an international forum to share ideas, problems, and solutions relating to agricultural robots and applica‐ tions in agriculture. In addition to the introductory chapter, this book also presents selected representative chapters covering infield navigation, pest and disease detection, field robots for mechanical weed control, unmanned aerial vehicles, hybrid powered autonomous ro‐

The first chapter, "Introductory chapter: recent developments and applications of agricultur‐ al robots," gives a brief overview of the recent developments and applications of agricultur‐ al robots. With the motivation of high efficiency, high automatization, and increase in the value of agricultural products to the consumer, worldwide research and development ef‐ forts in agricultural robot technology have continued to be developed over the last 40 years. Many excellent research papers have been published focused on harvesting robots, weeding robots, fertilizing robots, transplanting robots, spraying robots, grading robots, grafting ro‐

The second chapter, "Development of a field robot platform for mechanical weed control in greenhouse cultivation of cucumber," introduces a prototype robot that moves on a mono‐ rail along the greenhouse for weed elimination between cucumber plants. The use of a labor

#### **Section 3 Grading Robots for Fruit Industry 117**

Chapter 7 **Multimodal Classification of Mangoes 119** Son V.T. Dao

## Preface

Chapter 6 **Fundamental Research on Unmanned Aerial Vehicles to**

**Section 3 Grading Robots for Fruit Industry 117**

Chapter 7 **Multimodal Classification of Mangoes 119**

Yamin

**VI** Contents

Son V.T. Dao

**Support Precision Agriculture in Oil Palm Plantations 91** Redmond Ramin Shamshiri, Ibrahim A. Hameed, Siva K.

Balasundram, Desa Ahmad, Cornelia Weltzien and Muhammad

The robotics industry was originally developed to supplement or replace humans by doing dull, repetitive, dirty, or dangerous work. Robot systems have broad application prospects in industry, agriculture, defense, and other fields. Over the past few decades, extensive research has been conducted on the applications of agricultural robots and automation to a variety of field and greenhouse operations, and technical fundamentals and their feasibility have been also widely demonstrated. Due to the unstructured environment, adverse interference, and complicated and diversified operation process are the key of blocking its commercialization in robotic agricultural operations. Because of the development of automation techniques, smart sensors, and information techniques, some types of agricultural robots have achieved considerable success in recent years. Mapping and localization, navigation and guidance, ro‐ botic grasping, human/robot interaction, object recognition and location, as well as multiro‐ bot systems and collaboration operation are also research hotspots in agricultural robots.

*Agricultural Robots: Fundamentals and Applications* aims to present the recent developments and applications of agricultural robots, and to publish original articles from a wide variety of interdisciplinary perspectives concerning the theory and practice of robots in the fields of agriculture, food, and biosystem engineering. This book can also serve as an international forum to share ideas, problems, and solutions relating to agricultural robots and applica‐ tions in agriculture. In addition to the introductory chapter, this book also presents selected representative chapters covering infield navigation, pest and disease detection, field robots for mechanical weed control, unmanned aerial vehicles, hybrid powered autonomous ro‐ bots, and fruit grading robots.

#### **Overview of the recent developments and applications**

The first chapter, "Introductory chapter: recent developments and applications of agricultur‐ al robots," gives a brief overview of the recent developments and applications of agricultur‐ al robots. With the motivation of high efficiency, high automatization, and increase in the value of agricultural products to the consumer, worldwide research and development ef‐ forts in agricultural robot technology have continued to be developed over the last 40 years. Many excellent research papers have been published focused on harvesting robots, weeding robots, fertilizing robots, transplanting robots, spraying robots, grading robots, grafting ro‐ bots, phenotyping robots, unmanned aerial vehicles, etc.

#### **Agricultural robots for field and greenhouse farming**

The second chapter, "Development of a field robot platform for mechanical weed control in greenhouse cultivation of cucumber," introduces a prototype robot that moves on a mono‐ rail along the greenhouse for weed elimination between cucumber plants. The use of a labor force that manually pulls out weeds is not an efficient method. It is therefore necessary to select a proper method for effective weed control. In this study, the robot benefits from three arrays of ultrasonic sensors for weed detection and a PIC18 F4550-E/P microcontroller board for processing. Thus, the feedback from the sensors activates a robotic arm that moves inside the rows of cucumber plants and cuts the weeds using rotating blades. In this regard, agri‐ cultural robotic and automation technology plays an essential role in improving the interac‐ tions between human, machine, and plants. This study can contribute to the commercialization of a reliable and affordable robot for automated weed control in green‐ house cultivation of cucumber.

for autonomous robots devoted to weed and pest control in agriculture is modeled and eval‐ uated, and its exhaust emissions are compared with those of an internal combustion enginepowered system. The results demonstrate that the hybrid energy system reduced emissions

The sixth chapter, "Fundamental research on unmanned aerial vehicles (UAVs) to support precision agriculture in oil palm plantations," discusses several considerations and recom‐ mendations for using UAV drones in several precision agriculture applications for oil palms in Malaysia, including health assessment and disease detection, pest monitoring, yield mon‐ itoring, and creation of virtual plantations and dynamic web mapping. With the background of Malaysia as the second largest oil palm exporter, precision agriculture of oil palm is one of the largest markets in Malaysia to benefit from UAV and robotics. To solve these four aspects, the chapter proposes a solution of aerial scanning the plantation using a visible RGB camera and near-infrared, hyperspectral, and multispectral sensors. Furthermore, it relates to stabilizing a fixed-wing Osprey UAV. For this purpose, a case study is presented on stabi‐ lizing a fixed-wing Osprey crop surveillance drone that can be adapted as a research plat‐ form in UAV-based precision agriculture of oil palm. The objective was to design and implement three controllers, including PID, LQR with full-state feedback, and LQR plus an

The seventh chapter, "Multimodal classification of mangos," presents an integrated frame‐ work for the automated grading, sorting, and weighing system of Cat-Chu mangos using features including weight, size, and external defects. This topic is crucial for ensuring high quality from the earliest stages of the crop to postharvest storage and treatments. In the first part of this chapter, an algorithm for estimating mangos' masses is proposed based on the research of the simple, linear correlation between mass and sizes, and the estimation error is relatively small. A review of several methods for automatic selection of threshold values is given, especially Otsu's method and the Valley-Emphasis method. In addition, this chapter constructs an artificial neural network model to classify mangos having multiple types of external defects, and the research results indicate that the proposed system has great poten‐

This book is the result of many collaborating parties, and we gratefully acknowledge all the contributors to this book. We sincerely hope that you will find the chapters as useful and interesting as we did. We also look forward to seeing another technological breakthrough in

> **Jun Zhou and Baohua Zhang** College of Engineering

Nanjing Agricultural University

Nanjing

Preface IX

Jiangsu, PR China

observer to improve automatic flight control of the Osprey drone.

by up to approximately 50%.

**Grading robots for the fruit industry**

tial to be used in a real industrial setting.

agricultural robots in the near future.

The third chapter, "An evaluation of three different infield navigation algorithms," presents three different infield navigation algorithms based on the readings from a LIDAR sensor. The usual autonomously driving agriculture machinery requires a previously known path of movement. This is repeated with each iteration when the crop needs to be treated. This is not always possible if no prior GPS information exists, or an accurate GPS system is not always available. So, different algorithms should be compiled to autonomously drive the machines and evaluate the accuracy for infield navigation. The three different algorithms include a minimal row offset-based algorithm, a least-squares fit approach algorithm, and a trianglebased navigation algorithm. The algorithms are tested on a small field robot. The robot has to drive autonomously through the crop fields with the navigation algorithm, which relies on the distance measurements from the LIDAR sensor and then chooses the optimal path based on the readings. Once the robot reaches the end of the crop line it uses data from the on-board compass (IMU unit) and turns in the next row. Additionally, the accuracy of the three pro‐ posed navigation algorithms was investigated and compared in this work.

The fourth chapter, "Agricultural robot for intelligent detection of Pyralidae insects," intro‐ duces a new scheme to detect and identify Pyralidae insects by using the designed insect monitoring robot to solve the problem of maize borer detection. In this chapter, a robot plat‐ form for pest monitoring is first designed and fabricated. Then, a recognition scheme for Pyralidae insects is presented. The recognition scheme for Pyralidae insects is mainly divid‐ ed into input module, reference image-processing module, image segmentation module, contour extraction module, and target recognition module. Furthermore, the histogram re‐ verse-mapping method and the multitemplate image are used to obtain the general proba‐ bility image superposition. Color histograms are particularly useful for describing images that are difficult to segment automatically and the histogram reverse-mapping method can be used to segment the image or find interesting content in the image. Next, the image is segmented with the Otsu algorithm based on constrained space. Finally, the contour of Pyr‐ alidae insects can be recognized by using the contours and Hu moments to automatically screen and identify the contours. This chapter also compares the new scheme with the mul‐ tistructural element-based crop pest identification method proposed and the general histo‐ gram reverse-mapping method.

The fifth chapter, "Hybrid-powered autonomous robots for reducing both fuel consumption and pollution in precision agriculture tasks," investigates techniques to reduce fuel con‐ sumption in robotic vehicles and reduce atmospheric emissions from these automated sys‐ tems. Agricultural vehicles that use fossil fuels emit significant amounts of atmospheric pollutants. Thus, a solution is examined by designing and assessing a hybrid energy system that omits the alternators from the original vehicle and modifies the agricultural imple‐ ments. Therefore, the PTO power is replaced by electrical power. A hybrid energy system for autonomous robots devoted to weed and pest control in agriculture is modeled and eval‐ uated, and its exhaust emissions are compared with those of an internal combustion enginepowered system. The results demonstrate that the hybrid energy system reduced emissions by up to approximately 50%.

The sixth chapter, "Fundamental research on unmanned aerial vehicles (UAVs) to support precision agriculture in oil palm plantations," discusses several considerations and recom‐ mendations for using UAV drones in several precision agriculture applications for oil palms in Malaysia, including health assessment and disease detection, pest monitoring, yield mon‐ itoring, and creation of virtual plantations and dynamic web mapping. With the background of Malaysia as the second largest oil palm exporter, precision agriculture of oil palm is one of the largest markets in Malaysia to benefit from UAV and robotics. To solve these four aspects, the chapter proposes a solution of aerial scanning the plantation using a visible RGB camera and near-infrared, hyperspectral, and multispectral sensors. Furthermore, it relates to stabilizing a fixed-wing Osprey UAV. For this purpose, a case study is presented on stabi‐ lizing a fixed-wing Osprey crop surveillance drone that can be adapted as a research plat‐ form in UAV-based precision agriculture of oil palm. The objective was to design and implement three controllers, including PID, LQR with full-state feedback, and LQR plus an observer to improve automatic flight control of the Osprey drone.

#### **Grading robots for the fruit industry**

force that manually pulls out weeds is not an efficient method. It is therefore necessary to select a proper method for effective weed control. In this study, the robot benefits from three arrays of ultrasonic sensors for weed detection and a PIC18 F4550-E/P microcontroller board for processing. Thus, the feedback from the sensors activates a robotic arm that moves inside the rows of cucumber plants and cuts the weeds using rotating blades. In this regard, agri‐ cultural robotic and automation technology plays an essential role in improving the interac‐ tions between human, machine, and plants. This study can contribute to the commercialization of a reliable and affordable robot for automated weed control in green‐

The third chapter, "An evaluation of three different infield navigation algorithms," presents three different infield navigation algorithms based on the readings from a LIDAR sensor. The usual autonomously driving agriculture machinery requires a previously known path of movement. This is repeated with each iteration when the crop needs to be treated. This is not always possible if no prior GPS information exists, or an accurate GPS system is not always available. So, different algorithms should be compiled to autonomously drive the machines and evaluate the accuracy for infield navigation. The three different algorithms include a minimal row offset-based algorithm, a least-squares fit approach algorithm, and a trianglebased navigation algorithm. The algorithms are tested on a small field robot. The robot has to drive autonomously through the crop fields with the navigation algorithm, which relies on the distance measurements from the LIDAR sensor and then chooses the optimal path based on the readings. Once the robot reaches the end of the crop line it uses data from the on-board compass (IMU unit) and turns in the next row. Additionally, the accuracy of the three pro‐

The fourth chapter, "Agricultural robot for intelligent detection of Pyralidae insects," intro‐ duces a new scheme to detect and identify Pyralidae insects by using the designed insect monitoring robot to solve the problem of maize borer detection. In this chapter, a robot plat‐ form for pest monitoring is first designed and fabricated. Then, a recognition scheme for Pyralidae insects is presented. The recognition scheme for Pyralidae insects is mainly divid‐ ed into input module, reference image-processing module, image segmentation module, contour extraction module, and target recognition module. Furthermore, the histogram re‐ verse-mapping method and the multitemplate image are used to obtain the general proba‐ bility image superposition. Color histograms are particularly useful for describing images that are difficult to segment automatically and the histogram reverse-mapping method can be used to segment the image or find interesting content in the image. Next, the image is segmented with the Otsu algorithm based on constrained space. Finally, the contour of Pyr‐ alidae insects can be recognized by using the contours and Hu moments to automatically screen and identify the contours. This chapter also compares the new scheme with the mul‐ tistructural element-based crop pest identification method proposed and the general histo‐

The fifth chapter, "Hybrid-powered autonomous robots for reducing both fuel consumption and pollution in precision agriculture tasks," investigates techniques to reduce fuel con‐ sumption in robotic vehicles and reduce atmospheric emissions from these automated sys‐ tems. Agricultural vehicles that use fossil fuels emit significant amounts of atmospheric pollutants. Thus, a solution is examined by designing and assessing a hybrid energy system that omits the alternators from the original vehicle and modifies the agricultural imple‐ ments. Therefore, the PTO power is replaced by electrical power. A hybrid energy system

posed navigation algorithms was investigated and compared in this work.

house cultivation of cucumber.

VIII Preface

gram reverse-mapping method.

The seventh chapter, "Multimodal classification of mangos," presents an integrated frame‐ work for the automated grading, sorting, and weighing system of Cat-Chu mangos using features including weight, size, and external defects. This topic is crucial for ensuring high quality from the earliest stages of the crop to postharvest storage and treatments. In the first part of this chapter, an algorithm for estimating mangos' masses is proposed based on the research of the simple, linear correlation between mass and sizes, and the estimation error is relatively small. A review of several methods for automatic selection of threshold values is given, especially Otsu's method and the Valley-Emphasis method. In addition, this chapter constructs an artificial neural network model to classify mangos having multiple types of external defects, and the research results indicate that the proposed system has great poten‐ tial to be used in a real industrial setting.

This book is the result of many collaborating parties, and we gratefully acknowledge all the contributors to this book. We sincerely hope that you will find the chapters as useful and interesting as we did. We also look forward to seeing another technological breakthrough in agricultural robots in the near future.

#### **Jun Zhou and Baohua Zhang**

College of Engineering Nanjing Agricultural University Nanjing Jiangsu, PR China

**Section 1**

**Overview of the Recent Developments and**

**Applications**

**Overview of the Recent Developments and Applications**

**Chapter 1**

**Provisional chapter**

**Introductory Chapter: Recent Development and**

**Introductory Chapter: Recent Development and** 

DOI: 10.5772/intechopen.81149

The robotics industry was originally developed to supplement or replace humans by doing dull, repetitive, dirty, or dangerous work [1]. Robot systems have broad application prospects in industry, agriculture, defense, and other fields. In the past decades, extensive research has been conducted on the applications of agricultural robots and automation to a variety of field and greenhouse operations, and technical fundamentals and their feasibility have been also widely demonstrated. Due to the unstructured environment, adverse interferences, as well as the complicated and diversified operation process, are the key in blocking its commercializa-

It is well known that the introduction of robotics in agriculture has not had the same success as it has in the manufacturing industry [2]. This is mainly due to the fact that the agricultural environment is much less structured, and the operating agricultural objects are flexible and vary in shape, size, and materials [3]. Consequently, it is more difficult to adopt robots in the

As the development of automation techniques, smart sensors, and information techniques, some types of agricultural robot have achieved considerable success in recent years. Mapping and localization, navigation and guidance, robotic grasping, human-robot interaction, object recognition and location, and multi-robot systems and collaboration operation are the research

> © 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

© 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use,

distribution, and reproduction in any medium, provided the original work is properly cited.

**Applications of Agricultural Robots**

**Applications of Agricultural Robots**

Additional information is available at the end of the chapter

Baohua Zhang and Jun ZhouAdditional information is available at the end of the chapter

Baohua Zhang and Jun Zhou

**1. Introduction**

http://dx.doi.org/10.5772/intechopen.81149

tion in robotic agricultural operations.

hotspots in agricultural robots.

automation of different agricultural processes.

#### **Introductory Chapter: Recent Development and Applications of Agricultural Robots Introductory Chapter: Recent Development and Applications of Agricultural Robots**

DOI: 10.5772/intechopen.81149

Baohua Zhang and Jun Zhou Baohua Zhang and Jun Zhou

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.81149

**1. Introduction**

The robotics industry was originally developed to supplement or replace humans by doing dull, repetitive, dirty, or dangerous work [1]. Robot systems have broad application prospects in industry, agriculture, defense, and other fields. In the past decades, extensive research has been conducted on the applications of agricultural robots and automation to a variety of field and greenhouse operations, and technical fundamentals and their feasibility have been also widely demonstrated. Due to the unstructured environment, adverse interferences, as well as the complicated and diversified operation process, are the key in blocking its commercialization in robotic agricultural operations.

It is well known that the introduction of robotics in agriculture has not had the same success as it has in the manufacturing industry [2]. This is mainly due to the fact that the agricultural environment is much less structured, and the operating agricultural objects are flexible and vary in shape, size, and materials [3]. Consequently, it is more difficult to adopt robots in the automation of different agricultural processes.

As the development of automation techniques, smart sensors, and information techniques, some types of agricultural robot have achieved considerable success in recent years. Mapping and localization, navigation and guidance, robotic grasping, human-robot interaction, object recognition and location, and multi-robot systems and collaboration operation are the research hotspots in agricultural robots.

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. © 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

## **2. Hot research topics of agricultural robots**

#### **2.1. Object recognition for harvesting robots**

In the latest decades, the use of robot systems in agriculture has seen a sharp increment [4]. Object recognition is the precondition for robot grasping. Today, harvesting robot mainly utilizes the methods based on computer vision to achieve recognition and location of objects. Actually, the key step of computer vision is the processing procedures of vision images which were acquired by visual sensors including our object. Not only the object but also many of useless and interferential information were present in the acquired image. These redundant information extremely may slow down the speed and accuracy of recognition. Thus, how to extract useful information while detecting the fruit or vegetable became the most important part.

tends to be diversified, and more attention has been paid to the external and internal quality of agricultural products [11]. Due to the ever-growing demand for food safety and security, the automated grading of agricultural products is playing an increasingly important role in agricultural field. The automatic robots are able to remove the need for human operators to carry out heavy, monotonous, and dangerous operations and give future generations the possibility to achieve economic sustainability in small high value farming operations. The principle of the automated grading robots is to apply machine vision to detect the external defects of the fruits and vegetables and to use spectral/hyperspectral imaging technology to measure the internal quality [12]. Therefore, it can be said that operations in grading system for fruits and vegetables became highly automated with the machine vision, near infrared, and robotics technologies. The automated grading robots of fruits and vegetables have been

Introductory Chapter: Recent Development and Applications of Agricultural Robots

http://dx.doi.org/10.5772/intechopen.81149

5

Grasping and holding of objects are the fundamental capabilities and key tasks for robots and robotic manipulators. The grippers are the most important components of robots for many manipulation tasks, since they serve as mechanical interface between the robots and their environment. The fragile structure of the fruit or vegetable body makes them susceptible to bruising caused by the aggressiveness of harvest and postharvest processes. Grasping without damaging the fruits is a key barrier to the replacement of manual labor by robotic

The agricultural tasks are always carried out in an outdoor unstructured environment [13]; the agricultural operations and sequence of motions are different from one task to the other; the manipulating objects of robots and grippers are flexible and damageable plants or fruits, and they are highly variable in shapes, sizes, and structures. The abovementioned factors put forward higher requirement to the machinery of robotic grippers, smart sensors, and grasping

Robotic grippers design attempts to simulate the advantage aptitudes with the aim of grasping any kind of objects by copying human abilities such as sense of touch and visual perception [1]. By integrating various sensors, the robotic grippers can not only manipulate the workpiece but also analyze and the grippers can also conduct online decisionmaking based on the fusion sensory data. **Figure 1** shows the gripper development trend

As the requirement of the agricultural robots, the research topics have been extended to very large areas, including precision farming, cloud computing, human-robot interaction, sensing and control, robot design and optimization, multi-robot systems and collaboration operation, weeding robots, fertilizing robots, transplanting robots, spraying robots, grafting robots,

investigated over the past few years.

control strategy in agricultural tasks.

**2.5. Other hot research topics**

phenotyping robots, and unmanned aerial vehicles.

harvesting.

for sensors.

**2.4. Robotic grippers and grasping control**

In the past, researches gradually put forward the concept of feature engineering [5]. The available information in the object recognition is called feature. By extracting and using those features, the distinction between the object and their growing environment can be figured out. However, there are some uncontrollable and complex factories near the fruit and vegetable. The occlusion of leaves and stems, the illumination unevenness on fruit surface and the high variability of fruit color enhanced the difficulty of recognition and location. Therefore, a robust algorithm must be developed and applied to object recognition for harvesting robots.

#### **2.2. Simultaneous localization and mapping for agricultural environments**

Simultaneous localization and mapping (SLAM) is a critic problem in robotics automatic navigation and positioning, using multiple sensors to get external information, for the purpose of obtaining a consistent map of the environment and at the same time recognizing itself within this map [6]. From the choice of different sensors, the solutions to SLAM problem can be divided into three main fundamental categories: visual, lidar, and sensor fusion SLAM. Different kinds of SLAM algorithms have been in demand in many scenarios for a long time, such as VR/AR equipment, indoor autonomous mobile robots, and unmanned vehicle [7]. With the development and maturity of SLAM technology, SLAM has been applied to agriculture. It is a combination of traditional fields and new technologies. The most significant requirement for many applications in precision agriculture is the ability to accurately locate a moving vehicle [8]. By solving the problem of SLAM, the vehicle can map the targeted areas, locate itself, and fulfill tasks such as spraying, weeding, and mowing [8, 9]. As a consequence, the application of SLAM technology in agriculture can lighten framers' workload, complete dangerous tasks instead of human labor, and increase productivity [10]. Overall, SLAM technology has a bright prospect in the automation and intelligence of agriculture.

#### **2.3. Agricultural product quality sensing for grading robots**

Nowadays, fruits and vegetables have become an indispensable food in our daily life. Fruits and vegetables are not only important for people to adjust their taste but also rich in healthy vitamins, fiber, and trace elements. In recent years, consumer demand for fruits and vegetables tends to be diversified, and more attention has been paid to the external and internal quality of agricultural products [11]. Due to the ever-growing demand for food safety and security, the automated grading of agricultural products is playing an increasingly important role in agricultural field. The automatic robots are able to remove the need for human operators to carry out heavy, monotonous, and dangerous operations and give future generations the possibility to achieve economic sustainability in small high value farming operations. The principle of the automated grading robots is to apply machine vision to detect the external defects of the fruits and vegetables and to use spectral/hyperspectral imaging technology to measure the internal quality [12]. Therefore, it can be said that operations in grading system for fruits and vegetables became highly automated with the machine vision, near infrared, and robotics technologies. The automated grading robots of fruits and vegetables have been investigated over the past few years.

#### **2.4. Robotic grippers and grasping control**

**2. Hot research topics of agricultural robots**

In the latest decades, the use of robot systems in agriculture has seen a sharp increment [4]. Object recognition is the precondition for robot grasping. Today, harvesting robot mainly utilizes the methods based on computer vision to achieve recognition and location of objects. Actually, the key step of computer vision is the processing procedures of vision images which were acquired by visual sensors including our object. Not only the object but also many of useless and interferential information were present in the acquired image. These redundant information extremely may slow down the speed and accuracy of recognition. Thus, how to extract useful information while detecting the fruit or vegetable became the most important part.

In the past, researches gradually put forward the concept of feature engineering [5]. The available information in the object recognition is called feature. By extracting and using those features, the distinction between the object and their growing environment can be figured out. However, there are some uncontrollable and complex factories near the fruit and vegetable. The occlusion of leaves and stems, the illumination unevenness on fruit surface and the high variability of fruit color enhanced the difficulty of recognition and location. Therefore, a robust algorithm must be developed and applied to object recognition for harvesting robots.

Simultaneous localization and mapping (SLAM) is a critic problem in robotics automatic navigation and positioning, using multiple sensors to get external information, for the purpose of obtaining a consistent map of the environment and at the same time recognizing itself within this map [6]. From the choice of different sensors, the solutions to SLAM problem can be divided into three main fundamental categories: visual, lidar, and sensor fusion SLAM. Different kinds of SLAM algorithms have been in demand in many scenarios for a long time, such as VR/AR equipment, indoor autonomous mobile robots, and unmanned vehicle [7]. With the development and maturity of SLAM technology, SLAM has been applied to agriculture. It is a combination of traditional fields and new technologies. The most significant requirement for many applications in precision agriculture is the ability to accurately locate a moving vehicle [8]. By solving the problem of SLAM, the vehicle can map the targeted areas, locate itself, and fulfill tasks such as spraying, weeding, and mowing [8, 9]. As a consequence, the application of SLAM technology in agriculture can lighten framers' workload, complete dangerous tasks instead of human labor, and increase productivity [10]. Overall, SLAM tech-

**2.2. Simultaneous localization and mapping for agricultural environments**

nology has a bright prospect in the automation and intelligence of agriculture.

Nowadays, fruits and vegetables have become an indispensable food in our daily life. Fruits and vegetables are not only important for people to adjust their taste but also rich in healthy vitamins, fiber, and trace elements. In recent years, consumer demand for fruits and vegetables

**2.3. Agricultural product quality sensing for grading robots**

**2.1. Object recognition for harvesting robots**

4 Agricultural Robots - Fundamentals and Applications

Grasping and holding of objects are the fundamental capabilities and key tasks for robots and robotic manipulators. The grippers are the most important components of robots for many manipulation tasks, since they serve as mechanical interface between the robots and their environment. The fragile structure of the fruit or vegetable body makes them susceptible to bruising caused by the aggressiveness of harvest and postharvest processes. Grasping without damaging the fruits is a key barrier to the replacement of manual labor by robotic harvesting.

The agricultural tasks are always carried out in an outdoor unstructured environment [13]; the agricultural operations and sequence of motions are different from one task to the other; the manipulating objects of robots and grippers are flexible and damageable plants or fruits, and they are highly variable in shapes, sizes, and structures. The abovementioned factors put forward higher requirement to the machinery of robotic grippers, smart sensors, and grasping control strategy in agricultural tasks.

Robotic grippers design attempts to simulate the advantage aptitudes with the aim of grasping any kind of objects by copying human abilities such as sense of touch and visual perception [1]. By integrating various sensors, the robotic grippers can not only manipulate the workpiece but also analyze and the grippers can also conduct online decisionmaking based on the fusion sensory data. **Figure 1** shows the gripper development trend for sensors.

#### **2.5. Other hot research topics**

As the requirement of the agricultural robots, the research topics have been extended to very large areas, including precision farming, cloud computing, human-robot interaction, sensing and control, robot design and optimization, multi-robot systems and collaboration operation, weeding robots, fertilizing robots, transplanting robots, spraying robots, grafting robots, phenotyping robots, and unmanned aerial vehicles.

**References**

2011;**9**(4):1130

2009. pp. 1684-1689

pp. 3-21

Conference on. Vol. 2. IEEE; 2005

Journal of Systems and Software. 1999;**49**(1):3-15

Intelligent and Robotic Systems. 2009;**55**(4-5):299-321

ture. In: Asabe Annual International Meeting; 2010

Agriculture 9; 2009. pp. 857-866

International. 2014;**62**(62):326-343

choke harvesting. Robotics. 2013;**2**(4):217-230

2016;**112**:121-133

sequences. IEEE Transactions on Robotics. 2012;**28**(5):1188-1197

sugar beet field. Biosystems Engineering. 2011;**109**(4):357-368

[1] Blanes C, Mellado M, Ortiz C, et al. Technologies for robot grippers in pick and place operations for fresh fruits and vegetables. Spanish Journal of Agricultural Research.

Introductory Chapter: Recent Development and Applications of Agricultural Robots

http://dx.doi.org/10.5772/intechopen.81149

7

[2] Muscato G, Prestiflippo M. A fuzzy-PD for the position and attitude control of an underwater robot. Emerging Technologies and Factory Automation, ETFA 2005. 10th IEEE

[3] Sam R, Nefti S. Design and development of flexible robotic gripper for handling food products. In: International Conference on Control, Automation, Robotics and Vision. IEEE;

[4] Reis LP, Almeida F, Mota L, et al. Coordination in multi-robot systems: Applications in robotic soccer. In: Agents and Artificial Intelligence. Berlin Heidelberg: Springer; 2012.

[5] Turner CR, Fuggetta A, Lavazza L, et al. A conceptual basis for feature engineering.

[6] Galvez-López D, Tardos JD. Bags of binary words for fast place recognition in image

[7] Artieda J, Sebastian JM, Campoy P, et al. Visual 3-D SLAM from UAVs. Journal of

[8] Libby J, Kantor G. Accurate GPS-free positioning of utility vehicles for specialty agricul-

[9] Bakker T, Van AK, Bontsema J, et al. Autonomous navigation using a robot platform in a

[10] Griepentrog HW et al. Safe and reliable: Further development of a field robot. Precision

[11] Li J, Chen L, Huang W, et al. Multispectral detection of skin defects of bi-colored peaches based on vis–NIR hyperspectral imaging. Postharvest Biology and Technology.

[12] Zhang B, Huang W, Li J, et al. Principles, developments and applications of computer vision for external quality inspection of fruits and vegetables: A review. Food Research

[13] Longo D, Muscato G. Design and simulation of two robotic systems for automatic arti-

**Figure 1.** Gripper development trend for sensors.

## **3. Conclusions**

With the motivation of high efficiency, high automatization, and increasing the value of the agricultural products to the consumer, worldwide research and development efforts in agricultural robot technology continue to be developed in the last 40 years. Many excellent research papers have been published focused on harvesting robots, weeding robots, fertilizing robots, transplanting robots, spraying robots, grading robots, grafting robots, phenotyping robots, unmanned aerial vehicles, etc.

This book intends to provide the reader with a comprehensive overview of the current stateof-the-art agricultural robots, fundamentals, and applications in robotic agricultural operations. The challenges and directions of agricultural robots for future research and development will be also reported and formulated in this book.

### **Author details**

Baohua Zhang and Jun Zhou\*

\*Address all correspondence to: zhoujun@njau.edu.cn

College of Engineering, Nanjing Agricultural University, Nanjing, China

#### **References**

**3. Conclusions**

**Author details**

Baohua Zhang and Jun Zhou\*

robots, unmanned aerial vehicles, etc.

**Figure 1.** Gripper development trend for sensors.

6 Agricultural Robots - Fundamentals and Applications

will be also reported and formulated in this book.

\*Address all correspondence to: zhoujun@njau.edu.cn

College of Engineering, Nanjing Agricultural University, Nanjing, China

With the motivation of high efficiency, high automatization, and increasing the value of the agricultural products to the consumer, worldwide research and development efforts in agricultural robot technology continue to be developed in the last 40 years. Many excellent research papers have been published focused on harvesting robots, weeding robots, fertilizing robots, transplanting robots, spraying robots, grading robots, grafting robots, phenotyping

This book intends to provide the reader with a comprehensive overview of the current stateof-the-art agricultural robots, fundamentals, and applications in robotic agricultural operations. The challenges and directions of agricultural robots for future research and development


**Section 2**

**Agricultural Robots for Field and Greenhouse**

**Farming**

**Agricultural Robots for Field and Greenhouse Farming**

**Chapter 2**

**Provisional chapter**

**Development of a Field Robot Platform for Mechanical**

**Development of a Field Robot Platform for Mechanical** 

DOI: 10.5772/intechopen.80935

**Weed Control in Greenhouse Cultivation of Cucumber**

A prototype robot that moves on a monorail along the greenhouse for weed elimination between cucumber plants was designed and developed. The robot benefits from three arrays of ultrasonic sensors for weed detection and a PIC18 F4550-E/P microcontroller board for processing. The feedback from the sensors activates a robotic arm, which moves inside the rows of the cucumber plants for cutting the weeds using rotating blades. Several experiments were carried out inside a greenhouse to find the best combination of arm motor (AM) speed, blade rotation (BR) speed, and blade design. We assigned three BR speeds of 3500, 2500, and 1500 rpm, and two AM speed of 10 and 30 rpm to three blade designs of S-shape, triangular shape, and circular shape. Results indicated that different types of blades, different BR speed, and different AM speed had significant effects (P < 0.05) on the percentage of weeds cut (PWC); however, no significant interaction effects were observed. The comparison between the interaction effect of the factors (three blade designs, three BR speeds, and two AM speeds) showed that maximum mean PWC was equal to 78.2% with standard deviation of 3.9% and was achieved with the S-shape blade when the BR speed was 3500 rpm, and the AM speed was 10 rpm. Using this setting, the maximum PWC that the robot achieved in a random experiment was 95%. The lowest mean PWC was observed with the triangular-shaped blade (mean of 50.39% and SD = 1.86), which resulted from BR speed of 1500 rpm and AM speed of 30 rpm. This study can contribute to the commercialization of a reliable and affordable

robot for automated weed control in greenhouse cultivation of cucumber.

**Keywords:** agricultural robot, weed control, cucumber, greenhouse

**Weed Control in Greenhouse Cultivation of Cucumber**

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

© 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use,

distribution, and reproduction in any medium, provided the original work is properly cited.

Amid Heravi, Desa Ahmad, Ibrahim A. Hameed,

Amid Heravi, Desa Ahmad, Ibrahim A. Hameed, Redmond Ramin Shamshiri, Siva K. Balasundram

Siva K. Balasundram and Muhammad Yamin

Additional information is available at the end of the chapter

Additional information is available at the end of the chapter

Redmond Ramin Shamshiri,

and Muhammad Yamin

**Abstract**

http://dx.doi.org/10.5772/intechopen.80935

#### **Development of a Field Robot Platform for Mechanical Weed Control in Greenhouse Cultivation of Cucumber Development of a Field Robot Platform for Mechanical Weed Control in Greenhouse Cultivation of Cucumber**

DOI: 10.5772/intechopen.80935

Amid Heravi, Desa Ahmad, Ibrahim A. Hameed, Redmond Ramin Shamshiri, Siva K. Balasundram and Muhammad Yamin Amid Heravi, Desa Ahmad, Ibrahim A. Hameed, Redmond Ramin Shamshiri, Siva K. Balasundram and Muhammad Yamin

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.80935

#### **Abstract**

A prototype robot that moves on a monorail along the greenhouse for weed elimination between cucumber plants was designed and developed. The robot benefits from three arrays of ultrasonic sensors for weed detection and a PIC18 F4550-E/P microcontroller board for processing. The feedback from the sensors activates a robotic arm, which moves inside the rows of the cucumber plants for cutting the weeds using rotating blades. Several experiments were carried out inside a greenhouse to find the best combination of arm motor (AM) speed, blade rotation (BR) speed, and blade design. We assigned three BR speeds of 3500, 2500, and 1500 rpm, and two AM speed of 10 and 30 rpm to three blade designs of S-shape, triangular shape, and circular shape. Results indicated that different types of blades, different BR speed, and different AM speed had significant effects (P < 0.05) on the percentage of weeds cut (PWC); however, no significant interaction effects were observed. The comparison between the interaction effect of the factors (three blade designs, three BR speeds, and two AM speeds) showed that maximum mean PWC was equal to 78.2% with standard deviation of 3.9% and was achieved with the S-shape blade when the BR speed was 3500 rpm, and the AM speed was 10 rpm. Using this setting, the maximum PWC that the robot achieved in a random experiment was 95%. The lowest mean PWC was observed with the triangular-shaped blade (mean of 50.39% and SD = 1.86), which resulted from BR speed of 1500 rpm and AM speed of 30 rpm. This study can contribute to the commercialization of a reliable and affordable robot for automated weed control in greenhouse cultivation of cucumber.

**Keywords:** agricultural robot, weed control, cucumber, greenhouse

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. © 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### **1. Introduction**

The demand for off-season cultivation of fruits and vegetables require different aspects of automation and robotics in closed-field plant production environments like greenhouses [1]. Modern greenhouse bioproduction systems are required to exhibit integration of automation, biological culture practices, and control systems through the concept of Automation-Culture-Environmentoriented SYStems analysis (ACESYS) as defined in [2, 3]. The growth condition for Solanaceae vegetables in the greenhouse provides the leeway for the growth of other plants as well. In greenhouse cultivation of Cucumber (*Cucumis sativus*), the growth of weeds like cleavers, amaranth, camelthorn, grass quack, and oat wild decreases the final crop yield and quality. These weeds compete with cucumbers for nutrients, water, and photosynthesis. During the growing period, weeds use a large portion of water and nutrient, and because of their physiological properties, they grow simultaneously and rapidly with the original plant. It is, therefore, necessary to eliminate them before causing serious damage to the original plants. Various mechanical and chemical methods, as well as cultivation techniques, have been proposed to prevent the growth of weeds, including mechanical techniques, hand picking, spraying, environment heating, herbicides and biocontrols, and soilless cultural practices. For example, weed biocontrol is the suppression of weeds by insects and microorganisms that feed on the target plants or otherwise parasitize them. The success in this method is not always guaranteed because biocontrol is species specific, and there are hundreds of serious weed species. Cultural control includes those management practices that modify the agro-ecosystem to make the pasture, crop, or forest ecosystem resistant to weed establishment, i.e., integrating sheep or goats to browse brush species and fowl to graze herbs and grasses [4]. Prior to the development of modern herbicides, rancher and forest managers relied mainly on mechanical methods of weed control, such as grubbing, bulldozing, dragging, cabling, and mowing. Compared to mechanical weed control methods, herbicides are more effective at a lower cost. Herbicidal weed control results in greater grass production in pastures than does clipping of weeds [5]. In order to apply chemical directly to the weed's vascular tissue, a direct chemical application end effector is required to cut the weed's stem and spread the chemical on the cut surface. An example of such application can be found in [6] where a prototype weed control robot was developed to spray weeds in cotton plants in the seed line. A real-time intelligent weed control system was introduced in [7] for selective herbicide application to in-row weeds using machine vision and chemical application. A minirobot to perform spraying activities based on machine vision and fuzzy logic has been described in [8, 9]. More examples of autonomous vehicle robot for spraying the weeds can be found in [10–12].

the interactions between human, machine, and plants [15]. For example, the prevention of musculoskeletal disorders in manual harvesting operations in Dutch greenhouses has motivated various researchers for replacement of human labor by automatons robot for picking cucumber [16] and sweet pepper [13] fruits. Automation is a viable and sometimes necessary method to ensure maximum profits with minimum costs. In fact, one of the main purposes of agricultural automation has been always concerned with the substitution of human workforce by robots or mechanized systems that can handle the tasks more accurately and uniformly at

Development of a Field Robot Platform for Mechanical Weed Control in Greenhouse Cultivation…

http://dx.doi.org/10.5772/intechopen.80935

13

Research and development in agricultural robotics date back to 1980s, with Japan, the Netherlands, and the USA as the pioneer countries. Example of such research works included the works of [7, 23] for robotic weed control and automated harvesting of tomato. Development of an autonomous weeding machine requires a vision system capable of detecting and locating the position of the crop. Such vision system should be able to recognize the accurate position of the plant stem and protects it during the weed control [24]. A near-ground image capturing and processing technique to detect broad-leaved weeds in cereal crops under actual field conditions has been reported in the work of [25]. Here, the researchers proposed a method that uses color information to discriminate between vegetation and background, while shape analysis techniques were applied to distinguish between crop and weeds. Shape features of the radish plant and weed were investigated by [26]. They proposed a machine vision system using a charge coupled device camera for the weed detection in a radish farm

A combined method of color and shape features for sugar beet weed segmentation was proposed by [27] with 92% success rate in classification. This rate increased to 96% by adding two shape features. Another approach extracted a correlation between the three main color components R, G and B, which constitute weeds and sugar beet color classes by means of discriminant analysis [28]. Their method resulted in different classification success rates between 77 and 98%. The segmentation of weeds and soybean seedlings by CCD images in the field was studied by [29]. Texture features of weed species have been applied for distinguishing weed species by [30] with grass and broadleaf classification accuracies of 93 and 85%, respectively. Textural image analysis was used to detect weeds in the grass [31]. Gabor wavelet features of NIR images of apples were extracted for quality inspection and used as input to kernel PCA [32]. Kernel PCA first maps the nonlinear features to linear space, and then, PCA is applied to separate the image Gabor wavelet (5 scales and 8 orientations) combined with kernel PCA that had the highest recognition rate (90.5%). Improvements in vision-based control system [13, 33–36] have enabled several applications of robotic manipulators for greenhouse and orchard tasks and have contributed to the decrease in workload and labor's fatigue, while improving the efficiency and safety of the operations. These achievements were considered a challenge in the earlier agricultural robotics works [23, 37, 38]. For example, spray equipment for weed control has been developed with vertical spray booms that increase the deposition in the canopy [39–41]. Some of these alternatives are self-propelled vehicles such as Fumimatic® (IDM S.L, Almería, Spain) and Tizona (Carretillas Amate S.L., Almería, Spain), or autonomous vehicles such as Fitorobot (Universidad de Almería, Cadia S.L., Almería, Spain) that have been designed specifically to move without difficulty over loose soils and in spaces with

resulting 92% success rate of recognition for radish and 98% for weeds.

a lower cost and higher efficiency [17–22].

The use of labor force that manually pulls out the weeds is still practiced by local growers. This is, however, not an efficient method since the availability of the skilled workforce that accepts repetitive tasks in the harsh greenhouse and field conditions impose uncertainties and timeliness costs [13]. It is, therefore, necessary to select a proper method for effective weed control. The trends in the agricultural robotics in the past 10 years show that automation of plant trimming with simultaneous localization and mapping techniques will change the industry in future [14]. The available time, labor, equipment, costs, and types of weeds and the areas infested need to be considered when planning a weed control program. In this regard, agricultural robotic and automation technology plays an essential role in improving the interactions between human, machine, and plants [15]. For example, the prevention of musculoskeletal disorders in manual harvesting operations in Dutch greenhouses has motivated various researchers for replacement of human labor by automatons robot for picking cucumber [16] and sweet pepper [13] fruits. Automation is a viable and sometimes necessary method to ensure maximum profits with minimum costs. In fact, one of the main purposes of agricultural automation has been always concerned with the substitution of human workforce by robots or mechanized systems that can handle the tasks more accurately and uniformly at a lower cost and higher efficiency [17–22].

**1. Introduction**

12 Agricultural Robots - Fundamentals and Applications

The demand for off-season cultivation of fruits and vegetables require different aspects of automation and robotics in closed-field plant production environments like greenhouses [1]. Modern greenhouse bioproduction systems are required to exhibit integration of automation, biological culture practices, and control systems through the concept of Automation-Culture-Environmentoriented SYStems analysis (ACESYS) as defined in [2, 3]. The growth condition for Solanaceae vegetables in the greenhouse provides the leeway for the growth of other plants as well. In greenhouse cultivation of Cucumber (*Cucumis sativus*), the growth of weeds like cleavers, amaranth, camelthorn, grass quack, and oat wild decreases the final crop yield and quality. These weeds compete with cucumbers for nutrients, water, and photosynthesis. During the growing period, weeds use a large portion of water and nutrient, and because of their physiological properties, they grow simultaneously and rapidly with the original plant. It is, therefore, necessary to eliminate them before causing serious damage to the original plants. Various mechanical and chemical methods, as well as cultivation techniques, have been proposed to prevent the growth of weeds, including mechanical techniques, hand picking, spraying, environment heating, herbicides and biocontrols, and soilless cultural practices. For example, weed biocontrol is the suppression of weeds by insects and microorganisms that feed on the target plants or otherwise parasitize them. The success in this method is not always guaranteed because biocontrol is species specific, and there are hundreds of serious weed species. Cultural control includes those management practices that modify the agro-ecosystem to make the pasture, crop, or forest ecosystem resistant to weed establishment, i.e., integrating sheep or goats to browse brush species and fowl to graze herbs and grasses [4]. Prior to the development of modern herbicides, rancher and forest managers relied mainly on mechanical methods of weed control, such as grubbing, bulldozing, dragging, cabling, and mowing. Compared to mechanical weed control methods, herbicides are more effective at a lower cost. Herbicidal weed control results in greater grass production in pastures than does clipping of weeds [5]. In order to apply chemical directly to the weed's vascular tissue, a direct chemical application end effector is required to cut the weed's stem and spread the chemical on the cut surface. An example of such application can be found in [6] where a prototype weed control robot was developed to spray weeds in cotton plants in the seed line. A real-time intelligent weed control system was introduced in [7] for selective herbicide application to in-row weeds using machine vision and chemical application. A minirobot to perform spraying activities based on machine vision and fuzzy logic has been described in [8, 9]. More

examples of autonomous vehicle robot for spraying the weeds can be found in [10–12].

The use of labor force that manually pulls out the weeds is still practiced by local growers. This is, however, not an efficient method since the availability of the skilled workforce that accepts repetitive tasks in the harsh greenhouse and field conditions impose uncertainties and timeliness costs [13]. It is, therefore, necessary to select a proper method for effective weed control. The trends in the agricultural robotics in the past 10 years show that automation of plant trimming with simultaneous localization and mapping techniques will change the industry in future [14]. The available time, labor, equipment, costs, and types of weeds and the areas infested need to be considered when planning a weed control program. In this regard, agricultural robotic and automation technology plays an essential role in improving Research and development in agricultural robotics date back to 1980s, with Japan, the Netherlands, and the USA as the pioneer countries. Example of such research works included the works of [7, 23] for robotic weed control and automated harvesting of tomato. Development of an autonomous weeding machine requires a vision system capable of detecting and locating the position of the crop. Such vision system should be able to recognize the accurate position of the plant stem and protects it during the weed control [24]. A near-ground image capturing and processing technique to detect broad-leaved weeds in cereal crops under actual field conditions has been reported in the work of [25]. Here, the researchers proposed a method that uses color information to discriminate between vegetation and background, while shape analysis techniques were applied to distinguish between crop and weeds. Shape features of the radish plant and weed were investigated by [26]. They proposed a machine vision system using a charge coupled device camera for the weed detection in a radish farm resulting 92% success rate of recognition for radish and 98% for weeds.

A combined method of color and shape features for sugar beet weed segmentation was proposed by [27] with 92% success rate in classification. This rate increased to 96% by adding two shape features. Another approach extracted a correlation between the three main color components R, G and B, which constitute weeds and sugar beet color classes by means of discriminant analysis [28]. Their method resulted in different classification success rates between 77 and 98%. The segmentation of weeds and soybean seedlings by CCD images in the field was studied by [29]. Texture features of weed species have been applied for distinguishing weed species by [30] with grass and broadleaf classification accuracies of 93 and 85%, respectively. Textural image analysis was used to detect weeds in the grass [31]. Gabor wavelet features of NIR images of apples were extracted for quality inspection and used as input to kernel PCA [32]. Kernel PCA first maps the nonlinear features to linear space, and then, PCA is applied to separate the image Gabor wavelet (5 scales and 8 orientations) combined with kernel PCA that had the highest recognition rate (90.5%). Improvements in vision-based control system [13, 33–36] have enabled several applications of robotic manipulators for greenhouse and orchard tasks and have contributed to the decrease in workload and labor's fatigue, while improving the efficiency and safety of the operations. These achievements were considered a challenge in the earlier agricultural robotics works [23, 37, 38]. For example, spray equipment for weed control has been developed with vertical spray booms that increase the deposition in the canopy [39–41]. Some of these alternatives are self-propelled vehicles such as Fumimatic® (IDM S.L, Almería, Spain) and Tizona (Carretillas Amate S.L., Almería, Spain), or autonomous vehicles such as Fitorobot (Universidad de Almería, Cadia S.L., Almería, Spain) that have been designed specifically to move without difficulty over loose soils and in spaces with a large number of obstacles [41]. These vehicles rely on inductive sensors to follow metal pipes buried in the soil. Few studies have addressed the navigation problem of vehicles in greenhouses operating completely autonomously [9, 11, 15]. The main challenge of these systems is that localization approaches needed for feeding the closed-loop controllers would lead to inaccurate measurements after a few steps fail for long trajectories [42]. A stereovision system along with an image processing algorithm was used to recognize the weeds and also to estimate their location in the field. In order to experiment with vision sensors and agricultural robots, [13] created a completely simulated environment in V-REP, ROS, and MATLAB for improvement of plant/fruit scanning and visual servoing task through an easy testing and debugging of control algorithms with zero damage risk to the real robot and to the actual equipment. In another study, [43] designed a field survey mobile robot platform based for navigating inside greenhouses and open-field cultivation for automated image acquisition. A functional model shown in **Figure 1** was introduced by [44] in the field test of an autonomous robot for deleafing cucumber plants grown in a high-wire cultivation system. This model was also adapted and used by [13] for the robotic harvesting of sweet pepper and on a greenhouse field survey mobile platform [43]. Artificial neural networks have also been used by many researchers to discriminate weeds [45, 46] with machine vision as shown in **Figure 2**. A fixedposition weed robot was presented by [47], which is interfaced to a standard belt-conveyor displacement system and provides the robot with pallets containing the crops. These reviews indicate that a commercial robotic platform for the elimination of weeds in a cucumber greenhouse has not been materialized yet. In addition, most of the research works in the area of robotic weed control are applicable prior to the plant growth or in some cases when the main plant height is between 0.2 and 0.3 m.

The overall objective of this study was to design and develop an affordable robotic weed control system for application in greenhouse cultivation of cucumbers where plants can reach to a height of 10 m. Our design is based on mechanical weed removal techniques without using chemical materials. The specific objectives were to determine (i) the best blade design for cutting the weeds among cultivation rows, (ii) the best blade rotation (BR) speed, and (iii)

Development of a Field Robot Platform for Mechanical Weed Control in Greenhouse Cultivation…

http://dx.doi.org/10.5772/intechopen.80935

15

A flowchart of the methodology is shown in **Figure 3**. A prototype robot was designed using AutoCAD software 2011 v18.1 (Autodesk Inc., San Rafael, CA, USA). Schematic views of the prototype robot, as well as the corresponding dimensions and parts are shown and illustrated in **Figures 4** and **5**. The main mechanical components of the robotic platform consist of a monorail, main chassis, ball bearings, wheels, arms, blade, and adjusting mechanism. Major electrical components include DC motors, microswitches, a 12 V 7.2–9 amp sealed lead acid battery, SRF05 ultrasonic sensors, pic 18F4550 microcontroller, and 2 × 24 LCD monitor (**Figure 6**). We began with the design of a monorail that was responsible to support the robot navigations and stops between two cucumber rows inside the greenhouse. The monorail has a width of 0.06 m and was placed 0.4 m above the ground (**Figure 5A** and **B**). The algorithm for robot navigation between two consecutive stops points on the monorail is also illustrated in the flowchart of **Figure 5**. Right after the robot is switched on, it starts moving on the monorail that is fixed along the greenhouse from one row to another. Upon reaching the first

the best arm motor (AM) speed.

**2. Materials and methods**

**2.1. Overview of the prototype robot weed**

**Figure 2.** Mechatronic paradigm followed in this research, adapted from [41].

**Figure 1.** Task sequence during leaf picking of cucumber, adapted from [44].

Development of a Field Robot Platform for Mechanical Weed Control in Greenhouse Cultivation… http://dx.doi.org/10.5772/intechopen.80935 15

**Figure 2.** Mechatronic paradigm followed in this research, adapted from [41].

The overall objective of this study was to design and develop an affordable robotic weed control system for application in greenhouse cultivation of cucumbers where plants can reach to a height of 10 m. Our design is based on mechanical weed removal techniques without using chemical materials. The specific objectives were to determine (i) the best blade design for cutting the weeds among cultivation rows, (ii) the best blade rotation (BR) speed, and (iii) the best arm motor (AM) speed.

#### **2. Materials and methods**

a large number of obstacles [41]. These vehicles rely on inductive sensors to follow metal pipes buried in the soil. Few studies have addressed the navigation problem of vehicles in greenhouses operating completely autonomously [9, 11, 15]. The main challenge of these systems is that localization approaches needed for feeding the closed-loop controllers would lead to inaccurate measurements after a few steps fail for long trajectories [42]. A stereovision system along with an image processing algorithm was used to recognize the weeds and also to estimate their location in the field. In order to experiment with vision sensors and agricultural robots, [13] created a completely simulated environment in V-REP, ROS, and MATLAB for improvement of plant/fruit scanning and visual servoing task through an easy testing and debugging of control algorithms with zero damage risk to the real robot and to the actual equipment. In another study, [43] designed a field survey mobile robot platform based for navigating inside greenhouses and open-field cultivation for automated image acquisition. A functional model shown in **Figure 1** was introduced by [44] in the field test of an autonomous robot for deleafing cucumber plants grown in a high-wire cultivation system. This model was also adapted and used by [13] for the robotic harvesting of sweet pepper and on a greenhouse field survey mobile platform [43]. Artificial neural networks have also been used by many researchers to discriminate weeds [45, 46] with machine vision as shown in **Figure 2**. A fixedposition weed robot was presented by [47], which is interfaced to a standard belt-conveyor displacement system and provides the robot with pallets containing the crops. These reviews indicate that a commercial robotic platform for the elimination of weeds in a cucumber greenhouse has not been materialized yet. In addition, most of the research works in the area of robotic weed control are applicable prior to the plant growth or in some cases when the main

plant height is between 0.2 and 0.3 m.

14 Agricultural Robots - Fundamentals and Applications

**Figure 1.** Task sequence during leaf picking of cucumber, adapted from [44].

#### **2.1. Overview of the prototype robot weed**

A flowchart of the methodology is shown in **Figure 3**. A prototype robot was designed using AutoCAD software 2011 v18.1 (Autodesk Inc., San Rafael, CA, USA). Schematic views of the prototype robot, as well as the corresponding dimensions and parts are shown and illustrated in **Figures 4** and **5**. The main mechanical components of the robotic platform consist of a monorail, main chassis, ball bearings, wheels, arms, blade, and adjusting mechanism. Major electrical components include DC motors, microswitches, a 12 V 7.2–9 amp sealed lead acid battery, SRF05 ultrasonic sensors, pic 18F4550 microcontroller, and 2 × 24 LCD monitor (**Figure 6**). We began with the design of a monorail that was responsible to support the robot navigations and stops between two cucumber rows inside the greenhouse. The monorail has a width of 0.06 m and was placed 0.4 m above the ground (**Figure 5A** and **B**). The algorithm for robot navigation between two consecutive stops points on the monorail is also illustrated in the flowchart of **Figure 5**. Right after the robot is switched on, it starts moving on the monorail that is fixed along the greenhouse from one row to another. Upon reaching the first

**Figure 3.** Flowchart of the research methodology.

stopper point on the rail, the robot strikes the first microswitch, which sends a deactivation signal to the first motor responsible for moving the robot. While stopped between two cucumber plant rows, the robot scans for weeds and determines the distance between the detected weed and the blade arm using the ultrasonic sensors. Subsequently, a command signal is sent to the arm motor and blade motor for activating the blade rotation as illustrated in the flowchart of **Figure 5**.

#### **2.2. Design of the mechanical parts**

The moving mechanical arm consists of a chassis, a small arm, and the main arm. Two main criteria were considered in designing the robot frame including minimum weight (for increasing the motor efficiency), and strength (for standing vibrations). The frame was made from an iron band bearing with the dimensions of 0.02 × 0.18 × 0.005 m. In order to provide support for the battery, bearing bases, microswitches, and the main arm, we installed additional extensions to the frame in a way that the robot gravity center is placed on the monorail. The battery is the heaviest part of the robot and can power the robot for 2 h. It was installed on the central frame above the rails and wheels. The battery weight creates stability for the robot when the main arm is outstretched, and this weight and location for the battery can hold the spinning wheel implemented in place. We placed several holes on the frame to facilitate the installation of the motor, wheels, and the required electrical fragments (**Figure 5C** and **D**). The robot makes use of four ball bearings of diameter 0.02 m, out of which three were used to hold the robot to the rail and to facilitate a smooth movement (two bearings were placed

on the right and one on the left side). The fourth bearing was used to act as the second wheel for the robot. All the ball bearings have a diameter of 0.02 m and are installed on the central frame. The diameter of the robot main wheel is 0.04 m, and the ideal speed was determined using trial and errors and time-motion studies during the conducted tests. The arm frame is made of an iron band bearing with a dimension of 0.02 × 0.2 × 0.005 m. A blade was installed on the main arm that moves forward and enables robot access to the weeds between the main plants. A shank protector in one of the holes in the arm frame makes the movement and the

Development of a Field Robot Platform for Mechanical Weed Control in Greenhouse Cultivation…

http://dx.doi.org/10.5772/intechopen.80935

17

selection of the angle for smooth cutting.

**Figure 4.** The CAD model design of the weed control robot.

Development of a Field Robot Platform for Mechanical Weed Control in Greenhouse Cultivation… http://dx.doi.org/10.5772/intechopen.80935 17

**Figure 4.** The CAD model design of the weed control robot.

stopper point on the rail, the robot strikes the first microswitch, which sends a deactivation signal to the first motor responsible for moving the robot. While stopped between two cucumber plant rows, the robot scans for weeds and determines the distance between the detected weed and the blade arm using the ultrasonic sensors. Subsequently, a command signal is sent to the arm motor and blade motor for activating the blade rotation as illustrated in the

The moving mechanical arm consists of a chassis, a small arm, and the main arm. Two main criteria were considered in designing the robot frame including minimum weight (for increasing the motor efficiency), and strength (for standing vibrations). The frame was made from an iron band bearing with the dimensions of 0.02 × 0.18 × 0.005 m. In order to provide support for the battery, bearing bases, microswitches, and the main arm, we installed additional extensions to the frame in a way that the robot gravity center is placed on the monorail. The battery is the heaviest part of the robot and can power the robot for 2 h. It was installed on the central frame above the rails and wheels. The battery weight creates stability for the robot when the main arm is outstretched, and this weight and location for the battery can hold the spinning wheel implemented in place. We placed several holes on the frame to facilitate the installation of the motor, wheels, and the required electrical fragments (**Figure 5C** and **D**). The robot makes use of four ball bearings of diameter 0.02 m, out of which three were used to hold the robot to the rail and to facilitate a smooth movement (two bearings were placed

flowchart of **Figure 5**.

**2.2. Design of the mechanical parts**

**Figure 3.** Flowchart of the research methodology.

16 Agricultural Robots - Fundamentals and Applications

on the right and one on the left side). The fourth bearing was used to act as the second wheel for the robot. All the ball bearings have a diameter of 0.02 m and are installed on the central frame. The diameter of the robot main wheel is 0.04 m, and the ideal speed was determined using trial and errors and time-motion studies during the conducted tests. The arm frame is made of an iron band bearing with a dimension of 0.02 × 0.2 × 0.005 m. A blade was installed on the main arm that moves forward and enables robot access to the weeds between the main plants. A shank protector in one of the holes in the arm frame makes the movement and the selection of the angle for smooth cutting.

because sound waves that broadcast from transmitters of ultrasonic sensors are circular. When these sensors are close to the ground, the broadcasting waves that bounce off from the

Development of a Field Robot Platform for Mechanical Weed Control in Greenhouse Cultivation…

http://dx.doi.org/10.5772/intechopen.80935

19

The robot movements are supported by three 12 V, 0.89 A DC motors that are labeled for this paper by motor 1, 2, and 3. The first motor was fixed directly to the wheels in front of the robot and was responsible for the robot movement on the monorail. To select the optimum speed for the robot, six motor speeds of 30, 40, 50, 60, 80, and 120 rpm were tested. We found that the motor with 60 rpm, 1.358 N·m torque, 12 V, 0.89 A had the best performance in the greenhouse under study. The second motor was connected to the small arm and is responsible to rotate the big arm that moves the blade of the robot at a selected speed of 10 rpm and torque of 8.15 N·m. The third motor was fixed to the frame of the main arm for rotating the blade at a high speed of 3500 rpm for efficient weed cutting and removal. This frame can move up and down and can fix the distance between the blade and the ground level. It should be noted that the 3500 rpm blade rotation speed and the 10 rpm arm motor speed were found from the experiments.

Three types of blade, namely the S-shaped, the triangle-shaped, and the circular-shaped blade (**Figure 7**) were initially considered in the weed cutting experiments. We conducted several tests to find the best blade width (equal to 0.1 m) for matching the 0.4 m distance between two cucumber plants. Based on our field tests, we found that the S-shaped blade was the most efficient design for the purpose of weed cutting. The blade was built from double stainless steel material to resist the corrosion in high humidity greenhouse environment. Analysis and calculations were carried out for finding the blade tip speed and corresponding vector components according to the formulations given in [48]. The corresponding diagrams of this analysis are shown schematically in **Figure 7**. It can be observed from **Figure 7A** that the direction of the tip of the blade follows a cycloid curve on the ground level. The component of blade speed in the direction of robot forward speed vector, as well as the demonstration of vector gradient in the blade speeds, is shown in **Figure 7B**–**D**. Here, WB is the circular speed of the blade (rad/s), *Vf* is the forward speed of robot [m/s], *Vbf = Vf + Vb* is the ratio of the total speed of blade to ground [m/s], *vb = rb × WB* is the circumferential speed of blade [m/s], *rb* is the radius of blade [m], *U* represents the direction of the robot movement, and *V* is the linear speed of blade [m/s]. The speed of the tip of the blade on the ground is equal to the sum of robot forward speed and its circumferential speed. Having the direction of robot moving (*U*), the direction of the moving blade will be in the direction of *Vbf*, which changes its direction as the blade rotates in the time frame *t* [s]. Therefore, to find the components of *U* and *V*, the speed of the blade tip can be written as the component of blade speed in direction of moving *U* [48], that is *Vu = Vf – rb × WB Sin (θ)*, and *Vv = Vbv = Vb × Cosθ = rb × WBcos (θ)*, where *θ = WB × t* is the angle between blade and movement direction, *Vu* and *Vu* are the speed component [m/s], and *t* [s] is the measured time from the initial angle θ = 0. Therefore, the speed of the blade tip with respect to the ground is

on the weed stem. Here, the force *fb* [N] is the bending strength of the plant body, *fr* is the cutting force [N], Ip is the pant geometry hardness torques, and *mp* is the weight of the cutting part of the plant [kg]. During the trial and error experiments, it was found that a minimum

. **Figure 7E** shows forces and torque vectors of the cutting strike

ground are misinterpreted as weeds.

**2.4. Blade design and analysis**

calculated as |Vbf| = √

\_\_\_\_\_\_\_\_\_\_\_ |Vv| 2 +|Vu| 2

strike speed of between 50 and 75 m/s is required for cutting the weeds.

**Figure 5.** The mechanism and flowchart for the robot navigation and control on the monorail showing (A) the monorail layout, (B) the robot mounted on the monorail, (C) the robot main body and manipulator arm, and (D) the joint setup between the robot and the monorail.

**Figure 6.** Major electrical module and wiring connections of the weed control robot.

#### **2.3. Design of the electronic parts: sense and action mechanism**

Major electronic components of the robot are three sets of SRF05 ultrasonic sensors, a PIC18F4550 microcontroller, and a 2 × 24 LCD monitor (**Figure 6**). The ultrasonic sensors were placed in a row having 0.10 m distance from each other. The sensors are specially positioned in a way that they cover the space between two cucumber plants on the cultivation row. As mentioned earlier, upon receiving a signal indicating weed existence, the microcontroller program determines the distance between the weed and the sensors and whether the weed is on the left, right, or middle of the sensors. This signal activates the cutting mechanism. Finally, the information of the entire process, including the distance between weed and sensors, and the specific sensor that identified the weed are shown on the robot LCD. During the experimental phase, we considered several improvements and adjustment on the sensing part and corresponding microcontroller program. For example, we used a tube pipe cover for each of the ultrasonic sensors to change the circular waves to linear waves. This was necessary because sound waves that broadcast from transmitters of ultrasonic sensors are circular. When these sensors are close to the ground, the broadcasting waves that bounce off from the ground are misinterpreted as weeds.

The robot movements are supported by three 12 V, 0.89 A DC motors that are labeled for this paper by motor 1, 2, and 3. The first motor was fixed directly to the wheels in front of the robot and was responsible for the robot movement on the monorail. To select the optimum speed for the robot, six motor speeds of 30, 40, 50, 60, 80, and 120 rpm were tested. We found that the motor with 60 rpm, 1.358 N·m torque, 12 V, 0.89 A had the best performance in the greenhouse under study. The second motor was connected to the small arm and is responsible to rotate the big arm that moves the blade of the robot at a selected speed of 10 rpm and torque of 8.15 N·m. The third motor was fixed to the frame of the main arm for rotating the blade at a high speed of 3500 rpm for efficient weed cutting and removal. This frame can move up and down and can fix the distance between the blade and the ground level. It should be noted that the 3500 rpm blade rotation speed and the 10 rpm arm motor speed were found from the experiments.

#### **2.4. Blade design and analysis**

**2.3. Design of the electronic parts: sense and action mechanism**

**Figure 6.** Major electrical module and wiring connections of the weed control robot.

between the robot and the monorail.

18 Agricultural Robots - Fundamentals and Applications

Major electronic components of the robot are three sets of SRF05 ultrasonic sensors, a PIC18F4550 microcontroller, and a 2 × 24 LCD monitor (**Figure 6**). The ultrasonic sensors were placed in a row having 0.10 m distance from each other. The sensors are specially positioned in a way that they cover the space between two cucumber plants on the cultivation row. As mentioned earlier, upon receiving a signal indicating weed existence, the microcontroller program determines the distance between the weed and the sensors and whether the weed is on the left, right, or middle of the sensors. This signal activates the cutting mechanism. Finally, the information of the entire process, including the distance between weed and sensors, and the specific sensor that identified the weed are shown on the robot LCD. During the experimental phase, we considered several improvements and adjustment on the sensing part and corresponding microcontroller program. For example, we used a tube pipe cover for each of the ultrasonic sensors to change the circular waves to linear waves. This was necessary

**Figure 5.** The mechanism and flowchart for the robot navigation and control on the monorail showing (A) the monorail layout, (B) the robot mounted on the monorail, (C) the robot main body and manipulator arm, and (D) the joint setup

Three types of blade, namely the S-shaped, the triangle-shaped, and the circular-shaped blade (**Figure 7**) were initially considered in the weed cutting experiments. We conducted several tests to find the best blade width (equal to 0.1 m) for matching the 0.4 m distance between two cucumber plants. Based on our field tests, we found that the S-shaped blade was the most efficient design for the purpose of weed cutting. The blade was built from double stainless steel material to resist the corrosion in high humidity greenhouse environment. Analysis and calculations were carried out for finding the blade tip speed and corresponding vector components according to the formulations given in [48]. The corresponding diagrams of this analysis are shown schematically in **Figure 7**. It can be observed from **Figure 7A** that the direction of the tip of the blade follows a cycloid curve on the ground level. The component of blade speed in the direction of robot forward speed vector, as well as the demonstration of vector gradient in the blade speeds, is shown in **Figure 7B**–**D**. Here, WB is the circular speed of the blade (rad/s), *Vf* is the forward speed of robot [m/s], *Vbf = Vf + Vb* is the ratio of the total speed of blade to ground [m/s], *vb = rb × WB* is the circumferential speed of blade [m/s], *rb* is the radius of blade [m], *U* represents the direction of the robot movement, and *V* is the linear speed of blade [m/s]. The speed of the tip of the blade on the ground is equal to the sum of robot forward speed and its circumferential speed. Having the direction of robot moving (*U*), the direction of the moving blade will be in the direction of *Vbf*, which changes its direction as the blade rotates in the time frame *t* [s]. Therefore, to find the components of *U* and *V*, the speed of the blade tip can be written as the component of blade speed in direction of moving *U* [48], that is *Vu = Vf – rb × WB Sin (θ)*, and *Vv = Vbv = Vb × Cosθ = rb × WBcos (θ)*, where *θ = WB × t* is the angle between blade and movement direction, *Vu* and *Vu* are the speed component [m/s], and *t* [s] is the measured time from the initial angle θ = 0. Therefore, the speed of the blade tip with respect to the ground is calculated as |Vbf| = √ \_\_\_\_\_\_\_\_\_\_\_ |Vv| 2 +|Vu| 2 . **Figure 7E** shows forces and torque vectors of the cutting strike on the weed stem. Here, the force *fb* [N] is the bending strength of the plant body, *fr* is the cutting force [N], Ip is the pant geometry hardness torques, and *mp* is the weight of the cutting part of the plant [kg]. During the trial and error experiments, it was found that a minimum strike speed of between 50 and 75 m/s is required for cutting the weeds.

and the surrounding weeds were also 15 days old (these weeds usually have thin and very flexible stalks and are 10 cm high), (ii) during the vegetation and early fruiting stage, when the cucumber plants were 2 months old, and (iii) during the mature fruiting stage, when the plants were at their mature height. Three types of blades were selected, namely the S-shaped, triangular-shaped, and circular-shaped blade. For each blade, we assigned three blade rotation (BR) speeds of BR1 = 3500, BR2 = 2500, and BR3 = 1500 rpm with two arm motor (AM) speed of AM1 = 10 and AM2 = 30 rpm. A factorial design with two-way analysis of variance (ANOVA) was used to determine variation effects in the cutting weed performance of each blade due to BR speed, AM speed, and their interaction. For the kth blade type, under the ith level of blade speed and the jth level of arm speed factor, the two-way ANOVA model was

**Figure 8.** Outside and inside views of the experimental site (top), and corresponding factorial design of experiment

Development of a Field Robot Platform for Mechanical Weed Control in Greenhouse Cultivation…

http://dx.doi.org/10.5772/intechopen.80935

21

(bottom) for determining the best combination of blade type, blade speed, and arm speed.

age of weeds cut (PWC) in an experiment. Time and motion study was conducted for the robot to move from one stopper to another. For motor no. 1, with a typical rotational speed of 60 rpm, and the wheel diameter of 0.04 m, the forward speed of the robot (VF) becomes 0.1256 m/s. Hence, the required time T [s] for the robot to travel the distance of X = 0.40 [m] between two consecutive stoppers is equal to T = 3.2 s using Eq. (1). The possibility for the robot to pass through the two stoppers within a row was considered for the consequent calculations. For the arm motor, the typical speed is 10 rpm, which implies that it takes T = 6 s for the robot to

V*F*

Results of statistical analysis are summarized in **Tables 1**–**3** showing that the effects of blade type (T), blade rotation (BR) speed, and arm motor (AM) speed are significant at the 0.05 level. Moreover, it was found that the S-shaped blade with a mean (μ) of 67.8% and standard error (σ)

ij <sup>+</sup> <sup>ε</sup>ijk, where Yijk is the dependent variable representing the percent-

(1)

stated as Yijk <sup>=</sup> <sup>μ</sup> <sup>+</sup> bi <sup>+</sup> aj <sup>+</sup> (b . a)

**3. Results**

remove the weed between two plants.

*T* = \_\_\_\_\_\_ 3.6 <sup>×</sup> <sup>X</sup>

**Figure 7.** Design of the cutting blade, (A): calculating the velocity of rotating blade in stickles, (B): component of blade speed in direction of moving U, (C, D): calculating U and V, (E): components of forces, and (F): vector demonstration of the blade speed. Adapted from [48].

#### **2.5. Experiment setup**

The weed control robot was tested in a 5000 m2 greenhouse in Jiroft city (28°40′41″N 57°44′26″E) located to the south of Kerman province of Iran (**Figure 8**). We planted over 10,000 cucumber seeds in pots and placed them in the greenhouse with spaces between the two plants being 0.4 m. It should be noted that in order to manually remove the weeds from 1 ha of the greenhouse under study, four seasonal workers had to perform the task every day, for 8 months (equivalent to 832 man/hour). Three experiments were conducted at different growth stages as follows: (i) during the seedling and germination stage, 15 days after the crop was cultivated Development of a Field Robot Platform for Mechanical Weed Control in Greenhouse Cultivation… http://dx.doi.org/10.5772/intechopen.80935 21

**Figure 8.** Outside and inside views of the experimental site (top), and corresponding factorial design of experiment (bottom) for determining the best combination of blade type, blade speed, and arm speed.

and the surrounding weeds were also 15 days old (these weeds usually have thin and very flexible stalks and are 10 cm high), (ii) during the vegetation and early fruiting stage, when the cucumber plants were 2 months old, and (iii) during the mature fruiting stage, when the plants were at their mature height. Three types of blades were selected, namely the S-shaped, triangular-shaped, and circular-shaped blade. For each blade, we assigned three blade rotation (BR) speeds of BR1 = 3500, BR2 = 2500, and BR3 = 1500 rpm with two arm motor (AM) speed of AM1 = 10 and AM2 = 30 rpm. A factorial design with two-way analysis of variance (ANOVA) was used to determine variation effects in the cutting weed performance of each blade due to BR speed, AM speed, and their interaction. For the kth blade type, under the ith level of blade speed and the jth level of arm speed factor, the two-way ANOVA model was stated as Yijk <sup>=</sup> <sup>μ</sup> <sup>+</sup> bi <sup>+</sup> aj <sup>+</sup> (b . a) ij <sup>+</sup> <sup>ε</sup>ijk, where Yijk is the dependent variable representing the percentage of weeds cut (PWC) in an experiment. Time and motion study was conducted for the robot to move from one stopper to another. For motor no. 1, with a typical rotational speed of 60 rpm, and the wheel diameter of 0.04 m, the forward speed of the robot (VF) becomes 0.1256 m/s. Hence, the required time T [s] for the robot to travel the distance of X = 0.40 [m] between two consecutive stoppers is equal to T = 3.2 s using Eq. (1). The possibility for the robot to pass through the two stoppers within a row was considered for the consequent calculations. For the arm motor, the typical speed is 10 rpm, which implies that it takes T = 6 s for the robot to remove the weed between two plants.

$$T = \frac{3.6 \times \chi}{V\_r} \tag{1}$$

#### **3. Results**

**2.5. Experiment setup**

the blade speed. Adapted from [48].

20 Agricultural Robots - Fundamentals and Applications

The weed control robot was tested in a 5000 m2

greenhouse in Jiroft city (28°40′41″N 57°44′26″E)

located to the south of Kerman province of Iran (**Figure 8**). We planted over 10,000 cucumber seeds in pots and placed them in the greenhouse with spaces between the two plants being 0.4 m. It should be noted that in order to manually remove the weeds from 1 ha of the greenhouse under study, four seasonal workers had to perform the task every day, for 8 months (equivalent to 832 man/hour). Three experiments were conducted at different growth stages as follows: (i) during the seedling and germination stage, 15 days after the crop was cultivated

**Figure 7.** Design of the cutting blade, (A): calculating the velocity of rotating blade in stickles, (B): component of blade speed in direction of moving U, (C, D): calculating U and V, (E): components of forces, and (F): vector demonstration of

> Results of statistical analysis are summarized in **Tables 1**–**3** showing that the effects of blade type (T), blade rotation (BR) speed, and arm motor (AM) speed are significant at the 0.05 level. Moreover, it was found that the S-shaped blade with a mean (μ) of 67.8% and standard error (σ)


**Table 1.** Factor effects on the percentage of weeds cut.


effect on the PWC. The results provided in **Table 3** show that the difference between the two blades, S-shape and triangular shape is significant at the 0.05 level. In other words, the mean of weeds cut by these two blades are significantly different, and according to the mean differences column, the mean of the PWC by the S-shape blade is larger than the PWC by the triangularshaped blade. The mean difference between the S-shaped and the circular-shaped blade with P-value of 0.036 is also significant at the 0.05 level. This implies that the average PWC by these two blades are significantly different, and according to the mean differences column, the mean PWC by the S-shaped blade is larger than the mean of the PWC by the circular-shaped blade. It was found that the difference between the means of the triangular-shaped blade and circularshaped blade with the P-value of 0.076 is not significant at the 0.05 level, that is, the mean of the

**Table 3.** Comparison of significant difference between blade types (A: S-shape, B: Triangular shape, and C: Circular

A-B 6.4444 0.000 A-C 3.5000 0.036 B-C −2.9444 0.076

BR1-BR2 −14.5556 0.000 BR1-BR3 −27.8333 0.000 BR2-BR3 −13.2778 0.000

**Mean differences P-value**

http://dx.doi.org/10.5772/intechopen.80935

23

Development of a Field Robot Platform for Mechanical Weed Control in Greenhouse Cultivation…

Results of analysis of variance also showed that the mean differences between the BR speeds are significant, indicating that the resulted PWC with BR1 = 1500, BR2 = 2500, and BR3 = 3500 rpm are not equal. More specifically, the PWC in 1500 rpm was found to be smaller than those of 2500 and 3500 rpm. In addition, the mean PWC in 2500 rpm was also smaller than that of 3500 rpm. This can also be observed from the bar plots of **Figure 9**,

**Figure 9.** Comparison of the effects of various blade types on (left) and various blade rotation speeds (right) on the

PWC by these two blade types are not significantly different.

shape), and blades rotation speed (BR1: 1500, BR2: 2500, and BR3: 3500 rpm).

Blade type

Blade rotation (rpm)

percentage of weeds cut.

**Table 2.** Variance analysis and effects of the robot blade type (T), blade rotation (BR) speed, and arm motor (AM) speed on the percentage of weed cutting performance.

of 3.052% had the highest effect, and triangular-shaped blade with μ = 61.38% and σ = 3.083% had the lowest effect on the percentage of the weeds cut (PWC). The BR factor was significant at P < 0.05, indicating that blade rotation speed of 3500 rpm with μ = 78.23% and σ = 1.71% had the highest effect and the 1500 rpm with μ = 50.39 and σ = 1.86% had the lowest effect. The AM speed factor was also found to be significant at P < 0.05, which indicates that the speed of 10 rpm with μ = 69.1% and σ = 2.45% had the highest effect and the speed of 30 rpm with μ = 59.8% and σ = 2.46% has had the lowest effect on the PWC. It was found that (**Table 1**) different blade shapes with the AM speed of 10 rpm had a significant effect on the PWC. While the mean PWC by the S-shaped blades was the highest, increasing AM speed to 30 rpm reduced the efficiency of the S-shaped blade (as well as with the other two blades), resulting a mean PWC of 59.39%. According to the P-values in **Table 2**, while all of the main effects of blade type, BR, and AM speeds are significant at 0.05 level, their interactions were not found to have a significant Development of a Field Robot Platform for Mechanical Weed Control in Greenhouse Cultivation… http://dx.doi.org/10.5772/intechopen.80935 23


**Table 3.** Comparison of significant difference between blade types (A: S-shape, B: Triangular shape, and C: Circular shape), and blades rotation speed (BR1: 1500, BR2: 2500, and BR3: 3500 rpm).

effect on the PWC. The results provided in **Table 3** show that the difference between the two blades, S-shape and triangular shape is significant at the 0.05 level. In other words, the mean of weeds cut by these two blades are significantly different, and according to the mean differences column, the mean of the PWC by the S-shape blade is larger than the PWC by the triangularshaped blade. The mean difference between the S-shaped and the circular-shaped blade with P-value of 0.036 is also significant at the 0.05 level. This implies that the average PWC by these two blades are significantly different, and according to the mean differences column, the mean PWC by the S-shaped blade is larger than the mean of the PWC by the circular-shaped blade. It was found that the difference between the means of the triangular-shaped blade and circularshaped blade with the P-value of 0.076 is not significant at the 0.05 level, that is, the mean of the PWC by these two blade types are not significantly different.

Results of analysis of variance also showed that the mean differences between the BR speeds are significant, indicating that the resulted PWC with BR1 = 1500, BR2 = 2500, and BR3 = 3500 rpm are not equal. More specifically, the PWC in 1500 rpm was found to be smaller than those of 2500 and 3500 rpm. In addition, the mean PWC in 2500 rpm was also smaller than that of 3500 rpm. This can also be observed from the bar plots of **Figure 9**,

of 3.052% had the highest effect, and triangular-shaped blade with μ = 61.38% and σ = 3.083% had the lowest effect on the percentage of the weeds cut (PWC). The BR factor was significant at P < 0.05, indicating that blade rotation speed of 3500 rpm with μ = 78.23% and σ = 1.71% had the highest effect and the 1500 rpm with μ = 50.39 and σ = 1.86% had the lowest effect. The AM speed factor was also found to be significant at P < 0.05, which indicates that the speed of 10 rpm with μ = 69.1% and σ = 2.45% had the highest effect and the speed of 30 rpm with μ = 59.8% and σ = 2.46% has had the lowest effect on the PWC. It was found that (**Table 1**) different blade shapes with the AM speed of 10 rpm had a significant effect on the PWC. While the mean PWC by the S-shaped blades was the highest, increasing AM speed to 30 rpm reduced the efficiency of the S-shaped blade (as well as with the other two blades), resulting a mean PWC of 59.39%. According to the P-values in **Table 2**, while all of the main effects of blade type, BR, and AM speeds are significant at 0.05 level, their interactions were not found to have a significant

**Table 2.** Variance analysis and effects of the robot blade type (T), blade rotation (BR) speed, and arm motor (AM) speed

**Blade type μ: Mean percentage of weeds cut (%) σ: Std. error (%)**

A: S shaped 67.8 3.05 B: Triangular shape 61.38 3.08 C: Circular shape 64.3 3.38

1500 50.3 1.86 2500 64.9 1.51 3500 78.2 1.71

10 69.148 2.457 30 59.88 2.461

**Model Sum of squares Mean sum of squares P-value** Blade type (T) 110.3 110.3 0.0462 Blade rotation speed (BR) 6977.1 3488.6 0.000 Arm motor speed (AM) 1157.4 1157.4 0.000

Interaction types P-value T × BR 114.1 57.1 0.1272 T × AM 5.4 5.4 0.6493 BR × AM 73.6 36.8 0.2560 T × BR × AM 16 8 0.7356

Blade rotation speed (rpm)

22 Agricultural Robots - Fundamentals and Applications

Arm motor speed (rpm)

**Table 1.** Factor effects on the percentage of weeds cut.

on the percentage of weed cutting performance.

Error 1264.6 26.3

**Figure 9.** Comparison of the effects of various blade types on (left) and various blade rotation speeds (right) on the percentage of weeds cut.

showing that the mean PWC in 1500 rpm is the smallest (59.39%) and that of 3500 rpm was the largest (78.23%). The bar plots in **Figure 10** illustrate descriptive statistics and frequency of the PWC for the experiments with the robot using all factors (blade types A, B, C, blade rotation speeds of 1500, 2500, 3500 rpm, and arm motor speed of 10 and 30 rpm). It can be seen from **Figure 10** that the average PWC by the blades was significantly different. Consequently, the highest PWC cut was related to S-shaped at the blade rotation speed of 3500 rpm. In each motor arm speed, the increase in the rotational blade speed caused an increase in the PWC. In each rotational blade speed, if the motor arm speed increases, the PWC cut will decrease. Comparing the interactions between the three different types of blades, blade speed, and the speed of the arm the following results was obtained: the highest PWC in the entire experiment was 95%, which was obtained when the S-shaped blade at the rotational speed of 3500 rpm was used and motor speed was 10 rpm. The lowest PWC was 45%, which was obtained when the blade speed was 1500 rpm, AM speed was 30 rpm, and the blade type was triangular in shape. The analysis of the interaction of the BR speed and blade type showed that (i) none of the mutual interactions was significant in the variance test, (ii) t-test showed that if the rotational speed of the blade is low, the blade type will have a significant effect on the PWC, and (iii) for all the blade types, the highest PWC cut was at

Development of a Field Robot Platform for Mechanical Weed Control in Greenhouse Cultivation…

http://dx.doi.org/10.5772/intechopen.80935

25

In this study, we designed, developed, and fabricated a prototype robot for mechanical weed control in greenhouse cultivation of cucumber. Automatic weed cutting experiments that were carried using the robot consist of ultrasonic sensor, which senses the existence of weeds between the cucumber plants. The robot then moves between cucumber rows on a monorail in the greenhouse, with an arm that moves the blade between the plants for cutting the detected weeds. The entire process of weed detection, moving the arm and blades, and weeds cutting is carried out in 10 s. Among the three blade types tested (S-, triangular-, and circular shapes), it was concluded that the S-shape was the most efficient design. For the best blade rotation (BR) and arm motor (AM) speeds, it was concluded that as the AM speed increased, the percentage of weeds cut (PWC) reduces; therefore, the motor with 10 rpm, 8.15 N·m torque, 12 V, and 0.89 A was selected to for moving the arm. The average weeds cut at 10 and 30 rpm was 69.1 and 58.9%, respectively. Finally, it was concluded that the best robot performance corresponding to the highest percentage of weeds cut was achieved with the S-shaped blade when the BR speed was 3500 rpm, and the AM speed

BR speed of 3500 rpm.

**4. Conclusion**

was 10 rpm.

**Conflict of interest**

The authors declare no conflict of interest.

**Figure 10.** Bar plots describing percentage of weeds cut with different blade type, blade rotation speed, and robot arm speed.

showing that the mean PWC in 1500 rpm is the smallest (59.39%) and that of 3500 rpm was the largest (78.23%). The bar plots in **Figure 10** illustrate descriptive statistics and frequency of the PWC for the experiments with the robot using all factors (blade types A, B, C, blade rotation speeds of 1500, 2500, 3500 rpm, and arm motor speed of 10 and 30 rpm). It can be seen from **Figure 10** that the average PWC by the blades was significantly different. Consequently, the highest PWC cut was related to S-shaped at the blade rotation speed of 3500 rpm. In each motor arm speed, the increase in the rotational blade speed caused an increase in the PWC. In each rotational blade speed, if the motor arm speed increases, the PWC cut will decrease. Comparing the interactions between the three different types of blades, blade speed, and the speed of the arm the following results was obtained: the highest PWC in the entire experiment was 95%, which was obtained when the S-shaped blade at the rotational speed of 3500 rpm was used and motor speed was 10 rpm. The lowest PWC was 45%, which was obtained when the blade speed was 1500 rpm, AM speed was 30 rpm, and the blade type was triangular in shape. The analysis of the interaction of the BR speed and blade type showed that (i) none of the mutual interactions was significant in the variance test, (ii) t-test showed that if the rotational speed of the blade is low, the blade type will have a significant effect on the PWC, and (iii) for all the blade types, the highest PWC cut was at BR speed of 3500 rpm.

#### **4. Conclusion**

In this study, we designed, developed, and fabricated a prototype robot for mechanical weed control in greenhouse cultivation of cucumber. Automatic weed cutting experiments that were carried using the robot consist of ultrasonic sensor, which senses the existence of weeds between the cucumber plants. The robot then moves between cucumber rows on a monorail in the greenhouse, with an arm that moves the blade between the plants for cutting the detected weeds. The entire process of weed detection, moving the arm and blades, and weeds cutting is carried out in 10 s. Among the three blade types tested (S-, triangular-, and circular shapes), it was concluded that the S-shape was the most efficient design. For the best blade rotation (BR) and arm motor (AM) speeds, it was concluded that as the AM speed increased, the percentage of weeds cut (PWC) reduces; therefore, the motor with 10 rpm, 8.15 N·m torque, 12 V, and 0.89 A was selected to for moving the arm. The average weeds cut at 10 and 30 rpm was 69.1 and 58.9%, respectively. Finally, it was concluded that the best robot performance corresponding to the highest percentage of weeds cut was achieved with the S-shaped blade when the BR speed was 3500 rpm, and the AM speed was 10 rpm.

#### **Conflict of interest**

**Figure 10.** Bar plots describing percentage of weeds cut with different blade type, blade rotation speed, and robot arm

speed.

24 Agricultural Robots - Fundamentals and Applications

The authors declare no conflict of interest.

## **Author details**

Amid Heravi<sup>1</sup> , Desa Ahmad1 , Ibrahim A. Hameed<sup>2</sup> , Redmond Ramin Shamshiri1 \*, Siva K. Balasundram3 and Muhammad Yamin4

\*Address all correspondence to: raminshamshiri@upm.edu.my

1 Department of Biological and Agricultural Engineering, Faculty of Engineering, Universiti Putra Malaysia, Selangor, Malaysia

[9] Subramanian V. Autonomous Vehicle Guidance Using Machine Vision and Laser Radar for Agricultural Applications. Doctoral dissertation, University of Florida. 2005

Development of a Field Robot Platform for Mechanical Weed Control in Greenhouse Cultivation…

http://dx.doi.org/10.5772/intechopen.80935

27

[10] Ollero A, Mandow A, Muñoz VF, De Gabriel JG. Control architecture for mobile robot operation and navigation. Robotics and Computer-Integrated Manufacturing. 1994;**11**(4):

[11] Mandow A, Gomez-de-Gabriel JM, Martinez JL, Munoz VF, Ollero A, Garcia-Cerezo A. The autonomous mobile robot AURORA for greenhouse operation. IEEE Robotics and

[12] Martínez JL, Mandow A, Morales J, Pedraza S, García-Cerezo A. Approximating kinematics for tracked mobile robots. International Journal of Robotics Research. 2005;**24**(10):

[13] Shamshiri RR, Hameed IA, Karkee M, Weltzien C. Robotic harvesting of fruiting vegetables: A simulation approach in V-REP, ROS and MATLAB. In: Automation in Agriculture-Securing Food Supplies for Future Generations. Rijeka, Croatia: InTech; 2018

[14] Billingsley J, Visala A, Dunn M. Robotics in agriculture and forestry. In: Springer

[15] González R, Rodríguez F, Sánchez-Hermosilla J, Donaire JG. Navigation techniques for mobile robots in greenhouses. Applied Engineering in Agriculture. 2009;**25**(2):153-165

[16] Bontsema J, van Os EA, Van Henten EJ, Hemming J, van Tuijl BAJ, Kornet JG, et al. An autonomous robot for harvesting cucumbers in greenhouses. Journal of Autonomous

[17] Hopkins M. Automating in the 21st century career and technical education. Greenhouse

[18] Pilarski T, Happold M, Pangels H, Ollis M, Fitzpatrick K, Stentz A. The demeter system

[19] Sezen B. Modeling Automated Guided Vehicle Systems in Material Handling. Dogus

[20] Shamshiri R, Ismail WIW. A review of greenhouse climate control and automation systems in tropical regions. Journal of Agri-Food and Applied Sciences. 2013;**2**(3):176-183

[21] Bac CW, Roorda T, Reshef R, Berman S, Hemming J, van Henten EJ. Analysis of a motion planning problem for sweet-pepper harvesting in a dense obstacle environment.

[22] Bloch V, Degani A, Bechar A. A methodology of orchard architecture design for an opti-

[23] Monta M, Kondo N, Ting KC. End-effectors for tomato harvesting robot. In: Artificial Intelligence for Biology and Agriculture. Dordrecht: Springer; 1998. pp. 1-25

for automated harvesting. Autonomous Robots. 2002;**13**(1):9-20

Biosystems Engineering. 2016;**146**(Supplement C):85-97

mal harvesting robot. Biosystems Engineering. 2018;**166**:126-137

Handbook of Robotics. Berlin, Heidelberg: Springer; 2008. pp. 1065-1077

259-269

867-878

Robots. 2002;**13**:241-258

Oniversiiesi Dergisi. 2003;**4**(2):207-216

Grower. 2000:4-12

Automation Magazine. 1996;**3**(4):18-28

2 Department of ICT and Natural Sciences, Faculty of Information Technology and Electrical Engineering, NTNU, Ålesund, Norway

3 Department of Agriculture Technology, Faculty of Agriculture, Universiti Putra Malaysia, Selangor, Malaysia

4 Department of Farm Machinery and Power, University of Agriculture Faisalabad, Pakistan

#### **References**


[9] Subramanian V. Autonomous Vehicle Guidance Using Machine Vision and Laser Radar for Agricultural Applications. Doctoral dissertation, University of Florida. 2005

**Author details**

Siva K. Balasundram3

Selangor, Malaysia

**References**

, Desa Ahmad1

26 Agricultural Robots - Fundamentals and Applications

Putra Malaysia, Selangor, Malaysia

Engineering, NTNU, Ålesund, Norway

Engineering. 2018;**11**(1):1-22

Singapore: Springer Singapore; 2016. pp. 19-36

weed control measures. Weed Science. 1991;**193**:32-40

Automation Technology for Off-Road Equipment; 2004. p. 79

InTech; 2017

1996;**10**(1):217-231

tions of ASAE. 2002;**45**(1):231

Agriculture. 1999;**1**(1):95-113

, Ibrahim A. Hameed<sup>2</sup>

1 Department of Biological and Agricultural Engineering, Faculty of Engineering, Universiti

2 Department of ICT and Natural Sciences, Faculty of Information Technology and Electrical

3 Department of Agriculture Technology, Faculty of Agriculture, Universiti Putra Malaysia,

4 Department of Farm Machinery and Power, University of Agriculture Faisalabad, Pakistan

[1] Shamshiri RR, Kalantari F, Ting KC, Thorp KR, Hameed IA, Weltzien C, et al. Advances in greenhouse automation and controlled environment agriculture: A transition to plant factories and urban agriculture. International Journal of Agricultural and Biological

[2] Shamshiri RR, Mahadi MR, Thorp KR, Ismail WIW, Ahmad D, Man HC. Adaptive management framework for evaluating and adjusting microclimate parameters in tropical greenhouse crop production systems. In: Jurić S, editor. Plant Engineering. Rijeka:

[3] Ting KC, Lin T, Davidson PC. Integrated urban controlled environment agriculture systems. In: Kozai T, Fujiwara K, Runkle ES, editors. LED Lighting for Urban Agriculture.

[4] Popay I, Field R. Grazing animals as weed control agents. Weed Technology.

[5] Baumann PA, Bade DH, Biediger DL. Forage grass response to chemical and mechanical

[6] Lamm RD, Slaughter DC, Giles DK. Precision weed control system for cotton. Transac-

[7] Lee WS, Slaughter DC, Giles DK. Robotic weed control system for tomatoes. Precision

[8] Burks TF, Subramanian V, Singh S. Autonomous greenhouse sprayer vehicle using machine vision and ladar for steering control. In: Proceedings of the Conference of

and Muhammad Yamin4

\*Address all correspondence to: raminshamshiri@upm.edu.my

, Redmond Ramin Shamshiri1

\*,

Amid Heravi<sup>1</sup>


[24] Kiani S, Jafari A. Crop detection and positioning in the field using discriminant analysis and neural networks based on shape features. Journal of Agricultural Science and Technology. 2012;**14**:755-765

[38] Dario P, Sandini G, Allotta B, Bucci A, Buemi F, Massa M, Ferrari F, Magrassi M, Bosio L, Valleggi R. The Agrobot project for greenhouse automation. In: International Symposium

Development of a Field Robot Platform for Mechanical Weed Control in Greenhouse Cultivation…

http://dx.doi.org/10.5772/intechopen.80935

29

[39] Langenakens J, Vergauwe G, De Moor A. Comparing hand held spray guns and spray booms in lettuce crops in a greenhouse. Aspects of Applied Biology. 2002;**66**:123-128 [40] Nuyttens D, Windey S, Sonck B. Comparison of operator exposure for five different greenhouse spraying applications. Journal of Agricultural Safety and Health. 2004;**10**(3):187

[41] Sánchez-Hermosilla J, Rodríguez F, González R, Guzmán JL, Berenguel M. A mechatronic description of an autonomous mobile robot for agricultural tasks in greenhouses.

[42] Borenstein J, Everett HR, Feng L. Navigating Mobile Robots: Systems and Techniques.

[43] Shamshiri R, Ishak W, Ismail W. Design and simulation of control systems for a field survey mobile robot platform. Research Journal of Applied Sciences, Engineering and

[44] Van Henten EJ, Van Tuijl BAJ, Hoogakker GJ, Van Der Weerd MJ, Hemming J, Kornet JG, et al. An autonomous robot for de-leafing cucumber plants grown in a high-wire

[45] Burks TF, Shearer SA, Heath JR, Donohue KD. Evaluation of neural-network classifiers for weed species discrimination. Biosystems Engineering. 2005;**91**(3):293-304

[46] Granitto PM, Verdes PF, Ceccatto HA. Large-scale investigation of weed seed identification by machine vision. Computers and Electronics in Agriculture. 2005;**47**(1):15-24 [47] Belforte G, Deboli R, Gay P, Piccarolo P, Aimonino DR. Robot design and testing for

[48] Persson S. Mechanics of Cutting Plant Material. St. Joseph, Michigan, USA: American

on New Cultivation Systems in Greenhouse 361; 1993. pp. 85-92

In: Mobile Robots Navigation. Rijeka, Croatia: InTech; 2010

cultivation system. Biosystems Engineering. 2006;**94**(3):317-323

greenhouse applications. Biosystems Engineering. 2006;**95**(3):309-321

Wellesley, MA: AK Peters, Ltd.; 1996

Technology. 2013;**6**(13):2307-2315

Society of Agricultural Engineers; 1987


[38] Dario P, Sandini G, Allotta B, Bucci A, Buemi F, Massa M, Ferrari F, Magrassi M, Bosio L, Valleggi R. The Agrobot project for greenhouse automation. In: International Symposium on New Cultivation Systems in Greenhouse 361; 1993. pp. 85-92

[24] Kiani S, Jafari A. Crop detection and positioning in the field using discriminant analysis and neural networks based on shape features. Journal of Agricultural Science and

[25] Perez AJ, Lopez F, Benlloch JV, Christensen S. Colour and shape analysis techniques for weed detection in cereal fields. Computers and Electronics in Agriculture.

[26] Cho S, Lee DS, Jeong JY. AE—Automation and emerging technologies: Weed–plant discrimination by machine vision and artificial neural network. Biosystems Engineering.

[27] Åstrand B, Baerveldt A-J. An agricultural mobile robot with vision-based perception for

[28] Jafari A, Mohtasebi SS, Jahromi HE, Omid M. Weed detection in sugar beet fields using machine vision. International Journal of Agriculture and Biology. 2006;**8**(5):602-605 [29] Huang M, He Y. Crop and weed image recognition by morphological operations and ann model. In: Instrumentation and Measurement Technology Conference Proceedings

[30] Meyer GE, Mehta T, Kocher MF, Mortensen DA, Samal A. Textural imaging and discriminant analysis for distinguishing weeds for spot spraying. Transactions of ASAE.

[31] Polder G, van Evert FK, Lamaker A, de Jong A, van der Heijden GWAM, Lotz LAP, van der Zalm AJA, Kempenaar C. Weed detection using textural image analysis. EFITA/WCCA

[32] Zhu B, Jiang L, Luo Y, Tao Y. Gabor feature-based apple quality inspection using kernel principal component analysis. Journal of Food Engineering. 2007;**81**(4):741-749

[33] Mehta SS, Burks TF. Vision-based control of robotic manipulator for citrus harvesting.

[34] Zhao Y, Gong L, Huang Y, Liu C. A review of key techniques of vision-based control for harvesting robot. Computers and Electronics in Agriculture. 2016;**127**(Supplement C):

[35] Choi D, Lee WS, Ehsani R, Roka FM. A machine vision system for quantification of citrus fruit dropped on the ground under the canopy. Transactions of the ASABE. 2015;**58**(4):

[36] Mehta SS, MacKunis W, Burks TF. Robust visual servo control in the presence of fruit motion for robotic citrus harvesting. Computers and Electronics in Agriculture. 2016;**123**:

[37] Sandini G, Buemi F, Massa M, Zucchini M. Visually guided operations in green-houses. In: IEEE International Workshop on Intelligent Robots and Systems' 90. Towards a New

Frontier of Applications, Proceedings. IROS'90; 1990. pp. 279-285

Computers and Electronics in Agriculture. 2014;**102**:146-158

mechanical weed control. Autonomous Robots. 2002;**13**(1):21-35

Technology. 2012;**14**:755-765

28 Agricultural Robots - Fundamentals and Applications

(IMTC 2007). IEEE; 2007. pp. 1-4

2000;**25**(3):197-212

2002;**83**(3):275-280

1998;**41**(4):1189

conference; 2007

311-323

933-946

362-375


**Chapter 3**

**Provisional chapter**

**An Evaluation of Three Different Infield Navigation**

**An Evaluation of Three Different Infield Navigation** 

In this chapter, we present and evaluate three different infield navigation algorithms, based on the readings from a LIDAR sensor. All three algorithms are tested on a small field robot and used to autonomously drive the robot between the two adjacent rows of maze plants. The first algorithm is the simplest one and just takes distance readings from the left and right side. If robot is not in the center of the mid-row space, it adjusts its course by turning the robot in the right direction accordingly. The second approach groups the left and right readings into two vertical lines by using least-square fit approach. According to the calculated distance and orientation to both lines, it adjusts the course of the robot. The third approach tries to fit an optimal triangle between the robot and the plants, revealing the most optimal one. Based on its shape, the course of the robot is adjusted. All three algorithms are tested in a simulated (ROS stage) and then in an outdoor (maze test field) environment comparing the optimal line with the actual calculated position of the robot. The tests prove that all three approaches work with an error of 0.041 ± 0.034 m for the first algorithm, 0.07 ± 0.059 m for the second, and

**Keywords:** infield algorithms, navigation algorithms, LIDAR, field robot, ROS

The usual approach of autonomously driving agriculture machinery trough the field is by using a precise differential [1] or RTK global positioning systems [2]. In order to work, these systems require a previously known path of movement that is repeated with each iteration

> © 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

© 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use,

distribution, and reproduction in any medium, provided the original work is properly cited.

DOI: 10.5772/intechopen.79942

**Algorithms**

**Abstract**

**1. Introduction**

**Algorithms**

Peter Bernad, Peter Lepej, Črtomir Rozman,

Peter Bernad, Peter Lepej, Črtomir Rozman,

Additional information is available at the end of the chapter

Additional information is available at the end of the chapter

Karmen Pažek and Jurij Rakun

Karmen Pažek and Jurij Rakun

http://dx.doi.org/10.5772/intechopen.79942

0.078 ± 0.055 m error for the third.

#### **An Evaluation of Three Different Infield Navigation Algorithms An Evaluation of Three Different Infield Navigation Algorithms**

DOI: 10.5772/intechopen.79942

Peter Bernad, Peter Lepej, Črtomir Rozman, Karmen Pažek and Jurij Rakun Peter Bernad, Peter Lepej, Črtomir Rozman, Karmen Pažek and Jurij Rakun

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.79942

#### **Abstract**

In this chapter, we present and evaluate three different infield navigation algorithms, based on the readings from a LIDAR sensor. All three algorithms are tested on a small field robot and used to autonomously drive the robot between the two adjacent rows of maze plants. The first algorithm is the simplest one and just takes distance readings from the left and right side. If robot is not in the center of the mid-row space, it adjusts its course by turning the robot in the right direction accordingly. The second approach groups the left and right readings into two vertical lines by using least-square fit approach. According to the calculated distance and orientation to both lines, it adjusts the course of the robot. The third approach tries to fit an optimal triangle between the robot and the plants, revealing the most optimal one. Based on its shape, the course of the robot is adjusted. All three algorithms are tested in a simulated (ROS stage) and then in an outdoor (maze test field) environment comparing the optimal line with the actual calculated position of the robot. The tests prove that all three approaches work with an error of 0.041 ± 0.034 m for the first algorithm, 0.07 ± 0.059 m for the second, and 0.078 ± 0.055 m error for the third.

**Keywords:** infield algorithms, navigation algorithms, LIDAR, field robot, ROS

#### **1. Introduction**

The usual approach of autonomously driving agriculture machinery trough the field is by using a precise differential [1] or RTK global positioning systems [2]. In order to work, these systems require a previously known path of movement that is repeated with each iteration

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. © 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

when the crop needs to be treated [3]. This is not always possible, if no prior GPS information exists, or simply because an accurate GPS system is not always available. So, different systems should be used in these situations.

A possible solution to solve this is to use cameras to detect plant lines [4] or even stereo cameras to build a 3D cloud of points [5], describing the plants, and drive the agricultural machinery to drive between them. Another possible approach is the use of LIDAR systems [6] that take accurate measurements of the scene, by reading distances from the sensor to the first obstacle and repeating this for the whole range (usually 270° or 360° in 1°, 0.5° or even 2.25° steps).

With the help of these systems, the machines [7–9] can drive even in an unfamiliar field where they have not been applied before. They rely on the property of the field and the crops planted. The plants are in parallel to each other, and the machines can drive between the crop lines, in mid-row spaces, in order to not damage the plants. This can be done by using different approaches/algorithms to guide the machines.

More advance systems using different SLAM methods [10, 11] even build a map of the environment and localize the machines in a new environment that is only being discovered. Based on the map, it then constructs a path using path planner [12] and path follower [13] to follow the path. These approaches come in use in case of unfamiliar scenes but are not really necessary in infield situations where prior information regarding the pattern of the plants is known.

If the robot, as the one in **Figure 1**, has to drive autonomously through the crop fields, it must have a navigation algorithm. The navigation algorithm relies on the distance measurements from the LIDAR sensor. The experimental robot is equipped with the SICK TIM310 LIDAR, with a 270° area at a 1° angular resolution. It detects obstacles up to 4 m away and then chooses the optimal path based on the readings. Once the robot reaches the end of the crop

**Figure 1.** A small field robot FarmBeast while performing Task 1—Basic navigation at Field Robot Event 2018.

In this chapter, we investigate the accuracy of three different navigation algorithms and compare them, with left and right row distance as a reference for optimal path that should be in

The first algorithm is the simplest one. It takes 30 readings from left and 30 from right side,

From these two sets, it first eliminates ones that are too far away, that is more than 0.75 cm and belong to the other crop lines, and then calculates an average distance value for each side.

and *d*<sup>l</sup>

The value of the *offset* is then used to adjust the course of the robot, if necessary, as shown in

So, the current course of the robot is adjusted by the new value *orientation* which also cor-

for right and left side, respectively. These

An Evaluation of Three Different Infield Navigation Algorithms

http://dx.doi.org/10.5772/intechopen.79942

33

) (1)

) (2)

line, it uses data from the on-board compass (IMU unit) and turns in the next row.

the middle of the rows and with as little oscillations as possible.

**2.1. Minimal row offset-based algorithm**

These average distances can be written as *d*<sup>r</sup>

two are then used to calculate an offset, as shown in Eq. 1:

*Orientation* = *Offset*/(*dr* + *dl*

responds directly how much the wheels should turn in radians.

*Offset* = (*dr* − *dl*

as shown in **Figure 2**.

Eq. 2.

So, the purpose of this work is to first present three different algorithms that could be used to autonomously drive the machines and to evaluate their accuracy for infield navigation. All three were implemented as part of robotic operating system (ROS) [14] and applied on a small field robot, making it autonomous when driving through the field in mid-row spaces. Their purpose is not to plan and follow the path but to adjust the heading of the machines/robot at every measured location. The field robot that was used to test the algorithms is presented in Section 2, along with all three algorithms used, which are then evaluated in Section 3.

#### **2. Materials and methods**

The algorithms are tested on a small field robot [16] depicted in **Figure 1**. It is an electric driven robot with 50 × 50 cm in size, small enough to fit in 75 cm wide space between two parallel lines of maze plants. It has four in-wheel BLDC motors capable of delivering 200 W max peak power, equipped with four additional motors to individually turn each of the wheels, making possible to drive in different steering modes like skid steer or Ackermann. Besides the odometry from the wheels, the robot is equipped with three different additional sensors: two digital cameras, an IMU unit, and a LIDAR sensor. The data from the sensors are processed by two onboard computers, a low-level computer build around Raspberry PI 3B and Intel NUC i7 (gen. 7) computer, used as high-level processing unit. The two units run a Linux-based distribution with robotic (meta) operating system (ROS) on top, configured in multimachine mode to split the essential processing from time consuming, advance algorithms, making the robot as responsive as possible.

**Figure 1.** A small field robot FarmBeast while performing Task 1—Basic navigation at Field Robot Event 2018.

If the robot, as the one in **Figure 1**, has to drive autonomously through the crop fields, it must have a navigation algorithm. The navigation algorithm relies on the distance measurements from the LIDAR sensor. The experimental robot is equipped with the SICK TIM310 LIDAR, with a 270° area at a 1° angular resolution. It detects obstacles up to 4 m away and then chooses the optimal path based on the readings. Once the robot reaches the end of the crop line, it uses data from the on-board compass (IMU unit) and turns in the next row.

In this chapter, we investigate the accuracy of three different navigation algorithms and compare them, with left and right row distance as a reference for optimal path that should be in the middle of the rows and with as little oscillations as possible.

#### **2.1. Minimal row offset-based algorithm**

when the crop needs to be treated [3]. This is not always possible, if no prior GPS information exists, or simply because an accurate GPS system is not always available. So, different systems

A possible solution to solve this is to use cameras to detect plant lines [4] or even stereo cameras to build a 3D cloud of points [5], describing the plants, and drive the agricultural machinery to drive between them. Another possible approach is the use of LIDAR systems [6] that take accurate measurements of the scene, by reading distances from the sensor to the first obstacle and repeating this for the whole range (usually 270° or 360° in 1°, 0.5° or even

With the help of these systems, the machines [7–9] can drive even in an unfamiliar field where they have not been applied before. They rely on the property of the field and the crops planted. The plants are in parallel to each other, and the machines can drive between the crop lines, in mid-row spaces, in order to not damage the plants. This can be done by using differ-

More advance systems using different SLAM methods [10, 11] even build a map of the environment and localize the machines in a new environment that is only being discovered. Based on the map, it then constructs a path using path planner [12] and path follower [13] to follow the path. These approaches come in use in case of unfamiliar scenes but are not really necessary in infield situations where prior information regarding the pattern of the

So, the purpose of this work is to first present three different algorithms that could be used to autonomously drive the machines and to evaluate their accuracy for infield navigation. All three were implemented as part of robotic operating system (ROS) [14] and applied on a small field robot, making it autonomous when driving through the field in mid-row spaces. Their purpose is not to plan and follow the path but to adjust the heading of the machines/robot at every measured location. The field robot that was used to test the algorithms is presented in

The algorithms are tested on a small field robot [16] depicted in **Figure 1**. It is an electric driven robot with 50 × 50 cm in size, small enough to fit in 75 cm wide space between two parallel lines of maze plants. It has four in-wheel BLDC motors capable of delivering 200 W max peak power, equipped with four additional motors to individually turn each of the wheels, making possible to drive in different steering modes like skid steer or Ackermann. Besides the odometry from the wheels, the robot is equipped with three different additional sensors: two digital cameras, an IMU unit, and a LIDAR sensor. The data from the sensors are processed by two onboard computers, a low-level computer build around Raspberry PI 3B and Intel NUC i7 (gen. 7) computer, used as high-level processing unit. The two units run a Linux-based distribution with robotic (meta) operating system (ROS) on top, configured in multimachine mode to split the essential processing from time consuming, advance algorithms, making the

Section 2, along with all three algorithms used, which are then evaluated in Section 3.

should be used in these situations.

32 Agricultural Robots - Fundamentals and Applications

ent approaches/algorithms to guide the machines.

2.25° steps).

plants is known.

**2. Materials and methods**

robot as responsive as possible.

The first algorithm is the simplest one. It takes 30 readings from left and 30 from right side, as shown in **Figure 2**.

From these two sets, it first eliminates ones that are too far away, that is more than 0.75 cm and belong to the other crop lines, and then calculates an average distance value for each side. These average distances can be written as *d*<sup>r</sup> and *d*<sup>l</sup> for right and left side, respectively. These two are then used to calculate an offset, as shown in Eq. 1:

$$\text{Offset} = \{d\_r - d\_l\} \tag{1}$$

The value of the *offset* is then used to adjust the course of the robot, if necessary, as shown in Eq. 2.

$$\text{Orientation} = \text{Offset} / (d\_r + d\_l) \tag{2}$$

So, the current course of the robot is adjusted by the new value *orientation* which also corresponds directly how much the wheels should turn in radians.

#### **2.2. Least-square fit approach algorithm**

This second algorithm was designed to navigate the robot between two walls that are parallel to each other, this being either an artificial barrier, e.g., walls, or real crop lines, such as maze plants. The overview of the approach is depicted in **Figure 3**, explaining each sensing adjustment cycle.

(C) In the third step, a linear fit is used, for which a least squares method [17] was chosen. This way the slope and y-intercept of a linear equation describing each set for each side is

The least squares method allows us to linearly fit the measurements with a smaller number of heavy duty mathematical operations. For this, we need to define some additional parameters

Once these parameters are known, the slope (*k*) and y-intercept (*n*) can easily be calculated as

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ *number of elements* <sup>∗</sup> *<sup>x</sup>square sum* <sup>−</sup> *<sup>x</sup>sum*

(D) With the calculated slopes and y-intercepts, a crossing point is calculated where those two lines cross each other. This is the point, which depends on the rotation of the robot and its

Based on step (C), with calculated slope and intercept for each side, the following parameters

can be calculated using Eqs. (5) and (6):

*k*<sup>1</sup> − *k*<sup>2</sup>

*yL* = *k*<sup>1</sup> ∗ *xL* + *n*<sup>1</sup> (6)

<sup>2</sup> (3)

An Evaluation of Three Different Infield Navigation Algorithms

http://dx.doi.org/10.5772/intechopen.79942

35

<sup>2</sup> (4)

(5)

with which we then calculate the slope and y-intercept of each line:

*ysquare sum* − *sum of all the squared angles taken for the line* (*in degrees*),

*<sup>k</sup>* <sup>=</sup> *number of points* <sup>∗</sup> *yxsum* <sup>−</sup> *<sup>x</sup>sum* <sup>∗</sup> *<sup>y</sup>sum* \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_ *number of elements* <sup>∗</sup> *<sup>x</sup>square sum* <sup>−</sup> *<sup>x</sup>sum*

*yxsum* − *sum of the products of the angle and distance of each point*.

*<sup>n</sup>* <sup>=</sup> *<sup>x</sup>square sum* <sup>∗</sup> *<sup>y</sup>sum* <sup>−</sup> *<sup>x</sup>sum* <sup>∗</sup> *yxsum*

*xsum* − *sum of all the distances taken for the line*,

*xsquare sum* − *sum of all the squared distances taken for the line*,

*ysum* − *sum of all the angles taken for the line* (*in degrees*),

calculated.

shown in Eqs. 3 and 4:

position between the two walls.

kL—slope of the left side,

nL—y intercept of the left side,

nD—y intercept of the right side.

and *yL*

*xL* <sup>=</sup> *<sup>n</sup>*<sup>2</sup> <sup>−</sup> *<sup>n</sup>* \_\_\_\_\_<sup>1</sup>

kD—slope of the right side,

are defined:

and distances *xL*

(A) The sensor reads the data—the distances between the sensor and the obstacles for each degree—and triggers a callback function each time it completes the measuring sequence.

(B) The callback function first filters the data depending on the distance readings. The points that are too far away and points that are too near to count are discarded. The algorithm makes possible to set how many points should be included for each count for each side that corresponds to how many degrees will use in the subsequent steps of the algorithm. The useful readings are stored in two data sets, one for left and one for right side.

**Figure 2.** Robot standing in the middle of the row. Each double-sided arrow represents a measurement of the sensor, where the actual number of reading is 30 on each side. The sensor returns the distance for each degree it measures.

**Figure 3.** Performed steps in each cycle.

(C) In the third step, a linear fit is used, for which a least squares method [17] was chosen. This way the slope and y-intercept of a linear equation describing each set for each side is calculated.

The least squares method allows us to linearly fit the measurements with a smaller number of heavy duty mathematical operations. For this, we need to define some additional parameters with which we then calculate the slope and y-intercept of each line:

*xsum* − *sum of all the distances taken for the line*,

**2.2. Least-square fit approach algorithm**

34 Agricultural Robots - Fundamentals and Applications

adjustment cycle.

**Figure 3.** Performed steps in each cycle.

This second algorithm was designed to navigate the robot between two walls that are parallel to each other, this being either an artificial barrier, e.g., walls, or real crop lines, such as maze plants. The overview of the approach is depicted in **Figure 3**, explaining each sensing—

(A) The sensor reads the data—the distances between the sensor and the obstacles for each degree—and triggers a callback function each time it completes the measuring sequence.

(B) The callback function first filters the data depending on the distance readings. The points that are too far away and points that are too near to count are discarded. The algorithm makes possible to set how many points should be included for each count for each side that corresponds to how many degrees will use in the subsequent steps of the algorithm. The useful

**Figure 2.** Robot standing in the middle of the row. Each double-sided arrow represents a measurement of the sensor, where the actual number of reading is 30 on each side. The sensor returns the distance for each degree it measures.

readings are stored in two data sets, one for left and one for right side.

*xsquare sum* − *sum of all the squared distances taken for the line*,

*ysum* − *sum of all the angles taken for the line* (*in degrees*),

*ysquare sum* − *sum of all the squared angles taken for the line* (*in degrees*),

*yxsum* − *sum of the products of the angle and distance of each point*.

Once these parameters are known, the slope (*k*) and y-intercept (*n*) can easily be calculated as shown in Eqs. 3 and 4:

 $\text{showr in Eqs. 3 and 4:}\\

$$k = \frac{number \, of \, points \, \* \, yx\_{\text{sum}} - x\_{\text{sum}} \, \* \, y\_{\text{sum}}}{number \, of \, elements \, \* \, x\_{\text{sum}} - x\_{\text{sum}}^2} \tag{3}$$
$ 

$$\begin{array}{c} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \\\\ \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{1cm}} \text{\hspace{$$

(D) With the calculated slopes and y-intercepts, a crossing point is calculated where those two lines cross each other. This is the point, which depends on the rotation of the robot and its position between the two walls.

Based on step (C), with calculated slope and intercept for each side, the following parameters are defined:

kL—slope of the left side,

nL—y intercept of the left side,

kD—slope of the right side,

nD—y intercept of the right side.

and distances *xL* and *yL* can be calculated using Eqs. (5) and (6):

$$\mathbf{x}\_{L} = \frac{n\_{2} - n\_{1}}{k\_{1} - k\_{2}} \tag{5}$$

$$y\_L = k\_1 \* x\_L + n\_1 \tag{6}$$

(E) Once the information about the intersection from the two lines is known, the position of the robot is calculated. The distance yL stays constant, if the robot is aligned up with the row no matter which wall it is closer. The x<sup>L</sup> describes how far away it is from both of walls/lines. With just looking at the two distances, the problem is simplified and can be solved with Eqs. (1) and (2).

Based on the position of the robot, the described approach can describe two different situations. The first situation is when the robot is in the right location, as shown in **Figure 4**, and the second if the robot is not in the right location and needs adjustment, as shown in **Figure 5**. In both cases, the parameters of the two linear equations and the distance to the path of the robot are used.

Since the walls are always apart from each other with a constant width, the triangle covers the same surface. What changes is the orientation and position changes with the robot that produces different triangles. If the robot is not aligned in parallel to the walls or crop lines, an asymmetric triangle is constructed, as the one in **Figure 5**.

#### **2.3. Triangle-based navigation algorithm**

The third approach [15] of finding an optimal path for the robot consists out of multiple steps. As shown in **Figure 6**, the algorithm uses trigonometric functions to calculate the distance of a segment between every two sequential points from LIDAR sensor. The segment distance must be wide enough to drive the robot trough, and if they do not meet the criteria, they are disposed. This is depicted in **Figure 6** where red colored segment distances written as (d1 , d5, d6 , d<sup>8</sup> , d<sup>9</sup> , and d13) are disposed and green colored distances (d<sup>2</sup> , d3, d<sup>4</sup> , d7, d10, d11, and d12) are retained for further procedure.

**Figure 5.** This picture shows us what we can get from the position of the intersection of the lines (distance *y <sup>L</sup>*

**Figure 6.** The way the robot calculates each segment of distances between every two sequential points from LIDAR

sensor.

We can figure out the position and the rotation of the robot with the two lengths.

 and *x <sup>L</sup>* ). 37

An Evaluation of Three Different Infield Navigation Algorithms

http://dx.doi.org/10.5772/intechopen.79942

**Figure 4.** Arranged distances (blue) depicted on a graph that correspond to the length and the degree, it is located on. The red dotted line shows the two lines that are created out of these measurements. The green circle represents where the lines meet. This way a triangle is calculate and its shape depends on the position of the robot.

(E) Once the information about the intersection from the two lines is known, the position of the

looking at the two distances, the problem is simplified and can be solved with Eqs. (1) and (2). Based on the position of the robot, the described approach can describe two different situations. The first situation is when the robot is in the right location, as shown in **Figure 4**, and the second if the robot is not in the right location and needs adjustment, as shown in **Figure 5**. In both cases, the parameters of the two linear equations and the distance to the path of the

Since the walls are always apart from each other with a constant width, the triangle covers the same surface. What changes is the orientation and position changes with the robot that produces different triangles. If the robot is not aligned in parallel to the walls or crop lines, an

The third approach [15] of finding an optimal path for the robot consists out of multiple steps. As shown in **Figure 6**, the algorithm uses trigonometric functions to calculate the distance of a segment between every two sequential points from LIDAR sensor. The segment distance must be wide enough to drive the robot trough, and if they do not meet the criteria, they are disposed. This is depicted in **Figure 6** where red colored segment distances written as (d1

**Figure 4.** Arranged distances (blue) depicted on a graph that correspond to the length and the degree, it is located on. The red dotted line shows the two lines that are created out of these measurements. The green circle represents where

the lines meet. This way a triangle is calculate and its shape depends on the position of the robot.

asymmetric triangle is constructed, as the one in **Figure 5**.

, and d13) are disposed and green colored distances (d<sup>2</sup>

**2.3. Triangle-based navigation algorithm**

retained for further procedure.

stays constant, if the robot is aligned up with the row no mat-

, d3, d<sup>4</sup>

, d5,

, d7, d10, d11, and d12) are

describes how far away it is from both of walls/lines. With just

robot is calculated. The distance yL

36 Agricultural Robots - Fundamentals and Applications

ter which wall it is closer. The x<sup>L</sup>

robot are used.

d6 , d<sup>8</sup> , d<sup>9</sup>

**Figure 5.** This picture shows us what we can get from the position of the intersection of the lines (distance *y <sup>L</sup>* and *x <sup>L</sup>* ). We can figure out the position and the rotation of the robot with the two lengths.

**Figure 6.** The way the robot calculates each segment of distances between every two sequential points from LIDAR sensor.

**Figure 6** shows that only segment d7

is marked as green triangle.

neglectable with some algorithms.

mimicking two plant rows (**Figure 10**).

with readings depicted as colored dots.

**3.1. Simulated environment**

**3.2. Real environment**

segment.

**3. Results**

is appropriate for robot to drive trough. The procedure

An Evaluation of Three Different Infield Navigation Algorithms

http://dx.doi.org/10.5772/intechopen.79942

which pass the criteria

39

eliminates inappropriate segments as shown in **Figure 7**. Algorithm calculates the angles: α, β, and γ in triangle limited between two sequential points and LIDAR sensor. If any of the angles α or β is bigger than predefined threshold, set to 100° or more, which would produce a triangle that would not fit in field situations, the segment is disposed. Disposed segments

When the optimal segment is determined, the algorithm calculates the angle for the front wheels to turn. The midpoint of optimal segment is calculated for robot to drive to. Angle δ in **Figure 8** represents the angle for robot to turn front wheels to follow the midpoint of optimal

In order to test and compare all three algorithms from Section 2, two separate approaches are used. In the first, a simulated environment is built to test all three using precisely the same data sets in order to verify if they work. In the second approach, a real testing environment is

In contrast to the first experiment, where the algorithms were tested one time, the robot in the second approach is tested using one of three algorithms and repeated five times, for example, it drives between the two rows five times, in order to calculate an average absolute value with its standard deviation. The average value is computed for all three algorithms and evaluated as the average displacement from the center point and its standard deviation. The algorithm that preforms best should have the average closest to the real middle point, with as little deviation as possible, corresponding to a minimal oscillation pattern. **Figure 9** depicts an exaugurated principle of how the robot moves. The oscillation pattern is of course small and

A simulated environment was build using ROS stage simulator [18] in order to test all three algorithms before the application in real environment. In this experiment, all tests start at the same starting point, and the robot moves in semi-parallel direction to two artificial walls,

In the second experiment, the robot moves between two plant rows. In this experiment, the data sets are not precisely the same as the plants can move due to wind. Even in no wind situation, the data sets might differ due to quantization steps of the LIDAR that might not be at the same location each time. **Figure 11** depicts the environment used in this approach.

**Figure 12** depicts an example of how the both corn rows are sensed using a LIDAR sensor

are marked with red triangle in **Figure 7**, and the optimal segment d7

used in order to evaluate the approaches in uncontrolled environment.

**Figure 7.** The procedure eliminates inappropriate solutions.

**Figure 8.** Calculating front wheels turn angle δ.

**Figure 6** shows that only segment d7 is appropriate for robot to drive trough. The procedure eliminates inappropriate segments as shown in **Figure 7**. Algorithm calculates the angles: α, β, and γ in triangle limited between two sequential points and LIDAR sensor. If any of the angles α or β is bigger than predefined threshold, set to 100° or more, which would produce a triangle that would not fit in field situations, the segment is disposed. Disposed segments are marked with red triangle in **Figure 7**, and the optimal segment d7 which pass the criteria is marked as green triangle.

When the optimal segment is determined, the algorithm calculates the angle for the front wheels to turn. The midpoint of optimal segment is calculated for robot to drive to. Angle δ in **Figure 8** represents the angle for robot to turn front wheels to follow the midpoint of optimal segment.

## **3. Results**

**Figure 7.** The procedure eliminates inappropriate solutions.

38 Agricultural Robots - Fundamentals and Applications

**Figure 8.** Calculating front wheels turn angle δ.

In order to test and compare all three algorithms from Section 2, two separate approaches are used. In the first, a simulated environment is built to test all three using precisely the same data sets in order to verify if they work. In the second approach, a real testing environment is used in order to evaluate the approaches in uncontrolled environment.

In contrast to the first experiment, where the algorithms were tested one time, the robot in the second approach is tested using one of three algorithms and repeated five times, for example, it drives between the two rows five times, in order to calculate an average absolute value with its standard deviation. The average value is computed for all three algorithms and evaluated as the average displacement from the center point and its standard deviation. The algorithm that preforms best should have the average closest to the real middle point, with as little deviation as possible, corresponding to a minimal oscillation pattern. **Figure 9** depicts an exaugurated principle of how the robot moves. The oscillation pattern is of course small and neglectable with some algorithms.

#### **3.1. Simulated environment**

A simulated environment was build using ROS stage simulator [18] in order to test all three algorithms before the application in real environment. In this experiment, all tests start at the same starting point, and the robot moves in semi-parallel direction to two artificial walls, mimicking two plant rows (**Figure 10**).

#### **3.2. Real environment**

In the second experiment, the robot moves between two plant rows. In this experiment, the data sets are not precisely the same as the plants can move due to wind. Even in no wind situation, the data sets might differ due to quantization steps of the LIDAR that might not be at the same location each time. **Figure 11** depicts the environment used in this approach.

**Figure 12** depicts an example of how the both corn rows are sensed using a LIDAR sensor with readings depicted as colored dots.

**Figure 9.** An example how the robot moves with the simplest algorithm. The movement is depicted with red line, the optimal mid-row path with blue, and the plants with green markers.

**Figure 10.** A screenshot of the simulation environment STAGE used in this test.

The results of the second test are presented in **Table 1**, where in each iteration, the average distance from the mid-row path was calculated on 8 m long test runs, but this time using real plants in real environment.

The results in **Table 1** show that the best performing algorithm is the algorithm from Section 2.1 with an error of 0.041 ± 0.034 m, and the second and third are very close with an average error of 0.07 ± 0.059 m for the second from Section 2.2 and 0.078 ± 0.055 m error for the third from Section 2.3. It should be noted that all three performed well reaching the end of each row without any problems. The difference between the first and other two is that the first uses the values in small proximity to the sensor on either side, but the second and third use a bigger range on both sides making them more useful in situations when rows are not straight as in

**Figure 12.** The environment from **Figure 12** seen by LIDAR sensor mounted on the front of the robot. The dots at the

bottom center of the robot represent the wheels of the robot and are filtered out.

**Figure 11.** An environment with real plants with 8 m in length and 0.75 m in width.

An Evaluation of Three Different Infield Navigation Algorithms

http://dx.doi.org/10.5772/intechopen.79942

41

An Evaluation of Three Different Infield Navigation Algorithms http://dx.doi.org/10.5772/intechopen.79942 41

**Figure 11.** An environment with real plants with 8 m in length and 0.75 m in width.

**Figure 12.** The environment from **Figure 12** seen by LIDAR sensor mounted on the front of the robot. The dots at the bottom center of the robot represent the wheels of the robot and are filtered out.

The results in **Table 1** show that the best performing algorithm is the algorithm from Section 2.1 with an error of 0.041 ± 0.034 m, and the second and third are very close with an average error of 0.07 ± 0.059 m for the second from Section 2.2 and 0.078 ± 0.055 m error for the third from Section 2.3. It should be noted that all three performed well reaching the end of each row without any problems. The difference between the first and other two is that the first uses the values in small proximity to the sensor on either side, but the second and third use a bigger range on both sides making them more useful in situations when rows are not straight as in

The results of the second test are presented in **Table 1**, where in each iteration, the average distance from the mid-row path was calculated on 8 m long test runs, but this time using real

**Figure 9.** An example how the robot moves with the simplest algorithm. The movement is depicted with red line, the

optimal mid-row path with blue, and the plants with green markers.

40 Agricultural Robots - Fundamentals and Applications

**Figure 10.** A screenshot of the simulation environment STAGE used in this test.

plants in real environment.


**References**

2015. pp. 7-11

DOI: 10.5772/61199

10.1007/s10514-017-9667-4

ISBN-13: 978-1449323899

[1] Vazquez J, Lacarra E, Sanchez MA, Rioja J, Bruzual J. EDAS (EGNOS Data Access Service): Differential GPS corrections performance test with state-of-the-art precision agriculture system. In: 30th International Technical Meeting of The Satellite-Division-of-

An Evaluation of Three Different Infield Navigation Algorithms

http://dx.doi.org/10.5772/intechopen.79942

43

[2] Yang LL, Gao DH, Hoshino Y, Suzuki S, Cao Y, Yang SM. Evaluation of the accuracy of an auto-navigation system for a tractor in mountain areas. In: IEEE/SICE International Symposium on System Integration (SII). Taiwan. New York: IEEE; 2017. pp. 133-138 [3] Esau TJ, Zaman QU, Chang YK, Schumann AW, Percival DV, Farooque AA. Spotapplication of fungicide for wild blueberry using an automated prototype variable rate sprayer. Precision Agriculture. 2014;**15**(2):147-161. DOI: 10.1007/s11119-013-9319-4 [4] Jiang GQ, Wang ZH, Liu HM. Automatic detection of crop rows based on multi-ROIs. Expert systems with applications. 2015;**42**(5):2429-2441. DOI: 10.1016/j.eswa.2014.10.033 [5] Ball D, Upcroft B, Wyeth G, Corke P, English A, Ross P, Patten T, Fitch R, Sikkareih S, Bate A. Vision-based obstacle detection and navigation for an agricultural robot. Journal

[6] Kragh M, Jorgensen RN, Pedersen H. Object detection and terrain classification in agricultural fields using 3D Lidar data. In: 10th International Conference on Computer

[7] Reiser D, Ulrich G, Hübner R, Reihle D, Griepentrog HW. Hefty. In: Proceedings of the

[8] Kemmerling M, Sontag C, Schaub C, Wulferding H, Höverling S, Schröder T, Schneider L. Helios. In: Proceedings of the 14th Field Robot Event 2016. Germany; 2016. pp. 120-125 [9] Nachtigall L. Beteigeuze. In: Proceedings of the 13th Field Robot Event 2015. Slovenia;

[10] Kohlbrecher S, Meyer J, Graber T, Petersen K, Klingauf U, Stryk O. Hector Open Source Modules for Autonomous Mapping and Navigation with Rescue Robots. Germany:

[11] Lepej P, Rakun J. Simultaneous localisation and mapping in a complex field environ-

[12] Lepej P, Maurer J, Uran S, Steinbauer G. Dynamic arc fitting path follower for skidsteered mobile robots. International Journal of Advanced Robotic Systems. 2015;**12**:139.

[13] Shen TT, Radmard S, Chan A, Croft EA, Chesi G. Optimized vision-based robot motion planning from multiple demonstrations. Autonomous Robots. 2018;**42**(6):1117-1132. DOI:

[14] Quigley M, Gerkey B, Smart WD. Programming Robots with ROS: A Practical Introduction to the Robot Operating System. 1st ed. USA: O'Reilly Media Inc; 2015 448 p.

the-Institute-of-Navigation (ION GNSS+); Oregon; 2017. pp. 1988-1998

of Field Robotics. 2016;**33**(8):1107-1130. DOI: 10.1002/rob.21644

15th Field Robot Event 2017; United Kingdom; 2017. pp. 15-24

Vision Systems (ICVS). Denmark; 2015. pp. 188-197

Department of Computer Science, TU Darmstadt; 2013

ment. Biosystems Engineering. 2016;**150**:160-169. ISSN 1537-5110

**Table 1.** A performance comparison of three different algorithms from Section 2 using a real environment.

this experiment. The scenario in which the first algorithm might fail is in situation where the corn plants are bigger and the leaves from the corn over leap the mid-row space, obstructing a clear view for the sensor. In this case, the algorithms from Section 2.2 and 2.3 would be more efficient due to higher robustness in comparison to the first algorithm.

#### **4. Conclusion**

In this chapter, we presented three different algorithms for infield navigation and then tested them using first a simulated environment and, second, evaluated them in a real environment. The results from Section 3 show that all three algorithms perform good, with the best one in terms of optimal mid-row driving and minimal oscillation, the algorithm from Section 2.1 with an error of 0.041 ± 0.034 m, the second being algorithm from Section 2.2 with an error of 0.07 ± 0.059 m, and third, the algorithm from Section 2.3 with an error of 0.078 ± 0.055 m.

The algorithms presented in this chapter adjust the movement of the robot according to each iteration of measured distance sets recorded by LIDAR sensor. As a logical improvement to the best performing algorithms from Section 2, the accuracy could be improved by taking into account the measurements further away from the robot, measurements of the parallel lines, that are currently filtered out, as well as measurements of the previously driven row(s), as the rows are always in parallel to each other in which we get a prior information for the current rows. All this would further improve the performance of the algorithms.

#### **Author details**

Peter Bernad1 , Peter Lepej3 , Črtomir Rozman<sup>2</sup> , Karmen Pažek<sup>2</sup> and Jurij Rakun<sup>2</sup> \*

\*Address all correspondence to: jurij.rakun@um.si


#### **References**

this experiment. The scenario in which the first algorithm might fail is in situation where the corn plants are bigger and the leaves from the corn over leap the mid-row space, obstructing a clear view for the sensor. In this case, the algorithms from Section 2.2 and 2.3 would be more

**Table 1.** A performance comparison of three different algorithms from Section 2 using a real environment.

In this chapter, we presented three different algorithms for infield navigation and then tested them using first a simulated environment and, second, evaluated them in a real environment. The results from Section 3 show that all three algorithms perform good, with the best one in terms of optimal mid-row driving and minimal oscillation, the algorithm from Section 2.1 with an error of 0.041 ± 0.034 m, the second being algorithm from Section 2.2 with an error of 0.07 ± 0.059 m, and third, the algorithm from Section 2.3 with an error of 0.078 ± 0.055 m.

The algorithms presented in this chapter adjust the movement of the robot according to each iteration of measured distance sets recorded by LIDAR sensor. As a logical improvement to the best performing algorithms from Section 2, the accuracy could be improved by taking into account the measurements further away from the robot, measurements of the parallel lines, that are currently filtered out, as well as measurements of the previously driven row(s), as the rows are always in parallel to each other in which we get a prior information for the current

, Karmen Pažek<sup>2</sup>

and Jurij Rakun<sup>2</sup>

\*

efficient due to higher robustness in comparison to the first algorithm.

**Iteration Algorithm 2.1 Algorithm 2.2 Algorithm 2.3** First 0.04 ± 0.035 m 0.082 ± 0.06 m 0.069 ± 0.049 m Second 0.038 ± 0.03 m 0.063 ± 0.049 m 0.071 ± 0.045 m Third 0.045 ± 0.037 m 0.077 ± 0.073 m 0.073 ± 0.053 m Fourth 0.039 ± 0.03 m 0.063 ± 0.044 m 0.07 ± 0.052 m Fifth 0.042 ± 0.036 m 0.065 ± 0.056 m 0.099 ± 0.065 m AVERAGE: 0.041 ± 0.034 m 0.07 ± 0.059 m 0.078 ± 0.055 m

42 Agricultural Robots - Fundamentals and Applications

rows. All this would further improve the performance of the algorithms.

, Črtomir Rozman<sup>2</sup>

1 Faculty of Natural Sciences and Mathematics, University of Maribor, Hoče, Slovenia

2 Faculty of Agriculture and Life Sciences, University of Maribor, Hoče, Slovenia

**4. Conclusion**

**Author details**

, Peter Lepej3

3 Vistion d.o.o., Slovenska Bistrica, Slovenia

\*Address all correspondence to: jurij.rakun@um.si

Peter Bernad1


[15] Lakota M, Berk P, Rakun J, Kraner J. Cornstar. In: Proceedings of the 13th Field Robot Event. University of Maribor. Vol. 12; 2015. p. 20. ISBN 978-961-6317-48-1

**Chapter 4**

Provisional chapter

**Agricultural Robot for Intelligent Detection of**

Agricultural Robot for Intelligent Detection of Pyralidae

DOI: 10.5772/intechopen.79460

The Pyralidae insects are one of the main pests in economic crops. However, the manual detection and identification of Pyralidae insects are labor intensive and inefficient, and subjective factors can influence recognition accuracy. To address these shortcomings, an insect monitoring robot and a new method to recognize the Pyralidae insects are presented in this chapter. Firstly, the robot gets images by performing a fixed action and detects whether there are Pyralidae insects in the images. The recognition method obtains the total probability image by using reverse mapping of histogram and multi-template images, and then image contour can be extracted quickly and accurately by using constraint Otsu. Finally, according to the Hu moment characters, perimeter, and area characters, the contours can be filtrated, and recognition results with triangle mark can be obtained. According to the recognition results, the speed of the robot car and mechanical arm can be adjusted adaptively. The theoretical analysis and experimental results show that the proposed scheme has high timeliness and high recognition accuracy in the natural planting scene.

Keywords: pest detection and recognition, Pyralidae insects, reverse mapping,

The timely detection and identification of corn pests and diseases are one of the major tasks of agriculturists for social and environmental challenges, such as maintaining the stability of grain output and reducing environmental pollution caused by the use of pesticides. Pyralidae insects are one of the most common pests of maize [1], and it does great harm to the quality and yield of maize. The traditional manual monitoring not only requires a large amount of labor but also causes that detection is not timely due to human omissions. With the rapidly

> © 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited.

© 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use,

distribution, and reproduction in any medium, provided the original work is properly cited.

**Pyralidae Insects**

Abstract

1. Introduction

Insects

Zhuhua Hu, Boyi Liu and Yaochi Zhao

Zhuhua Hu, Boyi Liu and Yaochi Zhao

http://dx.doi.org/10.5772/intechopen.79460

Additional information is available at the end of the chapter

Additional information is available at the end of the chapter

multi-template matching, agricultural robot


#### **Agricultural Robot for Intelligent Detection of Pyralidae Insects** Agricultural Robot for Intelligent Detection of Pyralidae Insects

DOI: 10.5772/intechopen.79460

Zhuhua Hu, Boyi Liu and Yaochi Zhao Zhuhua Hu, Boyi Liu and Yaochi Zhao

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.79460

#### Abstract

[15] Lakota M, Berk P, Rakun J, Kraner J. Cornstar. In: Proceedings of the 13th Field Robot

[16] Bernad P, Zajc A, Rasl M, Lakota M, Rakun J. Farmbeast robot. In: Proceedings of the

[17] Zhao L, Ding J. Least squares approximations to lognormal sum distributions. IEEE Transactions on Vehicular Technology. 2007;**56**(2):991-997. DOI: 10.1109/TVT.2007.891467

[18] Gerkey BP, Vaughan RT, Howard A. The player/stage project: Tools for multi-robot and distributed sensor systems. In: Proceedings of the International Conference on Advanced

Event. University of Maribor. Vol. 12; 2015. p. 20. ISBN 978-961-6317-48-1

Field Robot Event. University of Hohenheim; 2018. (in press)

Robotics (ICAR 2003); 2003. pp. 317-323

44 Agricultural Robots - Fundamentals and Applications

The Pyralidae insects are one of the main pests in economic crops. However, the manual detection and identification of Pyralidae insects are labor intensive and inefficient, and subjective factors can influence recognition accuracy. To address these shortcomings, an insect monitoring robot and a new method to recognize the Pyralidae insects are presented in this chapter. Firstly, the robot gets images by performing a fixed action and detects whether there are Pyralidae insects in the images. The recognition method obtains the total probability image by using reverse mapping of histogram and multi-template images, and then image contour can be extracted quickly and accurately by using constraint Otsu. Finally, according to the Hu moment characters, perimeter, and area characters, the contours can be filtrated, and recognition results with triangle mark can be obtained. According to the recognition results, the speed of the robot car and mechanical arm can be adjusted adaptively. The theoretical analysis and experimental results show that the proposed scheme has high timeliness and high recognition accuracy in the natural planting scene.

Keywords: pest detection and recognition, Pyralidae insects, reverse mapping, multi-template matching, agricultural robot

#### 1. Introduction

The timely detection and identification of corn pests and diseases are one of the major tasks of agriculturists for social and environmental challenges, such as maintaining the stability of grain output and reducing environmental pollution caused by the use of pesticides. Pyralidae insects are one of the most common pests of maize [1], and it does great harm to the quality and yield of maize. The traditional manual monitoring not only requires a large amount of labor but also causes that detection is not timely due to human omissions. With the rapidly

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited. © 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

development of computer technology, the monitoring of diseases and insect pests based on computer vision has been feasible, which can greatly improve the real-time detection and recognition of pests [2].

common apple pests [23]. Based on the combination of an image processing algorithm and artificial neural networks, Espinoza et al. proposed an algorithm to detect and monitor adultstage whitefly (Bemisia tabaci) and thrip (Frankliniella occidentalis) in greenhouses, and the correct recognition rate reached above 0.92 [24]. Zhu et al. combined the color histogram with dual tree complex wavelet transform [25] and SVM [26] to recognize insects, which can improve the recognition rate of insects. Li et al. proposed a red spider recognition method based on k-means clustering, which transformed the image into Lab color space for clustering [27]. This method had a high accuracy rate to identify red spider with obvious red features. However, the method can be only applied in the situation that there is high color contrast

Agricultural Robot for Intelligent Detection of Pyralidae Insects

http://dx.doi.org/10.5772/intechopen.79460

47

In addition, the device for image acquisition is also necessary [7]. Johannes et al. presented a scheme to diagnosis wheat disease automatically by using mobile capture devices [28]. In his research, a novel image processing algorithm based on candidate hot-spot detection in combination with statistical inference methods is proposed to tackle disease identification in wild

From the literature analysis in recent years, the image processing and computer vision technology have been widely used for the detection and recognition of diseases and pests and have achieved good results. Generally, the researchers used an existing method combined with image processing techniques to detect and identify clustering method, neural network, texture analysis, wavelet transform, the level set method, etc. However, it is difficult to have a universal method to detect and identify all pests. In general, the algorithms are used to detect and identify one or a class of pests. Moreover, most of the existing studies are often aimed at the greenhouse environment, and the researchers usually do not build a practical verification system. Obviously, deep learning can achieve high recognition accuracy, but this trainingbased approach is difficult to guarantee real time and requires a large amount of existing data

At present, there are still relatively few studies on the detection and identification of Pyralidae insects. In order to detect and identify the Pyralidae insects automatically and accurately in real time, we have researched in the following aspects. Firstly, a robot platform for pest monitoring is designed and fabricated. Then, a recognition scheme for Pyralidae insects is presented, in which the color feature of the image is used. Moreover, the histogram reverse mapping method and the multi-template image are used to obtain the general probability image superposition. Next, the image is segmented with constraint Otsu. Finally, the contours and Hu moments are used to automatically screen and identify the contours; thus, the contour of Pyralidae insects can be recognized. The scheme proposed in this chapter can recognize the

The rest of this chapter is organized as follows. Section 2 shows data acquisition equipment and its structure and also gives the whole detailed description of detection and recognition algorithm. In Section 3, we verify the monitoring robot's work and the proposed scheme of detection and recognition. In addition, we also evaluate the proposed scheme and discuss the

single target and also has good recognition ability for multiple targets.

results of the experiment. Finally, Section 4 concludes the chapter.

between the objects and the scenes.

conditions.

to train the model.

Currently, there have existed some methods to detect plant diseases or insect with image processing and computer vision technologies [3]. For example, Ali et al. used color histogram and textural descriptors to detect citrus diseases [4]. They took the use of color difference to separate the area affected by disease. Lu et al. used spectroscopy technology to detect anthracnose crown rot in strawberry [5]. Xie et al. employed the hyper-spectral images to detect whether there was gray mold disease in tomato leaves [6]. In addition, the researchers constructed an automated detection and monitoring system for the detection of small pests in the greenhouse, such as whitefly, etc., which can effectively monitor the tiny insects and their densities [7–10]. Meanwhile, computer vision technology was also used for aphid detection and monitoring of its population [11]. For the parasites on strawberry plants, support vector machine (SVM) method combined with the image processing technique was successful in detecting the thrips with an error less than 2.5% in the greenhouse environment [12]. The incorporation k-means clustering methodology with image processing was used to segment the pests or any object from the image [13]. Dai and Man used a convolutional Riemannian texture with differential entropic active contours to distinguish the background regions and expose pest regions [14]. Zhao et al. obtained accurate contour of crop diseases and insect pests for the following recognition, taking the use of texture difference and active contour guided by the texture difference [15]. In their further research, they also proposed image segmentation method for fruits with diseases based on constraint Otsu and level set active contour [16]. However, they did not research on identification.

As for the recognition of insects and diseases, some recent research advances can be classified into the two categories. The first category focuses on the image processing and computer vision technologies without requiring data training. Pest recognition method based on sparse representation and multi-feature fusion was proposed, which mainly used to identify beetles [17]. Four methods for the diagnosis and classification of the diseases of corn leaf were presented by using image processing and machine vision techniques [18]. Martin et al. proposed an extended region growing algorithm, which can identify the pest and have the counting of the pest to predict the pesticide amount to be used [19]. Przybyłowicz et al. developed a technique based on wing measurements, which can be an effective tool for monitoring of the European corn borer [20].

The second category concentrated on the training of data models, which mainly used machine learning and neural network technology. The method based on difference of Gaussian filter and local configuration pattern algorithm was used to extract the invariant features of the pest images, and then these features were put to a linear SVM (support vector machine) for pest recognition with recognition rate of 89% [21]. Kohonen's Self-Organizing Maps neural network was used to identify the extracted insect pests caught by a sticky trap [22]. In addition, Boniecki et al. proposed a classification neural model using optimized learning sets acquired based on the information encoded, which can be used to accurately identify the six most common apple pests [23]. Based on the combination of an image processing algorithm and artificial neural networks, Espinoza et al. proposed an algorithm to detect and monitor adultstage whitefly (Bemisia tabaci) and thrip (Frankliniella occidentalis) in greenhouses, and the correct recognition rate reached above 0.92 [24]. Zhu et al. combined the color histogram with dual tree complex wavelet transform [25] and SVM [26] to recognize insects, which can improve the recognition rate of insects. Li et al. proposed a red spider recognition method based on k-means clustering, which transformed the image into Lab color space for clustering [27]. This method had a high accuracy rate to identify red spider with obvious red features. However, the method can be only applied in the situation that there is high color contrast between the objects and the scenes.

development of computer technology, the monitoring of diseases and insect pests based on computer vision has been feasible, which can greatly improve the real-time detection and

Currently, there have existed some methods to detect plant diseases or insect with image processing and computer vision technologies [3]. For example, Ali et al. used color histogram and textural descriptors to detect citrus diseases [4]. They took the use of color difference to separate the area affected by disease. Lu et al. used spectroscopy technology to detect anthracnose crown rot in strawberry [5]. Xie et al. employed the hyper-spectral images to detect whether there was gray mold disease in tomato leaves [6]. In addition, the researchers constructed an automated detection and monitoring system for the detection of small pests in the greenhouse, such as whitefly, etc., which can effectively monitor the tiny insects and their densities [7–10]. Meanwhile, computer vision technology was also used for aphid detection and monitoring of its population [11]. For the parasites on strawberry plants, support vector machine (SVM) method combined with the image processing technique was successful in detecting the thrips with an error less than 2.5% in the greenhouse environment [12]. The incorporation k-means clustering methodology with image processing was used to segment the pests or any object from the image [13]. Dai and Man used a convolutional Riemannian texture with differential entropic active contours to distinguish the background regions and expose pest regions [14]. Zhao et al. obtained accurate contour of crop diseases and insect pests for the following recognition, taking the use of texture difference and active contour guided by the texture difference [15]. In their further research, they also proposed image segmentation method for fruits with diseases based on constraint Otsu and level set active contour [16].

As for the recognition of insects and diseases, some recent research advances can be classified into the two categories. The first category focuses on the image processing and computer vision technologies without requiring data training. Pest recognition method based on sparse representation and multi-feature fusion was proposed, which mainly used to identify beetles [17]. Four methods for the diagnosis and classification of the diseases of corn leaf were presented by using image processing and machine vision techniques [18]. Martin et al. proposed an extended region growing algorithm, which can identify the pest and have the counting of the pest to predict the pesticide amount to be used [19]. Przybyłowicz et al. developed a technique based on wing measurements, which can be an effective tool for monitoring of the

The second category concentrated on the training of data models, which mainly used machine learning and neural network technology. The method based on difference of Gaussian filter and local configuration pattern algorithm was used to extract the invariant features of the pest images, and then these features were put to a linear SVM (support vector machine) for pest recognition with recognition rate of 89% [21]. Kohonen's Self-Organizing Maps neural network was used to identify the extracted insect pests caught by a sticky trap [22]. In addition, Boniecki et al. proposed a classification neural model using optimized learning sets acquired based on the information encoded, which can be used to accurately identify the six most

recognition of pests [2].

46 Agricultural Robots - Fundamentals and Applications

However, they did not research on identification.

European corn borer [20].

In addition, the device for image acquisition is also necessary [7]. Johannes et al. presented a scheme to diagnosis wheat disease automatically by using mobile capture devices [28]. In his research, a novel image processing algorithm based on candidate hot-spot detection in combination with statistical inference methods is proposed to tackle disease identification in wild conditions.

From the literature analysis in recent years, the image processing and computer vision technology have been widely used for the detection and recognition of diseases and pests and have achieved good results. Generally, the researchers used an existing method combined with image processing techniques to detect and identify clustering method, neural network, texture analysis, wavelet transform, the level set method, etc. However, it is difficult to have a universal method to detect and identify all pests. In general, the algorithms are used to detect and identify one or a class of pests. Moreover, most of the existing studies are often aimed at the greenhouse environment, and the researchers usually do not build a practical verification system. Obviously, deep learning can achieve high recognition accuracy, but this trainingbased approach is difficult to guarantee real time and requires a large amount of existing data to train the model.

At present, there are still relatively few studies on the detection and identification of Pyralidae insects. In order to detect and identify the Pyralidae insects automatically and accurately in real time, we have researched in the following aspects. Firstly, a robot platform for pest monitoring is designed and fabricated. Then, a recognition scheme for Pyralidae insects is presented, in which the color feature of the image is used. Moreover, the histogram reverse mapping method and the multi-template image are used to obtain the general probability image superposition. Next, the image is segmented with constraint Otsu. Finally, the contours and Hu moments are used to automatically screen and identify the contours; thus, the contour of Pyralidae insects can be recognized. The scheme proposed in this chapter can recognize the single target and also has good recognition ability for multiple targets.

The rest of this chapter is organized as follows. Section 2 shows data acquisition equipment and its structure and also gives the whole detailed description of detection and recognition algorithm. In Section 3, we verify the monitoring robot's work and the proposed scheme of detection and recognition. In addition, we also evaluate the proposed scheme and discuss the results of the experiment. Finally, Section 4 concludes the chapter.

#### 2. Materials and methods

#### 2.1. Acquisition of Pyralidae insect data source

The image data used in this study are collected by an Automatic Detection and Identification System for Pests and Diseases. The system has been installed at the zone of technology application and demonstration of Hainan University in Hainan province, China. The system prototype and structure diagram is shown in Figure 1. The basic structure of the system can be divided into five major parts: the camera sensor (automatic focusing, resolution 1600 1200 and camera model KS2A01AF) and display unit, trap unit, the power delivery unit, the intelligent detection and recognition unit, and the hardware bearer unit.

probability image is obtained by the principle of histogram reverse mapping using the H layer histogram of multiple template images, and then the module binarizes the probability image. Subsequently, in the contour extraction module, the method obtains the contour of the binary image with the help of the function named findContours() in OpenCV. The contours of the internal holes are removed by morphological methods, which are screened according to the circumference and area features. Finally, in the target recognition module, the scheme recognizes the contour by calculating the similarity between the contour obtained in the previous steps and the template contour. The outline of the contour larger than the threshold is considered to be the target contour, and finally we can get the recognition result. The pseudo-code

2.3. Probability image acquisition based on color histogram reverse projection and multi-

The adults of the Pyralidae insects are yellowish brown. The male moths are 10–13 mm long, and the wing can reach 20–30 mm. The back of the Pyralidae insects is yellowish brown, and the end of the abdomen is relatively thin and pointed. Usually, they have a pair of filamentous antennae, which are grayish brown. Meanwhile, its forewing is tan, with two brown wavy stripes, and there are two yellowish brown short patterns between the two lines. In addition,

� �, <sup>α</sup>2; <sup>β</sup><sup>2</sup>

� �, <sup>α</sup>3; <sup>β</sup><sup>3</sup> � �

Agricultural Robot for Intelligent Detection of Pyralidae Insects

http://dx.doi.org/10.5772/intechopen.79460

49

corresponding to the scheme is shown in Table 1.

Algorithm: Recognition scheme of Pyralidae insects Input: S (target image); Mx (reference image);

S=(V-min(R,G,B))�255÷V if V!=0, 0 otherwise

/\*Similarity(),Calculate the histogram similarity \*/

6: R = OSTU(G); /\* Binarize the image by Otsu method \*/

Triangle Approximate processing for triangle contour;

Table 1. The pseudo-code description of the proposed scheme.

ð Þ� G � B 60÷S if V ¼ R 180 þ ð Þ� B � R 60÷S if V ¼ G 240 þ ð Þ� R � G 60÷S if V ¼ B

Output: Three vertices of triangular markings on the Pyralidae insects α1; β<sup>1</sup>

The color histogram of each image is obtained by statistics: H Xi=Hpi÷(Hm�Hn);

Gij=Similarity(H of {Image blocks with the same size as Mx}, H of Mx)

7: C=findContours(R) /\*findContours() extracts the contours from binary images\*/

8: real\_match Based on Hu moment feature, calculating the similarity between C and the template contour

2: Setting: The threshold of Hu moments; Reference contour image Yimage

template matching

1: Initialize: (R, G, B) S, Mx

Normalized (H);

3: V=max(R, G, B);

8 >< >: 4: for i=0:1:255

end for 5: for i=0:1:m for j=0:1:n

> end for end for

9: if real\_match > match:

else:delete R

Output vertex coordinates

H ¼

#### 2.2. Description of proposed scheme

In this chapter, the recognition scheme for Pyralidae insects based on reverse mapping of histogram and contour template matching is mainly divided into input module, reference image processing module, image segmentation module, contour extraction module, and target recognition module. The input module firstly converts the experimental image into a matrix and initializes the parameters such as contour recognition threshold and the binarization threshold of the probability image. Then, the reference image processing module makes space conversion for the reference image, transforms the image from RGB space to HSV space, and extracts the histogram of the color layer (H layer). After that, the image segmentation module is to extract the color histogram of the experimental image. After normalization, the total

Figure 1. The intelligent recognition of robot car for Pyralidae insects. (1) Deep grooved wheel, (2) shell, (3) guardrail, (4) screen display, (5) camera, (6) mechanical arm, (7) vertical thread screw, (8) screw guardrail, (9) solar panels, (10) sensor integrator, (11) horizontal screw motor, (12) trap lamp, (13) the hardcore, (14) crossbar, (15) insect collecting board, (16) vertical thread screw-driven motor, (17) chassis, (18) car control buttons, (19) horizontal thread screw, and (20) trap top cover.

probability image is obtained by the principle of histogram reverse mapping using the H layer histogram of multiple template images, and then the module binarizes the probability image. Subsequently, in the contour extraction module, the method obtains the contour of the binary image with the help of the function named findContours() in OpenCV. The contours of the internal holes are removed by morphological methods, which are screened according to the circumference and area features. Finally, in the target recognition module, the scheme recognizes the contour by calculating the similarity between the contour obtained in the previous steps and the template contour. The outline of the contour larger than the threshold is considered to be the target contour, and finally we can get the recognition result. The pseudo-code corresponding to the scheme is shown in Table 1.

#### 2.3. Probability image acquisition based on color histogram reverse projection and multitemplate matching

The adults of the Pyralidae insects are yellowish brown. The male moths are 10–13 mm long, and the wing can reach 20–30 mm. The back of the Pyralidae insects is yellowish brown, and the end of the abdomen is relatively thin and pointed. Usually, they have a pair of filamentous antennae, which are grayish brown. Meanwhile, its forewing is tan, with two brown wavy stripes, and there are two yellowish brown short patterns between the two lines. In addition,

Algorithm: Recognition scheme of Pyralidae insects

```
Input: S (target image); Mx (reference image);
```
2. Materials and methods

48 Agricultural Robots - Fundamentals and Applications

2.2. Description of proposed scheme

2.1. Acquisition of Pyralidae insect data source

The image data used in this study are collected by an Automatic Detection and Identification System for Pests and Diseases. The system has been installed at the zone of technology application and demonstration of Hainan University in Hainan province, China. The system prototype and structure diagram is shown in Figure 1. The basic structure of the system can be divided into five major parts: the camera sensor (automatic focusing, resolution 1600 1200 and camera model KS2A01AF) and display unit, trap unit, the power delivery unit, the

In this chapter, the recognition scheme for Pyralidae insects based on reverse mapping of histogram and contour template matching is mainly divided into input module, reference image processing module, image segmentation module, contour extraction module, and target recognition module. The input module firstly converts the experimental image into a matrix and initializes the parameters such as contour recognition threshold and the binarization threshold of the probability image. Then, the reference image processing module makes space conversion for the reference image, transforms the image from RGB space to HSV space, and extracts the histogram of the color layer (H layer). After that, the image segmentation module is to extract the color histogram of the experimental image. After normalization, the total

Figure 1. The intelligent recognition of robot car for Pyralidae insects. (1) Deep grooved wheel, (2) shell, (3) guardrail, (4) screen display, (5) camera, (6) mechanical arm, (7) vertical thread screw, (8) screw guardrail, (9) solar panels, (10) sensor integrator, (11) horizontal screw motor, (12) trap lamp, (13) the hardcore, (14) crossbar, (15) insect collecting board, (16) vertical thread screw-driven motor, (17) chassis, (18) car control buttons, (19) horizontal thread screw, and (20) trap top cover.

intelligent detection and recognition unit, and the hardware bearer unit.

```
Output: Three vertices of triangular markings on the Pyralidae insects α1; β1
                                                                     � �, α2; β2
                                                                             � �, α3; β3
                                                                                     � �
1: Initialize: (R, G, B) S, Mx
2: Setting: The threshold of Hu moments; Reference contour image Yimage
3: V=max(R, G, B);
  S=(V-min(R,G,B))�255÷V if V!=0, 0 otherwise
  H ¼
         ð Þ� G � B 60÷S if V ¼ R
         180 þ ð Þ� B � R 60÷S if V ¼ G
         240 þ ð Þ� R � G 60÷S if V ¼ B
       8
       ><
       >:
4: for i=0:1:255
      The color histogram of each image is obtained by statistics: H Xi=Hpi÷(Hm�Hn);
      Normalized (H);
     end for
5: for i=0:1:m
    for j=0:1:n
     Gij=Similarity(H of {Image blocks with the same size as Mx}, H of Mx)
/*Similarity(),Calculate the histogram similarity */
    end for
  end for
6: R = OSTU(G); /* Binarize the image by Otsu method */
7: C=findContours(R) /*findContours() extracts the contours from binary images*/
8: real_match Based on Hu moment feature, calculating the similarity between C and the template contour
9: if real_match > match:
      Triangle Approximate processing for triangle contour;
      Output vertex coordinates
    else:delete R
```
Table 1. The pseudo-code description of the proposed scheme.

the hind wings of the Pyralidae insects are grayish brown; especially, female moths are similar in shape to male moths with lighter shades, yellowish veins, lightly brown texture, and obese abdomen. From these characteristics, the color characteristics of adult Pyralidae insects are obvious, and it is very effective to recognize the Pyralidae insects by color characteristics. Color histograms are often used to describe color features and are particularly useful for describing images that are difficult to segment automatically.

In addition, due to the differences in the color and texture between different Pyralidae insects in natural scenes, it is necessary to use a plurality of template images for histogram reverse projection processing, which can avoid the use of a template that cannot be adapted to a variety of different scene situations. As shown in Table 3, three template images are given. The total probability image obtained by this method is shown in Eq. (3), where M represents

> X M

Similarity Hð Þ <sup>1</sup>; Hm (3)

Agricultural Robot for Intelligent Detection of Pyralidae Insects

http://dx.doi.org/10.5772/intechopen.79460

51

μ ¼ μ<sup>1</sup> � ω<sup>1</sup> þ μ<sup>2</sup> � ω<sup>2</sup> s:t:ω<sup>1</sup> þ ω<sup>2</sup> ¼ 1 (4)

� �<sup>2</sup> n o (5)

� �<sup>2</sup> n o <sup>p</sup>min <sup>≤</sup> <sup>T</sup> <sup>≤</sup> <sup>p</sup>max (6)

� � � � , which indicates that the Otsu threshold is biased to a

� �<sup>2</sup> n o gotsu <sup>≤</sup> <sup>T</sup> <sup>≤</sup> <sup>p</sup>max (7)

� �<sup>2</sup> <sup>þ</sup> <sup>ω</sup><sup>2</sup> � <sup>μ</sup> � <sup>μ</sup><sup>2</sup>

m¼1

The Otsu algorithm is also known as maximum between-class variance method [30], sometimes called the Otsu algorithm, which is considered to be the best algorithm of selecting the threshold in image segmentation. For the image G xð Þ ; y , the split threshold is set as T, ω<sup>1</sup> is the proportion of foreground pixels, μ<sup>1</sup> is the average grayscale of foreground image, ω<sup>2</sup> is the proportion background pixels, μ<sup>2</sup> is the average grayscale of background image, μ is the total average grayscale of background image, and g is the maximum between-class variance. pmin and pmax are, respectively, the minimum and maximum values of the pixel values in the image.

Substitute Eq. (4) into Eq. (5), and then the Otsu solution expression for threshold is as below:

Finally, by using the method of traverse, the threshold of the maximum between-class variance of the image is obtained. Inspired by the literature [16], the variance of the similarity value of the background area is smaller because of the variance of the similarity degree of the Pyralidae insect area and the diversity of the natural scene. In addition, the similarity of the Pyralidae insects is larger than that of the background. Therefore, the Otsu threshold will be biased toward the background, which can lead smaller threshold compared with the actual optimal threshold. After that, the Otsu constrained spatial segmentation method is used to obtain the gotsu firstly, and then a threshold for maximizing the between-class variance is obtained in the constraint space (between gotsu and pmax), as shown in Eq. (7), where gotsu is a simple calculation

the number of template images. The results obtained are shown in Table 4.

Similarity Hð Þ¼ <sup>1</sup>

gotsu ¼ argmax ω<sup>1</sup> � μ � μ<sup>1</sup>

gotsu ¼ argmax ω<sup>1</sup> � ω<sup>2</sup> � μ<sup>1</sup> � μ<sup>2</sup>

<sup>2</sup> μ<sup>1</sup> þ μ<sup>2</sup>

larger variance for the image with a large difference between the two variance values:

goptimal ¼ argmax ω<sup>1</sup> � ω<sup>2</sup> � μ<sup>1</sup> � μ<sup>2</sup>

2.4. Otsu image segmentation based on constrained space

Then, we can get

method [31], that is, gotsu <sup>¼</sup> <sup>1</sup>

The inverse projection of the histogram is proposed by Michael J. Swain and Dana H. Ballard [29], which is a form of record that shows how the pixel or pixel block adapts to the histogram model allocation. It can be used to segment image or find interesting content in the image. The output of the algorithm is an image of the same size as the input image, where the value of the pixel represents the probability that it belongs to the target image. Therefore, it is possible to obtain a probability image by mapping the histogram in the target image by using the template image of the Pyralidae insects. Considering the Pyralidae insect's highlight color feature and the functional characteristics of histogram reflective algorithm, the scheme proposed in this chapter applies the image grayscale processing based on the reflection of the color histogram to the color feature extraction step. After the target image and the template image are converted into the HSV space and the color layer (i.e., the H component) is extracted, the image is grayed out by the method of histogram mapping. The gray image obtained in this way is a probability image that reflects the degree of similarity to the target color. Thus, it realizes the color distribution feature screening of the target image. The algorithm flow is shown below:


$$H\_k^{'} = H\_k(i) - \frac{1}{N} \times \sum\_{j}^{N} H\_k(j) \tag{1}$$

$$\text{Similarity}(H\_1, H\_2) = \frac{\sum\_{i}^{N} H\_1'(i) \times H\_2'(i)}{\sqrt{\sum\_{i}^{N} H\_1'^2(i) \times H\_2'^2(i)}} \tag{2}$$

In Eqs. (1) and (2), k∈ f g 1; 2 , i ¼ j∈ f g 1; 2; 3; …; N , N is the number of intervals in the histogram, and Hkð Þi is the value of the ith interval in the kth histogram. Similarity Hð <sup>1</sup> ; H2Þ is the similarity between histogram H<sup>1</sup> and histogram H2. The degree of similarity reflects the color characteristics of the pixel which are in line with the probability of Asian Pyralidae insects.

In addition, due to the differences in the color and texture between different Pyralidae insects in natural scenes, it is necessary to use a plurality of template images for histogram reverse projection processing, which can avoid the use of a template that cannot be adapted to a variety of different scene situations. As shown in Table 3, three template images are given. The total probability image obtained by this method is shown in Eq. (3), where M represents the number of template images. The results obtained are shown in Table 4.

$$\text{Similarity}(H\_1) = \sum\_{m=1}^{M} \text{Similarity}(H\_1, H\_m) \tag{3}$$

#### 2.4. Otsu image segmentation based on constrained space

the hind wings of the Pyralidae insects are grayish brown; especially, female moths are similar in shape to male moths with lighter shades, yellowish veins, lightly brown texture, and obese abdomen. From these characteristics, the color characteristics of adult Pyralidae insects are obvious, and it is very effective to recognize the Pyralidae insects by color characteristics. Color histograms are often used to describe color features and are particularly useful for describing

The inverse projection of the histogram is proposed by Michael J. Swain and Dana H. Ballard [29], which is a form of record that shows how the pixel or pixel block adapts to the histogram model allocation. It can be used to segment image or find interesting content in the image. The output of the algorithm is an image of the same size as the input image, where the value of the pixel represents the probability that it belongs to the target image. Therefore, it is possible to obtain a probability image by mapping the histogram in the target image by using the template image of the Pyralidae insects. Considering the Pyralidae insect's highlight color feature and the functional characteristics of histogram reflective algorithm, the scheme proposed in this chapter applies the image grayscale processing based on the reflection of the color histogram to the color feature extraction step. After the target image and the template image are converted into the HSV space and the color layer (i.e., the H component) is extracted, the image is grayed out by the method of histogram mapping. The gray image obtained in this way is a probability image that reflects the degree of similarity to the target color. Thus, it realizes the color distribution feature screening of the target image. The algorithm flow is

1. Convert the reference image into HSV space; extract the H spatial matrix, statistically,

2. Start from the first pixel (x, y) of the experimental image, and cut temporary image that is the same size as the reference image, where (x, y) is the center pixel of the temporary

3. Calculate the similarity between the color histogram of the detected image H<sup>1</sup> and the

N �<sup>X</sup> N

In Eqs. (1) and (2), k∈ f g 1; 2 , i ¼ j∈ f g 1; 2; 3; …; N , N is the number of intervals in the histogram, and Hkð Þi is the value of the ith interval in the kth histogram. Similarity Hð <sup>1</sup> ; H2Þ is the similarity between histogram H<sup>1</sup> and histogram H2. The degree of similarity reflects the color characteristics of the pixel which are in line with the probability of Asian Pyral-

P<sup>N</sup> <sup>i</sup> H<sup>0</sup>

P<sup>N</sup> <sup>i</sup> <sup>H</sup>0<sup>2</sup>

j

<sup>1</sup>ðÞ�i H<sup>0</sup> <sup>2</sup>ð Þ<sup>i</sup> ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

<sup>1</sup>ðÞ�<sup>i</sup> <sup>H</sup>0<sup>2</sup>

<sup>2</sup>ð Þi

<sup>q</sup> (2)

Hkð Þj (1)

image. Extract the H space matrix, statistically its histogram, and normalize it.

color histogram of the reference image H2. The result is Similarity (H1, H2):

<sup>k</sup> <sup>¼</sup> HkðÞ�<sup>i</sup> <sup>1</sup>

H0

Similarity Hð Þ¼ <sup>1</sup>; H<sup>2</sup>

images that are difficult to segment automatically.

50 Agricultural Robots - Fundamentals and Applications

shown below:

idae insects.

histogram; and normalize it.

The Otsu algorithm is also known as maximum between-class variance method [30], sometimes called the Otsu algorithm, which is considered to be the best algorithm of selecting the threshold in image segmentation. For the image G xð Þ ; y , the split threshold is set as T, ω<sup>1</sup> is the proportion of foreground pixels, μ<sup>1</sup> is the average grayscale of foreground image, ω<sup>2</sup> is the proportion background pixels, μ<sup>2</sup> is the average grayscale of background image, μ is the total average grayscale of background image, and g is the maximum between-class variance. pmin and pmax are, respectively, the minimum and maximum values of the pixel values in the image. Then, we can get

$$
\mu = \mu\_1 \times a\nu\_1 + \mu\_2 \times a\nu\_2 \\
\text{s.t.} \\
\mu\_1 + a\nu\_2 = 1 \tag{4}
$$

$$\mathbf{g}\_{\text{otsu}} = \mathbf{argmax} \left\{ \omega\_1 \times \left( \mu - \mu\_1 \right)^2 + \omega\_2 \times \left( \mu - \mu\_2 \right)^2 \right\} \tag{5}$$

Substitute Eq. (4) into Eq. (5), and then the Otsu solution expression for threshold is as below:

$$\mathbf{g\_{otsu}} = \mathbf{argmax}\left\{\omega\_1 \times \omega\_2 \times \left(\mu\_1 - \mu\_2\right)^2\right\}
\\
p\_{\min} \le T \le p\_{\max}\tag{6}$$

Finally, by using the method of traverse, the threshold of the maximum between-class variance of the image is obtained. Inspired by the literature [16], the variance of the similarity value of the background area is smaller because of the variance of the similarity degree of the Pyralidae insect area and the diversity of the natural scene. In addition, the similarity of the Pyralidae insects is larger than that of the background. Therefore, the Otsu threshold will be biased toward the background, which can lead smaller threshold compared with the actual optimal threshold. After that, the Otsu constrained spatial segmentation method is used to obtain the gotsu firstly, and then a threshold for maximizing the between-class variance is obtained in the constraint space (between gotsu and pmax), as shown in Eq. (7), where gotsu is a simple calculation method [31], that is, gotsu <sup>¼</sup> <sup>1</sup> <sup>2</sup> μ<sup>1</sup> þ μ<sup>2</sup> � � � � , which indicates that the Otsu threshold is biased to a larger variance for the image with a large difference between the two variance values:

$$\mathbf{g}\_{\text{optimal}} = \text{argmax}\left\{\omega\_1 \times \omega\_2 \times \left(\mu\_1 - \mu\_2\right)^2\right\} \\ \mathbf{g}\_{\text{obs}} \le T \le p\_{\text{max}} \tag{7}$$

#### 2.5. Target contour recognition based on Hu moments

The moment feature mainly characterizes the geometric characteristics of the image area, also known as the geometric moment. Because it has the invariant characteristic of the rotation, translation, scale, and so on, so it is also called the invariant moment. In image processing, geometric invariant moments can be used as an important feature to represent objects, which can be used to classify an image. Among them, the invariant moments commonly used in humanoid recognition are mainly composed of Hu moments, Zernike moments, and so on. Hu moment is first proposed by M.K. Hu [32], and he gave the definition of Hu moments, the basic properties, and seven invariant moments with translation, rotation, and scaling invariance.

Specifically, we assume that the gray distribution in the target D region is f(x, y). In order to describe the target, the gray distribution outside the region D is considered to be 0, and then the geometric moment and the regional moment of the p þ q order are, respectively, expressed as follows:

$$m\_{pq} = \iint\_D \mathbf{x}^p y^q f(\mathbf{x}, y) d\mathbf{x} dy \tag{8}$$

3. Results and discussions

main parts and functions of the robot are shown in Table 2.

3.1. Probabilistic image acquisition experiment and analysis

The hardware environment of this scheme includes PC (Inter(R) Core(TM) i3-2500 CPU @3.30GHZ and 4.00GB RAM), embedded master development board (NVIDIA Jetson TX1), embedded auxiliary control development board (2 Raspberry Pi B+ and 6 Arduino uno r3 expansion board), camera module (KS2A01AF), etc. The software experiment environment includes Window 7 operating system, Python 2.7, OpenCV 2.4.13, and embedded Linux operating system. The images used in the experiment are collected from cameras on the robot arm. We gather about more than 200 photos of the Pyralidae insects for experiments. Some result images of detection are shown in Table 4. The robot can perform a well-designed motion, capture the images well, and identify Pyralidae insect object from the images. The

Agricultural Robot for Intelligent Detection of Pyralidae Insects

http://dx.doi.org/10.5772/intechopen.79460

53

After the image is converted to HSV space, the next step is that histogram reverse mapping is conducted by using the three template images for target images, and then we can obtain the probability image. The probability images obtained in the experiment are shown in Table 3. As shown in Table 3, there are probability images obtained after three template images make original image from the histogram reverse mapping. The image of the first column of lines 2–4 is the template image used by the current line. The first line of the table is the original image of the

> The sliding guide is used to move the robot arm

The part marked by the white circle is the robotic arm

Display screen for real-time display of images

The chassis is used to support and secure the sliding rails on the vehicle and can also be used to move the equipment

The camera is used to acquire images

Table 2. Image acquisition equipment: pest identification and environmental monitoring robot.

$$\mu\_{pq} = \iint\limits\_{D} (\mathbf{x} - \overline{\mathbf{x}})^p (y - \overline{y})^p f(\mathbf{x}, y) d\mathbf{x} dy \tag{9}$$

As shown in the above equation, mpq represents the p þ q order geometric moments of the image, and μpq represents the p þ q order center moments of the image. Calculating the two features of the reference contour image and the experimental contour image, we can use these two features to represent the contour. The similarity between the experimental contour and the reference contour is compared, and the similarity less than the threshold value of the contour is removed. Then, the rest of the contour is the contour of the Pyralidae insects. Finally, by using the function named approxPolyDP() in the OpenCV and other contour approximation processing functions, the contour is approximated to a triangle and marked. Obviously, the marked contour is the result we want.

#### 2.6. Recognition algorithm combined with robot control

Combining with robot operations is one of the innovations of this chapter. Depending on the result of the similarity detection, the robot arm can adjust the speed. When the similarity is greater than 0.9, the robot arm will stop moving; meanwhile, camera sensors continue to collect image data, and the robot will give an alarm. When the similarity is between 0.7 and 0.9, the movement of the robot will slow down. Using robot and image recognition in a coordinated manner, we can reduce the false alarm rate and missed detection rate. Meanwhile, when there is interference of other insects, the robot arm will stop or slow down, which can reduce the probability of false positives. Only when the similarity of five consecutive insect images is greater than 0.9, we can make the final decision on the presence of Pyralidae insects. Using this method, it can be prevented from being mistaken for other insects, so as to improve the correct recognition rate.

## 3. Results and discussions

2.5. Target contour recognition based on Hu moments

52 Agricultural Robots - Fundamentals and Applications

invariance.

as follows:

The moment feature mainly characterizes the geometric characteristics of the image area, also known as the geometric moment. Because it has the invariant characteristic of the rotation, translation, scale, and so on, so it is also called the invariant moment. In image processing, geometric invariant moments can be used as an important feature to represent objects, which can be used to classify an image. Among them, the invariant moments commonly used in humanoid recognition are mainly composed of Hu moments, Zernike moments, and so on. Hu moment is first proposed by M.K. Hu [32], and he gave the definition of Hu moments, the basic properties, and seven invariant moments with translation, rotation, and scaling

Specifically, we assume that the gray distribution in the target D region is f(x, y). In order to describe the target, the gray distribution outside the region D is considered to be 0, and then the geometric moment and the regional moment of the p þ q order are, respectively, expressed

f xð Þ ; y dxdy (8)

f xð Þ ; y dxdy (9)

mpq ¼

ðð

D

μpq ¼

marked contour is the result we want.

2.6. Recognition algorithm combined with robot control

ðð

D xp yq

ð Þ <sup>x</sup> � <sup>x</sup> <sup>p</sup>

ð Þ <sup>y</sup> � <sup>y</sup> <sup>p</sup>

As shown in the above equation, mpq represents the p þ q order geometric moments of the image, and μpq represents the p þ q order center moments of the image. Calculating the two features of the reference contour image and the experimental contour image, we can use these two features to represent the contour. The similarity between the experimental contour and the reference contour is compared, and the similarity less than the threshold value of the contour is removed. Then, the rest of the contour is the contour of the Pyralidae insects. Finally, by using the function named approxPolyDP() in the OpenCV and other contour approximation processing functions, the contour is approximated to a triangle and marked. Obviously, the

Combining with robot operations is one of the innovations of this chapter. Depending on the result of the similarity detection, the robot arm can adjust the speed. When the similarity is greater than 0.9, the robot arm will stop moving; meanwhile, camera sensors continue to collect image data, and the robot will give an alarm. When the similarity is between 0.7 and 0.9, the movement of the robot will slow down. Using robot and image recognition in a coordinated manner, we can reduce the false alarm rate and missed detection rate. Meanwhile, when there is interference of other insects, the robot arm will stop or slow down, which can reduce the probability of false positives. Only when the similarity of five consecutive insect images is greater than 0.9, we can make the final decision on the presence of Pyralidae insects. Using this method, it can be prevented from being mistaken for other insects, so as to improve the correct recognition rate.

The hardware environment of this scheme includes PC (Inter(R) Core(TM) i3-2500 CPU @3.30GHZ and 4.00GB RAM), embedded master development board (NVIDIA Jetson TX1), embedded auxiliary control development board (2 Raspberry Pi B+ and 6 Arduino uno r3 expansion board), camera module (KS2A01AF), etc. The software experiment environment includes Window 7 operating system, Python 2.7, OpenCV 2.4.13, and embedded Linux operating system. The images used in the experiment are collected from cameras on the robot arm. We gather about more than 200 photos of the Pyralidae insects for experiments. Some result images of detection are shown in Table 4. The robot can perform a well-designed motion, capture the images well, and identify Pyralidae insect object from the images. The main parts and functions of the robot are shown in Table 2.

#### 3.1. Probabilistic image acquisition experiment and analysis

After the image is converted to HSV space, the next step is that histogram reverse mapping is conducted by using the three template images for target images, and then we can obtain the probability image. The probability images obtained in the experiment are shown in Table 3.

As shown in Table 3, there are probability images obtained after three template images make original image from the histogram reverse mapping. The image of the first column of lines 2–4 is the template image used by the current line. The first line of the table is the original image of the

Table 2. Image acquisition equipment: pest identification and environmental monitoring robot.

Table 3. The original images and the obtained probability image after histogram reverse mapping.

five images containing Asian Pyralidae insects. The second line of the table is the probability image obtained by mapping the backward histogram with the template image 1. The third line of the table is the probability image obtained by mapping the backward histogram with the template image 2. The fourth line of the table is the probability image obtained by mapping the backward histogram with the template image 3. The last line of the table is the total probability image obtained by logical or operation and image erosion with the above three probability images.

different recognition results, the speed of the robot arm can be adjusted adaptively to improve the detection accuracy. Subsequently, we make statistics on time consumption and other indicators in the experimental results. The processing time is about 1 s on every image. So,

Agricultural Robot for Intelligent Detection of Pyralidae Insects

http://dx.doi.org/10.5772/intechopen.79460

55

Currently, recognition method based on ELM and deep learning has a rapid development. In theory, the use of these methods can get a higher correct rate. Unfortunately, the capture and establishment of such pest images of maize borers are very difficult. By now, there are few useful pictures we can take, which are far less than the minimum requirement for the number of image to be trained. Certainly, we also try to collect images through the trap. However, the background of the resulting images is single, which cannot meet the requirements. In addition, ELM and deep learning all have relatively high computational complexity and cannot meet the needs of real-time detection. So, based on the two reasons mentioned above, they are not feasible. Conversely, through the artificial summary of the characteristics of Pyralidae insects, the robot adaptively adjusts the sampling frequency to detect, which can achieve better accu-

the method proposed in this chapter can achieve real-time processing.

3.3. Comparison and analysis

Table 4. The recognition results and the robot arm action.

racy and good practicability.

As can be found in Table 3, the proposed method can avoid the situation that only one image used cannot adapt to a variety of different scenarios. It can be seen from the results of the final image after erosion operation that the total probability image obtained by multi-template image's logical operation has better effect.

#### 3.2. Experiment and analysis of maize borer

After obtaining the probability image, the contour extraction, matching, screening, and recognition experiments are carried out in this chapter. At the same time, triangle mark is to identify the results of recognition for the characteristics of Pyralidae insects shape, and recognition results are shown in Table 4.

As can be seen from Table 4, the scheme proposed can better identify the target containing Pyralidae insect images. The number marked on the pictures indicates similarity. While we use the triangle to identify the results of identification, better results are achieved. According to

Table 4. The recognition results and the robot arm action.

different recognition results, the speed of the robot arm can be adjusted adaptively to improve the detection accuracy. Subsequently, we make statistics on time consumption and other indicators in the experimental results. The processing time is about 1 s on every image. So, the method proposed in this chapter can achieve real-time processing.

#### 3.3. Comparison and analysis

five images containing Asian Pyralidae insects. The second line of the table is the probability image obtained by mapping the backward histogram with the template image 1. The third line of the table is the probability image obtained by mapping the backward histogram with the template image 2. The fourth line of the table is the probability image obtained by mapping the backward histogram with the template image 3. The last line of the table is the total probability image obtained by logical or operation and image erosion with the above three probability images.

Table 3. The original images and the obtained probability image after histogram reverse mapping.

As can be found in Table 3, the proposed method can avoid the situation that only one image used cannot adapt to a variety of different scenarios. It can be seen from the results of the final image after erosion operation that the total probability image obtained by multi-template

After obtaining the probability image, the contour extraction, matching, screening, and recognition experiments are carried out in this chapter. At the same time, triangle mark is to identify the results of recognition for the characteristics of Pyralidae insects shape, and recognition

As can be seen from Table 4, the scheme proposed can better identify the target containing Pyralidae insect images. The number marked on the pictures indicates similarity. While we use the triangle to identify the results of identification, better results are achieved. According to

image's logical operation has better effect.

54 Agricultural Robots - Fundamentals and Applications

results are shown in Table 4.

3.2. Experiment and analysis of maize borer

Currently, recognition method based on ELM and deep learning has a rapid development. In theory, the use of these methods can get a higher correct rate. Unfortunately, the capture and establishment of such pest images of maize borers are very difficult. By now, there are few useful pictures we can take, which are far less than the minimum requirement for the number of image to be trained. Certainly, we also try to collect images through the trap. However, the background of the resulting images is single, which cannot meet the requirements. In addition, ELM and deep learning all have relatively high computational complexity and cannot meet the needs of real-time detection. So, based on the two reasons mentioned above, they are not feasible. Conversely, through the artificial summary of the characteristics of Pyralidae insects, the robot adaptively adjusts the sampling frequency to detect, which can achieve better accuracy and good practicability.


Table 5. Comparison results of different schemes.

Finally, the proposed method is compared with the multi-structural element-based crop pest identification method proposed in [33] and the general histogram reverse mapping method. The experimental results are shown in Table 5. As can be seen from Table 5, the scheme of maize borer recognition proposed in this chapter has higher recognition rate, lower false alarm rate, and good application prospects. Besides, it is not necessary to carry out a large amount of data analysis, which ensures that the average time consumption is not significantly increased. In Table 5, the recognition rate and the false-positive rate are calculated as follows:

$$\begin{aligned} \stackrel{\sum}{n} r\_{i\dot{\jmath}}\\ \beta = \frac{\sum\_{i=1}^{n} r\_{i\dot{\jmath}}}{n} \left( r\_{i\dot{\jmath}} = 0 \text{ or } r\_{i\dot{\jmath}} = 1 \right) \end{aligned} \tag{10}$$

morphology is used to preprocess the obtained image, and then the image is binarized by histogram reverse mapping. Next, the binary image is processed by contour extraction and preliminary screening. Then, combining the reference contour image, the contours of Asian Pyralidae insect characteristics are selected by using the Hu moment feature. In the end, this chapter makes a statistical analysis of the experimental results, and the correct rate of recognition based on multi-template matching can reach nearly 94.3%. Compared with other methods, the time complexity of this scheme is basically the same as that of those, which can meet the

Agricultural Robot for Intelligent Detection of Pyralidae Insects

http://dx.doi.org/10.5772/intechopen.79460

57

The contents of this chapter were supported by the Key R&D Project of Hainan Province (Grant no. ZDYF2018015), the Hainan Province Natural Science Foundation of China (Grant no. 617033), the Open Sub-project of State Key Laboratory of Marine Resource Utilization in South China Sea (Grant no. 2016013B), and the Oriented Project of State Key Laboratory of

The authors declare that there is no conflict of interests regarding the publication of this

1 College of Information Science and Technology, Hainan University, Haikou, China

2 State Key Laboratory of Marine Resource Utilization in South China Sea, Hainan University,

[1] Wei TS, Zhu WF, Pang MH, Liu YC, Wang ZY, Dong JG. Influence of the damage of cotton bollworm and corn borer to ear rot in corn. Journal of Maize Sciences. 2013;21(4):116-118

Marine Resource Utilization in South China Sea (Grant no. DX2017012).

requirement of real-time detection.

Acknowledgements

Conflict of interest

Author details

Haikou, China

References

(in Chinese)

Zhuhua Hu1,2\*, Boyi Liu1,3 and Yaochi Zhao<sup>1</sup>

\*Address all correspondence to: yaochizi@163.com

3 University of Chinese Academy of Science, Beijing, China

chapter.

$$\begin{aligned} \text{x} &= \frac{\text{x} - \sum\_{i=1, i \neq 1}^{n\_{\text{ref}}} r\_{i\text{j}}}{\text{x}} \left( r\_{i\text{j}} = 0 \text{ or } r\_{i\text{j}} = 1 \right) \end{aligned} \tag{11}$$

In formulae (10) and (11), β represents the recognition rate and δ represents the false-positive rate. rij is the jth contour of the ith Pyralidae insects (if exist, then 1, else 0). n represents the number of real Pyralidae insects in the image, x represents the total number of contours marked by the algorithm, and m represents the total number of contours marked by the algorithm for the ith Pyralidae insects in the image. Thus, the recognition rate reflects the ability of the algorithm to identify maize borers. The false alarm rate reflects the proportion of the error contours in all marker contours. Especially, the sum of these two probabilities is not necessarily equal to 1.

Our scheme and two other algorithms are used to test more than 200 images containing the Pyralidae insects, respectively. Then, we conducted a statistical analysis for the average time consumption, the recognition accuracy, and the false alarm rate. The results of the statistics are shown in Table 5.

#### 4. Conclusions

Pyralidae insects have a great influence on the quality and yield of maize and so on. In order to solve the problem of maize borer detection, this chapter presents a scheme for the detection and identification of Pyralidae insects by using the robot we designed. Firstly, the mathematical morphology is used to preprocess the obtained image, and then the image is binarized by histogram reverse mapping. Next, the binary image is processed by contour extraction and preliminary screening. Then, combining the reference contour image, the contours of Asian Pyralidae insect characteristics are selected by using the Hu moment feature. In the end, this chapter makes a statistical analysis of the experimental results, and the correct rate of recognition based on multi-template matching can reach nearly 94.3%. Compared with other methods, the time complexity of this scheme is basically the same as that of those, which can meet the requirement of real-time detection.

#### Acknowledgements

Finally, the proposed method is compared with the multi-structural element-based crop pest identification method proposed in [33] and the general histogram reverse mapping method. The experimental results are shown in Table 5. As can be seen from Table 5, the scheme of maize borer recognition proposed in this chapter has higher recognition rate, lower false alarm rate, and good application prospects. Besides, it is not necessary to carry out a large amount of data analysis, which ensures that the average time consumption is not significantly increased.

(%)

rij <sup>¼</sup> <sup>0</sup> or rij <sup>¼</sup> <sup>1</sup> � � (10)

False alarm rate

Average time consumption (s)

(%)

78.8 16.9 1.10

rij <sup>¼</sup> <sup>0</sup> or rij <sup>¼</sup> <sup>1</sup> � � (11)

In Table 5, the recognition rate and the false-positive rate are calculated as follows:

Our proposed scheme in this chapter 94.3 6.5 1.12 Histogram reverse mapping method 65.2 60.8 1.01

> Pn i¼1 rij

> > n

x

In formulae (10) and (11), β represents the recognition rate and δ represents the false-positive rate. rij is the jth contour of the ith Pyralidae insects (if exist, then 1, else 0). n represents the number of real Pyralidae insects in the image, x represents the total number of contours marked by the algorithm, and m represents the total number of contours marked by the algorithm for the ith Pyralidae insects in the image. Thus, the recognition rate reflects the ability of the algorithm to identify maize borers. The false alarm rate reflects the proportion of the error contours in all marker contours. Especially, the sum of these two probabilities is not

Our scheme and two other algorithms are used to test more than 200 images containing the Pyralidae insects, respectively. Then, we conducted a statistical analysis for the average time consumption, the recognition accuracy, and the false alarm rate. The results of the statistics are

Pyralidae insects have a great influence on the quality and yield of maize and so on. In order to solve the problem of maize borer detection, this chapter presents a scheme for the detection and identification of Pyralidae insects by using the robot we designed. Firstly, the mathematical

β ¼

x � n P, m <sup>i</sup>¼<sup>1</sup>, <sup>j</sup>¼<sup>1</sup> rij

δ ¼

Schemes Recognition rate

Multi-structural element-based crop pest identification

56 Agricultural Robots - Fundamentals and Applications

Table 5. Comparison results of different schemes.

method [33]

necessarily equal to 1.

shown in Table 5.

4. Conclusions

The contents of this chapter were supported by the Key R&D Project of Hainan Province (Grant no. ZDYF2018015), the Hainan Province Natural Science Foundation of China (Grant no. 617033), the Open Sub-project of State Key Laboratory of Marine Resource Utilization in South China Sea (Grant no. 2016013B), and the Oriented Project of State Key Laboratory of Marine Resource Utilization in South China Sea (Grant no. DX2017012).

#### Conflict of interest

The authors declare that there is no conflict of interests regarding the publication of this chapter.

#### Author details


2 State Key Laboratory of Marine Resource Utilization in South China Sea, Hainan University, Haikou, China

3 University of Chinese Academy of Science, Beijing, China

### References

[1] Wei TS, Zhu WF, Pang MH, Liu YC, Wang ZY, Dong JG. Influence of the damage of cotton bollworm and corn borer to ear rot in corn. Journal of Maize Sciences. 2013;21(4):116-118 (in Chinese)

[2] Wen C, Guyer D. Image-based orchard insect automated identification and classification method. Computers and Electronics in Agriculture. 2012;89:110-115

Speech and Signal Processing (ICASSP), New Orleans, LA, USA: IEEE; 5–9 March, 2017.

Agricultural Robot for Intelligent Detection of Pyralidae Insects

http://dx.doi.org/10.5772/intechopen.79460

59

[15] Zhao Y, Hu Z, Bai Y, Cao F. An accurate segmentation approach for disease and pest based on DRLSE guided by texture difference. Transactions of the Chinese Society for

[16] Zhao Y, Hu Z. Segmentation of fruit with diseases in natural scenes based on logarithmic similarity constraint Otsu. Transactions of the Chinese Society for Agriculture Machinery.

[17] Hu Y, Song L, Zhang J, Xie C, Li R. Pest image recognition of multi-feature fusion based on sparse representation. International Journal of Pattern Recognition and Artificial Intelli-

[18] Bayat M, Abbasi M, Yosefi A. Improvement of pest detection using histogram adjustment method and Gabor wavelet. Journal of Asian Scientific Research. 2016;6(2):24-33

[19] Martin A, Sathish D, Balachander C, Hariprasath T, Krishnamoorthi G. Identification and counting of pests using extended region grow algorithm. In: 2015 2nd International Conference on Electronics and Communication Systems (ICECS), Coimbatore, India:

[20] Przybyłowicz Ł, Pniak M, Tofilski A. Semiautomated identification of European corn borer (Lepidoptera: Crambidae). Journal of Economic Entomology. 2015;109(1):195-199

[21] Deng L, Yu R. Pest recognition system based on bio-inspired filtering and LCP features. In: 2015 12th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), Chengdu, China: IEEE; 18–20 December, 2015. pp.

[22] Miranda JL. Pest identification using image processing techniques in detecting image pattern through neural network. International Journal of Advances in Image Processing

[23] Boniecki P, Koszela K, Piekarska-Boniecka H, Weres J, Zaborowicz M, Kujawa S, Majewskic A, Rabaa B. Neural identification of selected apple pests. Computers and

[24] Espinoza K, Valera DL, Torres JA, López A, Molina-Aiz FD. Combination of image processing and artificial neural networks as a novel approach for the identification of Bemisia tabaci and Frankliniella occidentalis on sticky traps in greenhouse agriculture.

[25] Zhu L, Zhang Z, Zhang P. Image identification of insects based on color histogram and dual tree complex wavelet transform (DTCWT). Acta Entomologica Sinica. 2010;53(1):

[26] Zhu L, Zhang Z. Automatic insect classification based on local mean colour feature and

supported vector machines. Journal of Oriental Insects. 2012;46(3–4):260-269

Agriculture Machinery. 2015;46(2):14-19 (in Chinese)

pp. 1028-1032

202-204

Techniques. 2014;1(4):4-9

91-97 (in Chinese)

2015;46(11):9-15 (in Chinese)

gence. 2014;27(11):985-992 (in Chinese)

IEEE; 26–27 February, 2015. pp. 1229-1234

Electronics in Agriculture. 2015;110:9-16

Computers and Electronics in Agriculture. 2016;127:495-505


Speech and Signal Processing (ICASSP), New Orleans, LA, USA: IEEE; 5–9 March, 2017. pp. 1028-1032

[15] Zhao Y, Hu Z, Bai Y, Cao F. An accurate segmentation approach for disease and pest based on DRLSE guided by texture difference. Transactions of the Chinese Society for Agriculture Machinery. 2015;46(2):14-19 (in Chinese)

[2] Wen C, Guyer D. Image-based orchard insect automated identification and classification

[3] Rupanagudi SR, Ranjani BS, Nagaraj P, Bhat VG, Thippeswamy G. A novel cloud computing based smart farming system for early detection of borer insects in tomatoes. In: 2015 International Conference on Communication, Information & Computing Technology

[4] Ali H, Lali MI, Nawaz MZ, Sharif M, Saleem BA. Symptom based automated detection of citrus diseases using color histogram and textural descriptors. Computers and Electronics

[5] Lu J, Ehsani R, Shi Y, Abdulridha J, Castro AI, Xu Y. Field detection of anthracnose crown rot in strawberry using spectroscopy technology. Computers and Electronics in Agricul-

[6] Xie C, Yang C, He Y. Hyperspectral imaging for classification of healthy and gray mold diseased tomato leaves with different infection severities. Computers and Electronics in

[7] Xia C, Lee JM, Li Y, Chung BK, Chon TS. In situ detection of small-size insect pests sampled on traps using multifractal analysis. Optical Engineering. 2012;51(2):027001-1-

[8] Xia C, Chon TS, Ren Z, Lee JM. Automatic identification and counting of small size pests in greenhouse conditions with low computational cost. Ecological Informatics. 2015;29(9):

[9] Chung BK, Xia C, Song YH, Lee JM, Li Y, Kim H, Chon TS. Sampling of Bemisia tabaci adults using a pre-programmed autonomous pest control robot. Journal of Asia-Pacific

[10] Qing Y, Chen G, Zheng W, Zhang C, Yang B, Jian T. Automated detection and identification of white-backed plant hoppers in paddy fields using image processing. Journal of

[11] Liu T, Chen W, Wu W, Sun C, Guo W, Zhu X. Detection of aphids in wheat fields using a computer vision technique. Journal of Biosystems Engineering. 2016;141:82-93

[12] Ebrahimi MA, Khoshtaghaza MH, Minaei S, Jamshidi B. Vision-based pest detection based on SVM classification method. Computers and Electronics in Agriculture. 2017;

[13] Javed MH, Noor MH, Khan BY, Noor N, Arshad T. K-means based automatic pests detection and classification for pesticides spraying. International Journal of Advanced

[14] Dai S, Man H. A convolutional Riemannian texture model with differential entropic active contours for unsupervised pest detection. In: IEEE International Conference on Acoustics,

method. Computers and Electronics in Agriculture. 2012;89:110-115

(ICCICT), Mumbai, India: IEEE; 15–17 January, 2015. pp. 1-6

in Agriculture. 2017;138(C):92-104

58 Agricultural Robots - Fundamentals and Applications

ture. 2017;135(C):289-299

Agriculture. 2017;135:154-162

Entomology. 2014;17(4):737-743

Integrative Agriculture. 2017;16(7):1547-1557

Computer Science and Applications. 2017;8(11):236-240

027001-12

139-146

137:52-58


[27] Li Z, Hong T, Zeng X, Zheng J. Citrus red mite image target identification based on Kmeans clustering. Transactions of the Chinese Society of Agricultural Engineering. 2013; 28(23):147-153 (in Chinese)

**Chapter 5**

**Provisional chapter**

**Hybrid-Powered Autonomous Robots for Reducing**

**Hybrid-Powered Autonomous Robots for Reducing** 

Environmental contamination and the resulting climate change are major concerns worldwide. Agricultural vehicles that use fossil fuels emit significant amounts of atmospheric pollutants. Thus, this study investigates techniques to reduce fuel consumption in robotic vehicles used for agricultural tasks and therefore reduce atmospheric emissions from these automated systems. A hybrid energy system for autonomous robots devoted to weed and pest control in agriculture is modeled and evaluated, and its exhaust emissions are compared with those of an internal combustion engine-powered system. Agricultural implements require power for hydraulic pumps and fans; this energy is conventionally provided by power take-off (PTO) systems, which waste substantial amounts of energy. In this work, we examine a solution by designing and assessing a hybrid energy system that omits the alternators from the original vehicle and modifies the agricultural implements to replace the PTO power with electrical power. The hybrid energy system uses the original combustion engine of the tractor in combination with a new electrical energy system based on a hydrogen fuel cell. We analyze and compare the exhaust gases resulting from the use of (1) an internal combustion engine as the single power source and (2) the hybrid energy system. The results demonstrate that the hybrid energy system

**Keywords:** atmospheric emissions, exhaust gases, hybrid power, robotic vehicles,

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

© 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use,

distribution, and reproduction in any medium, provided the original work is properly cited.

DOI: 10.5772/intechopen.79875

**Both Fuel Consumption and Pollution in Precision**

**Both Fuel Consumption and Pollution in Precision** 

**Agriculture Tasks**

**Agriculture Tasks**

Pablo Gonzalez-de-Santos

Pablo Gonzalez-de-Santos

**Abstract**

http://dx.doi.org/10.5772/intechopen.79875

Mariano Gonzalez-de-Soto, Luis Emmi and

Mariano Gonzalez-de-Soto, Luis Emmi and

Additional information is available at the end of the chapter

Additional information is available at the end of the chapter

reduced emissions by up to approximately 50%.

precision agriculture


#### **Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution in Precision Agriculture Tasks Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution in Precision Agriculture Tasks**

DOI: 10.5772/intechopen.79875

Mariano Gonzalez-de-Soto, Luis Emmi and Pablo Gonzalez-de-Santos Mariano Gonzalez-de-Soto, Luis Emmi and Pablo Gonzalez-de-Santos

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.79875

#### **Abstract**

[27] Li Z, Hong T, Zeng X, Zheng J. Citrus red mite image target identification based on Kmeans clustering. Transactions of the Chinese Society of Agricultural Engineering. 2013;

[28] Johannes A, Picon A, Alvarez-Gila A, Echazarra J, Rodriguez-Vaamonde S, Navajas AD, Ortiz-Barredo A. Automatic plant disease diagnosis using mobile capture devices, applied

[29] Swain MJ, Ballard DH. Color indexing. International Journal of Computer Vision. 1991;

[30] Otsu N. A threshold selection method from gray-level histogram. IEEE Transactions on

[31] Xu X, Song E, Jin L. Characteristic analysis of threshold based on Otsu criterion. Acta

[32] Doyle W. Operations useful for similarity-invariant pattern recognition. Association for

[33] Liu J, Geng G, Ren Z. Plant pest recognition system based on multi-structure element morphology. Journal of Computational Design and Engineering. 2009;30(6):1488-1490

on a wheat use case. Computers and Electronics in Agriculture. 2017;138:200-209

28(23):147-153 (in Chinese)

60 Agricultural Robots - Fundamentals and Applications

Systems, Man, and Cybernetics. 1979;9(1):62-66

Computing Machinery. 1962;9(2):259-267

Entomologica Sinica. 2009;37(12):2716-2719 (in Chinese)

7(1):11-32

(in Chinese)

Environmental contamination and the resulting climate change are major concerns worldwide. Agricultural vehicles that use fossil fuels emit significant amounts of atmospheric pollutants. Thus, this study investigates techniques to reduce fuel consumption in robotic vehicles used for agricultural tasks and therefore reduce atmospheric emissions from these automated systems. A hybrid energy system for autonomous robots devoted to weed and pest control in agriculture is modeled and evaluated, and its exhaust emissions are compared with those of an internal combustion engine-powered system. Agricultural implements require power for hydraulic pumps and fans; this energy is conventionally provided by power take-off (PTO) systems, which waste substantial amounts of energy. In this work, we examine a solution by designing and assessing a hybrid energy system that omits the alternators from the original vehicle and modifies the agricultural implements to replace the PTO power with electrical power. The hybrid energy system uses the original combustion engine of the tractor in combination with a new electrical energy system based on a hydrogen fuel cell. We analyze and compare the exhaust gases resulting from the use of (1) an internal combustion engine as the single power source and (2) the hybrid energy system. The results demonstrate that the hybrid energy system reduced emissions by up to approximately 50%.

**Keywords:** atmospheric emissions, exhaust gases, hybrid power, robotic vehicles, precision agriculture

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. © 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### **1. Introduction**

Off-road vehicles based on internal combustion engines use large amounts of fossil fuels that emit large amounts of pollution into the atmosphere. According to the US Environmental Protection Agency (EPA) [1], internal combustion engines emit carbon dioxide (CO2 ), nitrogen oxide (NOX), carbon monoxide (CO), particulate matter and hydrocarbons. CO2 and NOX are greenhouse gases that contribute to global warming, whereas sulfur dioxide (SO2 ) and NOx emissions contribute to acid rain. Therefore, the use of internal combustion engines is a major environmental concern. Furthermore, these chemical compounds also cause health problems. For example, NOX may cause respiratory diseases and intensify existing heart disease; CO can reduce oxygen delivery to the body's tissues and organs, which reduces an individual's work capacity, mental skills and learning ability. Hydrocarbons are volatile organic compounds that can cause headaches, dizziness, and loss of consciousness, among other effects. Moreover, some of these substances, such as benzene, are carcinogenic and increase the likelihood of leukemia. Particle matter emitted from combustion engines (nitrates, sulfates, organic chemicals, metals, and dust particles) can also affect lung and heart functions, causing serious health problems.

production and distribution and presented engineering systems analyses of how such systems can reduce pollution [9]. Others have compared the theoretical maximum efficiencies of a fuel cell and a Carnot cycle using the same fuel to determine the net reaction [10]. They found that the maximum efficiencies of both systems are similar, but in practice, a fuel cell is more efficient because internal combustion engines cannot operate at their theoretical maximum efficiency. Other researchers have compared battery electrical vehicles, hydrogen fuel cell electrical vehicles and hydrogen fuel cell plug-in hybrid vehicles [11]. These researchers determined that battery electrical vehicles and hydrogen fuel cell plug-in hybrid vehicles have similar life cycle costs. The life cycle costs of these vehicles are higher than the costs of

Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution…

http://dx.doi.org/10.5772/intechopen.79875

63

The approach presented in this work originated from the observation that during precision agriculture tasks with robotized vehicles, the internal combustion engine frequently supplied more power than needed, particularly when the implement (a tool or utensil for performing agricultural work) used a power take-off (PTO) device as a power source. Thus, the objective of this work was to develop, implement and assess a hybrid energy system for agricultural robotic vehicles. The proposed energy system combines the use of batteries, a hydrogen fuel cell and photovoltaic cells with the original internal combustion engine of the tractor to

The rest of this work is organized as follows. In Section 2, the autonomous robot and agricultural implements used in this study are described. In Section 3, a path planning method for reducing the fuel consumption is presented. Then, in Section 4, the energy demanded in the selected agricultural tasks is analyzed, and the hybrid energy system designed to reduce the system energy consumption is described. Section 5 studies the energy requirements of the selected agricultural tasks and the required features of the hybrid energy system. Finally, the main results are discussed in Section 6, and the main conclusions are summarized in Section 7.

The energetic model derived in this work was tested in a system composed of an autonomous robot and three different implements, each designed for a different agricultural task. The

• A fuel consumption measurement system used to estimate the exhaust gases from the

achieve a substantial decrease in fossil fuel use, reducing the pollutant emissions.

internal combustion engines but could decrease by 2030.

**2. Systems description**

• A central controller

• A hydrogen fuel cell • A photovoltaic panel

• A set of batteries

• An internal combustion engine

• An energy management system

internal combustion engine

autonomous robot consisted of several subsystems:

Many efforts to mitigate these negative effects have conducted analyses of energy use and the pollution emitted by agricultural tractors. In the early 2000s, several research studies compared different methods and calculated the average absolute and specific emission values from agricultural tractors, concluding that the use of hydrocarbon fuels must be progressively replaced by cleaner fuels or electrical systems [2]. Other studies have proposed using a model of fossil fuel to simulate possible agricultural production scenarios to improve future techniques [3]. In recent years, researchers have analyzed how increasing the soil organic carbon content decreases the draft force in plowing, resulting in reduced fuel consumption and emissions [4].

Several studies have analyzed the exhaust gas emissions from internal combustion engines in the last two decades, and many such studies have focused on agricultural machines. For example, a mathematical model of a tractor was developed in [5] to analyze the fuel consumption and engine emissions for different engine control strategies and engine transmission characteristics, whereas in [6], the exhaust emissions and fuel composition of a real tractor during plowing were measured and correlated to the load factor of the tractor. These works concluded that fuel consumption and emissions depend on the engine speed and load conditions.

Many studies have analyzed the impact of alternative energy sources such as biofuels and have demonstrated that biofuels can benefit the environment and society [7]. However, many of these studies have proposed the use of batteries and have examined various battery technologies available for use in solar-assisted plug-in hybrid electrical tractors that can be used in light-duty agricultural operations [8]. These researchers have also conducted life cycle analyses of a solar-assisted plug-in hybrid electrical tractor and compared the results with that of a similar power output internal combustion engine tractor considering both economic costs and environmental emissions; they determined that the life cycle costs of solar-assisted plug-in hybrid electrical tractors are lower than those of internal combustion engines.

Another important alternative to batteries is fuel cells. For example, several researchers have proposed the use of environmentally benign fuel cells for power production in field crop production and distribution and presented engineering systems analyses of how such systems can reduce pollution [9]. Others have compared the theoretical maximum efficiencies of a fuel cell and a Carnot cycle using the same fuel to determine the net reaction [10]. They found that the maximum efficiencies of both systems are similar, but in practice, a fuel cell is more efficient because internal combustion engines cannot operate at their theoretical maximum efficiency. Other researchers have compared battery electrical vehicles, hydrogen fuel cell electrical vehicles and hydrogen fuel cell plug-in hybrid vehicles [11]. These researchers determined that battery electrical vehicles and hydrogen fuel cell plug-in hybrid vehicles have similar life cycle costs. The life cycle costs of these vehicles are higher than the costs of internal combustion engines but could decrease by 2030.

The approach presented in this work originated from the observation that during precision agriculture tasks with robotized vehicles, the internal combustion engine frequently supplied more power than needed, particularly when the implement (a tool or utensil for performing agricultural work) used a power take-off (PTO) device as a power source. Thus, the objective of this work was to develop, implement and assess a hybrid energy system for agricultural robotic vehicles. The proposed energy system combines the use of batteries, a hydrogen fuel cell and photovoltaic cells with the original internal combustion engine of the tractor to achieve a substantial decrease in fossil fuel use, reducing the pollutant emissions.

The rest of this work is organized as follows. In Section 2, the autonomous robot and agricultural implements used in this study are described. In Section 3, a path planning method for reducing the fuel consumption is presented. Then, in Section 4, the energy demanded in the selected agricultural tasks is analyzed, and the hybrid energy system designed to reduce the system energy consumption is described. Section 5 studies the energy requirements of the selected agricultural tasks and the required features of the hybrid energy system. Finally, the main results are discussed in Section 6, and the main conclusions are summarized in Section 7.

### **2. Systems description**

**1. Introduction**

62 Agricultural Robots - Fundamentals and Applications

Off-road vehicles based on internal combustion engines use large amounts of fossil fuels that emit large amounts of pollution into the atmosphere. According to the US Environmental

emissions contribute to acid rain. Therefore, the use of internal combustion engines is a major environmental concern. Furthermore, these chemical compounds also cause health problems. For example, NOX may cause respiratory diseases and intensify existing heart disease; CO can reduce oxygen delivery to the body's tissues and organs, which reduces an individual's work capacity, mental skills and learning ability. Hydrocarbons are volatile organic compounds that can cause headaches, dizziness, and loss of consciousness, among other effects. Moreover, some of these substances, such as benzene, are carcinogenic and increase the likelihood of leukemia. Particle matter emitted from combustion engines (nitrates, sulfates, organic chemicals, metals, and dust particles) can also affect lung and heart functions, causing serious health problems. Many efforts to mitigate these negative effects have conducted analyses of energy use and the pollution emitted by agricultural tractors. In the early 2000s, several research studies compared different methods and calculated the average absolute and specific emission values from agricultural tractors, concluding that the use of hydrocarbon fuels must be progressively replaced by cleaner fuels or electrical systems [2]. Other studies have proposed using a model of fossil fuel to simulate possible agricultural production scenarios to improve future techniques [3]. In recent years, researchers have analyzed how increasing the soil organic carbon content decreases the draft force in plowing, resulting in reduced fuel consumption and emissions [4]. Several studies have analyzed the exhaust gas emissions from internal combustion engines in the last two decades, and many such studies have focused on agricultural machines. For example, a mathematical model of a tractor was developed in [5] to analyze the fuel consumption and engine emissions for different engine control strategies and engine transmission characteristics, whereas in [6], the exhaust emissions and fuel composition of a real tractor during plowing were measured and correlated to the load factor of the tractor. These works concluded that fuel consumption and emissions depend on the engine speed and load conditions. Many studies have analyzed the impact of alternative energy sources such as biofuels and have demonstrated that biofuels can benefit the environment and society [7]. However, many of these studies have proposed the use of batteries and have examined various battery technologies available for use in solar-assisted plug-in hybrid electrical tractors that can be used in light-duty agricultural operations [8]. These researchers have also conducted life cycle analyses of a solar-assisted plug-in hybrid electrical tractor and compared the results with that of a similar power output internal combustion engine tractor considering both economic costs and environmental emissions; they determined that the life cycle costs of solar-assisted plug-in

), nitrogen

) and NOx

and NOX are

Protection Agency (EPA) [1], internal combustion engines emit carbon dioxide (CO2

greenhouse gases that contribute to global warming, whereas sulfur dioxide (SO2

oxide (NOX), carbon monoxide (CO), particulate matter and hydrocarbons. CO2

hybrid electrical tractors are lower than those of internal combustion engines.

Another important alternative to batteries is fuel cells. For example, several researchers have proposed the use of environmentally benign fuel cells for power production in field crop The energetic model derived in this work was tested in a system composed of an autonomous robot and three different implements, each designed for a different agricultural task. The autonomous robot consisted of several subsystems:


These subsystems and the implements are described below. The modifications made to the implements such that the power provided by the PTO device could be replaced with electrical power are also described, and the implement power features needed to design the energy model are detailed.

#### **2.1. Autonomous robot**

The autonomous robot was based on the commercial CNHi Boomer 3050 CVT tractor (CNHi, Zedelgem, Bruges, Belgium). This vehicle, with a weight of approximately 1700 kg and a gross power of approximately 33.6 kW, was mechanically, electrically and hydraulically modified. The robot power system, originally based on the tractor internal combustion engine, was improved with an additional electrical energy system consisting of (1) a photovoltaic panel, (2) a hydrogen fuel cell and (3) a set of batteries.

A main controller onboard the vehicle managed the main vehicle functions and a safety system that provided safety to the vehicle, the environment and, most importantly, any individuals nearby [12]. The final autonomous robot and the different subsystems attached to it, whose main features are detailed in the following sections, are depicted in **Figure 1**. The size, justification and assessment of the added electrical energy system are presented in Section 5.2.

The autonomous robot is equipped with a positioning system that consists of a global positioning system (GPS) receiver (Trimble Model BX982), with two antennas to measure the robot's heading with triangulation techniques. The system uses a real-time kinematic (RTK) signal correction provided by a GPS base station located next to the working field. The posi-

Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution…

http://dx.doi.org/10.5772/intechopen.79875

65

• the crop row detection system, which is responsible for detecting the rows as a reference

• the safety system, which comprised (1) an obstacle detection system based on the robot

A base station generated the mission, which consisted of a plan that defined the trajectories of the robot and a plan for managing the implements, both plans depended on the specific application. After generating the mission, the base station sent both plans to the robot controller and executed them. When the robot was working, the base station was responsible for supervising the status of both the robot and implement in real time and detecting malfunctions, such as service disruptions, incorrect working speeds, incorrect implement statuses and

• As the only power source providing the total power demanded by the agricultural task.

• As a part of the hybrid energy system, providing only a sufficient amount of power to

tioning system provides a location accuracy of approximately ±0.025 m.

camera, (2) a laser and (3) a remote controller used by the operator.

• the weed detection system, which is responsible for detecting weed patches

A vision system installed onboard the robot was used by

**Figure 2.** Main components of the autonomous robot controller.

The internal combustion engine can work in two ways:

move the autonomous robot with its implement.

for guiding the autonomous robot

the probability of collisions [15].

*2.1.2. Internal combustion engine*

#### *2.1.1. Autonomous robot controller*

The autonomous robot controller (see **Figure 2**) allowed the robot to apply an effective treatment with high precision. The autonomous robot controller is based on a hybrid architecture that relies on the main controller based on a CompactRIO-9082 (National Instruments Corporation) running a LabVIEW Real-Time operating system. The controller synchronized and processed the information received from different sensors and the external operator and selected the best behavior for the entire system depending on the current working situation, the perceived environment and the general mission requirements. The controller also communicated with every other subsystem via diverse communication protocols (Ethernet, serial and CAN bus) [13, 14]. The set of these systems (controllers, sensors and actuators) had an average power demand of approximately 170 W for 12-V devices and approximately 260 W for 24-V devices.

**Figure 1.** Autonomous robot.

Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution… http://dx.doi.org/10.5772/intechopen.79875 65

**Figure 2.** Main components of the autonomous robot controller.

These subsystems and the implements are described below. The modifications made to the implements such that the power provided by the PTO device could be replaced with electrical power are also described, and the implement power features needed to design the energy

The autonomous robot was based on the commercial CNHi Boomer 3050 CVT tractor (CNHi, Zedelgem, Bruges, Belgium). This vehicle, with a weight of approximately 1700 kg and a gross power of approximately 33.6 kW, was mechanically, electrically and hydraulically modified. The robot power system, originally based on the tractor internal combustion engine, was improved with an additional electrical energy system consisting of (1) a photovoltaic panel,

A main controller onboard the vehicle managed the main vehicle functions and a safety system that provided safety to the vehicle, the environment and, most importantly, any individuals nearby [12]. The final autonomous robot and the different subsystems attached to it, whose main features are detailed in the following sections, are depicted in **Figure 1**. The size, justification and assessment of the added electrical energy system are presented in Section 5.2.

The autonomous robot controller (see **Figure 2**) allowed the robot to apply an effective treatment with high precision. The autonomous robot controller is based on a hybrid architecture that relies on the main controller based on a CompactRIO-9082 (National Instruments Corporation) running a LabVIEW Real-Time operating system. The controller synchronized and processed the information received from different sensors and the external operator and selected the best behavior for the entire system depending on the current working situation, the perceived environment and the general mission requirements. The controller also communicated with every other subsystem via diverse communication protocols (Ethernet, serial and CAN bus) [13, 14]. The set of these systems (controllers, sensors and actuators) had an average power demand of

approximately 170 W for 12-V devices and approximately 260 W for 24-V devices.

model are detailed.

**2.1. Autonomous robot**

64 Agricultural Robots - Fundamentals and Applications

(2) a hydrogen fuel cell and (3) a set of batteries.

*2.1.1. Autonomous robot controller*

**Figure 1.** Autonomous robot.

The autonomous robot is equipped with a positioning system that consists of a global positioning system (GPS) receiver (Trimble Model BX982), with two antennas to measure the robot's heading with triangulation techniques. The system uses a real-time kinematic (RTK) signal correction provided by a GPS base station located next to the working field. The positioning system provides a location accuracy of approximately ±0.025 m.

A vision system installed onboard the robot was used by


A base station generated the mission, which consisted of a plan that defined the trajectories of the robot and a plan for managing the implements, both plans depended on the specific application. After generating the mission, the base station sent both plans to the robot controller and executed them. When the robot was working, the base station was responsible for supervising the status of both the robot and implement in real time and detecting malfunctions, such as service disruptions, incorrect working speeds, incorrect implement statuses and the probability of collisions [15].

#### *2.1.2. Internal combustion engine*

The internal combustion engine can work in two ways:


The engine was similar to that of the original tractor; however, the maximum ground speed was limited to 7 km h−1 for safety reasons. **Figure 3** shows the performance curves of the internal combustion engine provided by the manufacturer, which were used to calculate the exhaust gas emissions and implement the energy demand model. The torque, power curve and specific fuel consumption volume (*V*SFC) are shown as functions of the engine speed.

*2.1.4. Photovoltaic panel*

(see **Figure 1**).

*2.1.5. Batteries*

A set of batteries were used to • store excess electrical energy

implements were used (see **Figure 1**).

*2.1.6. Energy management system*

combustion engine

The photovoltaic panel was used as an additional system for fossil-free energy. This device charged the batteries whenever there was sufficient light, even when the robot was in a garage. The panel was situated on top of the robot to minimize shadows. Only the antennas and camera were attached higher on the robot to improve the signal transmission and camera vision. The panel was set horizontally to collect solar power independent of the orientation

Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution…

http://dx.doi.org/10.5772/intechopen.79875

67

• supply energy during periods of high demand, for example, the startup of the internal

Because the set of batteries was heavy, one group of batteries was placed over the rear vehicle shaft to reduce the slippage in tasks requiring draft forces. Another battery bank was placed inside the fuel cell box in front part of the tractor and acted as a counterweight, when heavy

The energy management system consisted of a controller to manage the electrical energy flow from the hydrogen fuel cell and the photovoltaic panel. This system was responsible for regulating and adapting the electrical power and supervising the electrical energy storage. To accomplish this task, the system collected data about the status of the batteries and hydrogen

Fuel consumption was measured using two flowmeters installed in the fuel supply line and return line (see **Figure 4**). The instantaneous fuel consumption was measured as the differ-

The flow sensors must be placed behind a fuel filter for protection; therefore, the return line flowmeter must be installed in the pipe between the injectors and the junction with the fuel filter return line. The other flowmeter can be installed between the fuel filter and lift pump or between the lift pump and injection pump. The best position is behind the lift pump; otherwise, low pressure can lead to additional problems resulting from small air bubbles in the sensor circuit. Additionally, a cooling device was added in the return line before flowmeter 2 because a substantial amount of noise was observed in the flowmeter 2 data as a result of the

tanks and controlled the power provided by the hydrogen fuel cell.

ence between the data from flowmeter 1 and the data from flowmeter 2.

*2.1.7. Fuel consumption measurement system and model*

high temperature of the fuel returned by the robot engine.

• ensure that the robot's energy management system had a continuous energy supply.

#### *2.1.3. Hydrogen fuel cell*

A hydrogen fuel cell was used because this device generates electrical power with high performance and can be rapidly refueled. The cell was attached to the front of the robot in a box containing the hydrogen tanks (see **Figure 1**). A proton exchange membrane fuel cell and metal hydride tanks were selected, which provided a power range from 0.5 to 5 kW with a specific hydrogen consumption of approximately 0.74 Nm<sup>3</sup> kW h−1. This value was used to estimate the hydrogen consumption [17]. Nm<sup>3</sup> denotes normal cubic meters, the volume of gas measured under the normal conditions of 0°C and 1.01325 × 105 Pa (1 atm) of pressure.

**Figure 3.** Performance curves of the internal combustion engine [16].

#### *2.1.4. Photovoltaic panel*

The engine was similar to that of the original tractor; however, the maximum ground speed was limited to 7 km h−1 for safety reasons. **Figure 3** shows the performance curves of the internal combustion engine provided by the manufacturer, which were used to calculate the exhaust gas emissions and implement the energy demand model. The torque, power curve and specific fuel consumption volume (*V*SFC) are shown as functions of the engine speed.

A hydrogen fuel cell was used because this device generates electrical power with high performance and can be rapidly refueled. The cell was attached to the front of the robot in a box containing the hydrogen tanks (see **Figure 1**). A proton exchange membrane fuel cell and metal hydride tanks were selected, which provided a power range from 0.5 to 5 kW

volume of gas measured under the normal conditions of 0°C and 1.01325 × 105 Pa (1 atm)

kW h−1. This value was

denotes normal cubic meters, the

with a specific hydrogen consumption of approximately 0.74 Nm<sup>3</sup>

used to estimate the hydrogen consumption [17]. Nm<sup>3</sup>

**Figure 3.** Performance curves of the internal combustion engine [16].

*2.1.3. Hydrogen fuel cell*

66 Agricultural Robots - Fundamentals and Applications

of pressure.

The photovoltaic panel was used as an additional system for fossil-free energy. This device charged the batteries whenever there was sufficient light, even when the robot was in a garage. The panel was situated on top of the robot to minimize shadows. Only the antennas and camera were attached higher on the robot to improve the signal transmission and camera vision. The panel was set horizontally to collect solar power independent of the orientation (see **Figure 1**).

#### *2.1.5. Batteries*

A set of batteries were used to


Because the set of batteries was heavy, one group of batteries was placed over the rear vehicle shaft to reduce the slippage in tasks requiring draft forces. Another battery bank was placed inside the fuel cell box in front part of the tractor and acted as a counterweight, when heavy implements were used (see **Figure 1**).

#### *2.1.6. Energy management system*

The energy management system consisted of a controller to manage the electrical energy flow from the hydrogen fuel cell and the photovoltaic panel. This system was responsible for regulating and adapting the electrical power and supervising the electrical energy storage. To accomplish this task, the system collected data about the status of the batteries and hydrogen tanks and controlled the power provided by the hydrogen fuel cell.

#### *2.1.7. Fuel consumption measurement system and model*

Fuel consumption was measured using two flowmeters installed in the fuel supply line and return line (see **Figure 4**). The instantaneous fuel consumption was measured as the difference between the data from flowmeter 1 and the data from flowmeter 2.

The flow sensors must be placed behind a fuel filter for protection; therefore, the return line flowmeter must be installed in the pipe between the injectors and the junction with the fuel filter return line. The other flowmeter can be installed between the fuel filter and lift pump or between the lift pump and injection pump. The best position is behind the lift pump; otherwise, low pressure can lead to additional problems resulting from small air bubbles in the sensor circuit. Additionally, a cooling device was added in the return line before flowmeter 2 because a substantial amount of noise was observed in the flowmeter 2 data as a result of the high temperature of the fuel returned by the robot engine.

**Figure 4.** Scheme of the fuel flow system.

These flowmeters must be suitable for measuring diesel fuel and support the robot fuel circuit conditions. According to the characteristics of the fuel system (see **Table 1**), the flowmeters do not need to support high pressures; the nominal flow must be approximately 12.54 L h−1, with a maximum flow of approximately 30 L h−1, which is adequate for oil with a low kinematic viscosity. Furthermore, the flowmeter must deliver a measure of the temperature of the fluid, and the return line flow sensor must be sensitive to a low flow rate.

Several mathematical models for tractor engine consumption have been proposed in the last few decades; these models show the interest in minimizing fuel consumption in agriculture tasks [21, 22]. In this study, fuel consumption was assumed to depend on the terrain surface and slope, engine speed, throttle position and load conditions, tractor drive type, total weight, drawbar, PTO, and hydraulic and electrical power. To estimate the individual contribution of these elements to fuel consumption, their relationships with energy expenditures must be examined. These relationships were based on estimates using standards, engineering practices, and data suggested by the American Society of Agricultural Engineers (ASAE) [24].

Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution…

http://dx.doi.org/10.5772/intechopen.79875

69

The total energy and fuel consumption can be related through the specific fuel consumption volume, *V*SFC, which is the fuel volume consumed per power provided, by computing the total

*<sup>T</sup> VSFC*(*t*) <sup>P</sup>*<sup>T</sup>*\_*PTOeq* (*t*)*dt* (1)

0

According to ASAE [24], the specific fuel consumption volume can be computed by

fuel consumed, *V*TFC, as follows:

**Figure 5.** PD400 flowmeter.

*VTFC* = ∫

**Feature Value** Flow rate 1–60 L h−1 Maximum working pressure 25,105 Pa K factor 1830 pulses L−1

Temperature range 0–60°C

**Table 2.** Main characteristics of the PD400 flowmeter [23].

where *P*T\_PTOeq is the total equivalent PTO demanded power.

Accuracy ±2.5% (at a density of 830 kg m−3)

An effective flowmeter for these applications was a small positive displacement oval gear flowmeter. The oval gear design ensures that the pressure loss across the sensor is very low (less than 1.5 × 10<sup>4</sup> Pa at full flow) and that the performance remains nearly constant over the entire temperature and viscosity range. The PD400 flowmeter from Titan Enterprises Ltd. (Sherborne Dorset, England, UK) was selected (see **Figure 5**) for both flow lines. **Table 2** presents the main features of the PD400 flowmeter, which has a small, oval, tooth-wheeled counter in addition to an easily replaceable filter that protects the sensor from any floating particles. The flowmeter accuracy was approximately ±2.5% with a low pressure loss of 10<sup>4</sup> Pa [20], and its recommended working temperature range was from 0 to 60°C, which was slightly lower than the temperature of the fluid in the return line; thus, the return line required a small radiator to cool the returned fuel. This fuel consumption measurement system was properly calibrated and validated experimentally. Considering the flowmeter accuracy and the rate of flow in the fuel line, we obtained an accuracy of approximately 0.3 L h−1.


**Table 1.** Main characteristics of the fuel and fuel systems [16, 18, 19].

Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution… http://dx.doi.org/10.5772/intechopen.79875 69

**Figure 5.** PD400 flowmeter.

These flowmeters must be suitable for measuring diesel fuel and support the robot fuel circuit conditions. According to the characteristics of the fuel system (see **Table 1**), the flowmeters do not need to support high pressures; the nominal flow must be approximately 12.54 L h−1, with a maximum flow of approximately 30 L h−1, which is adequate for oil with a low kinematic viscosity. Furthermore, the flowmeter must deliver a measure of the temperature of the fluid,

An effective flowmeter for these applications was a small positive displacement oval gear flowmeter. The oval gear design ensures that the pressure loss across the sensor is very low (less than 1.5 × 10<sup>4</sup> Pa at full flow) and that the performance remains nearly constant over the entire temperature and viscosity range. The PD400 flowmeter from Titan Enterprises Ltd. (Sherborne Dorset, England, UK) was selected (see **Figure 5**) for both flow lines. **Table 2** presents the main features of the PD400 flowmeter, which has a small, oval, tooth-wheeled counter in addition to an easily replaceable filter that protects the sensor from any floating particles. The flowmeter accuracy was approximately ±2.5% with a low pressure loss of 10<sup>4</sup> Pa [20], and its recommended working temperature range was from 0 to 60°C, which was slightly lower than the temperature of the fluid in the return line; thus, the return line required a small radiator to cool the returned fuel. This fuel consumption measurement system was properly calibrated and validated experimentally. Considering the flowmeter accuracy and the rate of

and the return line flow sensor must be sensitive to a low flow rate.

**Figure 4.** Scheme of the fuel flow system.

68 Agricultural Robots - Fundamentals and Applications

flow in the fuel line, we obtained an accuracy of approximately 0.3 L h−1.

**Feature Value** Lift pump rated flow 12.54 L h−1

Maximum flow ~30 L h−1 Density at 15°C 820–845 kg m−3 Gross calorific value 10.40–10.72 kWh L−1 Net calorific value 9.881–10.182 kWh L−1

**Table 1.** Main characteristics of the fuel and fuel systems [16, 18, 19].

CO2

Lift pump working pressure (0.2 ± 0.05) 105 Pa

emissions 2.616–2.696 kg L−1

Several mathematical models for tractor engine consumption have been proposed in the last few decades; these models show the interest in minimizing fuel consumption in agriculture tasks [21, 22]. In this study, fuel consumption was assumed to depend on the terrain surface and slope, engine speed, throttle position and load conditions, tractor drive type, total weight, drawbar, PTO, and hydraulic and electrical power. To estimate the individual contribution of these elements to fuel consumption, their relationships with energy expenditures must be examined. These relationships were based on estimates using standards, engineering practices, and data suggested by the American Society of Agricultural Engineers (ASAE) [24].

The total energy and fuel consumption can be related through the specific fuel consumption volume, *V*SFC, which is the fuel volume consumed per power provided, by computing the total fuel consumed, *V*TFC, as follows:

$$\boldsymbol{V}\_{\rm TFC} = \int\_{0}^{\boldsymbol{\gamma}} \boldsymbol{V}\_{\rm SFC}(t) \, \boldsymbol{P}\_{\boldsymbol{T}\_{\rm s} \rm PIOaq}(t) dt \tag{1}$$

where *P*T\_PTOeq is the total equivalent PTO demanded power.

According to ASAE [24], the specific fuel consumption volume can be computed by


**Table 2.** Main characteristics of the PD400 flowmeter [23].

$$V\_{\rm SIC}(t) = \left(0.22 + 0.096 \frac{P\_{\rm POL}(t)}{P\_{\rm POL}(t)}\right) \left(1 - \left(\frac{n\_{p\_{\rm T}}}{n\_{\rm IT}} - 1\right) \left(0.45 \frac{P\_{\rm POL}(t)}{P\_{\rm POL}(t)} - 0.877\right)\right) \tag{2}$$

and the total equivalent PTO demanded power is given by

$$P\_{T\_{\\_P TOoq}}(t) = \frac{D(t)\,\upsilon(t)}{E\_{\underline{\;\prime}}} + \frac{F\_{\rm att}}{Em} + P\_{\rm PIO} + P\_{\rm hqd} + P\_{\rm el} \tag{3}$$

**Figure 6** illustrates these three implements, and their main features and requirements are presented in the following sections. The modifications to the implements made to allow use

Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution…

http://dx.doi.org/10.5772/intechopen.79875

71

This task consisted of performing weed control using plowing and thermal treatments with a particular mechanical-thermal machine. The weed detection system detected weed patches by processing the images from a vision camera in real time. The autonomous robot was programmed to follow an initial predefined path, which fixed the initial and final points of each track (the path followed by the vehicle through the crop). However, the initial path was corrected with information provided by the row detection system based on the vision camera of the weed detection system. The area analyzed in each image was a 3-m-wide (4 rows) and 2-m-long rectangle. It was georeferenced with an accuracy of approximately 0.08 m and

The implement consisted of a row crop cultivator and thermal device. The row crop cultivator performed a mechanical treatment in the furrows (the spaces between crop rows, similar to a conventional row crop cultivator). The thermal device consisted of several burners that produced flames applied in each row for weed control. This implement (see **Figure 6a**) was used for wide rows of crops, with rows separated by approximately 0.75 m. This technique can be applied to crops that can withstand high temperatures over short periods of time, such as

**Figure 6.** Implements working: (a) thermal and row cultivator, (b) canopy sprayer and (c) patch sprayer.

with the hybrid energy system are also described.

divided into 0.25 × 0.25 m cells [14].

*2.2.1. Weed control with a thermal implement and cultivator implement*

where *P*PTOeq is the equivalent PTO power; *P*PTOrated is the rated PTO power; *n*PT and *n*FT are the partial and full throttle engine speed, respectively; *D* is the implement draft force; *F*MR is the motion resistance (i.e., the difference between the gross traction and net traction; *v* is the vehicle speed; *E*T is the traction efficiency; *E*M is the mechanical efficiency of the power transmission from the net flywheel to the PTO; and *P*PTO, *P*hyd and *P*el are the power requirements from the PTO, the hydraulic power and the electrical power, respectively. Details about the model's derivation are provided in [25].

#### *2.1.8. Calculation of the exhaust gas emissions*

The exhaust gas emissions were computed by considering the partial load and speed of the engine for a particular work regime according to the ISO 8178 standard [26] and the fuel features specified in [18]. To calculate the partial load, the values of wheel slippage (the difference between the ground speed provided by GPS and the wheel speed provided by the control system), PTO speed, engine speed, three-point hitch position (which determines the plowing depth when the cultivator is used), and terrain slope (obtained from the orthometric height in each point) were obtained. The partial load of the engine was calculated using these data and the equations from the ASABE standards [24, 27]. With this partial load, the work regime of the engine was obtained from the curves shown in **Figure 3**. Then, the corresponding emission factor for each exhaust substance was estimated with these data and the ISO 8178 standard [26]. The ISO standard defines the emissions factors of exhaust gases for agricultural combustion engines in eight individual work regimes based on the maximum power and the manufacturing year of the engine. For small engines, as in this case, the ISO standard defines the emission factors needed to calculate CO, particulate matter and NO<sup>X</sup> + hydrocarbons. Finally, the CO2 emissions were calculated using the chemical equation of combustion reaction (considering the other exhaust emission gases calculated with the emission factor of the ISO standard) and the measured fuel consumption.

#### **2.2. Agricultural tasks and implements**

Three different agricultural tasks were considered, each one requiring a specific agricultural implement:


**Figure 6** illustrates these three implements, and their main features and requirements are presented in the following sections. The modifications to the implements made to allow use with the hybrid energy system are also described.

#### *2.2.1. Weed control with a thermal implement and cultivator implement*

*VSFC*(*t*) <sup>=</sup> (0.22 <sup>+</sup> 0.096

70 Agricultural Robots - Fundamentals and Applications

*PPTOrated*(*t*) \_\_\_\_\_\_\_\_

and the total equivalent PTO demanded power is given by

*PT*\_*PTOeq* (*t*) <sup>=</sup> *<sup>D</sup>*(*t*) *<sup>v</sup>*(*t*) \_\_\_\_\_\_\_\_\_\_\_\_

model's derivation are provided in [25].

bons. Finally, the CO2

implement:

*2.1.8. Calculation of the exhaust gas emissions*

the ISO standard) and the measured fuel consumption.

• Weed control using an herbicide patch sprayer

• Weed control using a thermal and a row crop cultivator implement

**2.2. Agricultural tasks and implements**

• Pest control using a canopy sprayer.

*PPTOeq*(*t*) )(<sup>1</sup> <sup>−</sup> (

*ET* + *FMR v* \_\_\_\_\_\_\_\_\_\_

where *P*PTOeq is the equivalent PTO power; *P*PTOrated is the rated PTO power; *n*PT and *n*FT are the partial and full throttle engine speed, respectively; *D* is the implement draft force; *F*MR is the motion resistance (i.e., the difference between the gross traction and net traction; *v* is the vehicle speed; *E*T is the traction efficiency; *E*M is the mechanical efficiency of the power transmission from the net flywheel to the PTO; and *P*PTO, *P*hyd and *P*el are the power requirements from the PTO, the hydraulic power and the electrical power, respectively. Details about the

The exhaust gas emissions were computed by considering the partial load and speed of the engine for a particular work regime according to the ISO 8178 standard [26] and the fuel features specified in [18]. To calculate the partial load, the values of wheel slippage (the difference between the ground speed provided by GPS and the wheel speed provided by the control system), PTO speed, engine speed, three-point hitch position (which determines the plowing depth when the cultivator is used), and terrain slope (obtained from the orthometric height in each point) were obtained. The partial load of the engine was calculated using these data and the equations from the ASABE standards [24, 27]. With this partial load, the work regime of the engine was obtained from the curves shown in **Figure 3**. Then, the corresponding emission factor for each exhaust substance was estimated with these data and the ISO 8178 standard [26]. The ISO standard defines the emissions factors of exhaust gases for agricultural combustion engines in eight individual work regimes based on the maximum power and the manufacturing year of the engine. For small engines, as in this case, the ISO standard defines the emission factors needed to calculate CO, particulate matter and NO<sup>X</sup> + hydrocar-

reaction (considering the other exhaust emission gases calculated with the emission factor of

Three different agricultural tasks were considered, each one requiring a specific agricultural

*n*\_\_\_*PT nFT*

− 1)(0.45

emissions were calculated using the chemical equation of combustion

*PPTOeq*(*t*) \_\_\_\_\_\_\_\_

*Em* + *PPTO* + *Phyd* + *Pel* (3)

*PPTOrated*(*t*) <sup>−</sup> 0.877)) (2)

This task consisted of performing weed control using plowing and thermal treatments with a particular mechanical-thermal machine. The weed detection system detected weed patches by processing the images from a vision camera in real time. The autonomous robot was programmed to follow an initial predefined path, which fixed the initial and final points of each track (the path followed by the vehicle through the crop). However, the initial path was corrected with information provided by the row detection system based on the vision camera of the weed detection system. The area analyzed in each image was a 3-m-wide (4 rows) and 2-m-long rectangle. It was georeferenced with an accuracy of approximately 0.08 m and divided into 0.25 × 0.25 m cells [14].

The implement consisted of a row crop cultivator and thermal device. The row crop cultivator performed a mechanical treatment in the furrows (the spaces between crop rows, similar to a conventional row crop cultivator). The thermal device consisted of several burners that produced flames applied in each row for weed control. This implement (see **Figure 6a**) was used for wide rows of crops, with rows separated by approximately 0.75 m. This technique can be applied to crops that can withstand high temperatures over short periods of time, such as

**Figure 6.** Implements working: (a) thermal and row cultivator, (b) canopy sprayer and (c) patch sprayer.

maize, garlic, leek and onion. The implement was controlled from the main controller, which was able to regulate the gas pressure of each burner separately in three stages: zero (off), low and high. The basic features of this implement are provided in **Table 3** [28].

*2.2.2. Weed control with an herbicide patch sprayer*

cells depending on their cell weed indexes.

demand from the electrical energy system slightly.

*2.2.3. Pest control with a canopy sprayer*

This task consisted of spraying herbicides over weed patches of herbaceous crops. The weed patches were detected and localized using the remote weed detection system, which was a system based on vision cameras that acquired images using aerial robots and provided a weed map of the crop consisting of 0.25 × 0.25 m cells with weed indexes indicating the percentage of each cell's area covered by weeds with respect to the total cell area. This map is an input for the autonomous robot, which will open/close the implement nozzles over the

Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution…

http://dx.doi.org/10.5772/intechopen.79875

73

The implement was a conventional patch sprayer (see **Figure 6c**) modified to activate each nozzle separately and regulate the total flow of the applied product. Two electrical linear actuators extended and retracted the spraying booms that were controlled by the robot's cen-

Originally, this implement used a main pump that worked with the PTO using the internal combustion engine's power. The pump worked to a rated power whenever a valve was open and used a bypass line to return the product overflow, wasting a large amount of energy. To improve this system, the main pump was replaced with a set of small pumps, using one pump for each nozzle. The selected pump for this application was the model MG100 Micropump (TCS Micropumps Ltd., Faversham, Kent, UK), which was able to regulate the flow to provide sufficient flow and pressure. The implement control system was able to regulate the main herbicide flow (the total nozzle flow) to ensure correct operation. This modification generated a significant reduction in power demand from the combustion engine and increased the power

This task consisted of spraying insecticide into tree canopies for pest control. The robot path plan provided the initial and final points of each track, and the robot controller was responsible for interpolating the path to follow. The implement was a canopy sprayer (see **Figure 6b**) that sprayed a pesticide solution over the tree canopies and blew the spray along the entire canopy. The canopy sprayer was designed to spray trees planted in rows spaced approximately 4 m apart, a common row spacing in olive groves. The implement was autonomous, that is, capable of detecting the tree canopies and applying pesticide doses depending of the canopy dimensions. The autonomous robot central controller turned the implement on and off only at the start and end of the mission, respectively. The sprayer had four vertically placed diffusors on each side, four of which (the lower and upper) allowed the spray direction to the canopy to range from −15 to 15° with respect to its initial vertical position. Each diffusor was equipped with two nozzles and one air outlet, and the implement control system was able to activate each diffusor separately. Eight ultrasonic sensors were used to detect the tree canopy, activate the required diffusors and regulate the diffusor positions. In addition, the sprayer regulated the main flow of pesticide and

air using a main pump and fan. **Table 3** shows the main features of this implement [30].

Analogous to the patch sprayer, the canopy sprayer originally used the internal combustion engine power from the PTO to operate the main pump and the fan that diffuses the pesticide throughout the tree canopy. Because this implement was autonomous, the robot's central

tral controller. The main features of this implement are summarized in **Table 3** [29].

The implement originally had two hydraulic cylinders to allow the main bar to extend (for treatment) and retract (during transportation). These cylinders could be replaced by linear actuators with electrical motors (LINAK LA36, Guderup, Nordborg, Denmark), reducing the power demand from the internal combustion engine and increasing the power demand from the electrical energy system. The power demand from the electrical energy system was relatively small because plowing was the main energy demand of this task, and the energy required for plowing was supplied by the internal combustion engine. This implement used gas fuel for the burners, but the gas fuel was not considered in the energy analysis; the energy analysis considered only the electrical power used to light the burners. Any type of biogas could be used for the burners, which have negligible combustion emissions.


**Table 3.** Main features of the implements [12, 31–33].

#### *2.2.2. Weed control with an herbicide patch sprayer*

maize, garlic, leek and onion. The implement was controlled from the main controller, which was able to regulate the gas pressure of each burner separately in three stages: zero (off), low

The implement originally had two hydraulic cylinders to allow the main bar to extend (for treatment) and retract (during transportation). These cylinders could be replaced by linear actuators with electrical motors (LINAK LA36, Guderup, Nordborg, Denmark), reducing the power demand from the internal combustion engine and increasing the power demand from the electrical energy system. The power demand from the electrical energy system was relatively small because plowing was the main energy demand of this task, and the energy required for plowing was supplied by the internal combustion engine. This implement used gas fuel for the burners, but the gas fuel was not considered in the energy analysis; the energy analysis considered only the electrical power used to light the burners. Any type of biogas

> Number of burners (two per row) 8 Power of the valves and sensors <1 W

Number of nozzles 12

Number of diffusors 8

Nominal flow of the nozzles (two per diffusor) 0.066 L s−1 Nominal pressure of the nozzles 3 105 Pa Power of each pump 19 W (24 V) Power of the flow control system 24 W (24 V) Air flow per nozzle ~0.5 m<sup>3</sup> s−1 Power of each fan 105 W (24 V) Power of the ultrasonic sensors 12 W (24 V) Engine power of each angle regulator (×4) 36 W (24 V)

Power of each ignitor 144 W (24 V) Linear actuator engine power (×2) 240 W (24 V)

Nominal flow of the nozzles 0.0126 L s−1 Nominal pressure of the nozzles 2.76 105 Pa Power of each pump 16.5 W (24 V) Power of the flow control system 15 W (12 V) Engine power of each linear actuator (×2) 200 W (12 V)

and high. The basic features of this implement are provided in **Table 3** [28].

72 Agricultural Robots - Fundamentals and Applications

could be used for the burners, which have negligible combustion emissions.

**Implement Feature Value** Thermal and row crop cultivator Power of the implement controller 40 W (24 V)

Patch sprayer Power of the implement controller 40 W (24 V)

Canopy sprayer Power of the implement controller 40 W (24 V)

**Table 3.** Main features of the implements [12, 31–33].

This task consisted of spraying herbicides over weed patches of herbaceous crops. The weed patches were detected and localized using the remote weed detection system, which was a system based on vision cameras that acquired images using aerial robots and provided a weed map of the crop consisting of 0.25 × 0.25 m cells with weed indexes indicating the percentage of each cell's area covered by weeds with respect to the total cell area. This map is an input for the autonomous robot, which will open/close the implement nozzles over the cells depending on their cell weed indexes.

The implement was a conventional patch sprayer (see **Figure 6c**) modified to activate each nozzle separately and regulate the total flow of the applied product. Two electrical linear actuators extended and retracted the spraying booms that were controlled by the robot's central controller. The main features of this implement are summarized in **Table 3** [29].

Originally, this implement used a main pump that worked with the PTO using the internal combustion engine's power. The pump worked to a rated power whenever a valve was open and used a bypass line to return the product overflow, wasting a large amount of energy. To improve this system, the main pump was replaced with a set of small pumps, using one pump for each nozzle. The selected pump for this application was the model MG100 Micropump (TCS Micropumps Ltd., Faversham, Kent, UK), which was able to regulate the flow to provide sufficient flow and pressure. The implement control system was able to regulate the main herbicide flow (the total nozzle flow) to ensure correct operation. This modification generated a significant reduction in power demand from the combustion engine and increased the power demand from the electrical energy system slightly.

#### *2.2.3. Pest control with a canopy sprayer*

This task consisted of spraying insecticide into tree canopies for pest control. The robot path plan provided the initial and final points of each track, and the robot controller was responsible for interpolating the path to follow. The implement was a canopy sprayer (see **Figure 6b**) that sprayed a pesticide solution over the tree canopies and blew the spray along the entire canopy. The canopy sprayer was designed to spray trees planted in rows spaced approximately 4 m apart, a common row spacing in olive groves. The implement was autonomous, that is, capable of detecting the tree canopies and applying pesticide doses depending of the canopy dimensions. The autonomous robot central controller turned the implement on and off only at the start and end of the mission, respectively. The sprayer had four vertically placed diffusors on each side, four of which (the lower and upper) allowed the spray direction to the canopy to range from −15 to 15° with respect to its initial vertical position. Each diffusor was equipped with two nozzles and one air outlet, and the implement control system was able to activate each diffusor separately. Eight ultrasonic sensors were used to detect the tree canopy, activate the required diffusors and regulate the diffusor positions. In addition, the sprayer regulated the main flow of pesticide and air using a main pump and fan. **Table 3** shows the main features of this implement [30].

Analogous to the patch sprayer, the canopy sprayer originally used the internal combustion engine power from the PTO to operate the main pump and the fan that diffuses the pesticide throughout the tree canopy. Because this implement was autonomous, the robot's central controller could not determine the instantaneous power requirements of the task. Thus, the pump and fan worked continuously at the rated power, wasting large amounts of energy. The system could be improved using a similar process as the previous application: replacing the main pump with a series of small pumps, replacing the main fan with a set of small fans and using one pump and one fan in each diffusor. The pump model MG100 Micro Pump and the axial compact fan EC W1G250-HH37-52 (ebm-papst Group, Mulfingen, Baden-Württemberg, Germany) were used for this modification. In this implement, the pesticide flow was controlled while the fan was maintained at its rated power. The reduction in power demand from the internal combustion engine was the largest of the three tasks, as it was the total power consumed by electrical devices.

#### **3. Path planning to reduce the fuel consumption**

To find a path plan that minimizes the fuel consumption, a number of different possible paths must be simulated. These paths must consider all possible track angles, field's slopes and vehicle mass losses during the spraying task. **Figure 7** shows the block diagram of the procedure. The first step is to obtain (a) the field and crop data, (b) the digital elevation model and (c) the weed map, and to define potential angles for the first track of the path plans to accomplish the treatment. The procedure relies in calculating the required energy of each path, considering that we can start the treatment from both sides of the crop (right or left) and rejecting the track with no weed. Finally, the plan with the smallest fuel consumption is selected.

**Table 4** summarizes the results obtained with these methods, and **Figure 8** represents the best

**calculated**

Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution…

With optimization 0.31 L 0.43 L Reduction 36% 13%

With optimization 0.29 L 0.53 L Reduction 61% 41%

With optimization 0.35 L 0.43 L Reduction 6% 5%

**Fuel consumption measured**

75

http://dx.doi.org/10.5772/intechopen.79875

For the path sprayer case, where the areas to be treated are known in advance, the fuel reduction resulted the best. Thus, a weed detection system that provides the weed data in advance, with the consequent energy cost, is essential. However, this study considers that energy negligible with respect to the energy savings achieved. In the case of the thermal and row cultivator implement, we know the areas to be treated a few seconds before the treatment, but it suffices for applying some energy-saving actions to obtain an important energy

Finally, in the case of the canopy sprayer, an autonomous implement capable of detecting the areas to be treated a few milliseconds in advance is used. This implement does not enable the use of nearly any energy-saving actions, and the fuel reduction obtained is thus

path plan to reduce the fuel consumption.

**Figure 8.** Test field schemes with the best path planning.

**Table 4.** Results in fuel optimization.

**Implement Implement method Fuel consumption** 

Thermal and row cultivator Without optimization 0.48 L 0.50 L

Patch sprayer Without optimization 0.74 L 0.90 L

Canopy sprayer Without optimization 0.37 L 0.45 L

reduction.

the lowest.

Fuel consumption for all possible track angels, from 0 to 360°, must be calculated because for treatments with mass losses, the motion resistance at a given point may differ. Thus, the instantaneous motion resistance at each point depends on the path plan starting point [25].

The crop limits were defined as a function of the crop features and the weed map, if available, was provided by an external device. The weed maps were represented in gray scale images using eight bits per color channel; they have a pixel size of 0.5 m and were georeferenced by the position of the lower left pixel with geodesic coordinates that are translated to the (Universal transverse Mercator) UTM. The WGS84 standard Earth reference ellipsoid was used. To estimate the terrain elevation model, the GeoTIFF ASTER GDEM images obtained from the NASA website were used. These data were provided using a 1-arc-second (approximately 30 m at the Equator) grid and are referenced to the WGS84 [34].

**Figure 7.** Path plan selection.

Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution… http://dx.doi.org/10.5772/intechopen.79875 75


**Table 4.** Results in fuel optimization.

**Table 4** summarizes the results obtained with these methods, and **Figure 8** represents the best path plan to reduce the fuel consumption.

For the path sprayer case, where the areas to be treated are known in advance, the fuel reduction resulted the best. Thus, a weed detection system that provides the weed data in advance, with the consequent energy cost, is essential. However, this study considers that energy negligible with respect to the energy savings achieved. In the case of the thermal and row cultivator implement, we know the areas to be treated a few seconds before the treatment, but it suffices for applying some energy-saving actions to obtain an important energy reduction.

Finally, in the case of the canopy sprayer, an autonomous implement capable of detecting the areas to be treated a few milliseconds in advance is used. This implement does not enable the use of nearly any energy-saving actions, and the fuel reduction obtained is thus the lowest.

**Figure 8.** Test field schemes with the best path planning.

**Figure 7.** Path plan selection.

controller could not determine the instantaneous power requirements of the task. Thus, the pump and fan worked continuously at the rated power, wasting large amounts of energy. The system could be improved using a similar process as the previous application: replacing the main pump with a series of small pumps, replacing the main fan with a set of small fans and using one pump and one fan in each diffusor. The pump model MG100 Micro Pump and the axial compact fan EC W1G250-HH37-52 (ebm-papst Group, Mulfingen, Baden-Württemberg, Germany) were used for this modification. In this implement, the pesticide flow was controlled while the fan was maintained at its rated power. The reduction in power demand from the internal combustion engine was the largest of the three tasks, as it was the total power consumed by electrical devices.

To find a path plan that minimizes the fuel consumption, a number of different possible paths must be simulated. These paths must consider all possible track angles, field's slopes and vehicle mass losses during the spraying task. **Figure 7** shows the block diagram of the procedure. The first step is to obtain (a) the field and crop data, (b) the digital elevation model and (c) the weed map, and to define potential angles for the first track of the path plans to accomplish the treatment. The procedure relies in calculating the required energy of each path, considering that we can start the treatment from both sides of the crop (right or left) and rejecting the track

Fuel consumption for all possible track angels, from 0 to 360°, must be calculated because for treatments with mass losses, the motion resistance at a given point may differ. Thus, the instantaneous motion resistance at each point depends on the path plan starting point [25].

The crop limits were defined as a function of the crop features and the weed map, if available, was provided by an external device. The weed maps were represented in gray scale images using eight bits per color channel; they have a pixel size of 0.5 m and were georeferenced by the position of the lower left pixel with geodesic coordinates that are translated to the (Universal transverse Mercator) UTM. The WGS84 standard Earth reference ellipsoid was used. To estimate the terrain elevation model, the GeoTIFF ASTER GDEM images obtained from the NASA website were used. These data were provided using a 1-arc-second (approxi-

with no weed. Finally, the plan with the smallest fuel consumption is selected.

mately 30 m at the Equator) grid and are referenced to the WGS84 [34].

**3. Path planning to reduce the fuel consumption**

74 Agricultural Robots - Fundamentals and Applications

#### **4. Energy demand analysis**

To estimate the total energy consumed in each agriculture task, the instantaneous power, the time and the relationship with the energy source were all related. The robot had four energy sources: fuel, hydrogen, batteries and solar power (the instantaneous power provided by the Sun cannot be regulated). The energy demand has two main components:


$$E\_r = E\_{\rm tCC} + E\_{\rm tES} \tag{4}$$

**5. Energetic analysis**

This section studies the energy requirements of the aforementioned agricultural tasks and the

Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution…

http://dx.doi.org/10.5772/intechopen.79875

77

The energy required for the autonomous robot and implement in each of the aforementioned agricultural tasks was estimated using a representation of real crops (see **Figure 8**). The energy analysis described in Section 4 was used to estimate the total energy consumed, measured as both the maximum power demand and average power. The power was split into two values: (1) the power demanded by the 24-V DC system and (2) the power demanded by the 12-V DC system.

The electrical power demands of the 12-V DC devices, 24-V DC devices and their combined sum are shown in **Figure 9a**. In this application, the power demand from the 12-V DC devices was approximately constant, and the power demand of the 24-V DC system had abrupt and short peaks, which were generated by the ignition of the burners. **Table 5** presents the values

The total hydrogen consumed during a working shift of 8 h was calculated under the assumption that the hydrogen fuel cell, which was described in Section 2.1.3, supplied all of the electrical energy. This task represented the lowest power demand from the electrical energy system because the energy for plowing was supplied by the internal combustion engine and the gas burners did not require electrical power to work, only an ignition spark. Thus, the

**Figure 9b** illustrates the instantaneous power demand of the patch sprayer in a weed control task. The power consumed by the 12-V DC system was approximately constant because this power was mainly used to supply energy to the system controllers. However, the power demanded by the 24-V DC system, used to provide power to the implement devices, exhibited important variation that occurred when the pumps applied the treatment to weed patches. The maximum values of the power demand, their average values and the total hydrogen consumed during an 8-h work shift are shown in **Table 5**. These values were generally higher than those for the above case.

As in previous tasks, **Figure 9c** shows the power required for pest control using the autonomous canopy sprayer. These graphs are similar to the patch sprayer application but with larger values for the power demanded by the 24-V system, the only voltage used in this implement. The numerical values (and comparisons) for power demand in the pest control task, and other tasks are provided in **Table 5**. As expected, the pest control task had the highest

*5.1.1. Energy analysis of weed control using a thermal implement and cultivator implement*

electrical energy system was mainly used to power the electrical control systems.

of these peaks and the average values of each type of power demand.

*5.1.2. Energy analysis of weed control using a patch sprayer*

*5.1.3. Energy analysis of pest control tasks using a canopy sprayer*

required features and devices of the hybrid energy system.

**5.1. Energy demanded in the selected task**

The energy provided by the internal combustion engine, *E*ICE, can be computed as

$$E\_{\rm{KCE}} = \int\_0^\cdot P\_{\rm{MCE}}(t) \, dt = \int\_0^\cdot (D(t) + F\_{\rm{MR}}(t)) \nu(t) dt \tag{5}$$

where *v* is the system speed; *D* is the implement draft force, which depends on the dimensionless soil texture adjustment parameter and machine-specific parameters; *F*MR is the motion resistance force, which depends of the soil surfaces, terrain slope, wheel slippage, total system mass and vehicle tires; and *T* is the study period. Eq. (5) computes the energy obtained from an internal combustion engine, but it does not consider the loss of mechanical and traction efficiencies in the vehicle [24, 27].

The second term of Eq. (4), that is, the energy supplied by the electrical energy system, can be calculated by

$$E\_{\text{ESS}} = \int\_0^\cdot P\_{\text{ESS}}(t) \, dt = \int\_0^\cdot \left( P\_{\text{AR}\_{\text{untotal}}}(t) + P\_{\text{MP}\_{\text{untotal}}}(t) + n \, P\_{\text{Total}}(t) \right) \, dt \tag{6}$$

where *P*EES is the instantaneous power demanded to the electrical energy system, *P*AR\_control is the power used to supply the autonomous robot controller described in Section 2.1.1, *P*IMP\_control is the power consumed by the electrical system of the implement (e.g., controllers, sensors and position actuator), *n* is the number of active tools and *P*Tool is the electrical power consumed by each tool. A tool is defined as a set of systems that can be activated separately to apply the treatment correctly in a given zone. With the thermal and row cultivator implement, the tool was the set of two burners and hoes used in each crop row, which used two ignitors (only to light the burners) and two valves (only to change the tool status). In the patch sprayer, the tool was each nozzle, each of which used a pump. In the canopy sprayer, the tool was the set of two nozzles and the air outlet of each diffuser, each of which used a pump and fan. Eqs. (4)–(6) describe the model of the energy demand in the system for a specific task.

## **5. Energetic analysis**

**4. Energy demand analysis**

76 Agricultural Robots - Fundamentals and Applications

*EICE* = ∫

efficiencies in the vehicle [24, 27].

0

*<sup>T</sup> PEES*(*t*) *dt* = ∫

0 *T*

be calculated by

specific task.

*EEES* = ∫

used to move the robot and implement.

To estimate the total energy consumed in each agriculture task, the instantaneous power, the time and the relationship with the energy source were all related. The robot had four energy sources: fuel, hydrogen, batteries and solar power (the instantaneous power provided by the

• The energy demand supplied by the internal combustion engine, *E*ICE, which is the energy

*ET* = *EICE* + *EEES* (4)

0 *T*

where *v* is the system speed; *D* is the implement draft force, which depends on the dimensionless soil texture adjustment parameter and machine-specific parameters; *F*MR is the motion resistance force, which depends of the soil surfaces, terrain slope, wheel slippage, total system mass and vehicle tires; and *T* is the study period. Eq. (5) computes the energy obtained from an internal combustion engine, but it does not consider the loss of mechanical and traction

The second term of Eq. (4), that is, the energy supplied by the electrical energy system, can

where *P*EES is the instantaneous power demanded to the electrical energy system, *P*AR\_control is the power used to supply the autonomous robot controller described in Section 2.1.1, *P*IMP\_control is the power consumed by the electrical system of the implement (e.g., controllers, sensors and position actuator), *n* is the number of active tools and *P*Tool is the electrical power consumed by each tool. A tool is defined as a set of systems that can be activated separately to apply the treatment correctly in a given zone. With the thermal and row cultivator implement, the tool was the set of two burners and hoes used in each crop row, which used two ignitors (only to light the burners) and two valves (only to change the tool status). In the patch sprayer, the tool was each nozzle, each of which used a pump. In the canopy sprayer, the tool was the set of two nozzles and the air outlet of each diffuser, each of which used a pump and fan. Eqs. (4)–(6) describe the model of the energy demand in the system for a

(*P*AR\_*control* (*t*) + *P*IMP\_*control* (*t*) + *n PTool* (*t*)

(*D*(*t*) + *FMR*(*t*))*v*(*t*)*dt* (5)

) *dt* (6)

Sun cannot be regulated). The energy demand has two main components:

• The electrical energy demand supplied by the electrical energy system, *E*EES.

The energy provided by the internal combustion engine, *E*ICE, can be computed as

*<sup>T</sup> PICE*(*t*) *dt* = ∫

• Thus, the total energy consumed, *E*T, could be calculated as follows [35]:

0

This section studies the energy requirements of the aforementioned agricultural tasks and the required features and devices of the hybrid energy system.

#### **5.1. Energy demanded in the selected task**

The energy required for the autonomous robot and implement in each of the aforementioned agricultural tasks was estimated using a representation of real crops (see **Figure 8**). The energy analysis described in Section 4 was used to estimate the total energy consumed, measured as both the maximum power demand and average power. The power was split into two values: (1) the power demanded by the 24-V DC system and (2) the power demanded by the 12-V DC system.

#### *5.1.1. Energy analysis of weed control using a thermal implement and cultivator implement*

The electrical power demands of the 12-V DC devices, 24-V DC devices and their combined sum are shown in **Figure 9a**. In this application, the power demand from the 12-V DC devices was approximately constant, and the power demand of the 24-V DC system had abrupt and short peaks, which were generated by the ignition of the burners. **Table 5** presents the values of these peaks and the average values of each type of power demand.

The total hydrogen consumed during a working shift of 8 h was calculated under the assumption that the hydrogen fuel cell, which was described in Section 2.1.3, supplied all of the electrical energy. This task represented the lowest power demand from the electrical energy system because the energy for plowing was supplied by the internal combustion engine and the gas burners did not require electrical power to work, only an ignition spark. Thus, the electrical energy system was mainly used to power the electrical control systems.

#### *5.1.2. Energy analysis of weed control using a patch sprayer*

**Figure 9b** illustrates the instantaneous power demand of the patch sprayer in a weed control task. The power consumed by the 12-V DC system was approximately constant because this power was mainly used to supply energy to the system controllers. However, the power demanded by the 24-V DC system, used to provide power to the implement devices, exhibited important variation that occurred when the pumps applied the treatment to weed patches. The maximum values of the power demand, their average values and the total hydrogen consumed during an 8-h work shift are shown in **Table 5**. These values were generally higher than those for the above case.

#### *5.1.3. Energy analysis of pest control tasks using a canopy sprayer*

As in previous tasks, **Figure 9c** shows the power required for pest control using the autonomous canopy sprayer. These graphs are similar to the patch sprayer application but with larger values for the power demanded by the 24-V system, the only voltage used in this implement. The numerical values (and comparisons) for power demand in the pest control task, and other tasks are provided in **Table 5**. As expected, the pest control task had the highest

**5.2. Hybrid energy system**

H2

in **Table 5**.

required.

electrical energy required for each agricultural task.

**Table 5.** Power and hydrogen consumed in each application.

cell with a minimum power of 1.16 kW and at least 5.96 Nm<sup>3</sup>

**Implement**

12-V average power 0.26 kW 0.28 kW 0.26 kW 24-V average power 0.22 kW 0.28 kW 0.90 kW Total average power 0.48 kW 0.56 kW 1.16 kW 12-V maximum power 0.26 kW 0.40 kW 0.26 kW 24-V maximum power 0.79 kW 0.41 kW 1.39 kW Total maximum power 1.05 kW 0.68 kW 1.65 kW

consumed (for 8 h) 2.43 Nm<sup>3</sup> 2.84 Nm<sup>3</sup> 5.96 Nm<sup>3</sup>

used. Four tanks were used, each with a capacity of 3 Nm<sup>3</sup>

The energy system studied in this work used the original internal combustion engine of the tractor operating in parallel with an electrical energy system. The architecture of the resulting hybrid energy system is shown in **Figure 10**. The internal combustion engine was used to provide motion, overcoming the motion resistant force and possible draft forces generated by the implement, while the electrical energy system was used to power all electrical systems onboard the autonomous robot. The combustion engine had enough power and autonomy for the tasks analyzed in this work, but an electrical energy system was needed to supply the

**Thermal and row crop cultivator Patch sprayer Canopy sprayer**

Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution…

http://dx.doi.org/10.5772/intechopen.79875

79

The electrical energy system used hydrogen as the main energy source, with a small contribution from a photovoltaic panel, and used batteries to adapt the power supply to the energy requirements and store excess electrical energy generated by the photovoltaic panel when it was not in use. The electrical energy system was designed according to the maximum values

The hydrogen system was designed to supply the average power demanded by all electrical systems, for example, control systems, sensors and actuators. Therefore, a hybrid fuel

The hybrid fuel cell was based on the TROPICAL TB-1000 model. It was an unregulated DC power system based on a proton exchange membrane fuel cell (FCgen-1020ACS, Ballard Power Systems, Burnaby, British Columbia, Canada). The system had to be fueled with pure hydrogen and was able to deliver up to 1.4 kW of peak electrical power and 1.2 kW in a nominal continuous operation. A hydrogen storage system based on metal hydride tanks was

The photovoltaic panel was a Module EGM-185 (EGing PV Co., Ltd., Jintan, Jiangsu, P.R. C), which had a power rating of 183 W and an efficiency of approximately 15%. At the test site location (40°18′29″ N, 3°29′14″ W), this panel provided an average daily energy amount of

.

of hydrogen storage was

**Figure 9.** Power demands of the electrical energy system (ESS): (a) weed control using the thermal and row crop cultivator implement; (b) weed control using the sprayer implement and (c) pest control using the canopy sprayer. PEES 12, PEES 24 and PES*S* are the 12-V, 24-V and total power, respectively.

power demand from the electrical energy system, although the power demand from the 12-V DC system was similar to the other three cases because it was used mainly to supply the controllers. Thus, the controllers had a quasi-constant power demand and were not strongly influenced by the task.

Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution… http://dx.doi.org/10.5772/intechopen.79875 79


**Table 5.** Power and hydrogen consumed in each application.

#### **5.2. Hybrid energy system**

power demand from the electrical energy system, although the power demand from the 12-V DC system was similar to the other three cases because it was used mainly to supply the controllers. Thus, the controllers had a quasi-constant power demand and were not strongly

**Figure 9.** Power demands of the electrical energy system (ESS): (a) weed control using the thermal and row crop cultivator implement; (b) weed control using the sprayer implement and (c) pest control using the canopy sprayer. PEES 12, PEES 24 and

influenced by the task.

PES*S* are the 12-V, 24-V and total power, respectively.

78 Agricultural Robots - Fundamentals and Applications

The energy system studied in this work used the original internal combustion engine of the tractor operating in parallel with an electrical energy system. The architecture of the resulting hybrid energy system is shown in **Figure 10**. The internal combustion engine was used to provide motion, overcoming the motion resistant force and possible draft forces generated by the implement, while the electrical energy system was used to power all electrical systems onboard the autonomous robot. The combustion engine had enough power and autonomy for the tasks analyzed in this work, but an electrical energy system was needed to supply the electrical energy required for each agricultural task.

The electrical energy system used hydrogen as the main energy source, with a small contribution from a photovoltaic panel, and used batteries to adapt the power supply to the energy requirements and store excess electrical energy generated by the photovoltaic panel when it was not in use. The electrical energy system was designed according to the maximum values in **Table 5**.

The hydrogen system was designed to supply the average power demanded by all electrical systems, for example, control systems, sensors and actuators. Therefore, a hybrid fuel cell with a minimum power of 1.16 kW and at least 5.96 Nm<sup>3</sup> of hydrogen storage was required.

The hybrid fuel cell was based on the TROPICAL TB-1000 model. It was an unregulated DC power system based on a proton exchange membrane fuel cell (FCgen-1020ACS, Ballard Power Systems, Burnaby, British Columbia, Canada). The system had to be fueled with pure hydrogen and was able to deliver up to 1.4 kW of peak electrical power and 1.2 kW in a nominal continuous operation. A hydrogen storage system based on metal hydride tanks was used. Four tanks were used, each with a capacity of 3 Nm<sup>3</sup> .

The photovoltaic panel was a Module EGM-185 (EGing PV Co., Ltd., Jintan, Jiangsu, P.R. C), which had a power rating of 183 W and an efficiency of approximately 15%. At the test site location (40°18′29″ N, 3°29′14″ W), this panel provided an average daily energy amount of

**Figure 10.** Hybrid energy system.

0.88 kWh, with a maximum of 1.46 kWh per day in July, according to the irradiation data available from the Photovoltaic Geographical Information System of the Institute for Energy and Transport [36].

• Obtaining the maximum photovoltaic power.

**Figure 11.** Block diagram of the flow energy control.

when the robot was stationary.

**6. Results and discussion**

• Supervising the hydrogen storage, batteries status and photovoltaic power.

The energy management system used two solar-panel controllers (either 12-V batteries or 24-V batteries) with the maximum power point tracking (MPPT SS-MPPT-15 L, Morningstar,

Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution…

http://dx.doi.org/10.5772/intechopen.79875

81

Both 12-V and 24-V battery chargers (or power adapters) (BCD1015, Analytic Systems Ware Ltd., Delta, British Columbia, Canada) were required to adapt the power from the hybrid fuel cell. The energy management system was equipped with a controller that managed the energy flow. The block diagram of the energy management system is shown in **Figure 11**, where *C*12V and *C*24V are the charges of the 12-V and 24-V batteries, respectively; *C*12Vmin and *C*24Vmin are the minimum charges in these batteries with the hybrid fuel cell stopped; and *C*12Vmax and *C*24Vmax are the maximum charges in these batteries when the hybrid fuel cell is running. *C*12Vmin and *C*24Vmin were calculated to ensure correct operation during periods of high energy demand. The *C*12Vmax and *C*24Vmax values were calculated to create a hysteresis cycle for the hybrid fuel cell's operation with a value that is sufficiently high to avoid excessive start/stop in the hybrid fuel cell but sufficiently low to allow for the storage of the photovoltaic energy generated

This section presents the emission reductions obtained by using the hybrid energy system in real scenarios. To analyze the results, the emissions of the autonomous robot with the

Inc., Chicago, Illinois, US) to obtain the maximum power from the photovoltaic panel.

Deep-cycle lead-acid batteries, which could supply current levels greater than those provided by the hybrid fuel cell over short periods, were used. The lead-acid batteries were charged by both the hybrid fuel cell and photovoltaic panel and stored all unused photovoltaic energy. The batteries were divided into two banks to supply 12 V DC and 24 V DC. Each bank consisted of two batteries, each with a capacity of 2.2 kWh; this capacity was sufficient to store excess photovoltaic energy over several days of inactivity and could be used during sporadic periods of high energy demand, as shown in **Figure 11**. **Table 5** shows that the power demand of the 24-V DC system was higher, but this analysis did not consider the energy required to start the combustion engine. Furthermore, two or more batteries in parallel were required to start the internal combustion engine because deep-cycle batteries were used.

The energy management system was responsible for


Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution… http://dx.doi.org/10.5772/intechopen.79875 81

**Figure 11.** Block diagram of the flow energy control.

**Figure 10.** Hybrid energy system.

80 Agricultural Robots - Fundamentals and Applications

and Transport [36].

panel.

0.88 kWh, with a maximum of 1.46 kWh per day in July, according to the irradiation data available from the Photovoltaic Geographical Information System of the Institute for Energy

Deep-cycle lead-acid batteries, which could supply current levels greater than those provided by the hybrid fuel cell over short periods, were used. The lead-acid batteries were charged by both the hybrid fuel cell and photovoltaic panel and stored all unused photovoltaic energy. The batteries were divided into two banks to supply 12 V DC and 24 V DC. Each bank consisted of two batteries, each with a capacity of 2.2 kWh; this capacity was sufficient to store excess photovoltaic energy over several days of inactivity and could be used during sporadic periods of high energy demand, as shown in **Figure 11**. **Table 5** shows that the power demand of the 24-V DC system was higher, but this analysis did not consider the energy required to start the combustion engine. Furthermore, two or more batteries in parallel were required to

• Regulating and adapting the power provided by the hydrogen fuel cell and photovoltaic

start the internal combustion engine because deep-cycle batteries were used.

The energy management system was responsible for

• Assuring a minimum charge in the batteries.


The energy management system used two solar-panel controllers (either 12-V batteries or 24-V batteries) with the maximum power point tracking (MPPT SS-MPPT-15 L, Morningstar, Inc., Chicago, Illinois, US) to obtain the maximum power from the photovoltaic panel.

Both 12-V and 24-V battery chargers (or power adapters) (BCD1015, Analytic Systems Ware Ltd., Delta, British Columbia, Canada) were required to adapt the power from the hybrid fuel cell. The energy management system was equipped with a controller that managed the energy flow. The block diagram of the energy management system is shown in **Figure 11**, where *C*12V and *C*24V are the charges of the 12-V and 24-V batteries, respectively; *C*12Vmin and *C*24Vmin are the minimum charges in these batteries with the hybrid fuel cell stopped; and *C*12Vmax and *C*24Vmax are the maximum charges in these batteries when the hybrid fuel cell is running. *C*12Vmin and *C*24Vmin were calculated to ensure correct operation during periods of high energy demand. The *C*12Vmax and *C*24Vmax values were calculated to create a hysteresis cycle for the hybrid fuel cell's operation with a value that is sufficiently high to avoid excessive start/stop in the hybrid fuel cell but sufficiently low to allow for the storage of the photovoltaic energy generated when the robot was stationary.

#### **6. Results and discussion**

This section presents the emission reductions obtained by using the hybrid energy system in real scenarios. To analyze the results, the emissions of the autonomous robot with the internal combustion engine as the only power system were compared with the emissions of the same robot using the additional hybrid energy system described in Section 5.2. The same autonomous mobile robot was used in all tests, and the fuel consumption and emissions were measured as explained in Section 2.1.7. The experiments were carried out for the crops and tasks introduced in Section 2.2, and the results are described in the following sections.

#### **6.1. Hybrid power of weed control with a thermal implement and a cultivator implement**

This test was performed over the maize crop represented in **Figure 8a**, where the path followed by the robot is indicated in blue and the weed patches are indicated in green. As described in Section 2.2.1, the vision system onboard the robot detected the weed patches in real time; therefore, the trajectories had to cover the entire field because a priori knowledge of where the patches were located was not available. Although the burners were activated only over weed patches, the hoes plowed all furrows to kill weeds and aerate the soil. Therefore, the energy required to plow all tracks was high, and more exhaust gas was produced, particularly CO2 , as shown in **Figure 12a**, which shows the exhaust gas emitted as a function of the distance traveled. The required energy to plow was the main power demand supplied by the engine in both cases (using the internal combustion engine alone and using it along with the hybrid energy system). Consequently, the emission reduction obtained due to the use of the hybrid energy system was small.

**6.2. Hybrid power of weed control with an herbicide patch sprayer**

**Table 6.** Average values and comparison of exhaust gas emissions in the three applications.

to pass over only the weed patches, reducing energy consumption.

**Figure 13** shows the instantaneous emissions of CO2

**Figure 13.** Exhaust gas emissions in weed control using the patch sprayer.

This test was performed over the wheat crop represented in **Figure 8b**, where the path followed by the robot is indicated in blue and the weed patches are indicated in green. The weed map was acquired in advance using remote sensing. Thus, the robot did not need to move over the entire field, as in the task described in Section 2.2.2, and the path could be optimized

**Application Power system CO2 CO HC + NOX PM**

Thermal and cultivator Combustion engine only 11.32 0.1132 0.1478 0.0126

Patch sprayer Combustion engine only 9.86 0.1334 0.1852 0.013

Canopy sprayer Combustion engine only 10.74 0.1425 0.2047 0.0125

**(kg h−1)**

Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution…

Hybrid system 10.36 0.1129 0.1475 0.0126 Exhaust gas reduction 8.53% 0.2% 0.2% 0.0%

Hybrid system 6.25 0.110 0.1434 0.0123 Exhaust gas reduction 36.6% 17.8% 22.6% 5.4%

Hybrid system 5.64 0.1071 0.1393 0.012 Exhaust gas reduction 47.5% 24.8% 31.9% 3.8%

ulate matter obtained in this experiment, and **Table 6** shows their average values and the

, CO, hydrocarbons + NOX and partic-

http://dx.doi.org/10.5772/intechopen.79875

83

As shown in **Figures 12a**–**d** and **Table 6**, the reduction in air pollution during this task was lower than in the other analyzed tasks. The CO2 emissions were only slightly reduced as a result of the energy consumed by the burner ignitors and electrical control system, although the energy consumed by the ignitors was relatively negligible.

**Figure 12.** Exhaust gas emissions in weed control using the thermal and row crop cultivator implement.

Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution… http://dx.doi.org/10.5772/intechopen.79875 83


**Table 6.** Average values and comparison of exhaust gas emissions in the three applications.

internal combustion engine as the only power system were compared with the emissions of the same robot using the additional hybrid energy system described in Section 5.2. The same autonomous mobile robot was used in all tests, and the fuel consumption and emissions were measured as explained in Section 2.1.7. The experiments were carried out for the crops and

This test was performed over the maize crop represented in **Figure 8a**, where the path followed by the robot is indicated in blue and the weed patches are indicated in green. As described in Section 2.2.1, the vision system onboard the robot detected the weed patches in real time; therefore, the trajectories had to cover the entire field because a priori knowledge of where the patches were located was not available. Although the burners were activated only over weed patches, the hoes plowed all furrows to kill weeds and aerate the soil. Therefore, the energy required to plow all tracks was high, and more exhaust gas was produced, particularly CO2

as shown in **Figure 12a**, which shows the exhaust gas emitted as a function of the distance traveled. The required energy to plow was the main power demand supplied by the engine in both cases (using the internal combustion engine alone and using it along with the hybrid energy system). Consequently, the emission reduction obtained due to the use of the hybrid

As shown in **Figures 12a**–**d** and **Table 6**, the reduction in air pollution during this task was

result of the energy consumed by the burner ignitors and electrical control system, although

**Figure 12.** Exhaust gas emissions in weed control using the thermal and row crop cultivator implement.

emissions were only slightly reduced as a

,

tasks introduced in Section 2.2, and the results are described in the following sections.

**6.1. Hybrid power of weed control with a thermal implement and a cultivator** 

**implement**

82 Agricultural Robots - Fundamentals and Applications

energy system was small.

lower than in the other analyzed tasks. The CO2

the energy consumed by the ignitors was relatively negligible.

#### **6.2. Hybrid power of weed control with an herbicide patch sprayer**

This test was performed over the wheat crop represented in **Figure 8b**, where the path followed by the robot is indicated in blue and the weed patches are indicated in green. The weed map was acquired in advance using remote sensing. Thus, the robot did not need to move over the entire field, as in the task described in Section 2.2.2, and the path could be optimized to pass over only the weed patches, reducing energy consumption.

**Figure 13** shows the instantaneous emissions of CO2 , CO, hydrocarbons + NOX and particulate matter obtained in this experiment, and **Table 6** shows their average values and the

**Figure 13.** Exhaust gas emissions in weed control using the patch sprayer.

reductions obtained when using the hybrid energy system. **Figure 13a** shows a significant reduction in CO2 emissions because the hybrid energy system avoids the use of the PTO, resulting in a significant reduction in fuel consumption.

**7. Conclusions**

is able to treat.

This work demonstrates that using fuel optimization techniques with a good consumption model combined with field data (field limits and field elevation map) and crop needs can achieve fuel savings of approximately 50% in the best case. Furthermore, it is proved that to combine current agricultural machines, which use combustion engines for power, with new technologies that are based on clean energy sources to significantly reduce the emissions of atmospheric pollutants and greenhouse gases. This integration can be accomplished by offloading the combustion engine and adding this load to an additional electrical energy system. This technique was highly effective for tasks where the implement requires PTO power, as shown in Sections 6.2 and 6.3. The replacement of this PTO power is relatively simple; only small modifications were required in the implement, as described in Section 2.2. When the implement generated draft force (e.g., in plowing), this technique was not as effective as in the experiments analyzed in Sections 6.2 and 6.3; however, a reduction in the pollutant emissions was obtained from the robotic systems when electrical energy consumption was important.

Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution…

http://dx.doi.org/10.5772/intechopen.79875

85

The use of electrical energy systems allows small electrical actuators to be used, which are able to apply treatments to small areas with little power consumption. The use of such distributed systems is particularly important in precision agriculture, where treatments must be focused only on affected areas, which are often smaller than the total area that the implement

The greatest improvement was obtained by the autonomous implement analyzed in Section 6.3. In this example, the robot does not know the instantaneous power requirements of the implement, and thus, for the case in which the combustion engine is the only power source, the engine must supply the rated power to the implement, which is an inefficient use of energy. However, with the hybrid energy system, the implement uses the energy provided

The CO and particulate matter emissions present the least reduction because these concentrations of emissions were larger when the hybrid energy system was used due to the effect of decreasing engine speed. Similar emission results were obtained for these gases in many studies that analyzed internal combustion engine exhaust gases [5, 6, 37]. But, although the concentrations of CO and particulate matter in the exhaust gases were lesser for these engine speeds and loads, the total exhaust gases increased because the flow of exhaust gases emitted from the combustion engine was much greater than the gases from the hybrid energy system. The theoretical studies and experiments conducted in this work reveal that the use of a hybrid energy system in precision agriculture via autonomous robots improves the quality of the exhaust gases and decreases energy use. Compared with traditional tractors, robotic tractors have increased electric power consumption; therefore, an electrical energy system must be added when the agricultural vehicle is robotized because the use of alternators increases energy loss. Furthermore, an electrical energy system can be designed to supply some of the energy requirements of various agricultural tasks, as shown in this work. The hybrid energy system significantly reduced atmospheric pollutant emissions, including CO2

NOX, hydrocarbons and particulate matter. This work has demonstrated that hybrid energy

, CO,

by the electrical energy system, and it is able to manage and minimize its energy.

**Figure 13c** presents the reductions in hydrocarbon and NOX emissions, but these reductions were smaller than the reduction in CO2 because the NOX concentration in the exhaust gases decreased with engine speed but the concentration of hydrocarbons increased. Particulate matter emissions were highly similar in both cases (see **Figure 13d**) because their concentrations in the exhaust gases increased as the engine speed decreased, which occurred when the PTO was off or operating slowly. A similar result, but to a lesser extent, was obtained for CO, as shown in **Figure 13b** and **Table 6**.

#### **6.3. Hybrid power of pest control with a canopy sprayer**

This test was performed in the small olive grove represented in **Figure 8c**, where the path followed by the autonomous robot is indicated in blue and the olive trees are indicated in green. The implement used for this task was the autonomous canopy sprayer, as described in Section 2.2.3. This implement, as discussed in Section 5.1.3, demanded the majority of the energy from the electrical energy system out of the three experiments that were conducted. The highest reduction in power demand from the combustion engine was achieved in this experiment, resulting in the highest reduction in the exhaust gases, as illustrated in **Figures 14a**–**d** and **Table 6**. The reduction in CO2 emissions reached approximately 50%. The results are similar to the previous experiments (herbicide spraying) but, in general, with a greater reduction in emissions.

**Figure 14.** Exhaust gas emissions in weed control using the patch sprayer.

#### **7. Conclusions**

reductions obtained when using the hybrid energy system. **Figure 13a** shows a significant

**Figure 13c** presents the reductions in hydrocarbon and NOX emissions, but these reductions

decreased with engine speed but the concentration of hydrocarbons increased. Particulate matter emissions were highly similar in both cases (see **Figure 13d**) because their concentrations in the exhaust gases increased as the engine speed decreased, which occurred when the PTO was off or operating slowly. A similar result, but to a lesser extent, was obtained for CO,

This test was performed in the small olive grove represented in **Figure 8c**, where the path followed by the autonomous robot is indicated in blue and the olive trees are indicated in green. The implement used for this task was the autonomous canopy sprayer, as described in Section 2.2.3. This implement, as discussed in Section 5.1.3, demanded the majority of the energy from the electrical energy system out of the three experiments that were conducted. The highest reduction in power demand from the combustion engine was achieved in this experiment, resulting in the highest reduction in the exhaust gases, as illustrated in **Figures 14a**–**d** and

to the previous experiments (herbicide spraying) but, in general, with a greater reduction in

emissions because the hybrid energy system avoids the use of the PTO,

because the NOX concentration in the exhaust gases

emissions reached approximately 50%. The results are similar

reduction in CO2

resulting in a significant reduction in fuel consumption.

**6.3. Hybrid power of pest control with a canopy sprayer**

**Figure 14.** Exhaust gas emissions in weed control using the patch sprayer.

were smaller than the reduction in CO2

84 Agricultural Robots - Fundamentals and Applications

as shown in **Figure 13b** and **Table 6**.

**Table 6**. The reduction in CO2

emissions.

This work demonstrates that using fuel optimization techniques with a good consumption model combined with field data (field limits and field elevation map) and crop needs can achieve fuel savings of approximately 50% in the best case. Furthermore, it is proved that to combine current agricultural machines, which use combustion engines for power, with new technologies that are based on clean energy sources to significantly reduce the emissions of atmospheric pollutants and greenhouse gases. This integration can be accomplished by offloading the combustion engine and adding this load to an additional electrical energy system. This technique was highly effective for tasks where the implement requires PTO power, as shown in Sections 6.2 and 6.3. The replacement of this PTO power is relatively simple; only small modifications were required in the implement, as described in Section 2.2. When the implement generated draft force (e.g., in plowing), this technique was not as effective as in the experiments analyzed in Sections 6.2 and 6.3; however, a reduction in the pollutant emissions was obtained from the robotic systems when electrical energy consumption was important.

The use of electrical energy systems allows small electrical actuators to be used, which are able to apply treatments to small areas with little power consumption. The use of such distributed systems is particularly important in precision agriculture, where treatments must be focused only on affected areas, which are often smaller than the total area that the implement is able to treat.

The greatest improvement was obtained by the autonomous implement analyzed in Section 6.3. In this example, the robot does not know the instantaneous power requirements of the implement, and thus, for the case in which the combustion engine is the only power source, the engine must supply the rated power to the implement, which is an inefficient use of energy. However, with the hybrid energy system, the implement uses the energy provided by the electrical energy system, and it is able to manage and minimize its energy.

The CO and particulate matter emissions present the least reduction because these concentrations of emissions were larger when the hybrid energy system was used due to the effect of decreasing engine speed. Similar emission results were obtained for these gases in many studies that analyzed internal combustion engine exhaust gases [5, 6, 37]. But, although the concentrations of CO and particulate matter in the exhaust gases were lesser for these engine speeds and loads, the total exhaust gases increased because the flow of exhaust gases emitted from the combustion engine was much greater than the gases from the hybrid energy system. The theoretical studies and experiments conducted in this work reveal that the use of a hybrid energy system in precision agriculture via autonomous robots improves the quality of the exhaust gases and decreases energy use. Compared with traditional tractors, robotic tractors have increased electric power consumption; therefore, an electrical energy system must be added when the agricultural vehicle is robotized because the use of alternators increases energy loss. Furthermore, an electrical energy system can be designed to supply some of the energy requirements of various agricultural tasks, as shown in this work. The hybrid energy system significantly reduced atmospheric pollutant emissions, including CO2 , CO, NOX, hydrocarbons and particulate matter. This work has demonstrated that hybrid energy systems can be reliably and autonomously used in agricultural tasks with tractors or robots, decreasing (to various extents) the load on the internal combustion engine. This development can be regarded as an intermediate step toward the use of completely clean energy systems.

*n*FT full throttle engine speed (rpm)

NASA The National Aeronautics and Space Administration

Mariano Gonzalez-de-Soto, Luis Emmi and Pablo Gonzalez-de-Santos\*

[1] U.S. Environmental Protection Agency (EPA). http://www.epa.gov/ [Accessed: January 4,

Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution…

http://dx.doi.org/10.5772/intechopen.79875

87

[2] Hansson PA, Lindgren M, Noren O. PM-power and machinery: A comparison between different methods of calculating average engine emissions for agricultural tractors. Journal of Agricultural Engineering Research. 2001;**80**:37-43. DOI: 10.1006/jaer.

[3] Dalgaard T, Halberg N, Porter JR. A model for fossil energy use in Danish agriculture used to compare organic and conventional farming. Agriculture Ecosystems &

[4] Peltre C, Nyord T, Bruun S, Jensen LS, Magid J. Repeated soil application of organic waste amendments reduces draught force and fuel consumption for soil tillage. Agriculture

[5] Lindgren M, Arrhenius K, Larsson G, Bäfver L, Arvidsson H, Wetterberg C, et al. Analysis of unregulated emissions from an off-road diesel engine during realistic work operations. Atmospheric Environment. 2011;**45**:5394-5398. DOI: 10.1016/j.atmosenv.2011.06.046

Ecosystems & Environment. 2015;**211**:94-101. DOI: 10.1016/j.agee.2015.06.004

Environment. 2001;**87**:51-65. DOI: 10.1016/S0167-8809(00)00297-8

Centre for Automation and Robotics (UPM-CSIC), Madrid, Spain

\*Address all correspondence to: pablo.gonzalez@csic.es

*P*hyd hydraulic power (W) *P*el electrical power (W)

DC direct current

PTO power take-off

**Author details**

**References**

2018]

2001.0710

GPS global positioning system

UTM Universal transverse Mercator WGS84 World geodetic system of 1984

RTK real-time kinematic

*Acronyms*

#### **Acknowledgements**

The research leading to these results was funded by both CSIC (project BMCrop, ref. 201750E089) and the European Union's Seventh Framework Programme (Project RHEA, Grant Agreement no 245986).

#### **Nomenclature**

*Chemical components*


SO2 sulfur dioxide



Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution… http://dx.doi.org/10.5772/intechopen.79875 87

*n*FT full throttle engine speed (rpm)

*P*hyd hydraulic power (W)

*P*el electrical power (W)

*Acronyms*

systems can be reliably and autonomously used in agricultural tasks with tractors or robots, decreasing (to various extents) the load on the internal combustion engine. This development can be regarded as an intermediate step toward the use of completely clean energy systems.

The research leading to these results was funded by both CSIC (project BMCrop, ref. 201750E089) and the European Union's Seventh Framework Programme (Project RHEA, Grant Agreement

*E*ICE energy demand supplied by the internal combustion engine (Wh)

**Acknowledgements**

86 Agricultural Robots - Fundamentals and Applications

no 245986).

*Symbols*

**Nomenclature**

*Chemical components*

CO carbon monoxide

*D* implement draft force (N)

*F*MR total implement motion resistance (N)

*P*Tool electrical power of each implement tool (W)

*V*SFC specific fuel consumption volume (L Wh−1)

*P*PTO power requirement from the PTO shaft (W)

*P*T\_PTOeq total equivalent PTO demanded power (W)

*n*PT partial throttle engine speed (rpm)

*P*AR\_control power used to supply the autonomous robot controller (W)

*P*IMP\_control power consumed by the electrical system of the implement (W)

CO2 carbon dioxide NO<sup>X</sup> nitrogen oxides SO2 sulfur dioxide

*n* number of tools

*V*TFC total fuel consumed (L)

*P*PTOeq equivalent PTO power (W)

*P*PTOrated rated PTO power (W)

*E*<sup>T</sup> total energy (Wh)


#### **Author details**

Mariano Gonzalez-de-Soto, Luis Emmi and Pablo Gonzalez-de-Santos\*

\*Address all correspondence to: pablo.gonzalez@csic.es

Centre for Automation and Robotics (UPM-CSIC), Madrid, Spain

#### **References**


[6] Janulevičius A, Juostas A, Pupinis G. Tractor's engine performance and emission characteristics in the process of ploughing. Energy Conversion and Management. 2013;**75**:498- 508. DOI: 10.1016/j.enconman.2013.06.052

[18] Spanish Royal Order 61/2006 of January 31, which determines the requirements of gasoline, gasoline, fuels and liquefied petroleum gases and regulates the use of certain biofuels. Official State Gazette, February 17, 2006, no. 41. pp. 6342-6357. In Spanish

Hybrid-Powered Autonomous Robots for Reducing Both Fuel Consumption and Pollution…

http://dx.doi.org/10.5772/intechopen.79875

89

[19] International Energy Agency. Oil information: Documentation for beyond 2020 file; 2013

[20] Titan Enterprises Ltd. Titan flow meters for oil, boiler and diesel fuel applications— PD400 oil flowmeter. 2018. http://www.flowmeters.co.uk/pd\_pd400om [Accessed:

[21] Grisso R, Kocher M, Vaughan D. Predicting tractor fuel consumption. Biological Systems Engineering: Papers and Publications. University of Nebraska. 2004. https://pdfs.semanticscholar.org/79db/cec98eaa6ef92be5d1c590a5dd587121b1db.pdf [Accessed: January 4,

[22] Rahimi-Ajdadi F, Abbaspour-Gilandeh Y. Artificial neural network and stepwise multiple range regression methods for prediction of tractor fuel consumption. Measurement.

[23] Titan Enterprises Ltd-Titan flow meters for oil, boiler and diesel fuel applications— PD400 oil flowmeter. 2014. http://www.flowmeters.co.uk/ [Accessed: February 20, 2014]

[24] ASAE. D497.7 MAR2011 Agricultural Machinery Management Data. St. Joseph, MI,

[25] Gonzalez-de-Soto M, Emmi L, Garcia I, Gonzalez-de-Santos P. Reducing fuel consumption in weed and pest control using robotic tractors. Computers and Electronics in

[26] ISO 8178-4: 1996-Reciprocating internal combustion engines-exhaust emission measurement-Part 4: Test cycles for different engine applications. International Organisation of

[27] ANSI/ASAE. S296.4 DEC95 Agricultural Machinery Management Data. St. Joseph, MI,

[28] Frasconi C, Martelloni L, Fontanelli M, Raffaelli M, Emmi L, Pirchio M, Peruzzi A. Design and full realization of physical weed control (PWC) automatic machine within the RHEA project. In: Second International Conference on Robotics and Associated High-Technologies and Equipment for Agriculture and Forestry (RHEA-2014); May 21-23,

[29] Perez-Ruiz M, Gonzalez-de-Santos P, Ribeiro A, Fernandez-Quintanilla C, Peruzzi A, Vieri M, Tomic S, Agüera J. Highlights and preliminary results for autonomous crop protection. Computers and Electronics in Agriculture. 2015;**110**:150-161. DOI: 10.1016/j.

[30] Sarri D, Lisci R, Rimediotti M, Vieri M. RHEA airblast sprayer: Calibration indexes of the airjet vector related to canopy and foliage characteristics. In: Second International Conference

2011;**44**:2104-2111. DOI: 10.1016/j.measurement.2011.08.006

Agriculture. 2015;**114**:96-113. DOI: 10.1016/j.compag.2015.04.003

USA: ASAE Standards; 2011. pp. 372-380

USA: ASAE Standards; 1995. pp. 118-120

Standardisation; 1996

Madrid, Spain. 2014. pp. 3-11

compag.2014.11.010

January 4, 2018]

2018]


[18] Spanish Royal Order 61/2006 of January 31, which determines the requirements of gasoline, gasoline, fuels and liquefied petroleum gases and regulates the use of certain biofuels. Official State Gazette, February 17, 2006, no. 41. pp. 6342-6357. In Spanish

[6] Janulevičius A, Juostas A, Pupinis G. Tractor's engine performance and emission characteristics in the process of ploughing. Energy Conversion and Management. 2013;**75**:498-

[7] Gasparatos A, Stromberg P, Takeuchi K. Biofuels, ecosystem services and human wellbeing: Putting biofuels in the ecosystem services narrative. Agriculture Ecosystems &

[8] Mousazadeh H, Keyhani A, Javadi A, Mobli H, Abrinia K, Sharifi A. Life-cycle assessment of a solar assist plug-in hybrid electric tractor (SAPHT) in comparison with a conventional tractor. Energy Conversion and Management. 2011;**52**:1700-1710. DOI:

[9] Mulloney JA Jr. Mitigation of carbon dioxide releases from power production via "sustainable agri-power": The synergistic combination of controlled environmental agriculture (large commercial greenhouses) and disbursed fuel cell power plants. Energy conversion and management. In: Proceedings of the International Energy Agency Carbon Dioxide Disposal Symposium. Vol. 34. 1993. pp. 913-920. DOI: 10.1016/0196-8904(93)90036-A [10] Lutz AE, Larson RS, Keller JO. Thermodynamic comparison of fuel cells to the Carnot cycle. International Journal of Hydrogen Energy. 2002;**27**:1103-1111. DOI: 10.1016/S0360-

[11] Offer GJ, Howey D, Contestabile M, Clague R, Brandon NP. Comparative analysis of battery electric, hydrogen fuel cell and hybrid vehicles in a future sustainable road trans-

[12] Gonzalez-de-Santos P, Ribeiro A, Fernandez-Quintanilla C, Lopez-Granados F, Brandstoetter M, Tomic S, Pedrazzi S, Peruzzi A, Pajares G, Kaplanis G, Perez-Ruiz M, Valero C, del Cerro J, Vieri M, Rabatel G, Debilde B. Fleets of robots for environmentally-

[13] Emmi L, Gonzalez-de-Soto M, Pajares G, Gonzalez-de-Santos P. New trends in robotics for agriculture: Integration and assessment of a real fleet of robots. The Scientific World

[14] Emmi L, Gonzalez-de-Soto M, Pajares G, Gonzalez-de-Santos P. Integrating sensory/ actuation systems in agricultural vehicles. Sensors. 2014;**14**:4014-4049. DOI: 10.3390/

[15] Conesa-Muñoz J, Gonzalez-de-Soto M, Gonzalez-de-Santos P, Ribeiro A. Distributed multi-level supervision to effectively monitor the operations of a fleet of autonomous vehicles in agricultural tasks. Sensors. 2015;**15**:5402-5428. DOI: 10.3390/s150305402 [16] CNH America LLC. BOOMER 3040, 3045, 3050 CVT service manual complete contents;

[17] Tropical, Fuel Cell Power Gener. Hydrog. Technol. Electr. Automob. e Trop. SA Fuel Cell Hydrog. Technol. 2018. http://www.tropical.gr/index.php [Accessed: January 4, 2018]

port system. Energy Policy. 2010;**38**:24-29. DOI: 10.1016/j.enpol.2009.08.040

safe pest control in agriculture. Precision Agriculture. 2017;**18**(4):574-614

Journal. 2014;**2014**:1-21. Article ID 404059. DOI: 10.1155/2014/404059

Environment. 2011;**142**:111-128. DOI: 10.1016/j.agee.2011.04.020

508. DOI: 10.1016/j.enconman.2013.06.052

10.1016/j.enconman.2010.10.033

88 Agricultural Robots - Fundamentals and Applications

3199(02)00016-2

s140304014

2009


on Robotics and Associated High-Technologies and Equipment for Agriculture and Forestry; May 21-23; Madrid, Spain. 2014. pp. 73-81

**Chapter 6**

Provisional chapter

**Fundamental Research on Unmanned Aerial Vehicles**

Support Precision Agriculture in Oil Palm Plantations

Redmond Ramin Shamshiri, Ibrahim A. Hameed,

Redmond Ramin Shamshiri, Ibrahim A. Hameed,

Cornelia Weltzien and Muhammad Yamin

Cornelia Weltzien and Muhammad Yamin

Additional information is available at the end of the chapter

Additional information is available at the end of the chapter

complementary sensitivity was analyzed.

Siva K. Balasundram, Desa Ahmad,

Siva K. Balasundram, Desa Ahmad,

http://dx.doi.org/10.5772/intechopen.80936

Abstract

**to Support Precision Agriculture in Oil Palm Plantations**

Unmanned aerial vehicles carrying multimodal sensors for precision agriculture (PA) applications face adaptation challenges to satisfy reliability, accuracy, and timeliness. Unlike ground platforms, UAV/drones are subjected to additional considerations such as payload, flight time, stabilization, autonomous missions, and external disturbances. For instance, in oil palm plantations (OPP), accruing high resolution images to generate multidimensional maps necessitates lower altitude mission flights with greater stability. This chapter addresses various UAV-based smart farming and PA solutions for OPP including health assessment and disease detection, pest monitoring, yield estimation, creation of virtual plantations, and dynamic Web-mapping. Stabilization of UAVs was discussed as one of the key factors for acquiring high quality aerial images. For this purpose, a case study was presented on stabilizing a fixed-wing Osprey drone crop surveillance that can be adapted as a remote sensing research platform. The objective was to design three controllers (including PID, LQR with full state feedback, and LQR plus observer) to improve the automatic flight mission. Dynamic equations were decoupled into lateral and longitudinal directions, where the longitudinal dynamics were modeled as a fourth order two-inputs-two-outputs system. State variables were defined as velocity, angle of attack, pitch rate, and pitch angle, all assumed to be available to the controller. A special case was considered in which only velocity and pitch rate were measurable. The control objective was to stabilize the system for a velocity step input of 10m/s. The performance of noise effects, model error, and

Keywords: unmanned aerial vehicle, drone, flight control, oil palm, precision agriculture

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited.

© 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use,

distribution, and reproduction in any medium, provided the original work is properly cited.

Fundamental Research on Unmanned Aerial Vehicles to

DOI: 10.5772/intechopen.80936


#### **Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations** Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations

DOI: 10.5772/intechopen.80936

Redmond Ramin Shamshiri, Ibrahim A. Hameed, Siva K. Balasundram, Desa Ahmad, Cornelia Weltzien and Muhammad Yamin Redmond Ramin Shamshiri, Ibrahim A. Hameed, Siva K. Balasundram, Desa Ahmad, Cornelia Weltzien and Muhammad Yamin

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.80936

#### Abstract

on Robotics and Associated High-Technologies and Equipment for Agriculture and Forestry;

[31] Tecnológica Actuadores Lineales LINAK Actuadores SLuSpain Port. 2018. http://www.

[32] EBM-PAPST, World market leader for energy saving fans and motors. 2018. http://www.

[33] Micropumps-High Performance Miniature Pumps. 2018. http://www.micropumps.co.uk/

[34] EOSDIS Nasa's Earth Observing System Data Information System [WWW Document]. n.d. EOSDIS NASAs Earth Obs. Syst. http://reverb.echo.nasa.gov [Accessed: July 18,

[35] Gonzalez-de-Soto M, Emmi L, Benavides C, Garcia I, Gonzalez-de-Santos P. Reducing air pollution with hybrid-powered robotic tractors for precision agriculture. Biosystems

[36] Photovoltaic Geographical Information System, JRCs Inst. Energy Transp.—PVGIS— Eur. Comm. 2018. http://re.jrc.ec.europa.eu/pvgis/ [Accessed: January 4, 2018]

[37] Labeckas G, Slavinskas S. Performance and emission characteristics of a direct injection diesel engine operating on KDV synthetic diesel fuel. Energy Conversion and

Management. 2013;**66**:173-188. DOI: 10.1016/j.enconman.2012.10.004

May 21-23; Madrid, Spain. 2014. pp. 73-81

ebmpapst.com/en/ [Accessed: January 4, 2018]

linak.es/ [Accessed: January 4, 2018]

[Accessed: January 4, 2018]

90 Agricultural Robots - Fundamentals and Applications

Engineering. 2016;**143**:79-94

2013]

Unmanned aerial vehicles carrying multimodal sensors for precision agriculture (PA) applications face adaptation challenges to satisfy reliability, accuracy, and timeliness. Unlike ground platforms, UAV/drones are subjected to additional considerations such as payload, flight time, stabilization, autonomous missions, and external disturbances. For instance, in oil palm plantations (OPP), accruing high resolution images to generate multidimensional maps necessitates lower altitude mission flights with greater stability. This chapter addresses various UAV-based smart farming and PA solutions for OPP including health assessment and disease detection, pest monitoring, yield estimation, creation of virtual plantations, and dynamic Web-mapping. Stabilization of UAVs was discussed as one of the key factors for acquiring high quality aerial images. For this purpose, a case study was presented on stabilizing a fixed-wing Osprey drone crop surveillance that can be adapted as a remote sensing research platform. The objective was to design three controllers (including PID, LQR with full state feedback, and LQR plus observer) to improve the automatic flight mission. Dynamic equations were decoupled into lateral and longitudinal directions, where the longitudinal dynamics were modeled as a fourth order two-inputs-two-outputs system. State variables were defined as velocity, angle of attack, pitch rate, and pitch angle, all assumed to be available to the controller. A special case was considered in which only velocity and pitch rate were measurable. The control objective was to stabilize the system for a velocity step input of 10m/s. The performance of noise effects, model error, and complementary sensitivity was analyzed.

Keywords: unmanned aerial vehicle, drone, flight control, oil palm, precision agriculture

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited. © 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### 1. Introduction

Malaysia is the world's second largest exporter of palm oil (Figure 1) with approximately 5.08 million ha of land under cultivation [1]. Major percentage of these plantations is owned by small-scale private farmers that have huge demands to affordable low-cost autonomous platforms for applications, such as scouting, palm census, yield monitoring, spraying, and most importantly health assessment and disease detection. The ability to collect high spatial resolution aerial images using drones is changing the way the oil palm growers are approaching the business [2]. Conventional methods of practicing precision agriculture (PA) in oil palm plantations such as remote sensing and spraying are being replaced by integrated fixed-wing or multirotor unmanned aerial vehicles (UAV) [3], allowing collection of information to be instantly accessible for immediate decisions. Precision farming for increasing oil palm yield requires optimization of returns on inputs while preserving resources based on sensing, measuring, and health assessment of the plantations [4]. Relying on satellites images of palms, there is a substantial lag in terms of accessing the data quickly enough. Professionals have been using satellite and piloted airplane remote sensing platforms [5] for plantation scouting applications, such as vegetation cover assessment [6], vegetation mapping [7], crop monitoring [8], and forest fire applications [9]; however, the difference that drone technology [10] and agricultural robotics [11–13] have made is around the speed and accuracy of delivering that information. Digital agriculture [4] offers great opportunities for mechanization and automation of farming tasks in oil palm plantations through automation of data collection by means of ground or aerial surveillance and data processing software to predict or estimate palms yields.

comprehensive information because several parameters are ignored due to measurements difficulties (i.e., tasks that involve climbing trees, measuring canopy diameter, etc.). Other than the inaccuracy and biases statistics, manual scouting involves additional costs for each extra observation, hazards, and safety issues (i.e., falling from trees, bugs, snake bites, etc.). Satellite imaging services are extremely costly, and they can take images only once a day and have to be ordered in advance. The resolution of these images is low and can be influenced greatly with certain sky cloud conditions. Ground sensing platforms are also time consuming and are limited to small fields of view. Yield reduction due to high-density palm areas that cause etiolation is an issue in plantation management. Palm densities are an important and limiting factor for growth, nutritional status, fruiting, and hence for the plantation yield. Optimal palm densities depend on different factors, such as cultivars, climate, soil characteristics, and land preparation. Refilling of palm gaps and correction of nonoptimal densities are of high priority for a good plantation management. Conventional methods that are solely based on visual observation are inaccurate, particularly when coverage is large and dominant topography

Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations

http://dx.doi.org/10.5772/intechopen.80936

93

Figure 2. Tedious field work with conventional scouting of oil palm plantation.

Precision agriculture of oil palm is one of the largest markets in Malaysia that will be hit by UAV and robotics. These devices are the future of PA and are sometimes referred to as the next step in data-driven agriculture. UAV/drones carrying multi-spectral and multimodal data acquisition devices face adaptation challenges to satisfy information, accuracy, and timeliness as the bases of a successful precision agriculture (PA) operation. These platforms have contributed to significant reductions of in-field walking costs and observational experiments. UAVs are defined as "an aircraft that is equipped with necessary data processing units, sensors, automatic control, and communications systems and is capable of performing autonomous flight missions without the interference of a human pilot" [14]. The global market for agricultural UAV drones is estimated to reach 3.7 billion US dollars by the year 2022 (Source: Radiant Insight Research firm). Aerial photography from UAS has bridged the gap (see the schematic diagram shown in Figure 3) between ground-based observations and remotely sensed imagery of conventional aircraft and satellite platforms and has made possible great improvements in crop scouting, yield mapping, field boundary mapping, soil sampling and soil property mapping, weeds and pest control and mapping, vehicle's guidance, navigation control, and spraying. These devices are easy to use and are typically flexible, low cost, light-weight, and low airspeed aircraft. They have revolutionized smart farming and precision agriculture, from planting to harvesting, from seeding to sensing, and from scouting to spaying. UAS drones are widely available on demand, and their functionalities can be customized for different farming applications and can provide a cost-effective monitoring platform without requiring an expert

is hillocky.

Conventional scouting of oil palms on a regular basis (Figure 2), as well as palm census and quantification of the amount of fresh fruit bunches (FFB) for yield monitoring, is a laborintensive task that is either ignored or carried out manually by the use of hand counters. Traditional scouting of palms is an ineffective practice that requires expert knowledge and postprocessing lab equipment. It involves spending hours and hours of human observation inside the unpleasant hot and humid plantation and does not provide accurate and

Figure 1. Comparison between world exports of palm oil, with Malaysia as the second largest exporter. (data: [1]).

Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations http://dx.doi.org/10.5772/intechopen.80936 93

Figure 2. Tedious field work with conventional scouting of oil palm plantation.

1. Introduction

92 Agricultural Robots - Fundamentals and Applications

Malaysia is the world's second largest exporter of palm oil (Figure 1) with approximately 5.08 million ha of land under cultivation [1]. Major percentage of these plantations is owned by small-scale private farmers that have huge demands to affordable low-cost autonomous platforms for applications, such as scouting, palm census, yield monitoring, spraying, and most importantly health assessment and disease detection. The ability to collect high spatial resolution aerial images using drones is changing the way the oil palm growers are approaching the business [2]. Conventional methods of practicing precision agriculture (PA) in oil palm plantations such as remote sensing and spraying are being replaced by integrated fixed-wing or multirotor unmanned aerial vehicles (UAV) [3], allowing collection of information to be instantly accessible for immediate decisions. Precision farming for increasing oil palm yield requires optimization of returns on inputs while preserving resources based on sensing, measuring, and health assessment of the plantations [4]. Relying on satellites images of palms, there is a substantial lag in terms of accessing the data quickly enough. Professionals have been using satellite and piloted airplane remote sensing platforms [5] for plantation scouting applications, such as vegetation cover assessment [6], vegetation mapping [7], crop monitoring [8], and forest fire applications [9]; however, the difference that drone technology [10] and agricultural robotics [11–13] have made is around the speed and accuracy of delivering that information. Digital agriculture [4] offers great opportunities for mechanization and automation of farming tasks in oil palm plantations through automation of data collection by means of ground or aerial surveillance and data

Conventional scouting of oil palms on a regular basis (Figure 2), as well as palm census and quantification of the amount of fresh fruit bunches (FFB) for yield monitoring, is a laborintensive task that is either ignored or carried out manually by the use of hand counters. Traditional scouting of palms is an ineffective practice that requires expert knowledge and postprocessing lab equipment. It involves spending hours and hours of human observation inside the unpleasant hot and humid plantation and does not provide accurate and

Figure 1. Comparison between world exports of palm oil, with Malaysia as the second largest exporter. (data: [1]).

processing software to predict or estimate palms yields.

comprehensive information because several parameters are ignored due to measurements difficulties (i.e., tasks that involve climbing trees, measuring canopy diameter, etc.). Other than the inaccuracy and biases statistics, manual scouting involves additional costs for each extra observation, hazards, and safety issues (i.e., falling from trees, bugs, snake bites, etc.). Satellite imaging services are extremely costly, and they can take images only once a day and have to be ordered in advance. The resolution of these images is low and can be influenced greatly with certain sky cloud conditions. Ground sensing platforms are also time consuming and are limited to small fields of view. Yield reduction due to high-density palm areas that cause etiolation is an issue in plantation management. Palm densities are an important and limiting factor for growth, nutritional status, fruiting, and hence for the plantation yield. Optimal palm densities depend on different factors, such as cultivars, climate, soil characteristics, and land preparation. Refilling of palm gaps and correction of nonoptimal densities are of high priority for a good plantation management. Conventional methods that are solely based on visual observation are inaccurate, particularly when coverage is large and dominant topography is hillocky.

Precision agriculture of oil palm is one of the largest markets in Malaysia that will be hit by UAV and robotics. These devices are the future of PA and are sometimes referred to as the next step in data-driven agriculture. UAV/drones carrying multi-spectral and multimodal data acquisition devices face adaptation challenges to satisfy information, accuracy, and timeliness as the bases of a successful precision agriculture (PA) operation. These platforms have contributed to significant reductions of in-field walking costs and observational experiments. UAVs are defined as "an aircraft that is equipped with necessary data processing units, sensors, automatic control, and communications systems and is capable of performing autonomous flight missions without the interference of a human pilot" [14]. The global market for agricultural UAV drones is estimated to reach 3.7 billion US dollars by the year 2022 (Source: Radiant Insight Research firm). Aerial photography from UAS has bridged the gap (see the schematic diagram shown in Figure 3) between ground-based observations and remotely sensed imagery of conventional aircraft and satellite platforms and has made possible great improvements in crop scouting, yield mapping, field boundary mapping, soil sampling and soil property mapping, weeds and pest control and mapping, vehicle's guidance, navigation control, and spraying. These devices are easy to use and are typically flexible, low cost, light-weight, and low airspeed aircraft. They have revolutionized smart farming and precision agriculture, from planting to harvesting, from seeding to sensing, and from scouting to spaying. UAS drones are widely available on demand, and their functionalities can be customized for different farming applications and can provide a cost-effective monitoring platform without requiring an expert

plantation. Other than the selection of proper sensors, the stability and accuracy are vital to provide geo-referenced images for extraction of useful information. Adaptation of UAV technology for oil palm plantations involves integration of vision sensors, machine vision algorithms, and control system for (i) yield monitoring and yield mapping, (ii) automated airborne pest monitoring using thermal cameras, (iii) identification and counting of specific insects from very high-resolution optical images, (iv) development of decision support system (DSS) using geo-referenced images as a basis for a GIS-based system giving oil palm growers the possibility to incorporate data directly to their precision farming platforms, (v) identification and mapping of Ganoderma disease using hyperspectral camera, (vi) automated retrieving of oil palm canopy chlorophyll and nutrient content from multispectral and hyperspectral UAV acquired images, and (vii) dynamic Web mapping and inventory management of oil palm productivity using in situ sensors. This paper is the first of series reporting on design and development of an affordable fixed-wing UAV to be used as a flexible scouting test bed for oil palm plantations. Schematic diagram illustrating the early stages of technological development for introducing a UAV platform to local farmers and the general steps and procedure involved with setting up a UAV remote sensing platform for

Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations

http://dx.doi.org/10.5772/intechopen.80936

95

Figure 4. Schematic diagram illustrating the early stages of technological development for introducing a UAV platform

Figure 5. General steps and procedure involved with setting up a UAV remote sensing platform for agricultural applications.

agricultural applications are shown in Figures 4 and 5, respectively.

to local farmers (source: Adaptive AgroTech Consultancy International).

Figure 3. Typical components of a UAV-based remote sensing platform for precision agriculture of oil palm.

operator. With this technology, several problems associated with the data resolution from piloted aircraft and satellite imaging have been solved. They are capable of providing live data from a wide range of sensors, such as those shown in Figure 3 (multispectral, NIR, LiDAR, etc.) at precision resolutions measured as centimeters per pixel. Such information contributes to the in-depth analysis for the crop health assessment or the inventory management databases. With the UAV technology, the following can be achieved: information about accurate planted area for replanting or thinning, palm census for creating inventory database, calculating the total land area in use, finding distances between each palm to specific spots, calculating canopy diameter, palm height, and palm density, creating 2D, 3D, GIS, NDVI maps for plantation, identifying palm status based on Orthomosaics and digital elevation models, detecting healthy and unhealthy palms (stress assessment), monitoring exposed soil for variable rate technology application, quantification of fresh fruit bunches and mature fruits for yield calculation, monitoring chlorophyll content and nutrient estimation, and measuring leaf area index, drought assessment, biomass indication, weed detection, and inventory management. Data and information such as these are useful for developing decision support systems and yield prediction models.

#### 2. Adaptation of UAV for oil palm remote sensing

UAV drones can be well adapted for oil palm plantations, where field work is tedious. They allow observation of individual palm trees and can operate unnoticed and below cloud cover that prevents larger high-altitude aircraft and satellites from performing the same mission. Moreover, they can be deployed quickly and repeatedly, and they are less costly and safer than piloted aircraft, are flexible in terms of flying height and timing of missions, and can obtain very high-resolution imagery. As an aerial remote sensing platform, a UAV drone must be adapted to satisfy the basic requirements of image data collection from oil palm plantation. Other than the selection of proper sensors, the stability and accuracy are vital to provide geo-referenced images for extraction of useful information. Adaptation of UAV technology for oil palm plantations involves integration of vision sensors, machine vision algorithms, and control system for (i) yield monitoring and yield mapping, (ii) automated airborne pest monitoring using thermal cameras, (iii) identification and counting of specific insects from very high-resolution optical images, (iv) development of decision support system (DSS) using geo-referenced images as a basis for a GIS-based system giving oil palm growers the possibility to incorporate data directly to their precision farming platforms, (v) identification and mapping of Ganoderma disease using hyperspectral camera, (vi) automated retrieving of oil palm canopy chlorophyll and nutrient content from multispectral and hyperspectral UAV acquired images, and (vii) dynamic Web mapping and inventory management of oil palm productivity using in situ sensors. This paper is the first of series reporting on design and development of an affordable fixed-wing UAV to be used as a flexible scouting test bed for oil palm plantations. Schematic diagram illustrating the early stages of technological development for introducing a UAV platform to local farmers and the general steps and procedure involved with setting up a UAV remote sensing platform for agricultural applications are shown in Figures 4 and 5, respectively.

operator. With this technology, several problems associated with the data resolution from piloted aircraft and satellite imaging have been solved. They are capable of providing live data from a wide range of sensors, such as those shown in Figure 3 (multispectral, NIR, LiDAR, etc.) at precision resolutions measured as centimeters per pixel. Such information contributes to the in-depth analysis for the crop health assessment or the inventory management databases. With the UAV technology, the following can be achieved: information about accurate planted area for replanting or thinning, palm census for creating inventory database, calculating the total land area in use, finding distances between each palm to specific spots, calculating canopy diameter, palm height, and palm density, creating 2D, 3D, GIS, NDVI maps for plantation, identifying palm status based on Orthomosaics and digital elevation models, detecting healthy and unhealthy palms (stress assessment), monitoring exposed soil for variable rate technology application, quantification of fresh fruit bunches and mature fruits for yield calculation, monitoring chlorophyll content and nutrient estimation, and measuring leaf area index, drought assessment, biomass indication, weed detection, and inventory management. Data and information such as these are useful for developing decision support systems

Figure 3. Typical components of a UAV-based remote sensing platform for precision agriculture of oil palm.

UAV drones can be well adapted for oil palm plantations, where field work is tedious. They allow observation of individual palm trees and can operate unnoticed and below cloud cover that prevents larger high-altitude aircraft and satellites from performing the same mission. Moreover, they can be deployed quickly and repeatedly, and they are less costly and safer than piloted aircraft, are flexible in terms of flying height and timing of missions, and can obtain very high-resolution imagery. As an aerial remote sensing platform, a UAV drone must be adapted to satisfy the basic requirements of image data collection from oil palm

and yield prediction models.

94 Agricultural Robots - Fundamentals and Applications

2. Adaptation of UAV for oil palm remote sensing

Figure 4. Schematic diagram illustrating the early stages of technological development for introducing a UAV platform to local farmers (source: Adaptive AgroTech Consultancy International).

Figure 5. General steps and procedure involved with setting up a UAV remote sensing platform for agricultural applications.

#### 2.1. Recommendations for purchasing UAV for agricultural application

index describing the relative density and health of the palms, and thermal camera can show the heat signature of different spots in the plantations. A conceptual demonstration of a UAV remote sensing platform equipped with NDVI sensor for oil palm health assessment is shown

Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations

http://dx.doi.org/10.5772/intechopen.80936

97

The platform shown in Figure 7 can be customized and integrated with hyperspectral camera as shown in Figure 8, for the detection of Ganoderma boninense, which is a serious threat to oil palm plantations in Malaysia and has caused great losses to healthy palms. This disease causes both basal stem rot and upper stem rot and remains South East Asia's most devastating oil palm diseases, with direct loss of the stand, reduced yield of diseased palms, and the resultant requirement for earlier replanting. Using naked eye, the Ganoderma

Figure 6. Conceptual demonstration of a UAV-remote sensing platform for oil palm health assessment with NDVI camera.

Figure 7. Feasibility of using autonomous UAV-based hyperspectral imaging for detection of Ganoderma boninense disease

in Figure 6.

in oil palms.

A comprehensive document including recommendation for choosing the best UAV drone for precision agricultural and smart farming applications is available in [15]. Specifications of sample multirotor and fixed-wing UAV recommended for precision agriculture of oil palm are also provided in the Appendix. Compared with piloted airplanes and satellite imaging, the ability of UAVs in collecting higher resolution aerial images at a significantly lower cost can provide oil palm growers with more accurate information on palm height, crown size, and normalized difference vegetation index (NDVI), enabling practicing of data-driven techniques for early and accurate yield estimation and health assessment. While a typical UAV may cost as little as USD1000, it can be integrated with custom instrumentations, controllers, sensors, and software to operate as a flexible remote sensing or variable rate technology platform to contribute to plantation management, growth, and soil condition assessment mapping application (i.e., 2D, 3D, GIS, NDVI), risk/hazard/safety management, spraying application, and academic and research application. In specific, UAV remote sensing in oil palm precision agriculture can contribute to automatic palm detection and counting, automatic measurements of palm height and crown diameter measurements, calculation of planted and unplanted areas for replanting or thinning, analyzing palm status based on Orthomosaics and digital elevation models, inventory management and health assessment based on physical appearances and vegetation indices, model-based yield prediction, yield monitoring and mapping, rapid estimation of nutrient contents, and disease detection. It should be noted that agricultural UAV activity is considered commercial operation with a high-tech platform for data acquisition or spraying applications that should be carried out by licensed professionals or certified pilots. Price range for a complete package is between USD1500 to over USD25000 depending on the application. Multicopter drones can fly for 3–45 minutes on a one battery charge and are more suitable for regular use in small-scale plantation without the requirement to special takeoff and landing areas. Fixed-wing UAVs need to be planned for mission flights and reliable landing for use in larger plantations. It is better to purchase drones that can be controlled via mobile or tablets or are fully autonomous from takeoff to landing (i.e., the entire mission can be performed by a single start button). For a multicopter, it is also important to check for the live standstill view feed. This feature allows plantation managers to find specific spots and issues for closer inspection. One of the key considerations in purchasing scouting UAV is the NDVI and NIR camera options. For the sake of cost saving, an affordable regular 3D camera with two lenses can be purchased for less than USD300 and modified slightly with a blue plastic filter to produce NIR images. However, a more expensive UAV that can collect data faster will compensate the extra costs in a long run.

#### 2.2. Oil palm health assessment and disease detection

Health assessment in oil palm plantations is crucial for spotting fungal infection and bacterial disease on the palms. By aerial scanning the plantation using visible RGB camera, NIR, hyperspectral, and multispectral sensors, it is possible to identify temporal and spatial reflectance variations before they can be detected by naked eyes and associate these changes with palms heaths for an early response. For instance, NDVI cameras can calculate the vegetation index describing the relative density and health of the palms, and thermal camera can show the heat signature of different spots in the plantations. A conceptual demonstration of a UAV remote sensing platform equipped with NDVI sensor for oil palm health assessment is shown in Figure 6.

2.1. Recommendations for purchasing UAV for agricultural application

96 Agricultural Robots - Fundamentals and Applications

pensate the extra costs in a long run.

2.2. Oil palm health assessment and disease detection

Health assessment in oil palm plantations is crucial for spotting fungal infection and bacterial disease on the palms. By aerial scanning the plantation using visible RGB camera, NIR, hyperspectral, and multispectral sensors, it is possible to identify temporal and spatial reflectance variations before they can be detected by naked eyes and associate these changes with palms heaths for an early response. For instance, NDVI cameras can calculate the vegetation

A comprehensive document including recommendation for choosing the best UAV drone for precision agricultural and smart farming applications is available in [15]. Specifications of sample multirotor and fixed-wing UAV recommended for precision agriculture of oil palm are also provided in the Appendix. Compared with piloted airplanes and satellite imaging, the ability of UAVs in collecting higher resolution aerial images at a significantly lower cost can provide oil palm growers with more accurate information on palm height, crown size, and normalized difference vegetation index (NDVI), enabling practicing of data-driven techniques for early and accurate yield estimation and health assessment. While a typical UAV may cost as little as USD1000, it can be integrated with custom instrumentations, controllers, sensors, and software to operate as a flexible remote sensing or variable rate technology platform to contribute to plantation management, growth, and soil condition assessment mapping application (i.e., 2D, 3D, GIS, NDVI), risk/hazard/safety management, spraying application, and academic and research application. In specific, UAV remote sensing in oil palm precision agriculture can contribute to automatic palm detection and counting, automatic measurements of palm height and crown diameter measurements, calculation of planted and unplanted areas for replanting or thinning, analyzing palm status based on Orthomosaics and digital elevation models, inventory management and health assessment based on physical appearances and vegetation indices, model-based yield prediction, yield monitoring and mapping, rapid estimation of nutrient contents, and disease detection. It should be noted that agricultural UAV activity is considered commercial operation with a high-tech platform for data acquisition or spraying applications that should be carried out by licensed professionals or certified pilots. Price range for a complete package is between USD1500 to over USD25000 depending on the application. Multicopter drones can fly for 3–45 minutes on a one battery charge and are more suitable for regular use in small-scale plantation without the requirement to special takeoff and landing areas. Fixed-wing UAVs need to be planned for mission flights and reliable landing for use in larger plantations. It is better to purchase drones that can be controlled via mobile or tablets or are fully autonomous from takeoff to landing (i.e., the entire mission can be performed by a single start button). For a multicopter, it is also important to check for the live standstill view feed. This feature allows plantation managers to find specific spots and issues for closer inspection. One of the key considerations in purchasing scouting UAV is the NDVI and NIR camera options. For the sake of cost saving, an affordable regular 3D camera with two lenses can be purchased for less than USD300 and modified slightly with a blue plastic filter to produce NIR images. However, a more expensive UAV that can collect data faster will com-

The platform shown in Figure 7 can be customized and integrated with hyperspectral camera as shown in Figure 8, for the detection of Ganoderma boninense, which is a serious threat to oil palm plantations in Malaysia and has caused great losses to healthy palms. This disease causes both basal stem rot and upper stem rot and remains South East Asia's most devastating oil palm diseases, with direct loss of the stand, reduced yield of diseased palms, and the resultant requirement for earlier replanting. Using naked eye, the Ganoderma

Figure 6. Conceptual demonstration of a UAV-remote sensing platform for oil palm health assessment with NDVI camera.

Figure 7. Feasibility of using autonomous UAV-based hyperspectral imaging for detection of Ganoderma boninense disease in oil palms.

possibility of using a low-cost spectral radiometer for fast screening of Ganoderma-infected palms, (iv) developing an instrumented platform for collecting and geo-referencing hyperspectral images in the plantations, and (v) conducting a field trial to evaluate the effectiveness of hyperspectral imagery for detecting the disease in the plantations. Reflectance spectra of vegetation, measured in the visible and infrared region, contain information on plant pigment concentration, leaf cellular structure, and leaf moisture content. In this research, we propose to study the capability of hyperspectral imaging and spectroscopy in the range of 300-2500 nm for early detection of anomalies in oil palm trees as a result of Ganoderma infection. Preliminarily hyperspectral imaging data indicated that Ganodermainfected leaves have different spectral characteristics compared to healthy leaves. A quick and efficient method of detecting and mapping Ganoderma at the field level will assist growers to better manage and control this disease and can financially benefit growers. In the first year of the study, we will study the spectral characteristics of Ganoderma-infected oil palm leaves in laboratory conditions and compare them with other nutrient deficiency symptoms. Accordingly, we will develop a classification method to identify the symptoms of Ganoderma and separate it from plant stresses and other diseases with similar symptoms. Also, in the first year, we will study how well a low-cost spectral radiometer can detect Ganoderma symptoms. Based on the results from the first year of the study, we will develop an instrumented platform for collecting and geo-referencing hyperspectral images and evaluate the effectiveness of hyperspectral imagery for detecting suspicious Ganoderma-infected

Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations

http://dx.doi.org/10.5772/intechopen.80936

99

Oil Palm growers lose some portion of their yields to insects and pests infestation. Traditional methods of locating pests in thousands of hectare plantations are not effective. For example, early detection of an invasive pest like rats in palm plantations with labor requires a great amount of time and luck. Obviously, conventional methods are not accurate, and plantation managers have to make an educated guess before sending the crew to a large field to check for infested spots. For the purpose of pest monitoring, a solution is to have a UAV imagery platform equipped with a thermal camera and high-resolution RGB vision sensors for accurate identification of the spots in the oil palm plantations fields that are diagnosed with specific insects and pests. This approach may also involve development of a decision support system (DSS) using georeferenced insect count as a basis for a GIS-based system, giving plantation managers the possibility to incorporate data directly to their precision farming platforms. Specific steps involve (i) platform setup, that is integration of the UAV, vision sensor, and control system, (ii) perception which refers to the development of a real-time machine vision algorithm for pest monitoring (to refine the aerial images captured by the UAV in order to provide plantation managers with the most usable data), and (iii) action stage, which is the development of the DSS for creation of the prescription map. When pests are spotted, spraying UAV can be used for dropping a targeted load of pesticide. The spraying UAV can be equipped with distance-measuring and light detection sensors such as lasers, ultrasonic echoing, or LiDAR methods to scan the ground and adjust the flight altitude with the varying topography of the plantation and therefore apply the correct amount of spraying liquids for

palm trees in the grove.

2.3. Pest monitoring

Figure 8. Thermal camera and night vision (top row figures) and high-resolution RGB images approach (bottom row figures) for UAV based pest monitoring in oil palm plantations.

disease can only be recognized at a very late stage with serious symptoms of foliar chlorosis and breakage at older fronds, the presence of decayed tissues at palm base, and production of fruiting bodies. When symptoms of the disease appear on young palms, it is too late and younger palms die within 6 to 24 months, whereas mature palms may survive for 3 years. Reports also indicate that the basal stem rot can kill up to 80% of the total standing palms. Despite the several efforts in controlling this disease, the available methods are slow, and current strategies are still immature. To our knowledge, no effective method or a robust sensing instrumentation has been commercialized for early detection of this disease at an early stage. Research reports have highlighted that oil palm yields are highly correlated with most of the nutrients. There are extensive publications on the hyperspectral analysis of images with application in agriculture that shows promising methods to be adapted for early detection of Ganoderma disease in oil palm. In order to adapt a UAV remote sensing platform for this purpose, several questions should be addressed as follow: (i) at what stages of infection can the hyperspectral imaging detect the Ganoderma disease symptoms? (ii) what are the unique spectral characteristics of Ganoderma spectral reflectance data? (iii) what statistical or mathematical methods are the best for analyzing the Ganoderma spectral data? and (iv) how well can a low-cost multiband radiometer assist a scouting crew to detect the suspicious HLB-infected trees? We can begin with a hypothesis that wavelet analysis of reflectance data can improve detection of nutrient concentration in oil palm. This hypothesis can be studied by the use of the Matlab Wavelet CIR images Toolbox. Preliminary studies have demonstrated the potential of wavelet analysis for retrieving foliar nitrogen content and photosynthetic pigment concentrations from leaf and canopy reflectance spectra, but further research is needed to develop the approach. Our research will contribute to saving of more palm trees and consequently a higher yield which has a significant impact on large scale plantations and the economy of Malaysia. A project can be proposed with the longterm goal of developing a fast UAV-based screening technique that can assist oil palm growers in detecting suspicious Ganoderma-infected palms. Such a project may involve the following systematic steps and methodology: (i) study the spectral characteristics of GB in lab conditions, (ii) developing a classification method to identify the disease and separate it from other palm stresses and other diseases with similar symptoms, (iii) evaluating the

possibility of using a low-cost spectral radiometer for fast screening of Ganoderma-infected palms, (iv) developing an instrumented platform for collecting and geo-referencing hyperspectral images in the plantations, and (v) conducting a field trial to evaluate the effectiveness of hyperspectral imagery for detecting the disease in the plantations. Reflectance spectra of vegetation, measured in the visible and infrared region, contain information on plant pigment concentration, leaf cellular structure, and leaf moisture content. In this research, we propose to study the capability of hyperspectral imaging and spectroscopy in the range of 300-2500 nm for early detection of anomalies in oil palm trees as a result of Ganoderma infection. Preliminarily hyperspectral imaging data indicated that Ganodermainfected leaves have different spectral characteristics compared to healthy leaves. A quick and efficient method of detecting and mapping Ganoderma at the field level will assist growers to better manage and control this disease and can financially benefit growers. In the first year of the study, we will study the spectral characteristics of Ganoderma-infected oil palm leaves in laboratory conditions and compare them with other nutrient deficiency symptoms. Accordingly, we will develop a classification method to identify the symptoms of Ganoderma and separate it from plant stresses and other diseases with similar symptoms. Also, in the first year, we will study how well a low-cost spectral radiometer can detect Ganoderma symptoms. Based on the results from the first year of the study, we will develop an instrumented platform for collecting and geo-referencing hyperspectral images and evaluate the effectiveness of hyperspectral imagery for detecting suspicious Ganoderma-infected palm trees in the grove.

#### 2.3. Pest monitoring

disease can only be recognized at a very late stage with serious symptoms of foliar chlorosis and breakage at older fronds, the presence of decayed tissues at palm base, and production of fruiting bodies. When symptoms of the disease appear on young palms, it is too late and younger palms die within 6 to 24 months, whereas mature palms may survive for 3 years. Reports also indicate that the basal stem rot can kill up to 80% of the total standing palms. Despite the several efforts in controlling this disease, the available methods are slow, and current strategies are still immature. To our knowledge, no effective method or a robust sensing instrumentation has been commercialized for early detection of this disease at an early stage. Research reports have highlighted that oil palm yields are highly correlated with most of the nutrients. There are extensive publications on the hyperspectral analysis of images with application in agriculture that shows promising methods to be adapted for early detection of Ganoderma disease in oil palm. In order to adapt a UAV remote sensing platform for this purpose, several questions should be addressed as follow: (i) at what stages of infection can the hyperspectral imaging detect the Ganoderma disease symptoms? (ii) what are the unique spectral characteristics of Ganoderma spectral reflectance data? (iii) what statistical or mathematical methods are the best for analyzing the Ganoderma spectral data? and (iv) how well can a low-cost multiband radiometer assist a scouting crew to detect the suspicious HLB-infected trees? We can begin with a hypothesis that wavelet analysis of reflectance data can improve detection of nutrient concentration in oil palm. This hypothesis can be studied by the use of the Matlab Wavelet CIR images Toolbox. Preliminary studies have demonstrated the potential of wavelet analysis for retrieving foliar nitrogen content and photosynthetic pigment concentrations from leaf and canopy reflectance spectra, but further research is needed to develop the approach. Our research will contribute to saving of more palm trees and consequently a higher yield which has a significant impact on large scale plantations and the economy of Malaysia. A project can be proposed with the longterm goal of developing a fast UAV-based screening technique that can assist oil palm growers in detecting suspicious Ganoderma-infected palms. Such a project may involve the following systematic steps and methodology: (i) study the spectral characteristics of GB in lab conditions, (ii) developing a classification method to identify the disease and separate it from other palm stresses and other diseases with similar symptoms, (iii) evaluating the

Figure 8. Thermal camera and night vision (top row figures) and high-resolution RGB images approach (bottom row

figures) for UAV based pest monitoring in oil palm plantations.

98 Agricultural Robots - Fundamentals and Applications

Oil Palm growers lose some portion of their yields to insects and pests infestation. Traditional methods of locating pests in thousands of hectare plantations are not effective. For example, early detection of an invasive pest like rats in palm plantations with labor requires a great amount of time and luck. Obviously, conventional methods are not accurate, and plantation managers have to make an educated guess before sending the crew to a large field to check for infested spots. For the purpose of pest monitoring, a solution is to have a UAV imagery platform equipped with a thermal camera and high-resolution RGB vision sensors for accurate identification of the spots in the oil palm plantations fields that are diagnosed with specific insects and pests. This approach may also involve development of a decision support system (DSS) using georeferenced insect count as a basis for a GIS-based system, giving plantation managers the possibility to incorporate data directly to their precision farming platforms. Specific steps involve (i) platform setup, that is integration of the UAV, vision sensor, and control system, (ii) perception which refers to the development of a real-time machine vision algorithm for pest monitoring (to refine the aerial images captured by the UAV in order to provide plantation managers with the most usable data), and (iii) action stage, which is the development of the DSS for creation of the prescription map. When pests are spotted, spraying UAV can be used for dropping a targeted load of pesticide. The spraying UAV can be equipped with distance-measuring and light detection sensors such as lasers, ultrasonic echoing, or LiDAR methods to scan the ground and adjust the flight altitude with the varying topography of the plantation and therefore apply the correct amount of spraying liquids for even coverage and avoid collisions. This practice will result in an increased efficiency while reducing the amount of penetrating spray chemical in the soil and groundwater. It is estimated that UAV spraying is five times faster than conventional tractor and machinery equipment.

amount of FFB on the palms. Such technology is highly demanded by oil palm growers as a fast, accurate, and reliable tool for estimating palm numbers and FFB in large-scale plantations. In determining instantaneous oil palm yield, two parameters must be known, weight and coordination of FFB on each palm. The weight of the FFB can be estimated using a machine vision algorithm that quantifies the number of fruits on each palm (Figure 5). These estimated weights are then georeferenced with coordinates of the corresponding palm using computer programs for the creation of database and yield map. Collected data will be processed by custom-built GIS software for creation of yield map and inventory database. A conceptual illustration of integrated fixed-wing UAV-based inventory management and health assessment system with mobile application and cloud computing is shown in Figure 9.

Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations

http://dx.doi.org/10.5772/intechopen.80936

101

One of the limitations of doing research on oil palm plantation is the lack of accurate data and input variables for modeling and simulation purposes. UAV technology can be integrated with

Figure 9. Feasibility of UAV imaging system for yield monitoring of oil palm (top) and a proposed methodology for

UAV-based yield monitoring of apple and orange fruits using deep learning algorithms [22].

2.5. Virtual plantations and dynamic Web mapping

The FLIR Vue Pro thermal camera shown in Figure 8 is designed for small UAVs and can be used for agricultural applications. It has different lens options for different type of view and specific applications. The thermal sensor resolution of this camera is 640 by 512 pixels and records 30 frames per second for smooth video. The light weight and small size of this camera will not affect the UAV center of gravity during the flight or sacrifice the flight time. It comes with the mounting accessories that can be used with most UAV platforms. It can also be used with transmitters for live feeds. The FLIR Vue Pro thermal camera does not have a separate battery and can be charged through a 6 V power from the UAV. Image data are stored on a standard micro SD card. An application connects the camera with the computer via Bluetooth. The thermal imager Optris PI 640 shown in the figure is the smallest measuring VGA infrared camera available. With an optical resolution of 640 480 pixels, the PI 640 delivers pin-sharp radiometric pictures and videos in real time. With a body sized 45 56 90 mm and weighing only 320 grams (lens included), the optris PI 640 counts among the most compact thermal imaging cameras on the market. Temperature range is between 20 and 900C (optional up to 1500C), spectral range is between 7.5 and 13 μm, and frame rate up is to 125 Hz. For the purpose of validation, images taken at varying heights and resolutions will be compared with the ground truth pictures taken on the ground with a mobile device. The research findings may lead to new pest management strategies that use UAV and other imaging technologies for detecting invasive pests in other farm fields, e.g., oil palm plantations. The thermal camera can also be used for spotting the areas that are drier and require attention.

#### 2.4. Yield monitoring

Quantification of FFB from UAV stream images for yield map creation is the first step toward practicing PA in oil palm plantations. With the available high-tech imaging sensors and using real-time image processing and remote sensing techniques (i.e., pixel-based or object-based [16], template matching [17–19] image analysis, learning algorithms methods for classification [20, 21] and for extracting useful information from an image), it is possible to measure oil palm yield on much smaller scales. One of the benefits of using autonomous UAV is their affordable price and lower cost per each mission flight that make them suitable for academic research in yield monitoring applications. The idea is to evaluate the feasibility of having UAV agent robots that can fly over and inside oil palm plantations and collect high-resolution detailed photos from different angles for automated creation of yield maps. These maps can tell growers where and when to apply the optimal amount of inputs (i.e., fertilizer, pesticide, water) for creating further sustainability. Of course, mobile robots with camera and sensors mounted on top of them can also be used for such application; however as mentioned earlier, we are proposing a research idea that involves a swarm or fleet of small-scale UAVs similar to what is shown in the figure that simultaneously fly inside the plantation for image data collection. By using different sensor-based measurement and imaging techniques on each UAV, a real-time machine-vision system can be developed for accurate identification of the amount of FFB on the palms. Such technology is highly demanded by oil palm growers as a fast, accurate, and reliable tool for estimating palm numbers and FFB in large-scale plantations. In determining instantaneous oil palm yield, two parameters must be known, weight and coordination of FFB on each palm. The weight of the FFB can be estimated using a machine vision algorithm that quantifies the number of fruits on each palm (Figure 5). These estimated weights are then georeferenced with coordinates of the corresponding palm using computer programs for the creation of database and yield map. Collected data will be processed by custom-built GIS software for creation of yield map and inventory database. A conceptual illustration of integrated fixed-wing UAV-based inventory management and health assessment system with mobile application and cloud computing is shown in Figure 9.

#### 2.5. Virtual plantations and dynamic Web mapping

even coverage and avoid collisions. This practice will result in an increased efficiency while reducing the amount of penetrating spray chemical in the soil and groundwater. It is estimated that UAV spraying is five times faster than conventional tractor and machinery equipment.

The FLIR Vue Pro thermal camera shown in Figure 8 is designed for small UAVs and can be used for agricultural applications. It has different lens options for different type of view and specific applications. The thermal sensor resolution of this camera is 640 by 512 pixels and records 30 frames per second for smooth video. The light weight and small size of this camera will not affect the UAV center of gravity during the flight or sacrifice the flight time. It comes with the mounting accessories that can be used with most UAV platforms. It can also be used with transmitters for live feeds. The FLIR Vue Pro thermal camera does not have a separate battery and can be charged through a 6 V power from the UAV. Image data are stored on a standard micro SD card. An application connects the camera with the computer via Bluetooth. The thermal imager Optris PI 640 shown in the figure is the smallest measuring VGA infrared camera available. With an optical resolution of 640 480 pixels, the PI 640 delivers pin-sharp radiometric pictures and videos in real time. With a body sized 45 56 90 mm and weighing only 320 grams (lens included), the optris PI 640 counts among the most compact thermal imaging cameras on the market. Temperature range is between 20 and 900C (optional up to 1500C), spectral range is between 7.5 and 13 μm, and frame rate up is to 125 Hz. For the purpose of validation, images taken at varying heights and resolutions will be compared with the ground truth pictures taken on the ground with a mobile device. The research findings may lead to new pest management strategies that use UAV and other imaging technologies for detecting invasive pests in other farm fields, e.g., oil palm plantations. The thermal camera can

Quantification of FFB from UAV stream images for yield map creation is the first step toward practicing PA in oil palm plantations. With the available high-tech imaging sensors and using real-time image processing and remote sensing techniques (i.e., pixel-based or object-based [16], template matching [17–19] image analysis, learning algorithms methods for classification [20, 21] and for extracting useful information from an image), it is possible to measure oil palm yield on much smaller scales. One of the benefits of using autonomous UAV is their affordable price and lower cost per each mission flight that make them suitable for academic research in yield monitoring applications. The idea is to evaluate the feasibility of having UAV agent robots that can fly over and inside oil palm plantations and collect high-resolution detailed photos from different angles for automated creation of yield maps. These maps can tell growers where and when to apply the optimal amount of inputs (i.e., fertilizer, pesticide, water) for creating further sustainability. Of course, mobile robots with camera and sensors mounted on top of them can also be used for such application; however as mentioned earlier, we are proposing a research idea that involves a swarm or fleet of small-scale UAVs similar to what is shown in the figure that simultaneously fly inside the plantation for image data collection. By using different sensor-based measurement and imaging techniques on each UAV, a real-time machine-vision system can be developed for accurate identification of the

also be used for spotting the areas that are drier and require attention.

2.4. Yield monitoring

100 Agricultural Robots - Fundamentals and Applications

One of the limitations of doing research on oil palm plantation is the lack of accurate data and input variables for modeling and simulation purposes. UAV technology can be integrated with

Figure 9. Feasibility of UAV imaging system for yield monitoring of oil palm (top) and a proposed methodology for UAV-based yield monitoring of apple and orange fruits using deep learning algorithms [22].

image acquisition techniques for three-dimensional reconstruction of the environment and creation of virtual plantations. Examples of 3D reconstructed plantation are shown in Figure 10. The information extracted from these 3D models can lead to the development of dynamic Web inventory management and mapping system. A 3D reconstruction model of oil palm plantation can be created by using range data methods or depth map using laser range finder sensors and 3D scanner instrumentations. This approach is however costly and not affordable by local oil palm farmers. Alternatively, passive methods, also called image-based reconstruction methods (i.e., photogrammetry technique), have been introduced using a normal camera and image sensors, which do not interfere with the reconstructed object. In this method, a UAV equipped with a normal RGB camera will collect images of the oil palm plantations from different views and angles. Computer software will then process these images to create a 3D model, and filter specific wavelength to generate images that corresponds to vegetation index and palm health. For example, a red edge image can describe nitrogen content and water stress. The potential of UAV image data to simulate the physical process of palm photosynthesis as a result of different crown sizes and densities intercepting different amounts of light radiation can be evaluated using virtual plantations. A virtual plantation can be used to estimate palm height, crown size, and inventory database (Figure 11) for generating dynamic Web maps and yield prediction models. These maps can identify how different palm height, crown sizes, plantation densities, and row orientations in different locations can affect the water and fertilizer demand. Moreover, mathematical models can be established based on the validated information from virtual plantations for estimating nitrogen demand and fertilizer application. These maps also provide precision rich data for academic and educational purposes. Researchers can access to detailed measurements of palm trunk and crown size and the spacing between different palms, leaf area index, and crown density as a preliminary study for the possibility of autonomous variable rate applications and robotic harvesting.

For the purpose of a sensor Web-based approach for dynamic Web mapping, observations from a UAV can be combined with in situ sensor data to derive typical information offered by a dynamic Web mapping service (WMS). This will provide daily maps of vegetation productivity for oil palm plantation with a spatial resolution of 250 m. Results will present the vegetation productivity model, the sensor data sources, and the implementation of the automated processing facility. An evaluation will be made of the opportunities and limitations of sensor Web-based approaches for the development of Web services, which combine both UAV and in situ sensor sources. A conceptual illustration is provided in Figure 11. A yield estimation model can be developed by establishing performing regression analysis between palm height (x1), crown size (x2), palm age (x3), vegetation index (x4), nutrient content (x5), and soil parameters (x6): Yield = func(x1, x2,…, x6). This model will be based on comprehensive information of each palm location, size, and health, will provide managers with an estimation of yield, and make decisions for sustainable practices methods for production increase without

Figure 11. Conceptual illustration of a fixed-wing UAV Web mapping system integrated with mobile application and

Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations

http://dx.doi.org/10.5772/intechopen.80936

103

The fixed-wing Osprey drone shown in Figure 11 is a commercially available, low-cost experimental flight test bed manufactured by Unmanned Aerial Research (Florida, USA) that is suitable for investigating novel control approaches [24] and is a flexible platform for remote sensing research applications in precision agriculture of oil palm. An example application can be found in the work of [25], where the fixed-wing J-HAWK UAV was used for palm tree counting at Melaka Pindah oil palm plantation in Malaysia. This drone can carry large payloads while maintaining excellent performance with virtually no degradation in handling qualities. It is a well-constructed, durable aircraft with mission versatility and a cavernous payload volume that is easily accessible, featuring two long aluminum tracks on the floor for mounting payloads in limitless configurations. Some of the specifications according to the manufacturing website are as follows: payload capacity: 31.75 kg, empty weight: 15.87 kg,

necessary needs for expanding the plantation into natural forests.

cloud computing for yield prediction and inventory management in oil palm plantation.

3. Stabilizing a fixed-wing Osprey UAV

Figure 10. Example of virtual plantation generated by UAV imaging [23].

Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations http://dx.doi.org/10.5772/intechopen.80936 103

Figure 11. Conceptual illustration of a fixed-wing UAV Web mapping system integrated with mobile application and cloud computing for yield prediction and inventory management in oil palm plantation.

height (x1), crown size (x2), palm age (x3), vegetation index (x4), nutrient content (x5), and soil parameters (x6): Yield = func(x1, x2,…, x6). This model will be based on comprehensive information of each palm location, size, and health, will provide managers with an estimation of yield, and make decisions for sustainable practices methods for production increase without necessary needs for expanding the plantation into natural forests.

#### 3. Stabilizing a fixed-wing Osprey UAV

image acquisition techniques for three-dimensional reconstruction of the environment and creation of virtual plantations. Examples of 3D reconstructed plantation are shown in Figure 10. The information extracted from these 3D models can lead to the development of dynamic Web inventory management and mapping system. A 3D reconstruction model of oil palm plantation can be created by using range data methods or depth map using laser range finder sensors and 3D scanner instrumentations. This approach is however costly and not affordable by local oil palm farmers. Alternatively, passive methods, also called image-based reconstruction methods (i.e., photogrammetry technique), have been introduced using a normal camera and image sensors, which do not interfere with the reconstructed object. In this method, a UAV equipped with a normal RGB camera will collect images of the oil palm plantations from different views and angles. Computer software will then process these images to create a 3D model, and filter specific wavelength to generate images that corresponds to vegetation index and palm health. For example, a red edge image can describe nitrogen content and water stress. The potential of UAV image data to simulate the physical process of palm photosynthesis as a result of different crown sizes and densities intercepting different amounts of light radiation can be evaluated using virtual plantations. A virtual plantation can be used to estimate palm height, crown size, and inventory database (Figure 11) for generating dynamic Web maps and yield prediction models. These maps can identify how different palm height, crown sizes, plantation densities, and row orientations in different locations can affect the water and fertilizer demand. Moreover, mathematical models can be established based on the validated information from virtual plantations for estimating nitrogen demand and fertilizer application. These maps also provide precision rich data for academic and educational purposes. Researchers can access to detailed measurements of palm trunk and crown size and the spacing between different palms, leaf area index, and crown density as a preliminary study for the possibility of autonomous variable rate applications and

For the purpose of a sensor Web-based approach for dynamic Web mapping, observations from a UAV can be combined with in situ sensor data to derive typical information offered by a dynamic Web mapping service (WMS). This will provide daily maps of vegetation productivity for oil palm plantation with a spatial resolution of 250 m. Results will present the vegetation productivity model, the sensor data sources, and the implementation of the automated processing facility. An evaluation will be made of the opportunities and limitations of sensor Web-based approaches for the development of Web services, which combine both UAV and in situ sensor sources. A conceptual illustration is provided in Figure 11. A yield estimation model can be developed by establishing performing regression analysis between palm

Figure 10. Example of virtual plantation generated by UAV imaging [23].

robotic harvesting.

102 Agricultural Robots - Fundamentals and Applications

The fixed-wing Osprey drone shown in Figure 11 is a commercially available, low-cost experimental flight test bed manufactured by Unmanned Aerial Research (Florida, USA) that is suitable for investigating novel control approaches [24] and is a flexible platform for remote sensing research applications in precision agriculture of oil palm. An example application can be found in the work of [25], where the fixed-wing J-HAWK UAV was used for palm tree counting at Melaka Pindah oil palm plantation in Malaysia. This drone can carry large payloads while maintaining excellent performance with virtually no degradation in handling qualities. It is a well-constructed, durable aircraft with mission versatility and a cavernous payload volume that is easily accessible, featuring two long aluminum tracks on the floor for mounting payloads in limitless configurations. Some of the specifications according to the manufacturing website are as follows: payload capacity: 31.75 kg, empty weight: 15.87 kg,

A ¼

outputs give in (2).

�0:1470 11:0767 0:0841 �9:8065 �0:0316 �7:1712 0:8281 0 0 �37:3527 �9:9628 0 0 0 10

limited effect in both velocity and pitch rate.

3 7 7 7 5, <sup>B</sup> <sup>¼</sup>

Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations

where x ¼ ½ � V α q θ , with the state variables defined as V: velocity, α: angle of attack, q: pitch rate, and <sup>θ</sup>: pitch angle. The control inputs are <sup>u</sup> <sup>¼</sup> <sup>δ</sup>elev <sup>δ</sup>thrust ½ �<sup>T</sup>. Our controller design process begins with analyzing the mathematical model of the given dynamic system. The state-space model was first converted to a convenient transfer functions (TF) given in (1) and (2). Converting the SS model into TF form using MATLAB "tf(sys)" yields transfer function from input "δelev" to outputs given in (1), and the sets of transfer function from input "δthrust" to

H11ð Þ¼ <sup>s</sup> <sup>V</sup> <sup>¼</sup> <sup>0</sup>:03357 s<sup>3</sup> <sup>þ</sup> <sup>0</sup>:6577 s<sup>2</sup> <sup>þ</sup> <sup>3</sup>:407 s � <sup>68</sup>:<sup>91</sup>

H12ð Þ¼ <sup>s</sup> <sup>V</sup> <sup>¼</sup> <sup>0</sup>:06 s3 <sup>þ</sup> <sup>1</sup>:029 s<sup>2</sup> <sup>þ</sup> <sup>6</sup>:153 s <sup>þ</sup> <sup>0</sup>:<sup>03663</sup>

We first perform open-loop analysis to determine possible control strategies. The open-loop responses (Figure 13) from each of the four TFs were then analyzed individually. According to the TF in (1) and (2), the terms with the highest coupling can be obtained by considering the simple steady state case. Substituting jω = 0, in all the terms, it can be observed that the static gain relationship is high for δelev versus velocity output. This also makes physical sense as a change in the pitch would slowdown the Osprey. It is also noted that the pitch rate has zeros at origin. This suggests that the system has inherent derivative property and hence has a tendency to amplify noises. Based on the open-loop response shown in Figure 13, it can also be seen that the effect of δelev on the velocity is more than other inputs. From Figure 13, the following key points are helpful in controller design for the system, (i) δelev has more effect on velocity than any other input, (ii) the velocity falls sharply with input δelev, and (iii) δthrust has

For the PID controller design shown in Figure 14, the system was set at initial conditions [δelev ¼ 4 and V = 25 m/s]. A step input of 10 m/s was given at time t = 60 s. The following gains were used for the PID velocity controller: Kp = 200, Ki = 80, and Kd = 20. For the PID pitch controller, Kp, Ki, and Kd were respective selected as 6, 0.2, and 10. After introducing the noise, the new selected gains for the PID velocity controller were Kp = 50, Ki = 11 and Kd = 11. For the PID pitch controller, the new Kp, Ki, and Kd were chosen 100, 4, and 1, respectively. The decrease in Kp compared with the previous case for the velocity controller should be noticed.

<sup>H</sup>21ð Þ¼ <sup>s</sup> <sup>q</sup> <sup>¼</sup> <sup>0</sup>:<sup>98</sup> <sup>s</sup><sup>3</sup> <sup>þ</sup> <sup>7</sup>:<sup>171</sup> <sup>s</sup><sup>2</sup> <sup>þ</sup> <sup>1</sup>:<sup>416</sup> <sup>s</sup>

<sup>H</sup>22ð Þ¼ <sup>s</sup> <sup>q</sup> <sup>¼</sup> �0:<sup>003735</sup> <sup>s</sup><sup>2</sup> <sup>þ</sup> <sup>0</sup>:<sup>07027</sup> <sup>s</sup>

s4 þ 17:28 s3 þ 105:2 s2 þ 18:44 s þ 11:58

s4 þ 17:28 s3 þ 105:2 s2 þ 18:44 s þ 11:58

<sup>s</sup><sup>4</sup> <sup>þ</sup> <sup>17</sup>:<sup>28</sup> <sup>s</sup><sup>3</sup> <sup>þ</sup> <sup>105</sup>:<sup>2</sup> <sup>s</sup><sup>2</sup> <sup>þ</sup> <sup>18</sup>:<sup>44</sup> <sup>s</sup> <sup>þ</sup> <sup>11</sup>:<sup>58</sup> (1)

<sup>s</sup><sup>4</sup> <sup>þ</sup> <sup>17</sup>:<sup>28</sup> <sup>s</sup><sup>3</sup> <sup>þ</sup> <sup>105</sup>:<sup>2</sup> <sup>s</sup><sup>2</sup> <sup>þ</sup> <sup>18</sup>:<sup>44</sup> <sup>s</sup> <sup>þ</sup> <sup>11</sup>:<sup>58</sup> (2)

3:10�<sup>3</sup> 0:06 10�<sup>5</sup> 10�<sup>4</sup> 0:98 0 0 0

5, <sup>C</sup> <sup>¼</sup> 100 0 0010: � � 105

http://dx.doi.org/10.5772/intechopen.80936

Figure 12. Architecture, modules, and control system for a the proposed UAV in precision agriculture of oil palm.

payload volume 0.0566 m3 (0.203H � 0.304 W � 0.889 L), max cruise: 90 kts, landing speed (no flaps): 25 kts, power (DA-100): 10 hp by a reliable custom desert aircraft 100 cc motor with 3 blade carbon fiber propeller, wingspan 3.352 mm, and length 2.362 m. We begin with dynamic analysis and controller design for this drone in the presence of actuator limits and sensor noise for autonomous flight missions with greater accuracy and stability. The communication architecture, modules, and designed control system is shown in Figure 12.

For the purpose of this paper, we have concentrated our analysis on controller design for two outputs, velocity and pitch rate, by adjusting two control inputs, the elevator and the thrust. In specific, our control objective was to design a single controller, i.e., proportional-integralderivative (PID), Linear-quadratic regulator (LQR) full state feedback, (C ¼ I<sup>4</sup>�4), and LQR plus observer (with C defined by the dynamic model), that (i) stabilize the drone with a velocity step input of 10 m/s and (ii) minimize rise time, percentage overshoot, and steady state error over the widest possible initial conditions. Based on the field experiments data in the reviewed literature [24], the maximum δthrust and δelev and their rate of change were considered �<sup>200</sup> <sup>N</sup> and �30� and �<sup>200</sup> <sup>N</sup><sup>=</sup> sec and �300� =s, respectively. In addition, the noise for the velocity sensor and the pitch rate sensor were assumed to be �0:4 m=s and �1:7 � =s, respectively. The dynamics associated with this drone under standard aircraft assumptions were de-coupled into both lateral and longitudinal directions. For the sake of this control design, we only analyze the longitudinal dynamics. The longitudinal dynamics of the Osprey have been mathematically modeled as a fourth-order multiple-input multiple-output (MIMO) system with two inputs and two outputs [24]. The dynamics have been linearized for the Osprey aircraft flying at 25 m/s at an altitude of 60 m. In standard state-space form, they are given as:

Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations http://dx.doi.org/10.5772/intechopen.80936 105

$$A = \begin{bmatrix} -0.1470 & 11.0767 & 0.0841 & -9.8065 \\ -0.0316 & -7.1712 & 0.8281 & 0 \\ 0 & -37.3527 & -9.9628 & 0 \\ 0 & 0 & 1 & 0 \end{bmatrix}, B = \begin{bmatrix} 3.10^{-3} & 0.06 \\ 10^{-5} & 10^{-4} \\ 0.98 & 0 \\ 0 & 0 \end{bmatrix}, \mathbb{C} = \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{bmatrix}$$

where x ¼ ½ � V α q θ , with the state variables defined as V: velocity, α: angle of attack, q: pitch rate, and <sup>θ</sup>: pitch angle. The control inputs are <sup>u</sup> <sup>¼</sup> <sup>δ</sup>elev <sup>δ</sup>thrust ½ �<sup>T</sup>. Our controller design process begins with analyzing the mathematical model of the given dynamic system. The state-space model was first converted to a convenient transfer functions (TF) given in (1) and (2). Converting the SS model into TF form using MATLAB "tf(sys)" yields transfer function from input "δelev" to outputs given in (1), and the sets of transfer function from input "δthrust" to outputs give in (2).

$$H\_{11}(\mathbf{s}) = \mathbf{V} = \frac{0.03357 \ \mathbf{s}^3 + 0.6577 \ \mathbf{s}^2 + 3.407 \ \mathbf{s} - 68.91}{\mathbf{s}^4 + 17.28 \ \mathbf{s}^3 + 105.2 \ \mathbf{s}^2 + 18.44 \ \mathbf{s} + 11.58}$$

$$H\_{21}(\mathbf{s}) = q = \frac{0.98 \ \mathbf{s}^3 + 7.171 \ \mathbf{s}^2 + 1.416 \ \mathbf{s}}{\mathbf{s}^4 + 17.28 \ \mathbf{s}^3 + 105.2 \ \mathbf{s}^2 + 18.44 \ \mathbf{s} + 11.58} \tag{1}$$

$$0.06 \ \mathbf{s}^3 + 1.029 \ \mathbf{s}^2 + 6.153 \ \mathbf{s} + 0.03663}$$

$$H\_{12}(\mathbf{s}) = \mathbf{V} = \frac{0.06\,\mathrm{s}^3 + 1.029\,\mathrm{s}^2 + 6.153\,\mathrm{s} + 0.03663}{\mathrm{s}^4 + 17.28\,\mathrm{s}^3 + 105.2\,\mathrm{s}^2 + 18.44\,\mathrm{s} + 11.58}$$

$$H\_{22}(\mathbf{s}) = \eta = \frac{-0.003735\,\mathrm{s}^2 + 0.07027\,\mathrm{s}}{\mathrm{s}^4 + 17.28\,\mathrm{s}^3 + 105.2\,\mathrm{s}^2 + 18.44\,\mathrm{s} + 11.58} \tag{2}$$

We first perform open-loop analysis to determine possible control strategies. The open-loop responses (Figure 13) from each of the four TFs were then analyzed individually. According to the TF in (1) and (2), the terms with the highest coupling can be obtained by considering the simple steady state case. Substituting jω = 0, in all the terms, it can be observed that the static gain relationship is high for δelev versus velocity output. This also makes physical sense as a change in the pitch would slowdown the Osprey. It is also noted that the pitch rate has zeros at origin. This suggests that the system has inherent derivative property and hence has a tendency to amplify noises. Based on the open-loop response shown in Figure 13, it can also be seen that the effect of δelev on the velocity is more than other inputs. From Figure 13, the following key points are helpful in controller design for the system, (i) δelev has more effect on velocity than any other input, (ii) the velocity falls sharply with input δelev, and (iii) δthrust has limited effect in both velocity and pitch rate.

payload volume 0.0566 m3 (0.203H � 0.304 W � 0.889 L), max cruise: 90 kts, landing speed (no flaps): 25 kts, power (DA-100): 10 hp by a reliable custom desert aircraft 100 cc motor with 3 blade carbon fiber propeller, wingspan 3.352 mm, and length 2.362 m. We begin with dynamic analysis and controller design for this drone in the presence of actuator limits and sensor noise for autonomous flight missions with greater accuracy and stability. The communication archi-

Figure 12. Architecture, modules, and control system for a the proposed UAV in precision agriculture of oil palm.

For the purpose of this paper, we have concentrated our analysis on controller design for two outputs, velocity and pitch rate, by adjusting two control inputs, the elevator and the thrust. In specific, our control objective was to design a single controller, i.e., proportional-integralderivative (PID), Linear-quadratic regulator (LQR) full state feedback, (C ¼ I<sup>4</sup>�4), and LQR plus observer (with C defined by the dynamic model), that (i) stabilize the drone with a velocity step input of 10 m/s and (ii) minimize rise time, percentage overshoot, and steady state error over the widest possible initial conditions. Based on the field experiments data in the reviewed literature [24], the maximum δthrust and δelev and their rate of change were

and �<sup>200</sup> <sup>N</sup><sup>=</sup> sec and �300�

for the velocity sensor and the pitch rate sensor were assumed to be �0:4 m=s and �1:7

respectively. The dynamics associated with this drone under standard aircraft assumptions were de-coupled into both lateral and longitudinal directions. For the sake of this control design, we only analyze the longitudinal dynamics. The longitudinal dynamics of the Osprey have been mathematically modeled as a fourth-order multiple-input multiple-output (MIMO) system with two inputs and two outputs [24]. The dynamics have been linearized for the Osprey aircraft flying at 25 m/s at an altitude of 60 m. In standard state-space form, they are

=s, respectively. In addition, the noise

� =s,

tecture, modules, and designed control system is shown in Figure 12.

considered �<sup>200</sup> <sup>N</sup> and �30�

104 Agricultural Robots - Fundamentals and Applications

given as:

For the PID controller design shown in Figure 14, the system was set at initial conditions [δelev ¼ 4 and V = 25 m/s]. A step input of 10 m/s was given at time t = 60 s. The following gains were used for the PID velocity controller: Kp = 200, Ki = 80, and Kd = 20. For the PID pitch controller, Kp, Ki, and Kd were respective selected as 6, 0.2, and 10. After introducing the noise, the new selected gains for the PID velocity controller were Kp = 50, Ki = 11 and Kd = 11. For the PID pitch controller, the new Kp, Ki, and Kd were chosen 100, 4, and 1, respectively. The decrease in Kp compared with the previous case for the velocity controller should be noticed.

Hence, in this case, the gain of the pitch controller was driven high and the other low. Since pitch rate has very high impact on the other system variables, noise in the pitch rate influences the system heavily. Therefore to improve tuning the controller, a simple first-order TF = 2/ (S + 15) (low pass filter) was inserted in the loop (shown in the second Simulink block of

Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations

http://dx.doi.org/10.5772/intechopen.80936

107

The LQR controller is the solution of the optimization problem that optimizes the cost of errors and the cost of actuation effort, with appropriately weighted states. The optimization function is defined as <sup>J</sup> <sup>¼</sup> <sup>Ð</sup> <sup>x</sup>TQx <sup>þ</sup> uTRu � �dσ. In the state space form, the obtained LQR controller is expressed as u ¼ �Kx. For this solution, an LQR controller was first derived using the MATLAB "lqr" command. The cost weighting matrices Q and R were selected as unit matrices, and the LQR was realized. Simulink blocks for the designed LQR controller with full state feedback are shown in Figure 15. The weighting matrices used in this case were as follow:

It is noted that the control effort for pitch is the most optimized parameter in Q. This value was selected on the basis that pitch is the most influential state variable and controlling pitch translates control of all the other parameters. In addition, the weight for pitch rate is low because the effort to control pitch rate is harder and introduced more oscillations in the system. For the LQR controller with observer (Figure 16), the observer design allows controller to use full-state feedback techniques in situations where only a subset of states is available to the controller. The observer matrix L adds gain to the feedback loop, in order to ensure stability and quicker response of the state observer system. While this helps stability, the L gain adversely amplifies the sensor noise. Therefore, a trade-off has to be made on the noise resilience versus the system's robustness. The matrix L was determined through these steps: (i) the system output states were checked for controllability and observability using Matlab code "obsv" and "ctrb," (ii) the poles of the system were found and the system was found to be stable, (iii) for

, <sup>R</sup> <sup>¼</sup> 1 0 0 0:<sup>05</sup> � �

50 0 0 01 0 0 000:5 0 0 0 0 4000

Q ¼

Figure 15. Simulink blocks for the designed LQR controller with full state feedback.

Figure 14).

Figure 13. Open-loop step response analysis of the Osprey drone velocity and pitch rate for the elevator and thrust inputs.

Figure 14. Simulink blocks for the PID controller in the absence and presence of noise and actuator limits.

Hence, in this case, the gain of the pitch controller was driven high and the other low. Since pitch rate has very high impact on the other system variables, noise in the pitch rate influences the system heavily. Therefore to improve tuning the controller, a simple first-order TF = 2/ (S + 15) (low pass filter) was inserted in the loop (shown in the second Simulink block of Figure 14).

The LQR controller is the solution of the optimization problem that optimizes the cost of errors and the cost of actuation effort, with appropriately weighted states. The optimization function is defined as <sup>J</sup> <sup>¼</sup> <sup>Ð</sup> <sup>x</sup>TQx <sup>þ</sup> uTRu � �dσ. In the state space form, the obtained LQR controller is expressed as u ¼ �Kx. For this solution, an LQR controller was first derived using the MATLAB "lqr" command. The cost weighting matrices Q and R were selected as unit matrices, and the LQR was realized. Simulink blocks for the designed LQR controller with full state feedback are shown in Figure 15. The weighting matrices used in this case were as follow:

$$Q = \begin{bmatrix} 5 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0.5 & 0 \\ 0 & 0 & 0 & 4000 \end{bmatrix}, R = \begin{bmatrix} 1 & 0 \\ 0 & 0.05 \end{bmatrix}.$$

Figure 13. Open-loop step response analysis of the Osprey drone velocity and pitch rate for the elevator and thrust inputs.

106 Agricultural Robots - Fundamentals and Applications

Figure 14. Simulink blocks for the PID controller in the absence and presence of noise and actuator limits.

It is noted that the control effort for pitch is the most optimized parameter in Q. This value was selected on the basis that pitch is the most influential state variable and controlling pitch translates control of all the other parameters. In addition, the weight for pitch rate is low because the effort to control pitch rate is harder and introduced more oscillations in the system.

For the LQR controller with observer (Figure 16), the observer design allows controller to use full-state feedback techniques in situations where only a subset of states is available to the controller. The observer matrix L adds gain to the feedback loop, in order to ensure stability and quicker response of the state observer system. While this helps stability, the L gain adversely amplifies the sensor noise. Therefore, a trade-off has to be made on the noise resilience versus the system's robustness. The matrix L was determined through these steps: (i) the system output states were checked for controllability and observability using Matlab code "obsv" and "ctrb," (ii) the poles of the system were found and the system was found to be stable, (iii) for

Figure 15. Simulink blocks for the designed LQR controller with full state feedback.

Figure 16. LQR controller with observer block.

the observer system to be more stable and faster, the poles were multiplied by a factor of 3, (iv) these scaled poles were then used in the Ackermann's formula for pole-placement design to find L and design a mimic of the original system, the observer, and (v) the state output from the observer can now be fed to the LQR controller. The weighting matrices used in this case are:

$$Q = \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 5 & 0 \\ 0 & 0 & 0 & 1000 \end{bmatrix}, R = \begin{bmatrix} 1 & 0 \\ 0 & 0.001 \end{bmatrix}.$$

It can be seen that the value of pitch gains in the Q is four times smaller than the previous case. The gains were reduced to take control over noise in the system. In the other words, these reductions help eliminate the noise in the system. From step three of the observer design, we know that the observer matrix L adds gain to feedback loop. This gain helps amplifying the noise and then feeding them into the control loop back again. Noise introduces similar problems faced with the PID controller. With high gains, the noise amplifies and combined with actuator nonlinearities drives the system into instability. With lesser gains and actuator effort, noise is damped and absorbed by the system.

high, the pitch rate sensor noise distorts the response considerably. Moreover, since the tuned gains exploited the infinite actuator capabilities, the response of the system was quick and the steady state error was almost zero; however, due to the nonlinearities, the system had to be tuned again. Since elevation was directly related to the pitch rather than the pitch rate and to avoid the dynamics of the "rate" signal, pitch was compared against the elevator angle to generate the error signal. To accomplish this, the pitch rate was simply integrated using an

Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations

http://dx.doi.org/10.5772/intechopen.80936

109

It can be seen from the results that the state variables pitch and velocity are closely coupled variables. The coupling terms connecting these two quantities exhibit every high gains, hence the control design was challenging in regulating these variables independent of the other. This coupling needed special attention during control design. It should be noted that on the basis of tuning complexity, only two PID controllers were used in the control problem, as if the system was a weakly coupled system. Since PID control is ideally suited for singleinput single-output systems (SISO) and only for weakly coupled MIMO systems, a perfect performance was not expected to achieve with the two PID controllers. Nevertheless, a reasonable performance was still achieved when the system was considered ideal, i.e., free

ideal integrator (1/s).

Figure 17. PID performance without noise and actuator limits.

Figure 18. PID performance with noise and actuator nonlinearities.

#### 4. Simulation results and discussion

Results of the simulation for the designed controllers are shown in Figure 17 through Figure 20. It can be seen from Figure 17 that the step change applied at time 60 s has an effect on the pitch, and the PID controller is managed to minimize this effect. When noise is introduced to the system (Figure 18), because the coupling gain between pitch and velocity are very

Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations http://dx.doi.org/10.5772/intechopen.80936 109

Figure 17. PID performance without noise and actuator limits.

the observer system to be more stable and faster, the poles were multiplied by a factor of 3, (iv) these scaled poles were then used in the Ackermann's formula for pole-placement design to find L and design a mimic of the original system, the observer, and (v) the state output from the observer can now be fed to the LQR controller. The weighting matrices used in this case are:

It can be seen that the value of pitch gains in the Q is four times smaller than the previous case. The gains were reduced to take control over noise in the system. In the other words, these reductions help eliminate the noise in the system. From step three of the observer design, we know that the observer matrix L adds gain to feedback loop. This gain helps amplifying the noise and then feeding them into the control loop back again. Noise introduces similar problems faced with the PID controller. With high gains, the noise amplifies and combined with actuator nonlinearities drives the system into instability. With lesser gains and actuator effort,

Results of the simulation for the designed controllers are shown in Figure 17 through Figure 20. It can be seen from Figure 17 that the step change applied at time 60 s has an effect on the pitch, and the PID controller is managed to minimize this effect. When noise is introduced to the system (Figure 18), because the coupling gain between pitch and velocity are very

5, <sup>R</sup> <sup>¼</sup> 1 0

0 0:001 � �

100 0 010 0 005 0 0 0 0 1000

Q ¼

Figure 16. LQR controller with observer block.

108 Agricultural Robots - Fundamentals and Applications

noise is damped and absorbed by the system.

4. Simulation results and discussion

Figure 18. PID performance with noise and actuator nonlinearities.

high, the pitch rate sensor noise distorts the response considerably. Moreover, since the tuned gains exploited the infinite actuator capabilities, the response of the system was quick and the steady state error was almost zero; however, due to the nonlinearities, the system had to be tuned again. Since elevation was directly related to the pitch rather than the pitch rate and to avoid the dynamics of the "rate" signal, pitch was compared against the elevator angle to generate the error signal. To accomplish this, the pitch rate was simply integrated using an ideal integrator (1/s).

It can be seen from the results that the state variables pitch and velocity are closely coupled variables. The coupling terms connecting these two quantities exhibit every high gains, hence the control design was challenging in regulating these variables independent of the other. This coupling needed special attention during control design. It should be noted that on the basis of tuning complexity, only two PID controllers were used in the control problem, as if the system was a weakly coupled system. Since PID control is ideally suited for singleinput single-output systems (SISO) and only for weakly coupled MIMO systems, a perfect performance was not expected to achieve with the two PID controllers. Nevertheless, a reasonable performance was still achieved when the system was considered ideal, i.e., free from nonlinearities and noise. When noise was introduced to the system, the velocity suffered because of high noise content in the pitch signal. The noise also introduced dangerous oscillations in the system, limiting controller gains significantly and hence slowing down the overall system. Several instabilities caused due to the rate limit and saturation were evident. The integral gain of the PID acted on error build-up caused by saturation and hence pushing the system into instability. After reducing the gains in the loop, the controller was then tuned by trial and error procedures. The relative performance of PID with respect to other controllers is summarized in Table 1.

LQR controllers however work in the state-space and are suited for MIMO control. It assumes full state feedback; that is, all the system's states are available for the controller to take decisions, even though this might not be a case in reality. Therefore, we designed the observer to deal with this issue. The outputs of the LQR controlled system response with actuator dynamics are shown in Figures 19 and 20. Unlike the PID controller, the LQR handles actuator dynamics inconsequentially. Appropriate waiting matrices were assigned, and the LQR controller matrix was obtained by using the MATLAB "lqr" command. The LQR trivially performed well with actuator nonlinearities. By weighting the gains in the Q and R matrices, it was possible to avoid high actuation effort and thus saturation. But rate limit did affect the rise time. The LQR also suffered from oscillations, when noise was introduced.

From the plots of LQR with observer (Figure 20), it can be seen that the system is in the verge of instability and the noise content of the pitch signal disturbs the velocity severely. The relative stability of the given system can be discussed in terms of the gain margin and phase margin. Based on the Bode plots analysis of the open-loop system (plots not provided for the sake of paper page limits), the differential term in the elevator input to output relationships reduces the phase margin of system considerably. Model errors and disturbance in the pitch rate could easily drive the system to instability. This agrees with the findings in the controller design exercised.

Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations

http://dx.doi.org/10.5772/intechopen.80936

111

Health assessment and conventional scouting of oil palms on a regular basis, as well as palm census and quantification of the amount of fresh fruit bunches (FFB) for yield monitoring, are labor-intensive tasks that are either ignored in large scale plantations or are carried out manually by the use of labor force. Traditional scouting is not only an ineffective practice but also requires expert knowledge and post-processing lab equipment to provide useful information. Advances in aerospace engineering, control system, and computing have contributed significantly to the improvement of UAV-based remote sensing platforms. This paper discussed some of the potential applications of UAVs for precision agriculture of oil palm plantations. We also highlighted some of the adaptation challenges faced by UAV drones, including platform stability due to the flight dynamics parameters and winds, climate factors and light reflection degrading quality of the acquired images, and regulations and restrictions law by the Federal Aviation Administration. As a response to the needs of small-scale plantation owner for an affordable UAV platform, a fixed-wing Osprey drone was proposed and used in designing an auto-flight control. The aircraft can be externally actuated by controlling the thrust (δthrust) and the elevator (δelev). Initially, all states of the dynamic model were assumed to be available to the controller. A case was then considered when only velocity and pitch rate could be measured. We conclude that the MIMO control problem of the Osprey drone falls in the class of systems that exhibit high level of coupling between the inputs. We also conclude that the LQR design procedure was simple compared with the PID and performed better than PID in the presence of noise. Unlike PID, the

5. Conclusion

Figure 20. LQR with observer response with noise.


Table 1. A comparison between the proposed controllers.

Figure 19. LQR full state feedback response without noise.

Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations http://dx.doi.org/10.5772/intechopen.80936 111

Figure 20. LQR with observer response with noise.

From the plots of LQR with observer (Figure 20), it can be seen that the system is in the verge of instability and the noise content of the pitch signal disturbs the velocity severely. The relative stability of the given system can be discussed in terms of the gain margin and phase margin. Based on the Bode plots analysis of the open-loop system (plots not provided for the sake of paper page limits), the differential term in the elevator input to output relationships reduces the phase margin of system considerably. Model errors and disturbance in the pitch rate could easily drive the system to instability. This agrees with the findings in the controller design exercised.

#### 5. Conclusion

from nonlinearities and noise. When noise was introduced to the system, the velocity suffered because of high noise content in the pitch signal. The noise also introduced dangerous oscillations in the system, limiting controller gains significantly and hence slowing down the overall system. Several instabilities caused due to the rate limit and saturation were evident. The integral gain of the PID acted on error build-up caused by saturation and hence pushing the system into instability. After reducing the gains in the loop, the controller was then tuned by trial and error procedures. The relative performance of PID with respect to other control-

LQR controllers however work in the state-space and are suited for MIMO control. It assumes full state feedback; that is, all the system's states are available for the controller to take decisions, even though this might not be a case in reality. Therefore, we designed the observer to deal with this issue. The outputs of the LQR controlled system response with actuator dynamics are shown in Figures 19 and 20. Unlike the PID controller, the LQR handles actuator dynamics inconsequentially. Appropriate waiting matrices were assigned, and the LQR controller matrix was obtained by using the MATLAB "lqr" command. The LQR trivially performed well with actuator nonlinearities. By weighting the gains in the Q and R matrices, it was possible to avoid high actuation effort and thus saturation. But rate limit did affect the rise time. The LQR also suffered from oscillations, when noise was

Controller Noise Actuator limits Rise time (s) Settling time (s) Overshoot (%)

PID — — 0.61 0.95 4.2 PID Y Y 1.28 Inf 10.1 LQR — Y 1.01 1.9 2.1 LQR Y Y 3.3 3.95 0 Observer Y Y 1.9 Inf 34.5

lers is summarized in Table 1.

110 Agricultural Robots - Fundamentals and Applications

Table 1. A comparison between the proposed controllers.

Figure 19. LQR full state feedback response without noise.

introduced.

Health assessment and conventional scouting of oil palms on a regular basis, as well as palm census and quantification of the amount of fresh fruit bunches (FFB) for yield monitoring, are labor-intensive tasks that are either ignored in large scale plantations or are carried out manually by the use of labor force. Traditional scouting is not only an ineffective practice but also requires expert knowledge and post-processing lab equipment to provide useful information. Advances in aerospace engineering, control system, and computing have contributed significantly to the improvement of UAV-based remote sensing platforms. This paper discussed some of the potential applications of UAVs for precision agriculture of oil palm plantations. We also highlighted some of the adaptation challenges faced by UAV drones, including platform stability due to the flight dynamics parameters and winds, climate factors and light reflection degrading quality of the acquired images, and regulations and restrictions law by the Federal Aviation Administration. As a response to the needs of small-scale plantation owner for an affordable UAV platform, a fixed-wing Osprey drone was proposed and used in designing an auto-flight control. The aircraft can be externally actuated by controlling the thrust (δthrust) and the elevator (δelev). Initially, all states of the dynamic model were assumed to be available to the controller. A case was then considered when only velocity and pitch rate could be measured. We conclude that the MIMO control problem of the Osprey drone falls in the class of systems that exhibit high level of coupling between the inputs. We also conclude that the LQR design procedure was simple compared with the PID and performed better than PID in the presence of noise. Unlike PID, the LQR was more vulnerable to have steady state error. With changes in the δelev command, the velocity was affected considerably and would never recover unlike integral action of PID. The introduction of an observer in an already noisy system added more uncertainty in the system, thus pushing the system toward instability. The observer added to the gain of the feedback loop and hence amplifying noise. Even with various combinations of weighting matrices, the steady state oscillations were as high as 20%. In conclusion, it is observed that the LQR is a robust and effective controller for MIMO control. The LQR was found to be robust against noise and disturbance in the system too.

Model Price

Parrot Disco Pro AG Drone

DT-26 Crop mapper

E384 Mapping Drone

PrecisionHawk Lancaster 5

UAV drone crop sprayer

Yamaha's helicopters spray & survey

AG-UAV Sprayers1

AG-UAV Sprayers2

AG-UAV Sprayers3

DJI AGRAS MG-1 Sprayer

Hercules Heavy Lift UAV (HL6)

Hercules Heavy Lift UAV (HL10)

Hercules Heavy Lift UAV (HL20)

AgStar GoPro FPV Camera Payload

5 L

JMR-V1000 6-rotor

Xena thermo — Take-Off:

(\$)

Weight (Kg)

6875 UAV: 0.78 Take-Off: 0.94

2400 UAV: 2.5 Payload: 1

4.6

UAV: 9 Payload: 10 Take-Off: 13

UAV: 6.5 Take-Off: 18

130,000 UAV: 71 Payload: 30

— UAV: 8 Payload: 6

— UAV: 14.2 Payload: 20

— UAV: 9.5 Payload: 10

— UAV: 8 Payload: 6

— UAV: 9.5 Payload: 10

— UAV: 14 Payload: 20

665– 3799

AEE AP10 Drone 299 — — 1080 P Full HD

Size (mm) Camera

AgDrone UAS 10,000 — — 1080 P — 60 —

Quad Indigo 25,000 — — 1080 P — 45 —

Wing span: 1150 580 120

Wingspan: 1900 Length: 1300

800 800 70 (L.W.H)

875 1100 480 (L.W. H)

7999 Payload: 10 — — 7–10 Acres Per

Multirotor UAVs —— — — — 10–40 — —

1950 — — — —— —

resolution

RF70 UAV 3000 Payload: 3 — 1080 P 600 acres/hour 45–60 — 18

Xena observer — Take-Off: 5 —— — 27 5000 —

DJI drone sprayer 15,000 —— — 7–10 acres/hour — — 29

Video at 60 FPS

120,000 — — 1080 P — 60 — 110

Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations

— 1000 acres in

— Payload: 1 — 1 cm/pixel 300 acres/flight 45 — —

100 minutes at 5 cm resolution

—— — 32 5000 —

— — 10 acres —— —

Height: 650 — — 8–15 — —

Height: 650 — — 15–30 — —

Height: 650 — — 10–20 — —

Height: 660 — — 30 — 37

Height: 660 — — 30 — 37

Height: 660 — — 60 — 37

Hour

Coverage Flight

— — —— —

time (min)

— 25 500 71

— — 16 1000 —

— — 14–18 — 11–22

Max altitude (m)

http://dx.doi.org/10.5772/intechopen.80936

90 — 47

—— —

Flight speed (km/h) 113

#### A. Appendix

Names and specifications of sample multi-rotor and fixed-wing UAV recommended for precision agriculture of oil palm.

Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations http://dx.doi.org/10.5772/intechopen.80936 113

LQR was more vulnerable to have steady state error. With changes in the δelev command, the velocity was affected considerably and would never recover unlike integral action of PID. The introduction of an observer in an already noisy system added more uncertainty in the system, thus pushing the system toward instability. The observer added to the gain of the feedback loop and hence amplifying noise. Even with various combinations of weighting matrices, the steady state oscillations were as high as 20%. In conclusion, it is observed that the LQR is a robust and effective controller for MIMO control. The LQR was found to be robust against noise and

Names and specifications of sample multi-rotor and fixed-wing UAV recommended for

disturbance in the system too.

112 Agricultural Robots - Fundamentals and Applications

precision agriculture of oil palm.

A. Appendix



References

[1] Gennari P, Heyman A, Kainu M. FAO statistical pocketbook. World food and agriculture.

Fundamental Research on Unmanned Aerial Vehicles to Support Precision Agriculture in Oil Palm Plantations

http://dx.doi.org/10.5772/intechopen.80936

115

[2] Shamshiri RR. A Breakthrough in Oil Palm Precision Agriculture: Smart Management of Oil Palm Plantations with Autonomous UAV Imagery and Robust Machine Vision. In:

[3] Shamshiri RR. Unmanned Aerial Vehicles (UAV) to Support Precision Agriculture

[4] Shamshiri RR. Integration of smart sensors and robotics in increasing agricultural productivity with higher yields at lower costs. In: Asian Space Technology Summit. 2017

[5] Srestasathiern P, Rakwatin P. Oil palm tree detection with high resolution multi-spectral

[6] Breckenridge RP, Dakins M, Bunting S, Harbour JL, Lee RD. Using unmanned helicopters to assess vegetation cover in sagebrush steppe ecosystems. Rangeland Ecology & Man-

[7] Kalantar B, Bin Mansor S, Sameen MI, Pradhan B, Shafri HZM. Drone-based land-cover mapping using a fuzzy unordered rule induction algorithm integrated into object-based image analysis. International Journal of Remote Sensing. May 2017;38(8–10):2535-2556

[8] Chao H, Baumann M, Jensen A, Chen Y, Cao Y, Ren W, McKee M. Band-reconfigurable multi-UAV based cooperative remote sensing for real-time water management and dis-

[9] Ambrosia VG, Wegener SS, Sullivan DV, Buechel SW, Dunagan SE, Brass JA, et al. Demonstrating UAV-acquired real-time thermal data over fires. Photogrammetric Engineering

[10] Xiongkui H, Bonds J, Herbst A, Langenakens J. Recent development of unmanned aerial vehicle for plant protection in East Asia. International Journal of Agricultural and Biolog-

[11] Shamshiri R, Hameed IA, Balasundram SK, Weltzien C, Yule IJ, Grift TE, et al. Adapting simulation platforms and virtual environments for acceleration of agricultural robotics: A perspective of digital farming. International Journal of Agricultural and Biological Engi-

[12] Shamshiri R, Wan Ismail WI. Nonlinear tracking control of a two link oil palm harvesting manipulator. International Journal of Agricultural and Biological Engineering. 2012;5(2):9-19

[13] Shamshiri R, Ishak W, Ismail W. Design and simulation of control Systems for a Field Survey Mobile Robot Platform. Research Journal of Applied Sciences, Engineering and

tributed irrigation control. IFAC Proceedings Volumes. 2008;41(2):11744-11749

In: Food and Agriculture Organisation, United Nations, Rome, Italy. 2015

International Conference on Agricultural and Food Engineering; 2016

Research in Oil Palm Plantations. Kuala Lumpur; 2017

satellite imagery. Remote Sensing. 2014;6(10):9749-9774

agement. 2012;65(4):362-370

and Remote Sensing. 2003;69(4):391-402

ical Engineering. 2017;10(3):18

Technology. 2013;6(13):2307-2315

neering. 2018;11(4):1-25

#### Author details

Redmond Ramin Shamshiri1,4\*, Ibrahim A. Hameed2 , Siva K. Balasundram<sup>1</sup> , Desa Ahmad3 , Cornelia Weltzien<sup>4</sup> and Muhammad Yamin<sup>5</sup>

\*Address all correspondence to: raminshamshiri@upm.edu.my

1 Department of Agriculture Technology, Faculty of Agriculture, Universiti Putra Malaysia, Serdang, Selangor, Malaysia

2 Department of ICT and Natural Sciences, Faculty of Information Technology and Electrical Engineering, NTNU, Ålesund, Norway

3 Smart Farming Technology Research Center, Department of Biological and Agricultural Engineering, Faculty of Engineering, Universiti Putra Malaysia, Serdang, Selangor, Malaysia

4 Leibniz Institute for Agricultural Engineering and Bioeconomy, Potsdam-Bornim, Germany

5 Department of Farm Machinery and Power, University of Agriculture, Faisalabad, Pakistan

#### References

Author details

Model Price

Professional Electric Six Rotor Drone UA-8 Series

Yuneec H520 Hexacopter

Skytech TK110HW

JJRC H8D 5.8G FPV RTF RC

X810 Long Range Uav Sprayer

(\$)

114 Agricultural Robots - Fundamentals and Applications

Fixed Wing UG-II — UAV: 11

2500– 4500

Ag-drone AK-61 6999 Take-Off: 22

YM-6160 5000 Take-Off:

169– 175

4000– 6500

Syma X8C 68.99 — 508 508

Weight (Kg)

DJI Phantom 3 469 — — 2.7 K HD

Take-Off: 15

Payload: 10

UAV: 0.023 330 330 115

Payload: 10 2490 1645

 845 (L.W. H)

165 (L.W.H)

21.9 Payload: 10

— Payload: 3 860 860

Size (mm) Camera

2240 1600 650 (L.W. H)

— — 4 K/2 K/HD

540

resolution

videos, 12 MP photo

video or 20 MP images

2 MP HD Camera

32–52 — — 0.3 MP — 6–7 — —

Coverage Flight

time (min)

— 25 — —

— —— —

— — 180 — 65–110

— — 28 5000 36

—— — 10–15 0.5–5 m 18–36

—— — 10–15 0.5–5 m 18–36

— 8 — —

— — 25–40 — —

— 5–8 — —

Max altitude (m) Flight speed (km/h)

Serdang, Selangor, Malaysia

Redmond Ramin Shamshiri1,4\*, Ibrahim A. Hameed2

\*Address all correspondence to: raminshamshiri@upm.edu.my

1 Department of Agriculture Technology, Faculty of Agriculture, Universiti Putra Malaysia,

2 Department of ICT and Natural Sciences, Faculty of Information Technology and Electrical

3 Smart Farming Technology Research Center, Department of Biological and Agricultural Engineering, Faculty of Engineering, Universiti Putra Malaysia, Serdang, Selangor, Malaysia 4 Leibniz Institute for Agricultural Engineering and Bioeconomy, Potsdam-Bornim, Germany 5 Department of Farm Machinery and Power, University of Agriculture, Faisalabad, Pakistan

Cornelia Weltzien<sup>4</sup> and Muhammad Yamin<sup>5</sup>

Engineering, NTNU, Ålesund, Norway

, Siva K. Balasundram<sup>1</sup>

, Desa Ahmad3

,


[14] Cai G, Chen BM, Lee TH. Unmanned rotorcraft systems. New York: Springer Science & Business Media; 2011. pp. 01-267

**Section 3**

**Grading Robots for Fruit Industry**


**Grading Robots for Fruit Industry**

[14] Cai G, Chen BM, Lee TH. Unmanned rotorcraft systems. New York: Springer Science &

[15] Shamshiri RR. Choosing the Best UAV Drones for Precision Agriculture and Smart Farming: Agricultural drone buyer's guide for farmers and agriculture service professionals. Adaptive AgroTech Consultancy International; 2018. http://doi.org/10.13140/RG.2.2.19368.06409

[16] Blaschke T, Feizizadeh B, Hölbling D. Object-based image analysis and digital terrain analysis for locating landslides in the Urmia Lake Basin, Iran. IEEE Journal of Selected

[17] Yang Z. Fast template matching based on normalized cross correlation with centroid bounding. 2010 International Conference on Measuring Technology and Mechatronics

[18] Ahuja K, Tuli P. Object recognition by template matching using correlations and phase angle method. International Journal of Advanced Research in Computer Science and

[20] Tong S, Chang E. Support vector machine active learning for image retrieval. In: Proceedings of the Ninth ACM International Conference on Multimedia; 2001. pp. 107-118

[21] Kalantar B, Idrees MO, Mansor SB, Halin AA. Smart counting–oil palm tree inventory

[22] Chen SW, Shivakumar SS, Dcunha S, Das J, Okon E, Qu C, et al. Counting apples and oranges with deep learning: A data-driven approach. IEEE Robotics and Automation

[23] Li W, Guo Q, Jakubowski MK, Kelly M. A new method for segmenting individual trees from the lidar point cloud. Photogrammetric Engineering and Remote Sensing. 2012;78(1):

[24] MacKunis W, Kaiser MK, Patre PM, Dixon WE. Asymptotic tracking for aircraft via an uncertain dynamic inversion method. In: 2008 American Control Conference. 2008. pp.

[25] Kalantar B, Bin Mansor S, Halin AA, Zulhaidi H, Shafri M, Zand M. Multiple Moving Object Detection From UAV Videos Using Trajectories of Matched Regional Adjacency

[19] Lewis JP. Fast template matching. Vision interface. 1995;95(120123):15-19

Topics in Applied Earth Observations and Remote Sensing. 2014;7(12):4806-4817

Business Media; 2011. pp. 01-267

116 Agricultural Robots - Fundamentals and Applications

Automation. 2010;2:224-227

Electronics Engineering. 2013;2(3):1368-1373

with UAV. Coordinates. 2017:17-22

Letters. 2017;2(2):781-788

Graphs. 2017. pp. 1-16

75-84

3482-3487

**Chapter 7**

**Provisional chapter**

**Multimodal Classification of Mangoes**

**Multimodal Classification of Mangoes**

DOI: 10.5772/intechopen.81356

Grading, sorting, and classification of agricultural products are important steps to ensure a profitable and sustainable food industry. Human-intensive labors are replaced with better devices/machines that can be used in-line and generate sufficiently fast measurements for a high production volume. Most previous works focused on only one of the external quality parameters, such as color, size, mass, shape, and defects. In this work, we proposed an integrated machine vision system that can grade, sort, and classify mangoes using multiple features including weight, size, and external defects. We found that weight estimation using our proposed algorithm based on visual information was not statistically different from that of a conventional weight measurement using a static digital load cell; the estimation error is relatively small (4–5%). We also constructed an artificial neural network model to classify mango having multiple types of external defect; the classification error is less than 8% for the worst possible case. The results indicate that our system shows a great potential to be used in a real industrial setting. Future work will aim to investigate other features such as ripeness and bruises to increase the effectiveness

**Keywords:** mango, image processing, machine vision, weighing system,

Food standards are evolving both to ensure the sustainability of agriculture and to satisfy consumer needs. The reputation of producers and consequently their market share is based on the quality of the product, which makes quality controls very crucial. The market together with ever increasing social concerns about good agricultural practices, including environmental,

> © 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

© 2019 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use,

distribution, and reproduction in any medium, provided the original work is properly cited.

Additional information is available at the end of the chapter

Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.81356

and practicality of the system.

defect classification, real time, neural network

Son V.T. Dao

**Abstract**

**1. Introduction**

Son V.T. Dao

#### **Multimodal Classification of Mangoes Multimodal Classification of Mangoes**

#### Son V.T. Dao Son V.T. Dao

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.81356

#### **Abstract**

Grading, sorting, and classification of agricultural products are important steps to ensure a profitable and sustainable food industry. Human-intensive labors are replaced with better devices/machines that can be used in-line and generate sufficiently fast measurements for a high production volume. Most previous works focused on only one of the external quality parameters, such as color, size, mass, shape, and defects. In this work, we proposed an integrated machine vision system that can grade, sort, and classify mangoes using multiple features including weight, size, and external defects. We found that weight estimation using our proposed algorithm based on visual information was not statistically different from that of a conventional weight measurement using a static digital load cell; the estimation error is relatively small (4–5%). We also constructed an artificial neural network model to classify mango having multiple types of external defect; the classification error is less than 8% for the worst possible case. The results indicate that our system shows a great potential to be used in a real industrial setting. Future work will aim to investigate other features such as ripeness and bruises to increase the effectiveness and practicality of the system.

DOI: 10.5772/intechopen.81356

**Keywords:** mango, image processing, machine vision, weighing system, defect classification, real time, neural network

#### **1. Introduction**

Food standards are evolving both to ensure the sustainability of agriculture and to satisfy consumer needs. The reputation of producers and consequently their market share is based on the quality of the product, which makes quality controls very crucial. The market together with ever increasing social concerns about good agricultural practices, including environmental,

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. © 2019 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

economic, and social sustainability and traceability, require guarantees of high quality from the earliest stages of the crop to postharvest storage and treatments.

**2. Our research**

**2.1. Mass estimation**

cultivar (Chok Anan) from Thailand.

Consumers usually prefer fruits having almost uniform masses and shapes. This is also one of the requirements for export. However, one cannot easily model mango shapes which are not round or oval-shaped. Commonly accepted laboratory instruments are shown in **Figure 2** including a Vernier caliper for size/length measurements, a water replacement measurement setup to estimate volumes, and a planimeter to calculate areas. These methods are time-

Multimodal Classification of Mangoes http://dx.doi.org/10.5772/intechopen.81356 121

Several attempts have tried to formulate a relationship between mangoes' masses and their sizes [2–5]. Guzman-Estrada et al. [2] used a set of complicated geometrical parameters to estimate the mass of mangoes; most of the parameters can only be obtained using a mechanical measurement tool. Vasquez-Caicedo et al. [3] tried to use five parameters such as length, width, and thickness at maximum width and minimum width to estimate mango weight. Yimyam et al. [4] used four digital photographs to produce a three-dimensional model of Nam-Dokmai mangoes. Most of these methods did not provide easy-to-obtain parameters, except for Spreer et al. [5]; they provide an experimental weight-size correlation based on just three parameters—Length, Max Width (W), and Max Thickness (T) for a specific mango

Estimated mass\_ChoknanMango (grams) = 5.39 × 10−4 × L(mm) × W(mm) × T(mm) (1)

In this work, we will try to use Spreer's approach to find a meaningful relationship between shape parameters and masses of Cat-Chu mangoes. We used over 200 mangoes as a training

**Figure 2.** Equipment setup for measurement by lab instruments: (a) using a Vernier caliper—a device to measure size, (b) using water replacement method—a device to measure volume, and (c) using a planimeter—a device to calculate area.

consuming and not suitable to be implemented into a real production line.

The weight estimation method using Speer's method is shown as follows:

Optical sensors have been used extensively in the industry ranging from the automatic sorting of products into categories to the control of processes which are difficult to observe, for instance, because of their long duration [1]. At this point it is important to note that the quality of biological products is not easy to assess, as individuals of the same category may differ greatly from one to another in terms of color, shape, or size. Furthermore, because they are living products, their physiochemical properties evolve over time. Their inherent variability sometimes introduces a certain amount of subjectivity into quality control, thus increasing the difficulty involved in developing automated inspection systems. Addressing these challenges often requires research in advanced and multidisciplinary technologies and sometimes the use of expensive equipment.

In this work, our focus is on mango, *Mangifera indica*, especially the Cat-Chu cultivar due to the increasing export potential in our country—Vietnam.

Postharvest handling of mangoes is usually completed in several steps: washing, sorting, grading, packing, storage, and transportation as shown in the following **Figure 1**; among which, sorting and grading are considered the most important especially for fresh agricultural products.

Sorting of agricultural products is accomplished based on external quality parameters such as color, defects, shape, and sizes. Manual sorting is based on traditional visual quality inspection performed by trained human operators situated on one or both sides of a conveyor belt. They visually inspect the produce and remove those not satisfying the predetermined quality standards. Pieces are transported slowly enough to allow the workers to inspect all of them and even manipulate them to ensure the inspection of most of their surface. The process is normally tedious, time-consuming, subjective, slow, expensive, and nonconsistent. A cost-effective, consistent, faster, and accurate sorting can be achieved with a machine vision-assisted sorting.

In this work, we present an integrated machine vision-based inspection system including sorting, grading, and weighing of mangoes—particularly, the Cat-Chu cultivar.

**Figure 1.** Typical postharvest steps of agricultural products.

## **2. Our research**

economic, and social sustainability and traceability, require guarantees of high quality from

Optical sensors have been used extensively in the industry ranging from the automatic sorting of products into categories to the control of processes which are difficult to observe, for instance, because of their long duration [1]. At this point it is important to note that the quality of biological products is not easy to assess, as individuals of the same category may differ greatly from one to another in terms of color, shape, or size. Furthermore, because they are living products, their physiochemical properties evolve over time. Their inherent variability sometimes introduces a certain amount of subjectivity into quality control, thus increasing the difficulty involved in developing automated inspection systems. Addressing these challenges often requires research in advanced and multidisciplinary technologies and sometimes the use of expensive equipment. In this work, our focus is on mango, *Mangifera indica*, especially the Cat-Chu cultivar due to

Postharvest handling of mangoes is usually completed in several steps: washing, sorting, grading, packing, storage, and transportation as shown in the following **Figure 1**; among which, sorting and grading are considered the most important especially for fresh agricul-

Sorting of agricultural products is accomplished based on external quality parameters such as color, defects, shape, and sizes. Manual sorting is based on traditional visual quality inspection performed by trained human operators situated on one or both sides of a conveyor belt. They visually inspect the produce and remove those not satisfying the predetermined quality standards. Pieces are transported slowly enough to allow the workers to inspect all of them and even manipulate them to ensure the inspection of most of their surface. The process is normally tedious, time-consuming, subjective, slow, expensive, and nonconsistent. A cost-effective, consistent, faster, and accurate sorting can be achieved with a machine vision-assisted sorting.

In this work, we present an integrated machine vision-based inspection system including

sorting, grading, and weighing of mangoes—particularly, the Cat-Chu cultivar.

the earliest stages of the crop to postharvest storage and treatments.

120 Agricultural Robots - Fundamentals and Applications

the increasing export potential in our country—Vietnam.

**Figure 1.** Typical postharvest steps of agricultural products.

tural products.

#### **2.1. Mass estimation**

Consumers usually prefer fruits having almost uniform masses and shapes. This is also one of the requirements for export. However, one cannot easily model mango shapes which are not round or oval-shaped. Commonly accepted laboratory instruments are shown in **Figure 2** including a Vernier caliper for size/length measurements, a water replacement measurement setup to estimate volumes, and a planimeter to calculate areas. These methods are timeconsuming and not suitable to be implemented into a real production line.

Several attempts have tried to formulate a relationship between mangoes' masses and their sizes [2–5]. Guzman-Estrada et al. [2] used a set of complicated geometrical parameters to estimate the mass of mangoes; most of the parameters can only be obtained using a mechanical measurement tool. Vasquez-Caicedo et al. [3] tried to use five parameters such as length, width, and thickness at maximum width and minimum width to estimate mango weight. Yimyam et al. [4] used four digital photographs to produce a three-dimensional model of Nam-Dokmai mangoes. Most of these methods did not provide easy-to-obtain parameters, except for Spreer et al. [5]; they provide an experimental weight-size correlation based on just three parameters—Length, Max Width (W), and Max Thickness (T) for a specific mango cultivar (Chok Anan) from Thailand.

The weight estimation method using Speer's method is shown as follows:

Estimated mass\_ChoknanMango (grams) = 5.39 × 10−4 × L(mm) × W(mm) × T(mm) (1)

In this work, we will try to use Spreer's approach to find a meaningful relationship between shape parameters and masses of Cat-Chu mangoes. We used over 200 mangoes as a training

**Figure 2.** Equipment setup for measurement by lab instruments: (a) using a Vernier caliper—a device to measure size, (b) using water replacement method—a device to measure volume, and (c) using a planimeter—a device to calculate area.

dataset to establish the necessary weight-size relationship. Fortunately, we also obtain a linear relationship as shown in the following **Figure 3**. The constant in our case is 4.879 × 10−<sup>4</sup> . The obtained R<sup>2</sup> is about 97.6%.

Our estimated mass is:

$$\text{Estimated mass\\_CatChulMango (grams)} = 4.879 \, 39 \times 10^{-4} \, \text{L} \times \text{W} \times \text{T} \tag{2}$$

For a more general discussion regarding thresholding techniques, please read the reference

This used to be one of the de facto algorithms in image segmentation [7]. An image is a twodimensional matrix of N pixels, each with an intensity level between 0 and L-1, where L is the number of distinct gray levels. The number of pixels with a certain gray level i is denoted as

> *i*=0 *L*−1

which are formed by the area of pixels with gray levels [1,…,t] and [t,…L], respectively, where

and C<sup>2</sup>

*Pi* and *ω*<sup>2</sup>

(*t*) and *<sup>C</sup>*<sup>2</sup> <sup>=</sup> *<sup>P</sup>*\_\_\_\_*<sup>t</sup>*+1

(*t*) and *μ*<sup>2</sup> = ∑

Using discriminant analysis, Otsu [7] showed that the optimal threshold t\* can be determined

*ArgMax*{*σ <sup>B</sup>*

Otsu's method works well when the images have clear peaks and valleys—in other words, it works for images whose histograms show clear bimodal or multimodal distributions. There are times when histograms of images contain several different types with widely varied number of pixels, such as external defects; Otsu's method will not give the correct threshold level

<sup>2</sup> (*t*)

<sup>2</sup> + *ω*<sup>2</sup> · (*μ*<sup>2</sup> − *μT*)

are

*ω*2 (*t*) , *<sup>P</sup>*\_\_\_\_*<sup>t</sup>*+2 *ω*2 (*t*)

(*t*) = ∑ *i*=*t*+1 *L*

> *i*=*t*+1 *<sup>L</sup> <sup>i</sup>* · *<sup>P</sup>* \_\_\_\_*<sup>i</sup> ω*2

By segmenting the image using a single threshold, we get two disjoint regions C<sup>1</sup>

and C<sup>2</sup>

, …, *<sup>P</sup>*\_\_\_\_*<sup>t</sup> ω*1

(*t*) = ∑ *i*=1 *t*

> *i*=1 *<sup>t</sup> <sup>i</sup>* · *<sup>P</sup>* \_\_\_\_*<sup>i</sup> ω*1

<sup>2</sup> = *ω*<sup>1</sup> · (*μ*<sup>1</sup> − *μT*)

The mean gray-level values of the two classes can be computed as

/N (3)

Multimodal Classification of Mangoes http://dx.doi.org/10.5772/intechopen.81356 123

*i pi* (4)

correspond to the object of interest and the back-

…, *<sup>P</sup>*\_\_\_\_*<sup>L</sup> ω*2

 and C<sup>2</sup> ,

(*t*) (5)

*Pi* (6)

(*t*) (7)

} (8)

<sup>2</sup> (9)

, and the probability of occurrence of gray level i is given by

*ω*1 (*t*)

The average of the intensity level of the whole image can be calculated as

Pi = f<sup>i</sup>

*μT* = ∑

t is the threshold level. Normally, C<sup>1</sup>

*<sup>C</sup>*<sup>1</sup> <sup>=</sup> *<sup>P</sup>*\_\_\_\_*<sup>i</sup>*

*ω*<sup>1</sup>

*μ*<sup>1</sup> = ∑

*σ <sup>B</sup>*

as shown in the following **Figure 4**.

by maximizing the between-class variance, that is

*<sup>t</sup>*<sup>∗</sup> <sup>=</sup> <sup>0</sup> <sup>≤</sup> *<sup>t</sup>* <sup>&</sup>lt; *<sup>L</sup>*

where the between-class variance σB is defined as

ground. The probability distributions of C<sup>1</sup>

"Machine Vision" by Davies [6].

*2.2.1. Otsu's method*

f i

To validate our findings, we collected an additional 68 mangoes to be used as a validation dataset. The accuracy achieved is impressive, with an average error percentage of 3.23%. This further proves that the simple, linear correlation between mass and sizes can be used to estimate the corresponding mass effectively.

We also designed and constructed an image capturing platform to obtain the images from two different viewpoints (top and side views). The platform would also be used to test the algorithm's ability to estimate mangoes' masses solely based on their sizes. An algorithm was developed to capture and process the images while mangoes travel along a conveyor.

Top and side views of the mango were captured to estimate the mango mass using Eq. (2), and the result will also be compared with conventional mass measurement using a calibrated digital scale. We found that the difference between the masses estimated using this technique was not statistically different from the conventional method using a digital scale (p < 0.05). Classification result showed an accuracy of 95–96% when grading mangoes solely based on masses.

#### **2.2. Image segmentation**

In this section we review a few methods for automatic selection of threshold values; the most important methods that we will discuss are Otsu's method and the valley-emphasis method.

**Figure 3.** Correlation between mango masses and their sizes: L, T, and W.

For a more general discussion regarding thresholding techniques, please read the reference "Machine Vision" by Davies [6].

#### *2.2.1. Otsu's method*

. The

dataset to establish the necessary weight-size relationship. Fortunately, we also obtain a linear relationship as shown in the following **Figure 3**. The constant in our case is 4.879 × 10−<sup>4</sup>

Estimated mass\_CatChuMango (grams) = 4.879.39 × 10−4 × L × W × T (2)

To validate our findings, we collected an additional 68 mangoes to be used as a validation dataset. The accuracy achieved is impressive, with an average error percentage of 3.23%. This further proves that the simple, linear correlation between mass and sizes can be used to esti-

We also designed and constructed an image capturing platform to obtain the images from two different viewpoints (top and side views). The platform would also be used to test the algorithm's ability to estimate mangoes' masses solely based on their sizes. An algorithm was

Top and side views of the mango were captured to estimate the mango mass using Eq. (2), and the result will also be compared with conventional mass measurement using a calibrated digital scale. We found that the difference between the masses estimated using this technique was not statistically different from the conventional method using a digital scale (p < 0.05). Classification

In this section we review a few methods for automatic selection of threshold values; the most important methods that we will discuss are Otsu's method and the valley-emphasis method.

developed to capture and process the images while mangoes travel along a conveyor.

result showed an accuracy of 95–96% when grading mangoes solely based on masses.

**Figure 3.** Correlation between mango masses and their sizes: L, T, and W.

obtained R<sup>2</sup>

Our estimated mass is:

**2.2. Image segmentation**

is about 97.6%.

122 Agricultural Robots - Fundamentals and Applications

mate the corresponding mass effectively.

This used to be one of the de facto algorithms in image segmentation [7]. An image is a twodimensional matrix of N pixels, each with an intensity level between 0 and L-1, where L is the number of distinct gray levels. The number of pixels with a certain gray level i is denoted as f i , and the probability of occurrence of gray level i is given by

$$\mathbf{P}\_i = \mathbf{f}\_i / \mathbf{N} \tag{3}$$

The average of the intensity level of the whole image can be calculated as

$$
\mu\_T = \sum\_{i=0}^{\left\lfloor \sum i \right\rfloor} i \, p\_i \tag{4}
$$

By segmenting the image using a single threshold, we get two disjoint regions C<sup>1</sup> and C<sup>2</sup> , which are formed by the area of pixels with gray levels [1,…,t] and [t,…L], respectively, where t is the threshold level. Normally, C<sup>1</sup> and C<sup>2</sup> correspond to the object of interest and the background. The probability distributions of C<sup>1</sup> and C<sup>2</sup> are

$$\mathbf{C}\_{1} = \frac{P\_{i}}{\omega\_{1}\ell\ell'} \dots \frac{P\_{i}}{\omega\_{1}\ell\ell} \text{ and } \mathbf{C}\_{2} = \frac{P\_{\nu+1}}{\omega\_{2}\ell\ell'} \frac{P\_{\nu+2}}{\omega\_{2}\ell\ell} \dots \frac{P\_{\nu}}{\omega\_{2}\ell\ell} \tag{5}$$

$$
\omega\_1(t) = \sum\_{i=1}^{l} P\_i \quad \text{and} \quad \omega\_2(t) = \sum\_{i=l+1}^{L} P\_i \tag{6}
$$

The mean gray-level values of the two classes can be computed as

$$
\mu\_1 = \sum\_{i=1}^{l} \frac{i \cdot P\_i}{\omega\_1(t)} \quad \text{and} \quad \mu\_2 = \sum\_{i=1}^{l} \frac{i \cdot P\_i}{\omega\_2(t)} \tag{7}
$$

Using discriminant analysis, Otsu [7] showed that the optimal threshold t\* can be determined by maximizing the between-class variance, that is

$$t^\* = \mathop{\rm{AlgMax}}\_{0 \le t \le L} \left\{ \sigma^2\_{\mathcal{B}}(t) \right\} \tag{8}$$

where the between-class variance σB is defined as

$$
\sigma\_{\rm B}^2 = \omega\_1 \cdot (\mu\_1 - \mu\_1)^2 + \omega\_2 \cdot (\mu\_2 - \mu\_1)^2 \tag{9}
$$

Otsu's method works well when the images have clear peaks and valleys—in other words, it works for images whose histograms show clear bimodal or multimodal distributions. There are times when histograms of images contain several different types with widely varied number of pixels, such as external defects; Otsu's method will not give the correct threshold level as shown in the following **Figure 4**.

**Figure 4.** Segmentation result using Otsu's algorithm: (a) original image, (b) histogram with the Otsu threshold level (red), and (c) resulting image.

#### *2.2.2. Valley-emphasis method*

To improve drawbacks of Otsu's method, Ng et al. [8] proposed the valley-emphasis method. The idea of the valley-emphasis method is to select a threshold value that has a small probability of occurrence (valley in the gray-level histogram) and also maximize the between-group variance, as in Otsu's method. The formulation for the valley-emphasis method is

$$t^\* = \int\_{0 \le t \le L} {}^{AvgMax} \left\{ (1 - P\_i) \Big( \omega\_1(t) \, \mu\_1^2(t) + \omega\_2(t) \, \mu\_2^2(t) \Big) \right\} \tag{10}$$

**2.3. Defect isolation**

threshold level (red), and (c) resulting image.

*2.3.1. Defect isolation*

contrast enhancement.

Due to their green appearances, we use G channel as the main channel, since it will be much easier to observe defects. To make the defects stand out, we use a simple linear contrast enhancement as shown in [3]. The results shown in **Figure 6** illustrate the effectiveness of the

**Figure 6.** Contrast improvement after background removal: (a) before and (b) after.

**Figure 5.** Segmentation result using valley-emphasis method: (a) original image, (b) histogram with the valley-emphasis

Multimodal Classification of Mangoes http://dx.doi.org/10.5772/intechopen.81356 125

The addition of an extra weight factor, (1-P<sup>t</sup> ), ensures the calculated threshold having a small probability of occurrence Pt will always be selected. Hence, the name valley-emphasis because the threshold level will always reside at the valley of the histogram. For images that have apparent bimodal distribution, the valley-emphasis method should give a threshold value that is very close to the value generated by Otsu's method because both methods attempt to maximize the between-group variance of the histogram.

The same segmentation experiment done previously using Otsu's method is repeated using the valley-emphasis method as shown in **Figure 5**. We can clearly observe that the segmentation result is much better. And, the result can be utilized for further analysis steps.

**Figure 5.** Segmentation result using valley-emphasis method: (a) original image, (b) histogram with the valley-emphasis threshold level (red), and (c) resulting image.

#### **2.3. Defect isolation**

#### *2.3.1. Defect isolation*

*2.2.2. Valley-emphasis method*

124 Agricultural Robots - Fundamentals and Applications

(red), and (c) resulting image.

*<sup>t</sup>*<sup>∗</sup> <sup>=</sup> <sup>0</sup> <sup>≤</sup> *<sup>t</sup>* <sup>&</sup>lt; *<sup>L</sup>*

The addition of an extra weight factor, (1-P<sup>t</sup>

maximize the between-group variance of the histogram.

*ArgMax*

{(1 − *Pt*

method is

steps.

To improve drawbacks of Otsu's method, Ng et al. [8] proposed the valley-emphasis method. The idea of the valley-emphasis method is to select a threshold value that has a small probability of occurrence (valley in the gray-level histogram) and also maximize the between-group variance, as in Otsu's method. The formulation for the valley-emphasis

**Figure 4.** Segmentation result using Otsu's algorithm: (a) original image, (b) histogram with the Otsu threshold level

)(*ω*<sup>1</sup> (*t*) *μ*<sup>1</sup>

probability of occurrence Pt will always be selected. Hence, the name valley-emphasis because the threshold level will always reside at the valley of the histogram. For images that have apparent bimodal distribution, the valley-emphasis method should give a threshold value that is very close to the value generated by Otsu's method because both methods attempt to

The same segmentation experiment done previously using Otsu's method is repeated using the valley-emphasis method as shown in **Figure 5**. We can clearly observe that the segmentation result is much better. And, the result can be utilized for further analysis

<sup>2</sup>(*t*) + *ω*<sup>2</sup>

(*t*) *μ*<sup>2</sup>

), ensures the calculated threshold having a small

<sup>2</sup>(*t*))} (10)

Due to their green appearances, we use G channel as the main channel, since it will be much easier to observe defects. To make the defects stand out, we use a simple linear contrast enhancement as shown in [3]. The results shown in **Figure 6** illustrate the effectiveness of the contrast enhancement.

**Figure 6.** Contrast improvement after background removal: (a) before and (b) after.

After image enhancement, we apply another round of valley-emphasis segmentation on the area of the mango mask to isolate the defect zones. The result was illustrated as shown in **Figure 7**.

*2.3.2. Defect classification*

*2.3.2.1. Color features*

*2.3.2.2. Shape features*

There are many kinds of defects that negatively degrade mangoes' quality [9]. Among them, four kinds that are most commonly seen are shown in **Figure 9** including stripe-type scars, dark patches, sap burns, and small spots. The defect classification steps will help us know

Multimodal Classification of Mangoes http://dx.doi.org/10.5772/intechopen.81356 127

We use an artificial neural network with inputs as color features, shape features, and image statistical information. Li et al. [10] suggested that using HSV (HSI) instead of RGB color space improves segmentation results. In this research, there are 18H bins, 3S bins, and 3V

To calculate shape features, we used the moment invariant proposed by Hu [11] with practi-

how many kinds of defects are present on the fruit skin area as shown in **Figure 10**.

bins. Therefore, we will have 162 features in HSV space.

cal implementations by OpenCV as in [12].

**Figure 9.** Four typical types of defects (I) to (IV).

**Figure 10.** Proposed image processing procedures.

To simplify the calculation effort, we only concentrate on defects that are equal to or larger than 30 pixels. After segmenting the defect zones from the previous steps, we will use their sizes and locations on the original image to generate the new defect candidate for further classification steps as shown in **Figure 8**.

**Figure 7.** Defect zone isolation: (a) original image, (b) after background removal and contrast enhancement, and (c) defect isolation result.

**Figure 8.** Defect zones on the original image.

#### *2.3.2. Defect classification*

After image enhancement, we apply another round of valley-emphasis segmentation on the area of the mango mask to isolate the defect zones. The result was illustrated as shown in

To simplify the calculation effort, we only concentrate on defects that are equal to or larger than 30 pixels. After segmenting the defect zones from the previous steps, we will use their sizes and locations on the original image to generate the new defect candidate for further

**Figure 7.** Defect zone isolation: (a) original image, (b) after background removal and contrast enhancement, and (c)

**Figure 7**.

defect isolation result.

**Figure 8.** Defect zones on the original image.

classification steps as shown in **Figure 8**.

126 Agricultural Robots - Fundamentals and Applications

There are many kinds of defects that negatively degrade mangoes' quality [9]. Among them, four kinds that are most commonly seen are shown in **Figure 9** including stripe-type scars, dark patches, sap burns, and small spots. The defect classification steps will help us know how many kinds of defects are present on the fruit skin area as shown in **Figure 10**.

#### *2.3.2.1. Color features*

We use an artificial neural network with inputs as color features, shape features, and image statistical information. Li et al. [10] suggested that using HSV (HSI) instead of RGB color space improves segmentation results. In this research, there are 18H bins, 3S bins, and 3V bins. Therefore, we will have 162 features in HSV space.

#### *2.3.2.2. Shape features*

To calculate shape features, we used the moment invariant proposed by Hu [11] with practical implementations by OpenCV as in [12].

**Figure 9.** Four typical types of defects (I) to (IV).

#### *2.3.2.3. Histogram-based features*

The histogram-based features used in this work are first-order statistics that include mean, variance, skewness, and kurtosis for all R, G, and B channels. Let z be a random variable denoting image gray levels and p(zi), i = 0, 1, 2, 3, ……. L-1, be the corresponding histogram, where L is the number of distinct gray levels. The five following features for each color channel are calculated using the abovementioned histogram:

$$\text{Mean} = \sum\_{l=0}^{l-1} z\_l p(z\_l) \tag{11}$$

$$Variance = \sum\_{i=0}^{l-1} \left( \mathbf{z}\_i - Mean \right)^2 p(\mathbf{z}\_i) \tag{12}$$

$$\text{Skewness} = \sum\_{i=0}^{l-1} \{\mathbf{z}\_i - \text{Mean}\}^3 p(\mathbf{z}\_i) \tag{13}$$

$$Kurtosis = \sum\_{i=0}^{l-1} \left( \mathbf{z}\_i - Mean \right)^4 p(\mathbf{z}\_i) \tag{14}$$

$$Range = \frac{\max - \min}{255} \tag{15}$$

The classification results are summarized in **Table 1**. From the statistics, we can see that the classification accuracy reduces with an increasing number of defect zones and it also takes more computation time. The result is quite promising to be applicable to an automated sorting and grading system. In the current version, no acceleration techniques have been applied; in the near future, advanced parallel programming technique using graphics processing units (GPU) can be utilized to speed up the process, hopefully, to achieve a real-time performance level.

**Figure 11.** Our proposed feed-forward neural network with 193 inputs, 98 neurons in a hidden layer, and 4 outputs.

Number of photos 70 50 40 30 30 **220** Number of defects 70 106 272 425 722 **1595** Number of defects correctly identified 70 103 258 392 657 **1480** Number of wrong identifications 0 3 14 33 65 **115** Accuracy (%) 100 97.17 94.85 92.24 91.0 **92.79**

**Set 1 Set 2 Set 3 Set 4 Set 5 Total**

Multimodal Classification of Mangoes http://dx.doi.org/10.5772/intechopen.81356 129

In this work, we have established an integrated framework for an automated grading, sorting, and weighing system of Cat-Chu mangoes using features including weight, size, and external defects. We found a simple, easy-to-calculate formulation between simple parameters and mango mass. The estimation error is very small, less than 3% if we use a mechanical measurement tool and less than 5% if we use an optical measurement using top- and side-view image captures. We also proposed an innovative procedure to classify external defects based on an

**3. Conclusion**

**Table 1.** Summary of classification results.

#### *2.3.2.4. Manual labeling of training data*

We prepare our dataset with standardized defect templates of 20 × 20 using 193 abovementioned features: 162 color-based, 16 shape-based, and 15 histogram-based. We also manually label different images in the training dataset with different kinds of defects. For example, "Image 1" has nine defect zones, one is the first defect type and the rest are the fourth defect type. The procedure is applied similarly for the rest of the training images.

#### *2.3.2.5. Building a neural network model*

Our classification problem is a nonlinear one with 193 inputs corresponding to 193 chosen features and 4 outputs corresponding to four types of defects. Usually, a number of hidden layers are experimentally chosen to be a half of all the number of inputs and outputs [3]. Therefore, we chose 98 hidden neurons (=(193 + 4)/2). The neural network model is illustrated in **Figure 11**.

We split our dataset into five smaller ones with different characteristics:


**Figure 11.** Our proposed feed-forward neural network with 193 inputs, 98 neurons in a hidden layer, and 4 outputs.


**Table 1.** Summary of classification results.

The classification results are summarized in **Table 1**. From the statistics, we can see that the classification accuracy reduces with an increasing number of defect zones and it also takes more computation time. The result is quite promising to be applicable to an automated sorting and grading system. In the current version, no acceleration techniques have been applied; in the near future, advanced parallel programming technique using graphics processing units (GPU) can be utilized to speed up the process, hopefully, to achieve a real-time performance level.

#### **3. Conclusion**

*2.3.2.3. Histogram-based features*

128 Agricultural Robots - Fundamentals and Applications

nel are calculated using the abovementioned histogram:

*Mean* = ∑

*Variance* = ∑

*Skewness* = ∑

*Kurtosis* = ∑

*2.3.2.4. Manual labeling of training data*

*2.3.2.5. Building a neural network model*

• Set 1: All images show no defects.

• Set 2: Images show only one type of defects. • Set 3: Images show only two types of defects.

• Set 4: Images show three types of defects. • Set 5: Images have all four types of defects.

in **Figure 11**.

*Range* = \_\_\_\_\_\_\_\_\_

The histogram-based features used in this work are first-order statistics that include mean, variance, skewness, and kurtosis for all R, G, and B channels. Let z be a random variable denoting image gray levels and p(zi), i = 0, 1, 2, 3, ……. L-1, be the corresponding histogram, where L is the number of distinct gray levels. The five following features for each color chan-

> *i*=0 *L*−1 *zi p*(*zi*

(*zi* − *Mean*)<sup>2</sup> *p*(*zi*

(*zi* − *Mean*)<sup>3</sup> *p*(*zi*

(*zi* − *Mean*)<sup>4</sup> *p*(*zi*

*max*−*min*

*i*=0 *L*−1

> *i*=0 *L*−1

*i*=0 *L*−1

We prepare our dataset with standardized defect templates of 20 × 20 using 193 abovementioned features: 162 color-based, 16 shape-based, and 15 histogram-based. We also manually label different images in the training dataset with different kinds of defects. For example, "Image 1" has nine defect zones, one is the first defect type and the rest are the fourth defect

Our classification problem is a nonlinear one with 193 inputs corresponding to 193 chosen features and 4 outputs corresponding to four types of defects. Usually, a number of hidden layers are experimentally chosen to be a half of all the number of inputs and outputs [3]. Therefore, we chose 98 hidden neurons (=(193 + 4)/2). The neural network model is illustrated

type. The procedure is applied similarly for the rest of the training images.

We split our dataset into five smaller ones with different characteristics:

) (11)

) (12)

) (13)

) (14)

<sup>255</sup> (15)

In this work, we have established an integrated framework for an automated grading, sorting, and weighing system of Cat-Chu mangoes using features including weight, size, and external defects. We found a simple, easy-to-calculate formulation between simple parameters and mango mass. The estimation error is very small, less than 3% if we use a mechanical measurement tool and less than 5% if we use an optical measurement using top- and side-view image captures. We also proposed an innovative procedure to classify external defects based on an artificial neural network. The classification error is less than 8% for the worst possible case. The results indicate that our system has a great potential to be used in a real industrial setting. Future work will aim to investigate other features such as ripeness and bruises to increase the effectiveness and practicality of the system and possible speedup to real-time performance using advanced graphics processing unit (GPU) and further code parallelism.

[6] Davies ER. Machine Vision. 3rd ed. Morgan Kaufman; 2004. p. 938. DOI: 10.1016/B978-

Multimodal Classification of Mangoes http://dx.doi.org/10.5772/intechopen.81356 131

[7] Otsu N. A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics. 1979;**9**(1):62-66. DOI: 10.1109/TSMC.1979.4310076 [8] Ng HF. Automatic thresholding for defect detection. Pattern Recognition Letters. 2006;

[9] Mango Quality Standard. Available from: https://www.industry.mangoes.net.au/resourcecollection/2016/9/29/2016-mango-industry-quality-standards [Accessed: 31-08-2018] [10] Qingzhong L, Maohua W, Weikang G. Computer vision based system for apple surface defect detection. Computers and Electronics in Agriculture. 2002;**36**(2):215-223. DOI:

[11] Hu MK. Visual pattern recognition by moment invariants. IRE Transactions on Infor-

[12] OpenCV Tutorial. Available from: http://docs.opencv.org/doc/tutorials/tutorials.html

mation Theory. 1962;**8**(2):179-187. DOI: 10.1109/TIT.1962.1057692

0-12-206093-9.X5000-X

10.1016/s0168-1699(02)00093-5

[Accessed: 30-07-2018]

**27**(14):1644-1649. DOI: 10.1016/j.patrec.2006.03.009

### **Acknowledgements**

This work is being carried out at the International University, Vietnam National University Ho Chi Minh City, and being funded by Ho Chi Minh City Department of Science and Technology under the Contract Number 236/2017/HD-SKHCN. The project title is "Research, design, fabricate a prototype of a mango sorter for export."

## **Author details**

#### Son V.T. Dao

Address all correspondence to: dvtson@hcmiu.edu.vn

Department of Industrial and Systems Engineering, International University—Vietnam National University, Ho Chi Minh City, Vietnam

#### **References**


[6] Davies ER. Machine Vision. 3rd ed. Morgan Kaufman; 2004. p. 938. DOI: 10.1016/B978- 0-12-206093-9.X5000-X

artificial neural network. The classification error is less than 8% for the worst possible case. The results indicate that our system has a great potential to be used in a real industrial setting. Future work will aim to investigate other features such as ripeness and bruises to increase the effectiveness and practicality of the system and possible speedup to real-time performance

This work is being carried out at the International University, Vietnam National University Ho Chi Minh City, and being funded by Ho Chi Minh City Department of Science and Technology under the Contract Number 236/2017/HD-SKHCN. The project title is "Research,

Department of Industrial and Systems Engineering, International University—Vietnam

[1] Cubero S, Aleixos N, Moltó E, Gómez-Sanchis J, Blasco J. Advances in machine vision applications for automatic inspection and quality evaluation of fruits and vegetables. Food and Bioprocess Technology. 2011;**4**:487-504. DOI: 10.1007/s11947-010-0411-8

[2] Guzman-Estrada C, Mosqueda Vazques R, Alcalde-Blanco S, Marinez-Garza A. Equation to estimate volume and growth dynamics of mango 'Manila' fruit. Acta Horticulturae.

[3] Vasquez-Caicedo AL, Neidhart S, Pathomrungsiyounggul P, Wiriyacharee P, Chattrakul A, Sruamsiri P, et al. Physical, chemical and sensory properties of 9 Thai mango cultivars and evaluation of their technological and nutritional potential. In: Conference for Sustaining Food Security and Managing Natural Resources in Southeast Asia; 8-11

[4] Chalidabhongse T, Yimyam P, Sirisomboon P. 2D/3D vision-based mango's feature extraction and sorting. In: 9th International Conference on Control, Automation,

[5] Spreer W, Müller J. Estimating the mass of mango fruit (*Mangifera indica*, cv. Chok Anan) from its geometric dimensions by optical measurement. Computers and Electronics in

Robotics and Vision, ICARV'06; 5-8 December; Singapore. 2006. pp. 1-6

Agriculture. 2011;**75**:125-131. DOI: 10.1016/j.compag.2010.10.007

using advanced graphics processing unit (GPU) and further code parallelism.

design, fabricate a prototype of a mango sorter for export."

Address all correspondence to: dvtson@hcmiu.edu.vn

1997;**455**:449-454. DOI: 10.17660/ActaHortic.1997.455.57

National University, Ho Chi Minh City, Vietnam

January; Chiang Mai, Thailand. 2002

**Acknowledgements**

130 Agricultural Robots - Fundamentals and Applications

**Author details**

Son V.T. Dao

**References**


## *Edited by Jun Zhou and Baohua Zhang*

Over the past few decades, extensive research has been conducted on the applications of agricultural robots and automation to a variety of field and greenhouse operations, and technical fundamentals and their feasibility have also been widely demonstrated. Due to the unstructured environment, adverse interference and complicated and diversified operation process are the key of blocking its commercialization in robotic agricultural operations. Because of the development of automation techniques, smart sensors, and information techniques, some types of agricultural robots have achieved considerable success in recent years. This book intends to provide the reader with a comprehensive overview of the current state of the art in agricultural robots, fundamentals, and applications in robotic agricultural operations.

Published in London, UK © 2019 IntechOpen © Ekkasit919 / iStock

Agricultural Robots - Fundamentals and Applications

Agricultural Robots

Fundamentals and Applications

*Edited by Jun Zhou and Baohua Zhang*